Details
-
Improvement
-
Resolution: Resolved
-
Major
-
Morpheus
-
0
-
Analytics Sprint 15
Description
Currently, Analytics uses IVisitablePointable to allow functions (e.g., IPrintVisitor) to traverse and process (print for IPrintVisitor) nested values. The problem is that IVisitablePointable eagerly materializes all nested values into temporary buffers before a nested value could be processed, which is expensive.
For the columnar format, Analytics needs to infer the schema and columnize the rows during data ingestion, which requires traversing nested values. Using IVisitablePointable in this case could hinder the ingestion rate drastically.
Attachments
Issue Links
- links to