Details
-
Improvement
-
Resolution: Done
-
Major
-
5.5.0
-
CX Sprint 102
Description
Analytics supports the configuration of "iodevices". An iodevice is a unit of external storage that can be used independently of other iodevices (it's an abstraction of a disk).
Currently the number of iodevices determines the degree of parallelism for IO operations and all query plan operators above the IO operation up to the point where the plan needs to redistribute data.
For a simple aggregation query (e.g. select count(1) from ... where ...) redistribution is only needed in the final step and so the degree of parallelism of the entire query is limited by the number of iodevices.
Having an (easily configurable) single iodevice can therefore limit performance significantly, while confguring multiple iodevices makes the configuration more complicated and error prone.
We need a way to configure Analytics nodes easily without restricting the degree of parallelism unnecessarily.