Details
-
Task
-
Resolution: Done
-
Major
-
master
-
None
Description
Flusher and Compaction tasks in RocksDB are per database (i.e vBucket), not global - however we initially thought they were. As such, the existing code to control the number of flushers/compactors always sets the number of background threads to the same as the number of tasks which run on those threads.
That would be fine if we had a single RocksDB database - we would have:
- F flusher tasks running on F high priority threads, and
- C compaction tasks runnning on C high priority threads.
However, given we have one DB per vBucket, this means we end up with:
- F * V flusher tasks running on F threads, and
- C * V compaction tasks running on C threads.
Where V is the number of vBuckets.
As a result we cannot configure optimally in either direction - we either end up with too many tasks scheduled across too many threads, or too few tasks scheduled over too few threads.
Attachments
For Gerrit Dashboard: MB-26681 | ||||||
---|---|---|---|---|---|---|
# | Subject | Branch | Project | Status | CR | V |
85126,5 | MB-26681: RocksDB: Add config params for num background threads | master | kv_engine | Status: MERGED | +2 | +1 |