Details
-
Improvement
-
Resolution: Done
-
Major
-
7.0.0
-
1
-
KV-Engine Sprint 2021 August, KV-Engine-Sept-21
Description
EPEngine already allows any max number of checkpoints in the CM::CheckpointList:
"max_checkpoints": {
|
"default": "2",
|
"dynamic": true,
|
"type": "size_t"
|
},
|
The problem with increasing that number is that we have O(N = ckpt-list-size) code around that executes under CM::lock, so that possibly blocking frontend operations and degrading (massively, depending on the CheckpointList size) frontend throughput.
So essentially for allowing a higher number of checkpoints we need to remove any O(N = ckpt-list-size) code, which is what this MB tracks.
The most (in)famous example is the code in CheckpointManager::removeClosedUnrefCheckpoints, where we acquire the CM::lock and scan the entire CheckpointList.
At the time of writing the main purpose of this work is helping in the memory deallocation of checkpoints. Eg see MB-43299.
Also, this is a pre-requirement for MB-47388 (fine-grained locking in CM).
Attachments
For Gerrit Dashboard: MB-47386 | ||||||
---|---|---|---|---|---|---|
# | Subject | Branch | Project | Status | CR | V |
157970,2 | MB-47386: Remove CM::checkOpenCheckpoint | master | kv_engine | Status: NEW | -1 | 0 |
159474,3 | MB-47386: Make CM::getMemoryOverhead O(1) | master | kv_engine | Status: NEW | 0 | 0 |