Details
-
Bug
-
Status: Closed
-
Major
-
Resolution: Fixed
-
6.5.0, 6.5.1
-
Triaged
-
Unknown
-
KV Sprint 2020-April
Description
Discovered whilst investigating MB-38012 that we are currently tracking checkpoint memory overhead incorrectly. We appear to track the allocations of the checkpoint_index map, but not of the keys used in them.
Consider the case where we have a large disk checkpoint that is entirely non-resident in the checkpoint (i.e. everything has been expelled) then the keys are a substantial part of this. In the case of MB-38012 we are tracking approx 80 bytes overhead per item, but this does not include the keys which are ~48 bytes. We were tracking 27GB memory overhead in the checkpoint but in reality this is more like 40GB.
Attachments
Activity
Build couchbase-server-6.6.0-7589 contains kv_engine commit bbaf3ba with commit message:
MB-38173: Track key allocations as part of checkpoint overhead
Build couchbase-server-7.0.0-2030 contains kv_engine commit bbaf3ba with commit message:
MB-38173: Track key allocations as part of checkpoint overhead
Closing this ticket based on the unit_test results.
Commit ref for kv_engine: ee5212ee2faf739a70a342ade60a64eb0191e421
This is the problematic code (in Checkpoint)
size_t getMemoryOverhead() const {// All 3 indexes (preparedKey, committedKey and metaKey) share the// same allocator and therefore getting the bytes allocated for the// one will include the others.return sizeof(Checkpoint) +*(committedKeyIndex.get_allocator().getBytesAllocated()) +*(toWrite.get_allocator().getBytesAllocated());}