Details
-
Bug
-
Resolution: Fixed
-
Minor
-
7.1.0
-
Untriaged
-
1
-
Unknown
-
KV 2022-Jan
Description
When investigating MB-48070, it was observed that the Ephemeral memory recovery tasks (Eph tombstone hashtable cleaner, Eph tombstone stale item deleter) are running constantly during a number of ep_testsuite_dcp tests.
Investigation found that these tests only have a single vBucket in the replica state (no active vBuckets). When checking to see if memory should be recovered (via KVBucket::checkAndMaybeFreeMemory), the lack of any active vBuckets causes EphemeralBucket::getPageableMemHighWatermark() to return 0 - i.e. indicating that memory recovery should be attempted if there is any memory used:
// Trigger memory reduction (ItemPager) if we've exceeded the pageable high
|
// watermark.
|
void KVBucket::checkAndMaybeFreeMemory() { |
if (getPageableMemCurrent() > getPageableMemHighWatermark()) { |
attemptToFreeMemory();
|
}
|
}
|
When attemptToFreeMemory() is called and the aforementioned memory recovery tasks are run, they essentially find nothing to free. However the next call to checkAndMaybeFreeMemory results in the same - i.e. we end up pointlessly consuming NonIO threads attempting to recover memory when there is nothing to do.
Note while this scenario (zero active, >0 replica) vBuckets is unlikely to occur in a real system, it is possible during a failover scenario.
Additionally this scenario occurs frequently in unit tests; resulting in tests taking much longer than necessary (particulary under sanitizers).