Details
-
Bug
-
Resolution: Unresolved
-
Major
-
7.6.0, 7.2.5
-
Untriaged
-
0
-
Unknown
-
Plasma-Aug21-2024, Plasma-Sep9-2024
Description
The GetMemFrag is used in QuotaSwapper to decide burst eviction. Plasma's current GetMemFrag calculation is based on MemoryInUse which does not include the following memory components:
a)ReclaimPending
b)BufMemUsed
c)RecoveryBufMemUsed
https://github.com/couchbase/plasma/blob/neo/swapper.go#L1858C46-L1858C56
https://github.com/couchbase/plasma/blob/neo/swapper.go#L1230
"assigned_quota": 211634513510,
|
"current_quota": 211634513510,
|
"heap_limit": max,
|
"memory_resident": 201982799872,
|
"memory_jemalloc": 211671252992,
|
"memory_jemalloc_active": 207286284288,
|
"memory_jemalloc_allocated": 181987440728,
|
"estimated_mem_frag": 0.48372,
|
"memory_stats_size_total": 168869267034,
|
"memory_stats_size_global": 142663129879,
|
"memory_stats_size_page": 73443671438,
|
"memory_stats_size_index": 69220619704,
|
"memory_stats_size_bloom": 2517624064,
|
"memory_stats_size_delta": 2976645928,
|
"reclaim_pending": 5972638,
|
"reclaim_pending_global": 0,
|
"buf_memused": 23681379190,<-----------23GB
|
"under_swap": false,
|
"num_freeOS": 6308719,
|
If BufMemUsed/ReclaimPending is high, QuotaSwapper can overestimate plasma overall memory usage and throttle for a while even though memtuner has not decremented current quota. We can consider using jemalloc's fragmentation (allocated/resident) for computing GetMemFrag.
