Details
-
Improvement
-
Resolution: Fixed
-
Major
-
4.6.4, 5.0.0, 5.5.0
-
KV-Engine Sprint 2021 July
Description
While investigating / tuning the defragmenter for MB-29879 it was observed that the current defragmenter policy - where it runs constantly - means that we are aways paying a price for the defragmenter.
Specifically, we need to carefully balance how often and how long the defragmenter runs:
- Too often (e.g. running with defragmenter_interval=0) means it essentially consumes a whole NonIO thread; as it "greedily" runs regardless of the current memory usage / fragmentation.
- Too infrequently, and it may not be able to keep up with high throughput workloads (as seen in
MB-29879) and fail to defragment sufficiently.
If we would instead only schedule the defragmenter when we believe there is useful work for it to do, then we could afford to be more aggressive with the interval value when we do decide to run it.
For example, the defragmenter could be changed to only actually visit the Bucket if heap fragmentation (RSS / mem_used) exceeds a given threshold. It could run relatively aggressively then, but once the heap fragmentation drops we would stop visiting again.
Attachments
Issue Links
- causes
-
MB-49525 Defragmenter auto configuration too aggressive when few items but rss close to high-water-mark
- Closed
- relates to
-
MB-42346 CC: Very high CPU usage while under low load (memcached)
- Resolved
-
MB-29879 memcached is OOM-killed during data ingestion due to high heap fragmentation
- Closed
-
MB-23086 Use jemalloc arenas for bucket memory tracking
- Closed
- mentioned in
-
Page Loading...