Details
-
Bug
-
Status: Closed
-
Major
-
Resolution: Fixed
-
5.1.2
-
Triaged
-
No
Description
Currently plasma mem tuner logic decrements memory quota and forces GC when the indexer process RSS overshoots user quota by 15% but the increment logic does not examine RSS before choosing to restore the quota.
This results in spiky RSS and frequent forced garbage collection.
Better to use examine RSS and only increment if the RSS is below user quota.
Inspiration: PID controller
Attachments
Issue Links
For Gerrit Dashboard: MB-32433 | ||||||
---|---|---|---|---|---|---|
# | Subject | Branch | Project | Status | CR | V |
103130,3 | MB-32433 mem: Only increment if RSS < quota | unstable | plasma | Status: MERGED | +2 | +1 |
103264,1 | MB-32433 Backport of mem: Only increment if RSS < quota | alice | plasma | Status: MERGED | +2 | +1 |
104529,1 | MB-32433 mem: Only increment if RSS < quota | vulcan | plasma | Status: ABANDONED | 0 | 0 |
Need to examine average GC waste value -> gc_waste_avg (Available memory not released by go runtime and jemalloc)
Now we should lower plasma quota to a value less by gc_waste_avg
Take 5 samples of gc waste gc_waste_avg = RSS_before_GC_force - RSS_after_GC_waste
plasma_quota = 90% of indexer_quota - gc_waste_avg