Details
-
Improvement
-
Resolution: Done
-
Major
-
6.5.1, 6.6.0, 6.5.0
-
None
-
1
-
KV-Engine Sprint 2021 August, KV-Engine-Sept-21
Description
Investigate improve the expiry processing so that
- Its throughput is more predictable, so that we can perform reliable sizings
- It better scales to multicore, so that the available CPU can be utilized for expiry handling
Possible areas to investigate (after profiling) are:
- Performing expiration across multiple vBuckets in parallel.
- Looking at the locking here - is the ItemPager slowed when visiting due to having to hold HashTable lock; or is it the locking when we bulk-delete all expired items in a given vBucket?
- Allowing expiration to skip over HashTable locks which are currently locked - although this would mean StoredValues could be skipped from being considered in a given item pager pass.
Attachments
Issue Links
For Gerrit Dashboard: MB-41403 | ||||||
---|---|---|---|---|---|---|
# | Subject | Branch | Project | Status | CR | V |
162179,3 | MB-41403: Increase default and max NonIO threads | master | kv_engine | Status: NEW | 0 | -1 |
161943,3 | MB-41403: Simplify ItemPager test config string management | master | kv_engine | Status: MERGED | +2 | +1 |
161944,6 | MB-41403: Simplify ExpiredItemPager management | master | kv_engine | Status: MERGED | +2 | +1 |
161945,13 | MB-41403: Allow a configurable number of expiry PagingVisitors | master | kv_engine | Status: MERGED | +2 | +1 |
163008,13 | MB-48716: Make expiry pager config change update task waketime | master | kv_engine | Status: MERGED | +2 | +1 |