Details
-
Improvement
-
Resolution: Done
-
Critical
-
6.5.0, Cheshire-Cat
-
Security Level: Public
-
KV-Engine Sprint 2021 August, KV-Engine-Sept-21
Description
Problem
When doing a restore or a bulk load with a DGM dataset, the limiting factor is tmp_oom. Basically how fast the server can ejected documents. I have seen that by tuning the ejection parameters, restore and bulk load speeds can increase dramatically.
Expectation
To be able to do restore and bulk loads faster.
Suggestions
A few simple ideas
- Make ejection work faster
- Have a way for the client to inform that it's doing a bulk load
Attachments
Issue Links
For Gerrit Dashboard: MB-35075 | ||||||
---|---|---|---|---|---|---|
# | Subject | Branch | Project | Status | CR | V |
162179,3 | MB-41403: Increase default and max NonIO threads | master | kv_engine | Status: NEW | 0 | -1 |
150164,7 | MB-35075: Move ItemPager/PagingVisitor to use cb::Semaphore | master | kv_engine | Status: MERGED | +2 | +1 |
156356,8 | MB-35075: Add support for concurrent PagingVisitors | master | kv_engine | Status: MERGED | +2 | +1 |
156477,4 | MB-35075: Reduce nesting in PagingVisitor::visitBucket | master | kv_engine | Status: MERGED | +2 | +1 |
156478,5 | MB-35075: Allow Eviction PagingVisitor to pause between vbuckets | master | kv_engine | Status: MERGED | +2 | +1 |
156479,7 | MB-35075: Increase default concurrent item pagers to two | master | kv_engine | Status: MERGED | +2 | +1 |
157426,1 | MB-35075: Batch updates to EPStats evicted MFU histogram | master | kv_engine | Status: ABANDONED | 0 | +1 |