Details
-
Task
-
Resolution: Fixed
-
Major
-
None
-
None
-
1
-
Magma-July-5-2021, Magma-July-12-2021
Description
Currently, Magma does a lookup on every write to know whether a write is an update or insert, in order to maintain accurate item counts. This lookup can be avoided in a disk backfill scenario when a vbucket is being streamed right from the start (seqno 0). Since the snapshot won't have any duplicates, a lookup on the consumer side isn't required and we can safely do blind writes.
Trying this out in one of our rebalance tests, this gives a ~22% improvement in rebalance time from 16.5mins to 12.8mins.
(Rebalance-out (min), 3 nodes -> 2 nodes, 250M x 1KB, 15K ops/sec (90/10 R/W), 10% cache miss rate, 2.4% Resident Ratio, Magma)
Here's the rough patches used for the test:
kv_engine: http://review.couchbase.org/c/kv_engine/+/154756
magma: http://review.couchbase.org/c/magma/+/154755
Attachments
For Gerrit Dashboard: MB-46750 | ||||||
---|---|---|---|---|---|---|
# | Subject | Branch | Project | Status | CR | V |
156987,1 | MB-46750 Add DCP loopback tests | master | kv_engine | Status: NEW | 0 | 0 |