Details
-
Improvement
-
Resolution: Fixed
-
Critical
-
2.0.1
-
Security Level: Public
-
None
-
Release Note
-
2
Description
Optimize 2 types of state transitions:
1) If a state transition consists only of moving partitions
between the passive and active states (or vice-versa), that
is, no new partitions are added nor existing partitions are
marked for cleanup, don't restart the updater (at the expense
of more complex logic to correct snapshots received from a
currently running updater).
2) If a state transition only adds new partitions to the
passive state (as ns_server's rebalance frequently calls),
don't restart the updater, and send the updater the new
list of passive partitions, so that it has a chance to
iterate over those partitions' content and index them in
the same run.
As long as the rebalancer in ns_server doesn't keep changing
its logic regarding state transitions all the time, this
provides a significant reduction of the indexing time. However
for at least some scenarios, there's still significant portion
of rebalance time not spent on indexing nor index compaction.
Attachments
For Gerrit Dashboard: MB-7522 | ||||||
---|---|---|---|---|---|---|
# | Subject | Branch | Project | Status | CR | V |
23871,2 | MB-7522 Optimize index state transitions (rebalance) | 2.0.1 | couchdb | Status: MERGED | +2 | +1 |
23893,3 | MB-7522 Improve test 24-updater-add-more-passive-partitions.t | 2.0.1 | couchdb | Status: MERGED | +2 | +1 |
23895,2 | MB-7522 Fix race condition in test | 2.0.1 | couchdb | Status: MERGED | +2 | +1 |
23899,2 | MB-7522 Further test improvements/coverage | 2.0.1 | couchdb | Status: MERGED | +2 | +1 |
24014,2 | MB-7522 Do updater restarts if a pending transition exists | 2.0.1 | couchdb | Status: MERGED | +2 | +1 |
24534,4 | MB-7702 Fix groups received from the compactor | 2.0.1 | couchdb | Status: MERGED | +2 | +1 |
25050,3 | MB-7882 No id btree lookups for dynamically added partitions | 2.0.2 | couchdb | Status: MERGED | +2 | +1 |
25725,2 | MB-8109 Set pending transition for outdated updater groups | 2.0.2 | couchdb | Status: MERGED | +2 | +1 |