Details
-
Bug
-
Resolution: Incomplete
-
Major
-
6.5.0
-
Untriaged
-
Unknown
Description
Build 6.5.0-2082
Observed that in high bucket density test(with 30 buckets), rebalance after hard fail over of kv node is very slow.
In the test, it is 78% complete on kv nodes after ~14 hours. Please investigate if it is expected.
Note: Removing kv node(from 4 node to 3 node) from same cluster without failover and rebalance takes ~206 min.
Test:
Out of 3 kv nodes, 1 is hard failed over and then rebalance started without adding node back.
Buckets and docs: 32 buckets ~1M docs of 1KB per bucket.
Number of replicas: 1
XDCR: on
KV ops: ~200 for entire cluster
Cluster also had index, query, fts, eventing and analytics nodes.
Logs-
https://s3.amazonaws.com/bugdb/jira/mh_high_bkt_density_failover/collectinfo-2019-01-23T055648-ns_1%40172.23.97.12.zip
https://s3.amazonaws.com/bugdb/jira/mh_high_bkt_density_failover/collectinfo-2019-01-23T055648-ns_1%40172.23.97.13.zip