Details
-
Bug
-
Resolution: Duplicate
-
Major
-
6.5.0
-
Enterprise Edition 6.5.0 build 4380
-
Untriaged
-
Centos 64-bit
-
-
Unknown
Description
Steps to Reproduce:
- Create a 6 node cluster.
+----------------+----------+--------------+
| Nodes | Services | Status |
+----------------+----------+--------------+
| 172.23.106.134 | [u'kv'] | Cluster node |
| 172.23.106.136 | None | <--- IN --- |
| 172.23.106.137 | None | <--- IN --- |
| 172.23.106.138 | None | <--- IN --- |
| 172.23.105.168 | None | <--- IN --- |
| 172.23.106.82 | None | <--- IN --- |
+----------------+----------+--------------+
- Create a bucket with compression = Off, eviction policy=valueOnly, replicas=1.
- Load 1K docs with durability = MAJORITY.
- Rebalance In 1 node(172.23.106.83) with 200 creates, 400 updates in parallel with durability = MAJORITY.
- Rebalance Out 1 node(172.23.105.168) with 200 creates, 400 updates, 200 deletes in parallel with durability = MAJORITY.
- Rebalance In 2 nodes(172.23.106.85, 172.23.105.168) and Rebalance Out 1 node(172.23.106.83) with 200 creates, 400 updates, 200 deletes in parallel with durability = MAJORITY.
- Swap Rebalance 1 node(IN=172.23.106.86 ,OUT=172.23.106.82) with 200 creates, 400 updates, 200 deletes in parallel with durability = MAJORITY.
- Update Replica number of the bucket from 1 to 2.
- Rebalance In 1 node(172.23.106.82) with 200 creates, 400 updates, 200 deletes in parallel with durability = MAJORITY.
- Rebalance the cluster with 200 creates, 400 updates, 200 deletes in parallel with durability = MAJORITY.
- While Step 9 is in progress, Stop the memcached process and start it again after 20 seconds.
- Start 200 creates, 400 updates, 200 deletes with durability = MAJORITY.
- While Step 11 is in progress, failover a node (172.23.106.137).
- Rebalance Out the node failed over in Step 12.
- Rebalance In 1 node (172.23.106.83)
- Start 200 creates, 400 updates, 200 deletes with durability = MAJORITY.
- While Step 15 is in progress, failover a node (172.23.106.83)
- Fully Recover the node failed over in Step 16.
- Rebalance the cluster.
- Start 200 creates, 400 updates, 200 deletes with durability = MAJORITY.
- While Step 19 is in progress, failover a node (172.23.106.83)
- Delta Recovery the node failed over in Step 20.
- Rebalance the cluster.
- Decrement the replica number of the bucket from 2 to 1.
- Rebalance the cluster with 200 creates, 400 updates, 200 deletes in parallel with durability = MAJORITY.
Rebalance is stuck at 99.90234375%.
Attachments
Issue Links
- duplicates
-
MB-36094 Rebalance fails with unexpected_exit,badmatch,{error,closed} when memcached is killed and restarted
- Closed