Details
-
Bug
-
Resolution: Duplicate
-
Major
-
None
-
6.5.0
-
6.5.0-3821
-
Untriaged
-
-
Unknown
-
KV-Engine MH Beta part 2
Description
Steps to Reproduce
1. Create a 3 node cluster
+----------------+----------+--------------+
|
| Nodes | Services | Status |
|
+----------------+----------+--------------+
|
| 172.23.105.155 | [u'kv'] | Cluster node |
|
| 172.23.105.159 | None | <--- IN --- |
|
| 172.23.105.205 | None | <--- IN --- |
|
+----------------+----------+--------------+
|
2. Add a default bucket with replicas=0
3. Load 50k items to the bucket with durability=MAJORITY
Bucket statistics:
+---------+---------+----------+-----+-------+------------+-----------+-----------+
|
| Bucket | Type | Replicas | TTL | Items | RAM Quota | RAM Used | Disk Used |
|
+---------+---------+----------+-----+-------+------------+-----------+-----------+
|
| default | membase | 0 | 0 | 50000 | 6716129280 | 120314264 | 177073826 |
|
+---------+---------+----------+-----+-------+------------+-----------+-----------+
|
Cluster statistics:
+----------------+----------+-----------------+------------
|
| Node | Services | Active Items | Replica
|
+----------------+----------+-----------------+------------
|
| 172.23.105.205 | [u'kv'] | 16648 | 0
|
| 172.23.105.155 | [u'kv'] | 16684 | 0
|
| 172.23.105.159 | [u'kv'] | 16668 | 0
|
+----------------+----------+-----------------+------------
|
4. Rebalance out 1 node and start data loading in a separate thread
+----------------+----------+--------------+
|
| Nodes | Services | Status |
|
+----------------+----------+--------------+
|
| 172.23.105.205 | [u'kv'] | --- OUT ---> |
|
| 172.23.105.155 | [u'kv'] | Cluster node |
|
| 172.23.105.159 | [u'kv'] | Cluster node |
|
+----------------+----------+--------------+
|
Rebalance failed:
Rebalance exited with reason {mover_crashed,
|
{unexpected_exit,
|
{'EXIT',<0.14396.0>,
|
{{wait_seqno_persisted_failed,"default",972,
|
58,
|
[{'ns_1@172.23.105.159',
|
{'EXIT',
|
{socket_closed,
|
{gen_server,call,
|
[{'janitor_agent-default',
|
'ns_1@172.23.105.159'},
|
{if_rebalance,<0.6472.0>,
|
{wait_seqno_persisted,972,58}},
|
infinity]}}}}]},
|
[{ns_single_vbucket_mover,
|
'-wait_seqno_persisted_many/5-fun-2-',5,
|
[{file,"src/ns_single_vbucket_mover.erl"},
|
{line,488}]},
|
{proc_lib,init_p,3,
|
[{file,"proc_lib.erl"},{line,232}]}]}}}}.
|
Rebalance Operation Id = 89ccca0dadbb9290c709f4a87ec310b8
|
Attachments
Issue Links
- duplicates
-
MB-35208 Rebalance failed during replica update if disk backfill and memory snapshot contain same Prepare key
- Closed