Details
Description
Steps to Repro
1. Create a 3 node 6.5.2-6634 cluster(172.23.107.222,172.23.107.223,172.23.107.217)
2. do a swap rebalance of 6.6.6-10576 node(172.23.107.99) with 6.5.2-6634(172.23.107.223) node. This succeeds.
3. Graceful failover a 6.5.2-6634(172.23.107.217) node. Upgrade the build to 6.6.6-10576. Do a delta recovery and click on rebalance. This rebalance fails continuously on retry as shown below.
172.23.107.99 11:00:26 PM 19 Jan, 2023
Rebalance exited with reason {service_rebalance_failed,eventing,
|
{worker_died,
|
{'EXIT',<0.2411.13>,
|
{{badmatch,
|
{error,
|
{bad_nodes,eventing,prepare_rebalance,
|
[{'ns_1@172.23.107.99',
|
{error,
|
{unknown_error,
|
<<"Eventing Rebalance or Failover processing ongoing on nodeId: f368ef74e474bcd6adbb0a9d48b1102a">>}}}]}}},
|
[{service_rebalancer,rebalance_worker,1,
|
[{file,"src/service_rebalancer.erl"},
|
{line,164}]},
|
{proc_lib,init_p,3,
|
[{file,"proc_lib.erl"},{line,232}]}]}}}}.
|
Rebalance Operation Id = c7870e66deb80aa1c72d64fe66321e5a
|
Retried failed rebalance multiple times. Keeps failing.
cbcollect_info attached.
Attachments
Issue Links
- relates to
-
MB-44008 [System Test]: Failover and full recovery of KV made eventing rebalance hung
- Closed
Gerrit Reviews
For Gerrit Dashboard: MB-55191 | ||||||
---|---|---|---|---|---|---|
# | Subject | Branch | Project | Status | CR | V |
185103,2 | MB-55191: Skip vb takeover if current node is not owner of the vb | mad-hatter | eventing | Status: MERGED | +2 | +1 |