Details
Critical Description
Couchbase Server 7.0.0-5016 Enterprise Edition
After the failure in MB-45893, the new 7.0 node that was added to the cluster - 172.23.107.129 was removed. That rebalance operation went fine.
Next step was to perform a swap rebalance to take out 6.6.2 node 172.23.107.126 and bring in 172.23.107.129. This swap rebalance operation failed after 10 mins.
Rebalance exited with reason {mover_crashed,
|
{unexpected_exit,
|
{'EXIT',<0.27190.4>,
|
{{wait_seqno_persisted_failed,"default",469,
|
1553,
|
[{'ns_1@172.23.107.142',
|
{'EXIT',
|
{{{badmatch,{error,timeout}},
|
[{mc_client_binary,cmd_vocal_recv,5,
|
[{file,"src/mc_client_binary.erl"},
|
{line,152}]},
|
{mc_client_binary,select_bucket,2,
|
[{file,"src/mc_client_binary.erl"},
|
{line,344}]},
|
{ns_memcached_sockets_pool,
|
take_socket,2,
|
[{file,
|
"src/ns_memcached_sockets_pool.erl"},
|
{line,40}]},
|
{ns_memcached_sockets_pool,
|
'-executing_on_socket/3-fun-0-',3,
|
[{file,
|
"src/ns_memcached_sockets_pool.erl"},
|
{line,82}]},
|
{async,'-async_init/4-fun-1-',3,
|
[{file,"src/async.erl"},
|
{line,191}]}]},
|
{gen_server,call,
|
[{'janitor_agent-default',
|
'ns_1@172.23.107.142'},
|
{if_rebalance,<0.12682.4>,
|
{wait_seqno_persisted,469,1553}},
|
infinity]}}}}]},
|
[{ns_single_vbucket_mover,
|
'-wait_seqno_persisted_many/5-fun-2-',5,
|
[{file,"src/ns_single_vbucket_mover.erl"},
|
{line,482}]},
|
{proc_lib,init_p,3,
|
[{file,"proc_lib.erl"},{line,234}]}]}}}}.
|
Rebalance Operation Id = 81f43662ad33d73c76a515c82d352a89
|
Nodes in the cluster : 172.23.107.142 (7.0 node), 172.23.107.126 (outgoing 6.6.2 node), 172.23.107.129 (incoming 7.0 node).
Initial steps to setup the cluster are the same as MB-45893.
