Details
Description
Install cb server 4.0.0-3570 on 7 nodes
Create cluster of 7 nodees (5 kv, 1 index and 1 n1ql)
Load 34 million items with size 1K to default bucket (no view nor indexing)
Install cb server 4.0.0-3570 on other 7 nodes with same services setup above (5,1,1)
Addd new 7 nodes to cluster. Rebalance failed in few minutes.
Check log see memcached crashed.
<0.27899.12> exited with {unexpected_exit,
{'EXIT',<0.27907.12>,
{bulk_set_vbucket_state_failed,
[{'ns_1@10.145.15.54',
{'EXIT',
{{{{case_clause,
{error,
{{{badmatch,
{error,
badmatch,{error,etimedout,
[{mc_replication,connect,1,
[
{mc_replication,connect,1,
[{file,"src/mc_replication.erl"}
,
{line,49}]},{dcp_proxy,connect,4,
[{file,"src/dcp_proxy.erl"}, {line,174}]},
{dcp_proxy,maybe_connect,1,
[{file,"src/dcp_proxy.erl"}, {line,161}]},
{dcp_producer_conn,init,2,
[{file,"src/dcp_producer_conn.erl"}, {line,30}]},
{dcp_proxy,init,1,
[{file,"src/dcp_proxy.erl"}, {line,46}]},
{gen_server,init_it,6,
[{file,"gen_server.erl"}, {line,304}]},
{proc_lib,init_p_do_apply,3,
[{file,"proc_lib.erl"}, {line,239}]}]}}},
[{dcp_replicator,init,1,
[{file,"src/dcp_replicator.erl"},{line,49}
]},
{gen_server,init_it,6,
[
{proc_lib,init_p_do_apply,3,
[{file,"proc_lib.erl"},{line,239}]}]},
{child,undefined,'ns_1@10.164.216.81', {dcp_replicator,start_link, ['ns_1@10.164.216.81',"default"]},
temporary,60000,worker,
[dcp_replicator]}}}},
[{dcp_sup,start_replicator,2,
[{file,"src/dcp_sup.erl"},{line,53}]},
{dcp_sup,
'
[{file,"src/dcp_sup.erl"},{line,69}]},
{dcp_replication_manager,handle_call,3,
[{file,"src/dcp_replication_manager.erl"}, {line,78}]},
{gen_server,handle_msg,5,
[{file,"gen_server.erl"}
,
{line,585}]},
{proc_lib,init_p_do_apply,3,
[
,
{line,239}]}]},
{gen_server,call,
['dcp_replication_manager-default',
,
infinity]}},
{gen_server,call,
[
,
{if_rebalance,<0.9128.11>,
{update_vbucket_state,458,replica,passive,
'ns_1@10.165.120.25'}},
infinity]}}}}]}}}
ns_vbucket_mover000
ns_1@10.233.98.253
09:28:38 - Fri Jul 31, 2015
Node 'ns_1@10.233.170.117' saw that node 'ns_1@10.164.216.81' came up. Tags: []
ns_node_disco004
ns_1@10.233.170.117
09:02:44 - Fri Jul 31, 2015
Node 'ns_1@10.164.216.81' saw that node 'ns_1@10.233.170.117' came up. Tags: []
ns_node_disco004
ns_1@10.164.216.81
09:02:44 - Fri Jul 31, 2015
Bucket "default" rebalance does not seem to be swap rebalance
ns_vbucket_mover000
ns_1@10.233.98.253
09:01:38 - Fri Jul 31, 2015
Attachments
Issue Links
- blocks
-
MB-15886 3.1.1 Minor Release
- Resolved