Details
-
Bug
-
Resolution: Fixed
-
Critical
-
4.0.0
-
Security Level: Public
-
4.0.0-3402
-
Untriaged
-
Ubuntu 64-bit
-
Unknown
-
Mar 9 - Mar 27
Description
system tests with 2 buckets * 2 views
rebalance 7-1 node:
Shutting down bucket "saslbucket" on 'ns_1@172.23.105.22' for deletion ns_memcached000 ns_1@172.23.105.22 07:19:15 - Wed Jul 15, 2015
Bucket "saslbucket" loaded on node 'ns_1@172.23.105.157' in 134 seconds. ns_memcached000 ns_1@172.23.105.157 07:18:07 - Wed Jul 15, 2015
Bucket "default" loaded on node 'ns_1@172.23.105.157' in 88 seconds. ns_memcached000 ns_1@172.23.105.157 07:17:22 - Wed Jul 15, 2015
Rebalance exited with reason {unexpected_exit,
{'EXIT',<0.2752.10>,
badmatch,{error,closed,
{gen_server,call,
[
,
{if_rebalance,<0.26733.9>,
{dcp_takeover,'ns_1@172.23.105.22',504}},
infinity]}}}}
ns_orchestrator002 ns_1@172.23.105.156 07:16:28 - Wed Jul 15, 2015
<0.2150.10> exited with {unexpected_exit,
{'EXIT',<0.2752.10>,
badmatch,{error,closed,
{gen_server,call,
[
,
{if_rebalance,<0.26733.9>,
{dcp_takeover,'ns_1@172.23.105.22',504}},
infinity]}}}} ns_vbucket_mover000 ns_1@172.23.105.156 07:16:28 - Wed Jul 15, 2015
Control connection to memcached on 'ns_1@172.23.105.157' disconnected: {{badmatch,
{error,
closed}},
[{mc_client_binary,
cmd_vocal_recv,
5,
[
{mc_client_binary,
select_bucket,
2,
[{file, "src/mc_client_binary.erl"}
,
{line, 351}]},{ns_memcached,
ensure_bucket,
2,
[{file, "src/ns_memcached.erl"}, {line, 1292}]},
{ns_memcached,
handle_info,
2,
[{file, "src/ns_memcached.erl"}, {line, 745}]},
{gen_server,
handle_msg,
5,
[{file, "gen_server.erl"}, {line, 604}]},
{ns_memcached,
init,
1,
[{file, "src/ns_memcached.erl"}, {line, 174}]},
{gen_server,
init_it,
6,
[{file, "gen_server.erl"}, {line, 304}]},
{proc_lib,
init_p_do_apply,
3,
[{file, "proc_lib.erl"}, {line, 239}]}]} (repeated 1 times) ns_memcached000 ns_1@172.23.105.157 07:16:09 - Wed Jul 15, 2015
Service 'memcached' exited with status 134. Restarting. Messages: 2015-07-15T07:15:33.412636-07:00 WARNING /opt/couchbase/lib/memcached/ep.so() [0x7f3905f4e2a7]
2015-07-15T07:15:33.412664-07:00 WARNING /opt/couchbase/lib/memcached/ep.so() [0x7f3905f4e8f9]
2015-07-15T07:15:33.412690-07:00 WARNING /opt/couchbase/bin/../lib/libplatform.so.0.1.0() [0x7f39115bbbb8]
2015-07-15T07:15:33.412723-07:00 WARNING /lib64/libpthread.so.0() [0x7f390fd26851]
2015-07-15T07:15:33.412782-07:00 WARNING /lib64/libc.so.6(clone+0x6d) [0x7f390eec890d] ns_log000 ns_1@172.23.105.157 07:15:50 - Wed Jul 15, 2015
Control connection to memcached on 'ns_1@172.23.105.157' disconnected: {{badmatch,
{error,
closed}},
[{mc_client_binary,
cmd_vocal_recv,
5,
[{file, "src/mc_client_binary.erl"}, {line, 156}]},
{mc_client_binary,
select_bucket,
2,
[{file, "src/mc_client_binary.erl"},{line, 351}
]},
{ns_memcached,
ensure_bucket,
2,
[
{ns_memcached,
handle_info,
2,
[{file, "src/ns_memcached.erl"}
,
{line, 745}]},
{gen_server,
handle_msg,
5,
[
{ns_memcached,
init,
1,
[{file, "src/ns_memcached.erl"}, {line, 174}]},
{gen_server,
init_it,
6,
[{file, "gen_server.erl"}
,
{line, 304}]},
{proc_lib,
init_p_do_apply,
3,
[
,
{line, 239}]}]} ns_memcached000 ns_1@172.23.105.157 07:15:50 - Wed Jul 15, 2015
Bucket "default" rebalance does not seem to be swap rebalance ns_vbucket_mover000 ns_1@172.23.105.156 07:14:31 - Wed Jul 15, 2015
Started rebalancing bucket default ns_rebalancer000 ns_1@172.23.105.156 07:14:27 - Wed Jul 15, 2015
Node 'ns_1@172.23.105.207' saw that node 'ns_1@172.23.105.159' went down. Details: [
Node 'ns_1@172.23.105.22' saw that node 'ns_1@172.23.105.159' went down. Details: [{nodedown_reason, connection_closed}
] ns_node_disco005 ns_1@172.23.105.22 07:05:33 - Wed Jul 15, 2015
Bucket "saslbucket" rebalance does not seem to be swap rebalance ns_vbucket_mover000 ns_1@172.23.105.156 07:01:32 - Wed Jul 15, 2015
Started rebalancing bucket saslbucket ns_rebalancer000 ns_1@172.23.105.156 07:01:32 - Wed Jul 15, 2015
Starting rebalance, KeepNodes = ['ns_1@172.23.105.160','ns_1@172.23.105.156',
'ns_1@172.23.105.206','ns_1@172.23.105.158',
'ns_1@172.23.105.157','ns_1@172.23.105.207'], EjectNodes = ['ns_1@172.23.105.22'], Failed over and being ejected nodes = []; no delta recovery nodes
ns_orchestrator004 ns_1@172.23.105.156 07:01:32 - Wed Jul 15
will provide logs soon