Details
-
Bug
-
Resolution: Duplicate
-
Major
-
4.5.0
-
4.5.0-2488
-
Untriaged
-
Unknown
Description
delta recovery on src cluster:
Bucket "RevAB" loaded on node 'ns_1@172.23.105.94' in 0 seconds. ns_memcached 000 ns_1@172.23.105.94 5:47:26 AM Mon May 16, 2016
|
Control connection to memcached on 'ns_1@172.23.105.94' disconnected: {error,
|
timeout} ns_memcached 000 ns_1@172.23.105.94 5:45:25 AM Mon May 16, 2016
|
Control connection to memcached on 'ns_1@172.23.105.94' disconnected: {{badmatch,
|
{error,
|
timeout}},
|
[{mc_client_binary,
|
cmd_vocal_recv,
|
5,
|
[{file,
|
"src/mc_client_binary.erl"},
|
{line,
|
156}]},
|
{mc_client_binary,
|
select_bucket,
|
2,
|
[{file,
|
"src/mc_client_binary.erl"},
|
{line,
|
351}]},
|
{ns_memcached,
|
ensure_bucket,
|
2,
|
[{file,
|
"src/ns_memcached.erl"},
|
{line,
|
1299}]},
|
{ns_memcached,
|
handle_info,
|
2,
|
[{file,
|
"src/ns_memcached.erl"},
|
{line,
|
748}]},
|
{gen_server,
|
handle_msg,
|
5,
|
[{file,
|
"gen_server.erl"},
|
{line,
|
604}]},
|
{ns_memcached,
|
init,
|
1,
|
[{file,
|
"src/ns_memcached.erl"},
|
{line,
|
177}]},
|
{gen_server,
|
init_it,
|
6,
|
[{file,
|
"gen_server.erl"},
|
{line,
|
304}]},
|
{proc_lib,
|
init_p_do_apply,
|
3,
|
[{file,
|
"proc_lib.erl"},
|
{line,
|
239}]}]} (repeated 1 times) ns_memcached 000 ns_1@172.23.105.94 5:43:35 AM Mon May 16, 2016
|
Bucket "MsgsCalls" loaded on node 'ns_1@172.23.105.94' in 0 seconds. ns_memcached 000 ns_1@172.23.105.94 5:43:03 AM Mon May 16, 2016
|
Bucket "UserInfo" loaded on node 'ns_1@172.23.105.94' in 0 seconds. ns_memcached 000 ns_1@172.23.105.94 5:43:00 AM Mon May 16, 2016
|
Control connection to memcached on 'ns_1@172.23.105.94' disconnected: {{badmatch,
|
{error,
|
timeout}},
|
[{mc_client_binary,
|
cmd_vocal_recv,
|
5,
|
[{file,
|
"src/mc_client_binary.erl"},
|
{line,
|
156}]},
|
{mc_client_binary,
|
select_bucket,
|
2,
|
[{file,
|
"src/mc_client_binary.erl"},
|
{line,
|
351}]},
|
{ns_memcached,
|
ensure_bucket,
|
2,
|
[{file,
|
"src/ns_memcached.erl"},
|
{line,
|
1299}]},
|
{ns_memcached,
|
handle_info,
|
2,
|
[{file,
|
"src/ns_memcached.erl"},
|
{line,
|
748}]},
|
{gen_server,
|
handle_msg,
|
5,
|
[{file,
|
"gen_server.erl"},
|
{line,
|
604}]},
|
{ns_memcached,
|
init,
|
1,
|
[{file,
|
"src/ns_memcached.erl"},
|
{line,
|
177}]},
|
{gen_server,
|
init_it,
|
6,
|
[{file,
|
"gen_server.erl"},
|
{line,
|
304}]},
|
{proc_lib,
|
init_p_do_apply,
|
3,
|
[{file,
|
"proc_lib.erl"},
|
{line,
|
239}]}]} ns_memcached 000 ns_1@172.23.105.94 5:43:00 AM Mon May 16, 2016
|
Bucket "UserInfo" loaded on node 'ns_1@172.23.107.48' in 28 seconds. ns_memcached 000 ns_1@172.23.107.48 4:46:04 AM Mon May 16, 2016
|
Rebalance exited with reason {badmatch,fail}
|
ns_orchestrator 002 ns_1@172.23.105.87 4:36:36 AM Mon May 16, 2016
|
Starting rebalance, KeepNodes = ['ns_1@172.23.105.87','ns_1@172.23.105.94',
|
'ns_1@172.23.107.48','ns_1@172.23.107.85'], EjectNodes = [], Failed over and being ejected nodes = [], Delta recovery nodes = ['ns_1@172.23.107.48'], Delta recovery buckets = all ns_orchestrator 004 ns_1@172.23.105.87 4:35:36 AM Mon May 16, 2016
|
Service 'goxdcr' exited with status 1. Restarting. Messages: runtime.goexit()
|
C:/Users/vagrant/cbdepscache/exploded/amd64/go-1.6/go/src/runtime/asm_amd64.s:1998 +0x1 fp=0xc083023fa0 sp=0xc083023f98
|
created by github.com/couchbase/goxdcr/gen_server.(*GenServer).Start_server
|
C:/Jenkins/workspace/watson-windows/couchbase/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:70 +0x7f
|
[goport] 2016/05/16 04:35:09 c:/Program Files/Couchbase/Server/bin/goxdcr.exe terminated: exit status 2 ns_log 000 ns_1@172.23.105.94 4:35:10 AM Mon May 16, 2016
|
Shutting down bucket "UserInfo" on 'ns_1@172.23.107.48' for deletion ns_memcached 000 ns_1@172.23.107.48 4:31:53 AM Mon May 16, 2016
|
Failed over 'ns_1@172.23.107.48': ok ns_rebalancer 000 ns_1@172.23.105.87 4:31:53 AM Mon May 16, 2016
|
Starting failing over 'ns_1@172.23.107.48' ns_rebalancer 000 ns_1@172.23.105.87 4:31:51 AM Mon May 16, 2016
|
Bucket "MsgsCalls" rebalance does not seem to be swap rebalance ns_vbucket_mover 000 ns_1@172.23.105.87 4:10:55 AM Mon May 16, 2016
|
Bucket "RevAB" rebalance does not seem to be swap rebalance ns_vbucket_mover 000 ns_1@172.23.105.87 4:02:54 AM Mon May 16, 2016
|
Bucket "UserInfo" rebalance does not seem to be swap rebalance ns_vbucket_mover 000 ns_1@172.23.105.87 4:01:15 AM Mon May 16, 2016
|
Bucket "AbRegNums" rebalance does not seem to be swap rebalance ns_vbucket_mover 000 ns_1@172.23.105.87 3:57:59 AM Mon May 16, 2016
|
Starting vbucket moves for graceful failover of 'ns_1@172.23.107.48' ns_rebalancer 000 ns_1@172.23.105.87 3:57:58 AM Mon May 16, 2016
|
Service 'goxdcr' exited with status 1. Restarting. Messages: runtime.goexit()
|
C:/Users/vagrant/cbdepscache/exploded/amd64/go-1.6/go/src/runtime/asm_amd64.s:1998 +0x1 fp=0xc082103fa0 sp=0xc082103f98
|
created by net/http.(*Transport).dialConn
|
C:/Users/vagrant/cbdepscache/exploded/amd64/go-1.6/go/src/net/http/transport.go:853 +0x10ad
|
[goport] 2016/05/16 03:57:32 c:/Program Files/Couchbase/Server/bin/goxdcr.exe terminated: exit status 2 ns_log 000 ns_1@172.23.107.85 3:57:32 AM Mon May 16, 2016
|
Service 'goxdcr' exited with status 1. Restarting. Messages: runtime.goexit()
|
C:/Users/vagrant/cbdepscache/exploded/amd64/go-1.6/go/src/runtime/asm_amd64.s:1998 +0x1 fp=0xc084323f88 sp=0xc084323f80
|
created by github.com/couchbase/goxdcr/pipeline_svc.(*CheckpointManager).massCheckVBOpaquesJob
|
C:/Jenkins/workspace/watson-windows/couchbase/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/checkpoint_manager.go:896 +0x278
|
[goport] 2016/05/16 03:55:16 c:/Program Files/Couchbase/Server/bin/goxdcr.exe terminated: exit status 2 ns_log 000 ns_1@172.23.107.48 3:55:16 AM Mon May 16, 2016
|
Service 'goxdcr' exited with status 1. Restarting. Messages: runtime.goexit()
|
C:/Users/vagrant/cbdepscache/exploded/amd64/go-1.6/go/src/runtime/asm_amd64.s:1998 +0x1 fp=0xc085613f98 sp=0xc085613f90
|
created by github.com/couchbase/gomemcached/client.(*Client).NewUprFeed
|
C:/Jenkins/workspace/watson-windows/couchbase/godeps/src/github.com/couchbase/gomemcached/client/upr_feed.go:186 +0x1b8
|
[goport] 2016/05/16 03:08:06 c:/Program Files/Couchbase/Server/bin/goxdcr.exe terminated: exit status 2 ns_log 000 ns_1@172.23.107.85 3:08:06 AM Mon May 16, 2016
|
Reset auto-failover count auto_failover 000 ns_1@172.23.105.87 2:45:17 AM Mon May 16, 2016
|