Details
-
Bug
-
Resolution: Fixed
-
Critical
-
4.0.0
-
Security Level: Public
-
4.0.0-4047
-
Untriaged
-
Unknown
Description
rebelance in 3+2 nodes on src cluster
Service 'goxdcr' exited with status 1. Restarting. Messages: github.com/couchbase/goxdcr/simple_utils.funcĀ·001(0xc082be8600)
C:/Jenkins/workspace/sherlock-windows/couchbase/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:38 +0x60
created by github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout
C:/Jenkins/workspace/sherlock-windows/couchbase/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:40 +0x189
[goport] 2015/08/19 14:49:16 c:/Program Files/Couchbase/Server/bin/goxdcr.exe terminated: exit status 2 ns_log000 ns_1@172.23.105.39 14:49:16 - Wed Aug 19, 2015
Rebalance exited with reason {unexpected_exit,
{'EXIT',<0.16496.64>,
{bulk_set_vbucket_state_failed,
[{'ns_1@172.23.105.39',
{'EXIT',
{{timeout,
{gen_server,call,
[<19384.29431.25>,
{gen_server,call,
[{'janitor_agent-RevAB', 'ns_1@172.23.105.39'},
{if_rebalance,<0.21498.62>,
{update_vbucket_state,517,replica,
undefined,undefined}},
infinity]}}}}]}}}
ns_orchestrator002 ns_1@172.23.105.192 14:49:15 - Wed Aug 19, 2015
<0.15455.64> exited with {unexpected_exit,
{'EXIT',<0.16496.64>,
{bulk_set_vbucket_state_failed,
[{'ns_1@172.23.105.39',
{'EXIT',
{{timeout,
{gen_server,call,
[<19384.29431.25>,{register,#Ref<19384.0.30.197786>, <19384.2738.27>}
]}},
{gen_server,call,
[
,
{if_rebalance,<0.21498.62>,
{update_vbucket_state,517,replica,
undefined,undefined}},
infinity]}}}}]}}} ns_vbucket_mover000 ns_1@172.23.105.192 14:49:15 - Wed Aug 19, 2015
Haven't heard from a higher priority node or a master, so I'm taking over. mb_master000 ns_1@172.23.105.39 14:49:04 - Wed Aug 19, 2015
IP address seems to have changed. Unable to listen on 'ns_1@172.23.106.57'. menelaus_web_alerts_srv000 ns_1@172.23.106.57 14:48:53 - Wed Aug 19, 2015
IP address seems to have changed. Unable to listen on 'ns_1@172.23.106.142'. menelaus_web_alerts_srv000 ns_1@172.23.106.142 14:40:01 - Wed Aug 19, 2015
Bucket "RevAB" rebalance does not seem to be swap rebalance ns_vbucket_mover000 ns_1@172.23.105.192 14:17:26 - Wed Aug 19, 2015
Bucket "RevAB" loaded on node 'ns_1@172.23.105.39' in 0 seconds. ns_memcached000 ns_1@172.23.105.39 14:17:25 - Wed Aug 19, 2015
Bucket "RevAB" loaded on node 'ns_1@172.23.106.142' in 0 seconds. ns_memcached000 ns_1@172.23.106.142 14:17:25 - Wed Aug 19, 2015
Started rebalancing bucket RevAB ns_rebalancer000 ns_1@172.23.105.192 14:17:24 - Wed Aug 19, 2015
Bucket "UserInfo" rebalance does not seem to be swap rebalance ns_vbucket_mover000 ns_1@172.23.105.192 14:15:35 - Wed Aug 19, 2015
Bucket "UserInfo" loaded on node 'ns_1@172.23.105.39' in 0 seconds. ns_memcached000 ns_1@172.23.105.39 14:15:34 - Wed Aug 19, 2015
Bucket "UserInfo" loaded on node 'ns_1@172.23.106.142' in 0 seconds. ns_memcached000 ns_1@172.23.106.142 14:15:34 - Wed Aug 19, 2015
Started rebalancing bucket UserInfo ns_rebalancer000 ns_1@172.23.105.192 14:15:34 - Wed Aug 19, 2015
Deleting old data files of bucket "AbRegNums" ns_storage_conf000 ns_1@172.23.106.142 14:15:34 - Wed Aug 19, 2015
Deleting old data files of bucket "AbRegNums" ns_storage_conf000 ns_1@172.23.105.39 14:15:34 - Wed Aug 19, 2015
Deleting old data files of bucket "MsgsCalls" ns_storage_conf000 ns_1@172.23.105.39 14:15:34 - Wed Aug 19, 2015
Deleting old data files of bucket "RevAB" ns_storage_conf000 ns_1@172.23.105.39 14:15:34 - Wed Aug 19, 2015
Deleting old data files of bucket "UserInfo" ns_storage_conf000 ns_1@172.23.105.39 14:15:34 - Wed Aug 19, 2015
Deleting old data files of bucket "MsgsCalls" ns_storage_conf000 ns_1@172.23.106.142 14:15:34 - Wed Aug 19, 2015
Deleting old data files of bucket "RevAB" ns_storage_conf000 ns_1@172.23.106.142 14:15:34 - Wed Aug 19, 2015
Deleting old data files of bucket "UserInfo" ns_storage_conf000 ns_1@172.23.106.142 14:15:34 - Wed Aug 19, 2015
Starting rebalance, KeepNodes = ['ns_1@172.23.105.192','ns_1@172.23.105.204',
'ns_1@172.23.106.57','ns_1@172.23.105.39',
'ns_1@172.23.106.142'], EjectNodes = [], Failed over and being ejected nodes = []; no delta recovery nodes
ns_orchestrator004 ns_1@172.23.105.192 14:15:33 - Wed Aug 19, 2015
Attachments
Issue Links
- relates to
-
MB-15963 [system tests windows] Resetting rebalance status since it's not really running
- Closed