Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-34187

rebalance failure in build sanity

    XMLWordPrintable

Details

    • Bug
    • Resolution: Fixed
    • Major
    • 6.5.0
    • 6.5.0
    • qe
    • Untriaged
    • Yes

    Description

      6.5.0-3245 - build sanity tests failing with following rebalance failure:

      ERROR -

      {u'status': u'none', u'errorMessage': u'Rebalance failed. See logs for detailed reason. You can try again.'}

      - rebalance failed
      16:58:51 [2019-05-14 16:58:51,690] - [rest_client:3237] INFO - Latest logs from UI on 172.23.107.160:
      16:58:51 [2019-05-14 16:58:51,691] - [rest_client:3238] ERROR - {u'node': u'ns_1@cb.local', u'code': 0, u'text': u'Rebalance exited with reason {{badmatch,failed},\n [{ns_rebalancer,rebalance_body,5,\n [

      {file,"src/ns_rebalancer.erl"}

      ,\n

      {line,756}

      ]},\n {async,\'async_init/4-fun-2\',3,\n [

      {file,"src/async.erl"}

      ,

      {line,211}

      ]}]}. Operation Id = cfba9a8393658e15b8ebf50932e28944', u'shortText': u'message', u'serverTime': u'2019-05-14T16:58:46.666Z', u'module': u'ns_orchestrator', u'tstamp': 1557878326666, u'type': u'critical'}
      16:58:51 [2019-05-14 16:58:51,691] - [rest_client:3238] ERROR -

      {u'node': u'ns_1@cb.local', u'code': 0, u'text': u"Starting rebalance, KeepNodes = ['ns_1@cb.local'], EjectNodes = [], Failed over and being ejected nodes = []; no delta recovery nodes; Operation Id = cfba9a8393658e15b8ebf50932e28944", u'shortText': u'message', u'serverTime': u'2019-05-14T16:58:01.533Z', u'module': u'ns_orchestrator', u'tstamp': 1557878281533, u'type': u'info'}

      16:58:51 [2019-05-14 16:58:51,691] - [rest_client:3238] ERROR -

      {u'node': u'ns_1@cb.local', u'code': 102, u'text': u'Client-side error-report for user "Administrator" on node \'ns_1@cb.local\':\nUser-Agent:Python-httplib2/$Rev: 259 $\n2019-05-14 16:57:57.217311 : test_simple_ui_request finished \n', u'shortText': u'client-side error report', u'serverTime': u'2019-05-14T16:57:57.224Z', u'module': u'menelaus_web', u'tstamp': 1557878277224, u'type': u'warning'}

      16:58:51 [2019-05-14 16:58:51,692] - [rest_client:3238] ERROR -

      {u'node': u'ns_1@cb.local', u'code': 102, u'text': u'Client-side error-report for user "Administrator" on node \'ns_1@cb.local\':\nUser-Agent:Python-httplib2/$Rev: 259 $\n2019-05-14 16:57:55.073870 : test_simple_ui_request started \n', u'shortText': u'client-side error report', u'serverTime': u'2019-05-14T16:57:55.083Z', u'module': u'menelaus_web', u'tstamp': 1557878275083, u'type': u'warning'}

      16:58:51 [2019-05-14 16:58:51,692] - [rest_client:3238] ERROR -

      {u'node': u'ns_1@cb.local', u'code': 102, u'text': u'Client-side error-report for user "Administrator" on node \'ns_1@cb.local\':\nUser-Agent:Python-httplib2/$Rev: 259 $\n2019-05-14 16:57:51.549088 : test_simple_ui_request finished \n', u'shortText': u'client-side error report', u'serverTime': u'2019-05-14T16:57:51.557Z', u'module': u'menelaus_web', u'tstamp': 1557878271557, u'type': u'warning'}

      16:58:51 [2019-05-14 16:58:51,692] - [rest_client:3238] ERROR -

      {u'node': u'ns_1@cb.local', u'code': 0, u'text': u'Enabled auto-failover with timeout 120 and max count 1', u'shortText': u'message', u'serverTime': u'2019-05-14T16:57:41.554Z', u'module': u'auto_failover', u'tstamp': 1557878261554, u'type': u'info'}

      16:58:51 [2019-05-14 16:58:51,693] - [rest_client:3238] ERROR -

      {u'node': u'ns_1@cb.local', u'code': 0, u'text': u'Hot-reloaded memcached.json for config change of the following keys: [<<"client_cert_auth">>,\n <<"datatype_snappy">>,\n <<"scramsha_fallback_salt">>,\n <<"xattr_enabled">>]', u'shortText': u'message', u'serverTime': u'2019-05-14T16:57:41.550Z', u'module': u'memcached_config_mgr', u'tstamp': 1557878261550, u'type': u'info'}

      16:58:51 [2019-05-14 16:58:51,693] - [rest_client:3238] ERROR -

      {u'node': u'ns_1@cb.local', u'code': 0, u'text': u'Changed cluster compat mode from undefined to [6,5]', u'shortText': u'message', u'serverTime': u'2019-05-14T16:57:41.501Z', u'module': u'compat_mode_manager', u'tstamp': 1557878261501, u'type': u'warning'}

      16:58:51 [2019-05-14 16:58:51,693] - [rest_client:3238] ERROR -

      {u'node': u'ns_1@cb.local', u'code': 0, u'text': u"I'm the only node, so I'm the master.", u'shortText': u'message', u'serverTime': u'2019-05-14T16:57:41.428Z', u'module': u'mb_master', u'tstamp': 1557878261428, u'type': u'info'}

      16:58:51 [2019-05-14 16:58:51,694] - [rest_client:3238] ERROR -

      {u'node': u'ns_1@cb.local', u'code': 1, u'text': u'Couchbase Server has started on web port 8091 on node \'ns_1@cb.local\'. Version: "6.5.0-3245-enterprise".', u'shortText': u'web start ok', u'serverTime': u'2019-05-14T16:57:41.238Z', u'module': u'menelaus_sup', u'tstamp': 1557878261238, u'type': u'info'}

      Last good build: 6.5.0-3242

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            ritam.sharma Ritam Sharma
            arunkumar Arunkumar Senthilnathan (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              PagerDuty