Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-5289

Rebalance fails (noproc, ns_single_vbucket_mover:mover/6)

    XMLWordPrintable

Details

    • Bug
    • Resolution: Fixed
    • Major
    • 2.0-beta
    • 2.0-developer-preview-4
    • ns_server
    • Security Level: Public
    • None
    • Quite recent checkout (1 day old or so), Linux VM

    Description

      The testrunner spatial test
      spatialrebalancetests.SpatialRebalanceTests.test_insert_x_docs_during_rebalance,num-docs=10000

      fails sometimes. When it fails, it does a rebalance and then returns an error (see below). I've attached the debug log from node 0 (there are 4 nodes) from the time when i happened. The crash report mentions ns_single_vbucket_mover:mover/6.

      [2012-05-11 03:42:58,288] - [spatial_helper:120] INFO - inserted 10000 json documents
      [2012-05-11 03:42:58,294] - [rest_client:770] INFO - rebalance percentage : 7.8125 %
      [2012-05-11 03:43:00,299] - [rest_client:770] INFO - rebalance percentage : 12.5 %
      [2012-05-11 03:43:02,305] - [rest_client:770] INFO - rebalance percentage : 18.75 %
      [2012-05-11 03:43:04,313] - [rest_client:770] INFO - rebalance percentage : 23.4375 %
      [2012-05-11 03:43:06,318] - [rest_client:770] INFO - rebalance percentage : 25.78125 %
      [2012-05-11 03:43:08,339] - [rest_client:770] INFO - rebalance percentage : 28.90625 %
      [2012-05-11 03:43:10,346] - [rest_client:770] INFO - rebalance percentage : 32.03125 %
      [2012-05-11 03:43:12,352] - [rest_client:770] INFO - rebalance percentage : 35.15625 %
      [2012-05-11 03:43:14,355] - [rest_client:770] INFO - rebalance percentage : 38.28125 %
      [2012-05-11 03:43:16,361] - [rest_client:770] INFO - rebalance percentage : 41.40625 %
      [2012-05-11 03:43:18,365] - [rest_client:770] INFO - rebalance percentage : 44.53125 %
      [2012-05-11 03:43:20,370] - [rest_client:770] INFO - rebalance percentage : 47.65625 %
      [2012-05-11 03:43:22,410] - [rest_client:770] INFO - rebalance percentage : 51.5625 %
      [2012-05-11 03:43:24,432] - [rest_client:770] INFO - rebalance percentage : 54.6875 %
      [2012-05-11 03:43:26,436] - [rest_client:770] INFO - rebalance percentage : 62.5 %
      [2012-05-11 03:43:28,459] - [rest_client:770] INFO - rebalance percentage : 71.875 %
      [2012-05-11 03:43:30,469] - [rest_client:770] INFO - rebalance percentage : 81.25 %
      [2012-05-11 03:43:32,477] - [rest_client:770] INFO - rebalance percentage : 89.0625 %
      [2012-05-11 03:43:34,481] - [rest_client:770] INFO - rebalance percentage : 100.0 %
      [2012-05-11 03:43:36,486] - [rest_client:770] INFO - rebalance percentage : 100.0 %
      [2012-05-11 03:43:38,491] - [rest_client:770] INFO - rebalance percentage : 100.0 %
      [2012-05-11 03:43:40,495] - [rest_client:770] INFO - rebalance percentage : 100.0 %
      [2012-05-11 03:43:42,499] - [rest_client:770] INFO - rebalance percentage : 100.0 %
      [2012-05-11 03:43:44,503] - [rest_client:770] INFO - rebalance percentage : 100.0 %
      [2012-05-11 03:43:46,506] - [rest_client:763] ERROR -

      {u'status': u'none', u'errorMessage': u'Rebalance failed. See logs for detailed reason. You can try rebalance again.'}

      - rebalance failed
      ERROR

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            alkondratenko Aleksey Kondratenko (Inactive)
            vmx Volker Mische
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty