Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-11786

{UPR}:: Rebalance-out hangs due to indexing stuck

    XMLWordPrintable

Details

    • Bug
    • Resolution: Fixed
    • Blocker
    • 3.0
    • 3.0
    • Security Level: Public
    • None
    • Ubuntu 64-bit
    • Yes
    • June 30 - July 18

    Description

      Seeing this issue in 991

      1. Create 7 node cluster (10.6.2.144-150)
      2. Create default Bucket
      3. Add 1K items
      4. Create 5 views and query
      5. Rebalance out node 10.6.2.150

      Step 4 and 5 are run in parallel

      We see the rebalance hanging

      I am seeing the following issue couchdb log in 10.6.2.150

      [couchdb:error,2014-07-22T13:37:52.699,ns_1@10.6.2.150:<0.217.0>:couch_log:error:44]View merger, revision mismatch for design document `_design/ddoc1', wanted 5-3275804e, got 5-3275804e
      [couchdb:error,2014-07-22T13:37:52.699,ns_1@10.6.2.150:<0.217.0>:couch_log:error:44]Uncaught error in HTTP request: {throw,{error,revision_mismatch}}

      [couchdb:error,2014-07-22T13:37:52.699,ns_1@10.6.2.150:<0.217.0>:couch_log:error:44]View merger, revision mismatch for design document `_design/ddoc1', wanted 5-3275804e, got 5-3275804e
      [couchdb:error,2014-07-22T13:37:52.699,ns_1@10.6.2.150:<0.217.0>:couch_log:error:44]Uncaught error in HTTP request: {throw,{error,revision_mismatch}}

      Stacktrace: [{couch_index_merger,query_index,3,
      [

      {file, "/buildbot/build_slave/centos-5-x64-300-builder/build/build/couchdb/src/couch_index_merger/src/couch_index_merger.erl"}

      ,

      {line,75}

      ]},
      {couch_httpd,handle_request,6,
      [

      {file, "/buildbot/build_slave/centos-5-x64-300-builder/build/build/couchdb/src/couchdb/couch_httpd.erl"}

      ,

      {line,222}

      ]},
      {mochiweb_http,headers,5,

      Will attach logs ASAP

      Test Case:: ./testrunner i ubuntu_x64109_00-Rebalance-Out.ini get-cbcollect-info=False,get-logs=False,stop-on-failure=False,get-coredumps=True,force_kill_memached=False,verify_unacked_bytes=True,dgm=True,total_vbuckets=128,std_vbuckets=5 -t rebalance.rebalanceout.RebalanceOutTests.rebalance_out_with_queries,nodes_out=1,blob_generator=False,value_size=1024,GROUP=OUT;BASIC;P0;FROM_2_0

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              parag Parag Agarwal (Inactive)
              parag Parag Agarwal (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty