Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-11980

{DCP}:: Bad Replicas leads to Rebalance-out for Max Buckets (failure:: Exit) after a series of Rebalance-in/out operations (with 1, 2, 3, 4 node)

    XMLWordPrintable

Details

    • Bug
    • Resolution: Fixed
    • Blocker
    • 3.0
    • 3.0
    • couchbase-bucket
    • Security Level: Public
    • None
    • 10.6.2.144-10.6.2.150

    Description

      1166, 3.0.0

      1. Create 1 node cluster
      2. Create max buckets allowed (10 buckets)
      3. Add 100 K items per buckets
      4. Rebalance-out 1 Node and with mutation running
      5. Rebalance-in 1 Node and with mutation running
      6. Rebalance-out 2 Node and with mutation running
      7. Rebalance-in 2 Node and with mutation running
      8. Rebalance-out 3 Node and with mutation running
      9. Rebalance-in 3 Node and with mutation running
      10. Rebalance-out 4 Node and with mutation running
      11. Rebalance-in 4 Node and with mutation running

      After Step 11 was complete. Was trying to do a rebalance-out operation for 1 node and the rebalance would exit with Bad Replicas issue.

      Bad replicators after rebalance:
      Missing = [

      {'ns_1@10.6.2.146','ns_1@10.6.2.148',11}

      ]
      Extras = [] (repeated 1 times)

      Test Case::

      ./testrunner -i ~/run_tests/palm.ini get-cbcollect-info=False,get-logs=False,stop-on-failure=False,get-coredumps=True,force_kill_memached=True,verify_unacked_bytes=True,total_vbuckets=128,std_vbuckets_dist=5,chk_max_items=1000,chk_period=30 -t rebalance.rebalanceinout.RebalanceInOutTests.incremental_rebalance_in_out_with_max_buckets_number,items=10000,default_bucket=False,skip_cleanup=True,vbuckets=128,GROUP=IN_OUT_LONG;P2

      This test case will pass but the issue comes after we run this scenario. So its a good way of finding the issue. We tested it for a simple rebalance-out operation without clean up. Note that this test case would fail if we removed skip_cleanup=True parameter as well.

      Marking the bug as regression. Wasn't expecting 1 node rebalance-out to fail for bad replicas.

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            mikew Mike Wiederhold [X] (Inactive)
            parag Parag Agarwal (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty