Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-5886

During Rebalance-in , the xdcr halts while rebalancing is on and item count fluctuates (> expected count) till xdcr completes the replication of available items.

    XMLWordPrintable

Details

    • Bug
    • Resolution: Won't Fix
    • Major
    • 2.0
    • 2.0
    • XDCR
    • Security Level: Public
    • None

    Description

      description
      There are two clusters (C1, C2) with the unidirectional xdcr enabled (C1 -> C2).
      Load is generated on cluster C1 (source)
      If a node-x is added on C1 (source) and rebalancing started, the xdcr is halted.
      However, xdcr resumes after rebalacing is over. While the XDCR is going on, the items count keeps increasing (more than expected count) on source cluster though there is no new load added to it.
      Till the xdcr replication is in process, the item count keeps on fluctuating (higher side than expected) and once the xdcr completes replication of items from C1 to C2, the item count comes back to original count (expected items).

      Set-up
      The cluster configuration of setup can be found in attached vm_cluster_2-n1-n1-f1.ini file.

      The command to initialize the vm servers is following and it provides the info like vbuckets, version , etc.

      python scripts/install.py -p product=cb,version=2.0.0-1427-rel,parallel=True,vbuckets=1024 -i b/resources/xdcr/vm_7n.ini

      Manual Steps
      1. Create cluster as per .ini file. It is a 1:1 cluster
      2. load data using following mcsoda command (10,000)
      pytests/performance/mcsoda.py localhost:11442 vbuckets=1024 doc-gen=0 doc-cache=0 ratio-creates=1 ratio-sets=1 min-value-size=256,512 max-items=4000000 exit-after-creates=1 prefix=70one
      3. Add a node (f_node) to cluster1 and start rebalancing.

      Expected Result
      The xdcr should have continued while rebalancing was on.
      The item count should remain same (10,000) on source cluster node (cluster1.xd_c1_node1) once loading is complete.

      Actual Result
      XDCR was halted till rebalancing was over.
      The item count kept on fluctuating (> 10,000) for the time xdcr was performing replication of items

      For example:
      Starting a 2 cluster unidirectional run, setting it up to 4 threads and 1 Million max items,
      the number of items on the source side has exceeded 1.5 million and whenever an extra node is added on the source cluster, two-thirds through the rebalance, the ops-per-sec on the destination node declines to an almost zero value.

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            suhailkhaki Suhail Khaki
            suhailkhaki Suhail Khaki
            Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty