Details
-
Bug
-
Resolution: Won't Fix
-
Major
-
2.0
-
Security Level: Public
-
None
-
General Configuration
Virtual Machine (VM)
LSB Version: :core-4.0-amd64:core-4.0-ia32:core-4.0-noarch:graphics-4.0-amd64:graphics-4.0-ia32:graphics-4.0-noarch:printing-4.0-amd64:printing-4.0-ia32:printing-4.0-noarch
Distributor ID: CentOS
Description: CentOS release 5.8 (Final)
Release: 5.8
Codename: FinalGeneral Configuration Virtual Machine (VM) LSB Version: :core-4.0-amd64:core-4.0-ia32:core-4.0-noarch:graphics-4.0-amd64:graphics-4.0-ia32:graphics-4.0-noarch:printing-4.0-amd64:printing-4.0-ia32:printing-4.0-noarch Distributor ID: CentOS Description: CentOS release 5.8 (Final) Release: 5.8 Codename: Final
Description
description
There are two clusters (C1, C2) with the unidirectional xdcr enabled (C1 -> C2).
Load is generated on cluster C1 (source)
If a node-x is added on C1 (source) and rebalancing started, the xdcr is halted.
However, xdcr resumes after rebalacing is over. While the XDCR is going on, the items count keeps increasing (more than expected count) on source cluster though there is no new load added to it.
Till the xdcr replication is in process, the item count keeps on fluctuating (higher side than expected) and once the xdcr completes replication of items from C1 to C2, the item count comes back to original count (expected items).
Set-up
The cluster configuration of setup can be found in attached vm_cluster_2-n1-n1-f1.ini file.
The command to initialize the vm servers is following and it provides the info like vbuckets, version , etc.
python scripts/install.py -p product=cb,version=2.0.0-1427-rel,parallel=True,vbuckets=1024 -i b/resources/xdcr/vm_7n.ini
Manual Steps
1. Create cluster as per .ini file. It is a 1:1 cluster
2. load data using following mcsoda command (10,000)
pytests/performance/mcsoda.py localhost:11442 vbuckets=1024 doc-gen=0 doc-cache=0 ratio-creates=1 ratio-sets=1 min-value-size=256,512 max-items=4000000 exit-after-creates=1 prefix=70one
3. Add a node (f_node) to cluster1 and start rebalancing.
Expected Result
The xdcr should have continued while rebalancing was on.
The item count should remain same (10,000) on source cluster node (cluster1.xd_c1_node1) once loading is complete.
Actual Result
XDCR was halted till rebalancing was over.
The item count kept on fluctuating (> 10,000) for the time xdcr was performing replication of items
For example:
Starting a 2 cluster unidirectional run, setting it up to 4 threads and 1 Million max items,
the number of items on the source side has exceeded 1.5 million and whenever an extra node is added on the source cluster, two-thirds through the rebalance, the ops-per-sec on the destination node declines to an almost zero value.