Details
Description
[Jenkins]
http://qa.hq.northscale.net/job/centos_x64--31_01--uniXDCR-P1/29/consoleFull
[Test]
./testrunner -i /tmp/ubuntu-64-2.0-uniXDCR.ini GROUP=CHAIN,num_items=50000,get-cbcollect-info=True -t xdcr.uniXDCR.unidirectional.load_with_async_ops,items=100000,rdirection=unidirection,ctopology=chain,expires=60,standard_buckets=1,sasl_buckets=2,default_bucket=False,doc-ops=delete,GROUP=CHAIN;P1
[Test Logs]
[2014-04-18 03:54:11,221] - [rest_client:790] INFO - adding remote cluster hostname:10.3.4.177:8091 with username:password Administrator:password name:cluster1
[2014-04-18 03:54:11,295] - [rest_client:836] INFO - starting replication type:continuous from sasl_bucket_1 to sasl_bucket_1 in the remote cluster cluster1
[2014-04-18 03:54:11,506] - [xdcrbasetests:355] INFO - sleep for 5 secs. ...
[2014-04-18 03:54:16,513] - [rest_client:836] INFO - starting replication type:continuous from sasl_bucket_2 to sasl_bucket_2 in the remote cluster cluster1
[2014-04-18 03:54:16,653] - [xdcrbasetests:355] INFO - sleep for 5 secs. ...
[2014-04-18 03:54:21,659] - [rest_client:836] INFO - starting replication type:continuous from standard_bucket_1 to standard_bucket_1 in the remote cluster cluster1
[2014-04-18 03:54:21,824] - [xdcrbasetests:355] INFO - sleep for 5 secs. ...
..
..
..
2014-04-18 04:11:16,490] - [task:420] WARNING - Not Ready: curr_items 70001 == 70000 expected on '10.3.4.177:8091''10.3.3.208:8091''10.3.121.62:8091''10.3.2.204:8091', sasl_bucket_1 bucket
[2014-04-18 04:11:16,542] - [task:420] WARNING - Not Ready: vb_active_curr_items 70001 == 70000 expected on '10.3.4.177:8091''10.3.3.208:8091''10.3.121.62:8091''10.3.2.204:8091', sasl_bucket_1 bucket
[2014-04-18 04:11:21,585] - [task:420] WARNING - Not Ready: curr_items 70001 == 70000 expected on '10.3.4.177:8091''10.3.3.208:8091''10.3.121.62:8091''10.3.2.204:8091', sasl_bucket_1 bucket
[2014-04-18 04:11:21,622] - [task:420] WARNING - Not Ready: vb_active_curr_items 70001 == 70000 expected on '10.3.4.177:8091''10.3.3.208:8091''10.3.121.62:8091''10.3.2.204:8091', sasl_bucket_1 bucket
[Test Steps] - No UPR.
1. Create 4-4 Nodes SRC and DEST.
2. Create 1 sasl, 2 standard buckets.
3. Configure XMEM Uni XDCR for both buckets.
4. Load 1M Items each bucket.
5. Delete 30% items on each bucket.
6. expected items on Destination = 70000 while 70001 items on destination cluster (for sasl_bucket_1). Following key loadOne97310 has mismatch. Keys shows deleted from Source but not deleted from Destination, also it shows higher sequence number of Source, showing mutations were not replicated properly.
2014-04-18 04:18:16,419] - [task:1169] ERROR - ===== Verifying rev_ids failed for key: loadOne97310 =====
[2014-04-18 04:18:16,419] - [task:1170] ERROR - deleted mismatch: Source deleted:1, Destination deleted:0, Error Count:1
[2014-04-18 04:18:16,420] - [task:1170] ERROR - seqno mismatch: Source seqno:2, Destination seqno:1, Error Count:2
[2014-04-18 04:18:16,421] - [task:1170] ERROR - cas mismatch: Source cas:17748657181694094, Destination cas:17748657181694093, Error Count:3
[2014-04-18 04:18:16,423] - [task:1171] ERROR - Source meta data:
[2014-04-18 04:18:16,424] - [task:1172] ERROR - Dest meta data:
{'deleted': 0, 'seqno': 1, 'cas': 17748657181694093, 'flags': 0, 'expiration': 0}Items were properly replicated on other 2 buckets i.e. standard_bucekt_1, sasl_bucket_2. But not sasl_bucket_1.
Attachments
Issue Links
- relates to
-
MB-10856 Persistence and internal replication(TAP/UPR) are broken , affects XDCR
- Closed