Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-13528

Go-XDCR: One key missing update following rebalance-out

    XMLWordPrintable

Details

    • Bug
    • Resolution: Fixed
    • Critical
    • 4.0.0
    • 4.0.0
    • XDCR
    • Security Level: Public
    • None
    • centOS 6.x

    Description

      Build
      ------
      3.5.0-1270

      Testcase
      ------------
      ./testrunner -i INI_FILE.ini get-cbcollect-info=True,get-logs=False,stop-on-failure=False,replication_type=xmem,enable_goxdcr=True,checkpoint_interval=3600,GROUP=ALL -t xdcr.rebalanceXDCR.Rebalance.async_rebalance_out_master,items=100000,rdirection=bidirection,ctopology=chain,update=C1-C2,delete=C1-C2,rebalance=C1-C2,GROUP=P1

      test#32 in http://qa.sc.couchbase.com/view/BY%20COMPONENT/job/cen006-p0-xxdcr-vsetST-02-goxdcr-rebalance/18/consoleFull

      Steps
      -------
      1. C1 [.51,.47]<--> C2 [.48, .49]
      2. Load 100K keys on both clusters
      3. Rebalance out .51 and .48.
      4. Update 30% of keys on C1 and C2.
      5. Delete 30% of keys on C1 and C2.
      6. Verify correctness.

      C1-key-2185 on C1 is missing an update from C2.

      [2015-02-17 21:51:56,120] - [xdcrnewbasetests:2860] ERROR - Key: C1-key-2185 Bad hash result: 20575 != 12618
      [2015-02-17 21:51:56,121] - [xdcrnewbasetests:2721] INFO - Verifying RevIds for Couchbase Cluster: C1, Master Ip: 172.23.105.47 -> Couchbase Cluster: C2, Master Ip: 172.23.105.49, bucket <bound method XDCReplication.get_src_bucket of <xdcr.xdcrnewbasetests.XDCReplication instance at 0x7ffd37eb6128>>
      [2015-02-17 21:51:56,242] - [data_helper:295] INFO - creating direct client 172.23.105.47:11210 default
      [2015-02-17 21:51:56,454] - [data_helper:295] INFO - creating direct client 172.23.105.47:11210 default
      [2015-02-17 21:51:56,589] - [data_helper:295] INFO - creating direct client 172.23.105.49:11210 default
      [2015-02-17 21:52:53,628] - [task:1309] INFO - 50000 items have been verified
      [2015-02-17 21:52:59,530] - [task:1361] ERROR - ===== Verifying rev_ids failed for key: C1-key-2185 =====
      [2015-02-17 21:52:59,531] - [task:1362] ERROR - seqno mismatch: Source seqno:2, Destination seqno:1, Error Count:1
      [2015-02-17 21:52:59,531] - [task:1362] ERROR - cas mismatch: Source cas:1424237923300802560, Destination cas:1424237703915372544, Error Count:2
      [2015-02-17 21:52:59,532] - [task:1363] ERROR - Source meta data:

      {'deleted': 0, 'seqno': 2, 'cas': 1424237923300802560, 'flags': 0, 'expiration': 0}

      [2015-02-17 21:52:59,532] - [task:1364] ERROR - Dest meta data:

      {'deleted': 0, 'seqno': 1, 'cas': 1424237703915372544, 'flags': 0, 'expiration': 0}

      Logs attached.

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            apiravi Aruna Piravi (Inactive)
            apiravi Aruna Piravi (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty