Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-11573

replica items count mismatch on source cluster

    XMLWordPrintable

Details

    • Bug
    • Resolution: Fixed
    • Blocker
    • 3.0
    • 3.0
    • couchbase-bucket
    • Security Level: Public
    • None
    • Build 3.0.0-855

      Ubuntu 12.04

    Description

      http://qa.hq.northscale.net/job/ubuntu_x64--01_02--rebalanceXDCR-P0/17/consoleFull

      [Test]
      ./testrunner i ubuntu_x6401_02-rebalanceXDCR-P0.ini get-cbcollect-info=True,get-logs=False,stop-on-failure=False,get-coredumps=True -t xdcr.rebalanceXDCR.Rebalance.async_rebalance_in,items=100000,rdirection=bidirection,ctopology=chain,doc-ops=update-delete,doc-ops-dest=update-delete,expires=60,rebalance=destination,num_rebalance=1,GROUP=P1

      [Test Error]
      [2014-06-23 11:46:00,302] - [task:440] WARNING - Not Ready: vb_replica_curr_items 80001 == 80000 expected on '10.3.3.146:8091''10.3.3.144:8091''10.3.3.147:8091', default bucket
      [2014-06-23 11:46:05,339] - [task:440] WARNING - Not Ready: vb_replica_curr_items 80001 == 80000 expected on '10.3.3.146:8091''10.3.3.144:8091''10.3.3.147:8091', default bucket
      [2014-06-23 11:46:10,379] - [task:440] WARNING - Not Ready: vb_replica_curr_items 80001 == 80000 expected on '10.3.3.146:8091''10.3.3.144:8091''10.3.3.147:8091', default bucket
      [2014-06-23 11:46:15,433] - [task:440] WARNING - Not Ready: vb_replica_curr_items 80001 == 80000 expected on '10.3.3.146:8091''10.3.3.144:8091''10.3.3.147:8091', default bucket
      [2014-06-23 11:46:20,463] - [task:440] WARNING - Not Ready: vb_replica_curr_items 80001 == 80000 expected on '10.3.3.146:8091''10.3.3.144:8091''10.3.3.147:8091', default bucket
      [2014-06-23 11:46:25,498] - [task:440] WARNING - Not Ready: vb_replica_curr_items 80001 == 80000 expected on '10.3.3.146:8091''10.3.3.144:8091''10.3.3.147:8091', default bucket
      [2014-06-23 11:46:31,528] - [task:440] WARNING - Not Ready: vb_replica_curr_items 80001 == 80000 expected on '10.3.3.146:8091''10.3.3.144:8091''10.3.3.147:8091', default bucket
      [2014-06-23 11:46:36,566] - [task:440] WARNING - Not Ready: vb_replica_curr_items 80001 == 80000 expected on '10.3.3.146:8091''10.3.3.144:8091''10.3.3.147:8091', default bucket

      [Test Steps]
      1. Setup 3-3 node Src and Dest cluster.
      2. Setup CAPI mode replication between default bucket.
      3. Load 1M items on each cluster.
      4. Add one node on Destination cluster.
      5. Perform 30-30% update and delete on each cluster. Update items with expiration time of 60 seconds.
      6. Wait for expiration time of 60 seconds.
      8. Expecting 80000 items on each side.
      9. 1 replica item on Source side was extra.

      I ran this test twice on the same cluster, but couldn't reproduce solely. Please see if you find anything suspected from the logs.

      Attachments

        Issue Links

          For Gerrit Dashboard: MB-11573
          # Subject Branch Project Status CR V

          Activity

            People

              chiyoung Chiyoung Seo (Inactive)
              sangharsh Sangharsh Agarwal
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty