Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-7660

curr_items_tot & vb_replica_curr_items mismatch when rebalancing new 1.8.1 and 2.0.1 nodes at once to 1.8.1 cluster that was rebooted before

    XMLWordPrintable

Details

    • Bug
    • Resolution: Won't Fix
    • Major
    • None
    • 2.0.1
    • couchbase-bucket
    • Security Level: Public
    • None

    Description

      http://qa.hq.northscale.net/job/centos-64-2.0-new-rebalance-mixed-cluster/25/consoleFull
      ./testrunner -i /tmp/rebalance_in.ini -t rebalance.rebalancein.RebalanceInTests.rebalance_in_with_warming_up,nodes_in=3,items=100000,replicas=2,get-logs=True,wait_timeout=180,get-cbcollect-info=True,skip_cleanup=True

      steps:
      1. cluster with 1 node 1.8.1-937-rel(10.3.3.92), default bucket( replica=2) with 100000 items
      2. stop 10.3.3.92, sleep 20 secs, start node
      3. add 2 1.8.1-937-rel nodes(10.3.3.91, 10.3.3.93) and 1 2.0.1-146-rel (10.3.3.99) and rebalance cluster

      result:
      Not Ready: curr_items_tot 299376 == 300000 expected on '10.3.3.92:8091''10.3.3.99:8091''10.3.3.93:8091''10.3.3.91:8091'
      Not Ready: vb_replica_curr_items 199376 == 200000 expected on '10.3.3.92:8091''10.3.3.99:8091''10.3.3.93:8091''10.3.3.91:8091'

      [root@caper-007 bin]# ./cbstats 10.3.3.91:11210 all | grep items
      curr_items: 24999
      curr_items_tot: 74997
      curr_temp_items: 0
      ep_access_scanner_num_items: 0
      ep_chk_max_items: 5000
      ep_diskqueue_items: 0
      ep_items_rm_from_checkpoints: 0
      ep_total_del_items: 0
      ep_total_new_items: 147039
      ep_uncommitted_items: 0
      ep_warmup_min_items_threshold: 100
      vb_active_curr_items: 24999
      vb_active_num_ref_items: 24999
      vb_pending_curr_items: 0
      vb_pending_num_ref_items: 0
      vb_replica_curr_items: 49998
      vb_replica_num_ref_items: 0
      [root@caper-007 bin]# ./cbstats 10.3.3.92:11210 all | grep items
      curr_items: 25001
      curr_items_tot: 74874
      ep_diskqueue_items: 0
      ep_items_rm_from_checkpoints: 2
      ep_total_del_items: 0
      ep_total_new_items: 48296
      ep_uncommitted_items: 0
      vb_active_curr_items: 25001
      vb_pending_curr_items: 0
      vb_replica_curr_items: 49873
      [root@caper-007 bin]# ./cbstats 10.3.3.93:11210 all | grep items
      curr_items: 25001
      curr_items_tot: 74904
      ep_diskqueue_items: 0
      ep_items_rm_from_checkpoints: 6
      ep_total_del_items: 0
      ep_total_new_items: 90619
      ep_uncommitted_items: 0
      vb_active_curr_items: 25001
      vb_pending_curr_items: 0
      vb_replica_curr_items: 49903
      [root@caper-007 bin]# ./cbstats 10.3.3.99:11210 all | grep items
      curr_items: 24999
      curr_items_tot: 74601
      curr_temp_items: 0
      ep_access_scanner_num_items: 0
      ep_chk_max_items: 5000
      ep_diskqueue_items: 0
      ep_items_rm_from_checkpoints: 0
      ep_total_del_items: 100
      ep_total_new_items: 145143
      ep_uncommitted_items: 0
      ep_warmup_min_items_threshold: 100
      vb_active_curr_items: 24999
      vb_active_num_ref_items: 24999
      vb_pending_curr_items: 0
      vb_pending_num_ref_items: 0
      vb_replica_curr_items: 49602
      vb_replica_num_ref_items: 0

      so, sum of curr_items_tot = 74997+ 74874+ 74904+ 74601=299376 != 300000

      Chiyoung, is the test valid to check for mismatch these stats?

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            xiaoqin Xiaoqin Ma (Inactive)
            andreibaranouski Andrei Baranouski
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty