Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-14473

Stats inconsistent in Suse 11

    XMLWordPrintable

Details

    • Bug
    • Resolution: Cannot Reproduce
    • Critical
    • 4.0.0
    • 4.0.0
    • ns_server
    • Security Level: Public
    • None
    • Suse 11
    • Untriaged
    • Unknown

    Description

      Build

      4.0.0-1841

      Testcase
      --------
      ./testrunner -i /tmp/four-opensuse-nodes.ini -t xdcr.biXDCR.bidirectional.load_with_async_ops,replicas=1,items=10000,ctopology=chain,rdirection=bidirection

      Steps

      1. C1[.84,.85] <--> C2 [.86,.87]
      2. Load 10000 keys on C1 and C2
      3. Verify keys

      This cannot be reproduced consistently.

      After waiting for 6 (3 for active and 3 for replica) mins, active, replica key count did not match expected value.

      test log - https://friendpaste.com/68dIv86jV3Cp9EMUWDLbI0
      in http://qa.sc.couchbase.com/job/suse11-p0-sanit-vset09-00/64/consoleFull

      2015-04-13 07:50:02 | INFO | MainProcess | test_thread | [xdcrnewbasetests.verify_items_count] Saw: vb_active_curr_items 20000 == 20000 expected on C1, default bucket
      2015-04-13 07:50:02 | INFO | MainProcess | test_thread | [xdcrnewbasetests.verify_items_count] Saw: vb_replica_curr_items 20000 == 20000 expected on C1, default bucket
      2015-04-13 07:50:02 | WARNING | MainProcess | test_thread | [xdcrnewbasetests.verify_items_count] Not Ready: vb_active_curr_items 3055 == 20000 expected on C2, default bucket
      2015-04-13 07:50:07 | WARNING | MainProcess | test_thread | [xdcrnewbasetests.verify_items_count] Not Ready: vb_active_curr_items 3055 == 20000 expected on C2, default bucket
      2015-04-13 07:50:12 | WARNING | MainProcess | test_thread | [xdcrnewbasetests.verify_items_count] Not Ready: vb_active_curr_items 3055 == 20000 expected on C2, default bucket
      2015-04-13 07:50:17 | WARNING | MainProcess | test_thread | [xdcrnewbasetests.verify_items_count] Not Ready: vb_active_curr_items 3055 == 20000 expected on C2, default bucket
      2015-04-13 07:50:22 | WARNING | MainProcess | test_thread | [xdcrnewbasetests.verify_items_count] Not Ready: vb_active_curr_items 3055 == 20000 expected on C2, default bucket
      2015-04-13 07:50:27 | WARNING | MainProcess | test_thread | [xdcrnewbasetests.verify_items_count] Not Ready: vb_active_curr_items 3055 == 20000 expected on C2, default bucket
      2015-04-13 07:50:32 | WARNING | MainProcess | test_thread | [xdcrnewbasetests.verify_items_count] Not Ready: vb_active_curr_items 3055 == 20000 expected on C2, default bucket
      2015-04-13 07:50:38 | WARNING | MainProcess | test_thread | [xdcrnewbasetests.verify_items_count] Not Ready: vb_active_curr_items 3055 == 20000 expected on C2, default bucket
      2015-04-13 07:50:43 | WARNING | MainProcess | test_thread | [xdcrnewbasetests.verify_items_count] Not Ready: vb_active_curr_items 3055 == 20000 expected on C2, default bucket
      2015-04-13 07:50:48 | WARNING | MainProcess | test_thread | [xdcrnewbasetests.verify_items_count] Not Ready: vb_active_curr_items 3055 == 20000 expected on C2, default bucket
      2015-04-13 07:50:53 | WARNING | MainProcess | test_thread | [xdcrnewbasetests.verify_items_count] Not Ready: vb_active_curr_items 3055 == 20000 expected on C2, default bucket
      2015-04-13 07:50:58 | WARNING | MainProcess | test_thread | [xdcrnewbasetests.verify_items_count] Not Ready: vb_active_curr_items 3055 == 20000 expected on C2, default bucket
      2015-04-13 07:51:03 | WARNING | MainProcess | test_thread | [xdcrnewbasetests.verify_items_count] Not Ready: vb_active_curr_items 3055 == 20000 expected on C2, default bucket
      2015-04-13 07:51:08 | WARNING | MainProcess | test_thread | [xdcrnewbasetests.verify_items_count] Not Ready: vb_active_curr_items 3055 == 20000 expected on C2, default bucket
      2015-04-13 07:51:13 | WARNING | MainProcess | test_thread | [xdcrnewbasetests.verify_items_count] Not Ready: vb_active_curr_items 3055 == 20000 expected on C2, default bucket
      2015-04-13 07:51:18 | WARNING | MainProcess | test_thread | [xdcrnewbasetests.verify_items_count] Not Ready: vb_active_curr_items 3055 == 20000 expected on C2, default bucket
      2015-04-13 07:51:23 | WARNING | MainProcess | test_thread | [xdcrnewbasetests.verify_items_count] Not Ready: vb_active_curr_items 3055 == 20000 expected on C2, default bucket
      2015-04-13 07:51:28 | WARNING | MainProcess | test_thread | [xdcrnewbasetests.verify_items_count] Not Ready: vb_active_curr_items 3055 == 20000 expected on C2, default bucket
      2015-04-13 07:51:33 | WARNING | MainProcess | test_thread | [xdcrnewbasetests.verify_items_count] Not Ready: vb_active_curr_items 3055 == 20000 expected on C2, default bucket
      2015-04-13 07:51:38 | WARNING | MainProcess | test_thread | [xdcrnewbasetests.verify_items_count] Not Ready: vb_active_curr_items 3055 == 20000 expected on C2, default bucket
      2015-04-13 07:51:43 | WARNING | MainProcess | test_thread | [xdcrnewbasetests.verify_items_count] Not Ready: vb_active_curr_items 3055 == 20000 expected on C2, default bucket
      2015-04-13 07:51:49 | WARNING | MainProcess | test_thread | [xdcrnewbasetests.verify_items_count] Not Ready: vb_active_curr_items 3055 == 20000 expected on C2, default bucket
      2015-04-13 07:51:54 | WARNING | MainProcess | test_thread | [xdcrnewbasetests.verify_items_count] Not Ready: vb_active_curr_items 3055 == 20000 expected on C2, default bucket
      2015-04-13 07:51:59 | WARNING | MainProcess | test_thread | [xdcrnewbasetests.verify_items_count] Not Ready: vb_active_curr_items 3055 == 20000 expected on C2, default bucket
      2015-04-13 07:52:04 | WARNING | MainProcess | test_thread | [xdcrnewbasetests.verify_items_count] Not Ready: vb_active_curr_items 3055 == 20000 expected on C2, default bucket
      2015-04-13 07:52:09 | WARNING | MainProcess | test_thread | [xdcrnewbasetests.verify_items_count] Not Ready: vb_active_curr_items 3055 == 20000 expected on C2, default bucket
      2015-04-13 07:52:14 | WARNING | MainProcess | test_thread | [xdcrnewbasetests.verify_items_count] Not Ready: vb_active_curr_items 3055 == 20000 expected on C2, default bucket
      2015-04-13 07:52:19 | WARNING | MainProcess | test_thread | [xdcrnewbasetests.verify_items_count] Not Ready: vb_active_curr_items 3055 == 20000 expected on C2, default bucket
      2015-04-13 07:52:24 | WARNING | MainProcess | test_thread | [xdcrnewbasetests.verify_items_count] Not Ready: vb_active_curr_items 3055 == 20000 expected on C2, default bucket
      2015-04-13 07:52:29 | WARNING | MainProcess | test_thread | [xdcrnewbasetests.verify_items_count] Not Ready: vb_active_curr_items 3055 == 20000 expected on C2, default bucket
      2015-04-13 07:52:34 | WARNING | MainProcess | test_thread | [xdcrnewbasetests.verify_items_count] Not Ready: vb_active_curr_items 3055 == 20000 expected on C2, default bucket
      2015-04-13 07:52:39 | WARNING | MainProcess | test_thread | [xdcrnewbasetests.verify_items_count] Not Ready: vb_active_curr_items 3055 == 20000 expected on C2, default bucket
      2015-04-13 07:52:44 | WARNING | MainProcess | test_thread | [xdcrnewbasetests.verify_items_count] Not Ready: vb_active_curr_items 3055 == 20000 expected on C2, default bucket
      2015-04-13 07:52:49 | WARNING | MainProcess | test_thread | [xdcrnewbasetests.verify_items_count] Not Ready: vb_active_curr_items 3055 == 20000 expected on C2, default bucket
      2015-04-13 07:52:54 | WARNING | MainProcess | test_thread | [xdcrnewbasetests.verify_items_count] Not Ready: vb_active_curr_items 3055 == 20000 expected on C2, default bucket
      2015-04-13 07:52:59 | WARNING | MainProcess | test_thread | [xdcrnewbasetests.verify_items_count] Not Ready: vb_active_curr_items 3055 == 20000 expected on C2, default bucket
      2015-04-13 07:53:04 | ERROR | MainProcess | test_thread | [xdcrnewbasetests.verify_items_count] ERROR: Timed-out waiting for active item count to match
      2015-04-13 07:53:05 | WARNING | MainProcess | test_thread | [xdcrnewbasetests.verify_items_count] Not Ready: vb_replica_curr_items 3056 == 20000 expected on C2, default bucket
      2015-04-13 07:53:08 | WARNING | MainProcess | test_thread | [xdcrnewbasetests.verify_items_count] Not Ready: vb_replica_curr_items 3056 == 20000 expected on C2, default bucket
      2015-04-13 07:53:11 | WARNING | MainProcess | test_thread | [xdcrnewbasetests.verify_items_count] Not Ready: vb_replica_curr_items 3056 == 20000 expected on C2, default bucket
      2015-04-13 07:53:14 | WARNING | MainProcess | test_thread | [xdcrnewbasetests.verify_items_count] Not Ready: vb_replica_curr_items 3056 == 20000 expected on C2, default bucket
      2015-04-13 07:53:17 | WARNING | MainProcess | test_thread | [xdcrnewbasetests.verify_items_count] Not Ready: vb_replica_curr_items 3056 == 20000 expected on C2, default bucket
      2015-04-13 07:53:20 | WARNING | MainProcess | test_thread | [xdcrnewbasetests.verify_items_count] Not Ready: vb_replica_curr_items 3056 == 20000 expected on C2, default bucket
      2015-04-13 07:53:23 | WARNING | MainProcess | test_thread | [xdcrnewbasetests.verify_items_count] Not Ready: vb_replica_curr_items 3056 == 20000 expected on C2, default bucket
      2015-04-13 07:53:26 | WARNING | MainProcess | test_thread | [xdcrnewbasetests.verify_items_count] Not Ready: vb_replica_curr_items 3056 == 20000 expected on C2, default bucket
      2015-04-13 07:53:29 | WARNING | MainProcess | test_thread | [xdcrnewbasetests.verify_items_count] Not Ready: vb_replica_curr_items 3056 == 20000 expected on C2, default bucket
      2015-04-13 07:53:32 | WARNING | MainProcess | test_thread | [xdcrnewbasetests.verify_items_count] Not Ready: vb_replica_curr_items 3056 == 20000 expected on C2, default bucket
      2015-04-13 07:53:35 | WARNING | MainProcess | test_thread | [xdcrnewbasetests.verify_items_count] Not Ready: vb_replica_curr_items 3056 == 20000 expected on C2, default bucket
      2015-04-13 07:53:38 | WARNING | MainProcess | test_thread | [xdcrnewbasetests.verify_items_count] Not Ready: vb_replica_curr_items 3056 == 20000 expected on C2, default bucket
      2015-04-13 07:53:41 | WARNING | MainProcess | test_thread | [xdcrnewbasetests.verify_items_count] Not Ready: vb_replica_curr_items 3056 == 20000 expected on C2, default bucket
      2015-04-13 07:53:44 | WARNING | MainProcess | test_thread | [xdcrnewbasetests.verify_items_count] Not Ready: vb_replica_curr_items 3056 == 20000 expected on C2, default bucket
      2015-04-13 07:53:47 | WARNING | MainProcess | test_thread | [xdcrnewbasetests.verify_items_count] Not Ready: vb_replica_curr_items 3056 == 20000 expected on C2, default bucket
      2015-04-13 07:53:50 | WARNING | MainProcess | test_thread | [xdcrnewbasetests.verify_items_count] Not Ready: vb_replica_curr_items 3056 == 20000 expected on C2, default bucket
      2015-04-13 07:53:54 | WARNING | MainProcess | test_thread | [xdcrnewbasetests.verify_items_count] Not Ready: vb_replica_curr_items 3056 == 20000 expected on C2, default bucket
      2015-04-13 07:53:57 | WARNING | MainProcess | test_thread | [xdcrnewbasetests.verify_items_count] Not Ready: vb_replica_curr_items 3056 == 20000 expected on C2, default bucket
      2015-04-13 07:54:00 | WARNING | MainProcess | test_thread | [xdcrnewbasetests.verify_items_count] Not Ready: vb_replica_curr_items 3056 == 20000 expected on C2, default bucket
      2015-04-13 07:54:03 | WARNING | MainProcess | test_thread | [xdcrnewbasetests.verify_items_count] Not Ready: vb_replica_curr_items 3056 == 20000 expected on C2, default bucket
      2015-04-13 07:54:06 | WARNING | MainProcess | test_thread | [xdcrnewbasetests.verify_items_count] Not Ready: vb_replica_curr_items 3056 == 20000 expected on C2, default bucket
      2015-04-13 07:54:09 | WARNING | MainProcess | test_thread | [xdcrnewbasetests.verify_items_count] Not Ready: vb_replica_curr_items 3056 == 20000 expected on C2, default bucket
      2015-04-13 07:54:12 | WARNING | MainProcess | test_thread | [xdcrnewbasetests.verify_items_count] Not Ready: vb_replica_curr_items 3056 == 20000 expected on C2, default bucket
      2015-04-13 07:54:15 | WARNING | MainProcess | test_thread | [xdcrnewbasetests.verify_items_count] Not Ready: vb_replica_curr_items 3056 == 20000 expected on C2, default bucket
      2015-04-13 07:54:18 | WARNING | MainProcess | test_thread | [xdcrnewbasetests.verify_items_count] Not Ready: vb_replica_curr_items 3056 == 20000 expected on C2, default bucket
      2015-04-13 07:54:21 | WARNING | MainProcess | test_thread | [xdcrnewbasetests.verify_items_count] Not Ready: vb_replica_curr_items 3056 == 20000 expected on C2, default bucket
      2015-04-13 07:54:24 | WARNING | MainProcess | test_thread | [xdcrnewbasetests.verify_items_count] Not Ready: vb_replica_curr_items 3056 == 20000 expected on C2, default bucket
      2015-04-13 07:54:27 | WARNING | MainProcess | test_thread | [xdcrnewbasetests.verify_items_count] Not Ready: vb_replica_curr_items 3056 == 20000 expected on C2, default bucket
      2015-04-13 07:54:30 | WARNING | MainProcess | test_thread | [xdcrnewbasetests.verify_items_count] Not Ready: vb_replica_curr_items 3056 == 20000 expected on C2, default bucket
      2015-04-13 07:54:33 | WARNING | MainProcess | test_thread | [xdcrnewbasetests.verify_items_count] Not Ready: vb_replica_curr_items 3056 == 20000 expected on C2, default bucket
      2015-04-13 07:54:36 | WARNING | MainProcess | test_thread | [xdcrnewbasetests.verify_items_count] Not Ready: vb_replica_curr_items 3056 == 20000 expected on C2, default bucket
      2015-04-13 07:54:40 | WARNING | MainProcess | test_thread | [xdcrnewbasetests.verify_items_count] Not Ready: vb_replica_curr_items 3056 == 20000 expected on C2, default bucket
      2015-04-13 07:54:43 | WARNING | MainProcess | test_thread | [xdcrnewbasetests.verify_items_count] Not Ready: vb_replica_curr_items 3056 == 20000 expected on C2, default bucket
      2015-04-13 07:54:46 | WARNING | MainProcess | test_thread | [xdcrnewbasetests.verify_items_count] Not Ready: vb_replica_curr_items 3056 == 20000 expected on C2, default bucket
      2015-04-13 07:54:49 | WARNING | MainProcess | test_thread | [xdcrnewbasetests.verify_items_count] Not Ready: vb_replica_curr_items 3056 == 20000 expected on C2, default bucket
      2015-04-13 07:54:52 | WARNING | MainProcess | test_thread | [xdcrnewbasetests.verify_items_count] Not Ready: vb_replica_curr_items 3056 == 20000 expected on C2, default bucket
      2015-04-13 07:54:55 | WARNING | MainProcess | test_thread | [xdcrnewbasetests.verify_items_count] Not Ready: vb_replica_curr_items 3056 == 20000 expected on C2, default bucket
      2015-04-13 07:54:58 | WARNING | MainProcess | test_thread | [xdcrnewbasetests.verify_items_count] Not Ready: vb_replica_curr_items 3056 == 20000 expected on C2, default bucket
      2015-04-13 07:55:01 | WARNING | MainProcess | test_thread | [xdcrnewbasetests.verify_items_count] Not Ready: vb_replica_curr_items 3056 == 20000 expected on C2, default bucket
      2015-04-13 07:55:04 | WARNING | MainProcess | test_thread | [xdcrnewbasetests.verify_items_count] Not Ready: vb_replica_curr_items 3056 == 20000 expected on C2, default bucket
      2015-04-13 07:55:07 | WARNING | MainProcess | test_thread | [xdcrnewbasetests.verify_items_count] Not Ready: vb_replica_curr_items 3056 == 20000 expected on C2, default bucket
      2015-04-13 07:55:10 | WARNING | MainProcess | test_thread | [xdcrnewbasetests.verify_items_count] Not Ready: vb_replica_curr_items 3056 == 20000 expected on C2, default bucket
      2015-04-13 07:55:13 | WARNING | MainProcess | test_thread | [xdcrnewbasetests.verify_items_count] Not Ready: vb_replica_curr_items 3056 == 20000 expected on C2, default bucket
      2015-04-13 07:55:16 | WARNING | MainProcess | test_thread | [xdcrnewbasetests.verify_items_count] Not Ready: vb_replica_curr_items 3056 == 20000 expected on C2, default bucket
      2015-04-13 07:55:19 | WARNING | MainProcess | test_thread | [xdcrnewbasetests.verify_items_count] Not Ready: vb_replica_curr_items 3056 == 20000 expected on C2, default bucket
      2015-04-13 07:55:22 | WARNING | MainProcess | test_thread | [xdcrnewbasetests.verify_items_count] Not Ready: vb_replica_curr_items 3056 == 20000 expected on C2, default bucket
      2015-04-13 07:55:26 | WARNING | MainProcess | test_thread | [xdcrnewbasetests.verify_items_count] Not Ready: vb_replica_curr_items 3056 == 20000 expected on C2, default bucket
      2015-04-13 07:55:29 | WARNING | MainProcess | test_thread | [xdcrnewbasetests.verify_items_count] Not Ready: vb_replica_curr_items 3056 == 20000 expected on C2, default bucket
      2015-04-13 07:55:32 | WARNING | MainProcess | test_thread | [xdcrnewbasetests.verify_items_count] Not Ready: vb_replica_curr_items 3056 == 20000 expected on C2, default bucket
      2015-04-13 07:55:35 | WARNING | MainProcess | test_thread | [xdcrnewbasetests.verify_items_count] Not Ready: vb_replica_curr_items 3056 == 20000 expected on C2, default bucket
      2015-04-13 07:55:38 | WARNING | MainProcess | test_thread | [xdcrnewbasetests.verify_items_count] Not Ready: vb_replica_curr_items 3056 == 20000 expected on C2, default bucket
      2015-04-13 07:55:41 | WARNING | MainProcess | test_thread | [xdcrnewbasetests.verify_items_count] Not Ready: vb_replica_curr_items 3056 == 20000 expected on C2, default bucket
      2015-04-13 07:55:44 | WARNING | MainProcess | test_thread | [xdcrnewbasetests.verify_items_count] Not Ready: vb_replica_curr_items 3056 == 20000 expected on C2, default bucket
      2015-04-13 07:55:47 | WARNING | MainProcess | test_thread | [xdcrnewbasetests.verify_items_count] Not Ready: vb_replica_curr_items 3056 == 20000 expected on C2, default bucket
      2015-04-13 07:55:50 | WARNING | MainProcess | test_thread | [xdcrnewbasetests.verify_items_count] Not Ready: vb_replica_curr_items 3056 == 20000 expected on C2, default bucket
      2015-04-13 07:55:53 | WARNING | MainProcess | test_thread | [xdcrnewbasetests.verify_items_count] Not Ready: vb_replica_curr_items 3056 == 20000 expected on C2, default bucket
      2015-04-13 07:55:56 | WARNING | MainProcess | test_thread | [xdcrnewbasetests.verify_items_count] Not Ready: vb_replica_curr_items 3056 == 20000 expected on C2, default bucket
      2015-04-13 07:55:59 | WARNING | MainProcess | test_thread | [xdcrnewbasetests.verify_items_count] Not Ready: vb_replica_curr_items 3056 == 20000 expected on C2, default bucket
      2015-04-13 07:56:02 | WARNING | MainProcess | test_thread | [xdcrnewbasetests.verify_items_count] Not Ready: vb_replica_curr_items 3056 == 20000 expected on C2, default bucket

      Attaching logs. Don't see anything suspicious in logs though. Both .84 and .85 seem to have replicated 5000 docs each.

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            apiravi Aruna Piravi (Inactive)
            apiravi Aruna Piravi (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty