Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-6708

Docs not getting persisted to disk with consistent views enabled

    XMLWordPrintable

Details

    Description

      Test to reproduce the issue:

      ./testrunner -i ../../ini/vm-4nodes-2.ini -t rebalance.rebalancein.RebalanceInTests.rebalance_in_with_ops,nodes_in=1,replicas=1,items=50000,doc_ops=create;update;delete,disabled_consistent_view=False

      The test is rebalancing in 1 node and inserting/mutating docs while rebalance is in progress.

      After rebalance is finished, ep_queue_size is always > 0 and doesn't get drained even after waiting for a very long time e.g.

      2012-09-21 05:57:36,465 - root - INFO - Not Ready: ep_queue_size 2042 == 0 expected on '10.3.2.57'
      2012-09-21 05:57:41,484 - root - INFO - Not Ready: ep_queue_size 2042 == 0 expected on '10.3.2.57'
      2012-09-21 05:57:46,500 - root - INFO - Not Ready: ep_queue_size 2042 == 0 expected on '10.3.2.57'
      2012-09-21 05:57:51,518 - root - INFO - Not Ready: ep_queue_size 2042 == 0 expected on '10.3.2.57'
      2012-09-21 05:57:56,536 - root - INFO - Not Ready: ep_queue_size 2042 == 0 expected on '10.3.2.57'
      2012-09-21 05:58:01,556 - root - INFO - Not Ready: ep_queue_size 2042 == 0 expected on '10.3.2.57'
      2012-09-21 05:58:06,574 - root - INFO - Not Ready: ep_queue_size 2042 == 0 expected on '10.3.2.57'
      2012-09-21 05:58:11,591 - root - INFO - Not Ready: ep_queue_size 2042 == 0 expected on '10.3.2.57'
      2012-09-21 05:58:16,610 - root - INFO - Not Ready: ep_queue_size 2042 == 0 expected on '10.3.2.57'

      In the logs following message can be seen repeating:

      memcached<0.404.0>: Fri Sep 21 06:01:40.846677 PDT 3: Retry notify CouchDB of update, vbid=312 rev=1
      memcached<0.404.0>: Fri Sep 21 06:01:40.849414 PDT 3: Retry notify CouchDB of update, vbid=312 rev=1
      memcached<0.404.0>: Fri Sep 21 06:01:40.852653 PDT 3: Retry notify CouchDB of update, vbid=312 rev=1
      memcached<0.404.0>: Fri Sep 21 06:01:40.855700 PDT 3: Retry notify CouchDB of update, vbid=312 rev=1
      memcached<0.404.0>: Fri Sep 21 06:01:40.858190 PDT 3: Retry notify CouchDB of update, vbid=312 rev=1
      memcached<0.404.0>: Fri Sep 21 06:01:40.860517 PDT 3: Retry notify CouchDB of update, vbid=312 rev=1
      memcached<0.404.0>: Fri Sep 21 06:01:40.863663 PDT 3: Retry notify CouchDB of update, vbid=312 rev=1
      memcached<0.404.0>: Fri Sep 21 06:01:40.867291 PDT 3: Retry notify CouchDB of update, vbid=312 rev=1
      memcached<0.404.0>: Fri Sep 21 06:01:40.870259 PDT 3: Retry notify CouchDB of update, vbid=312 rev=1
      memcached<0.404.0>: Fri Sep 21 06:01:40.872901 PDT 3: Retry notify CouchDB of update, vbid=312 rev=1
      memcached<0.404.0>: Fri Sep 21 06:01:40.876232 PDT 3: Retry notify CouchDB of update, vbid=312 rev=1
      memcached<0.404.0>: Fri Sep 21 06:01:40.878934 PDT 3: Retry notify CouchDB of update, vbid=312 rev=1

      Diagnostics are attached.

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            chiyoung Chiyoung Seo (Inactive)
            deepkaran.salooja Deepkaran Salooja
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty