Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-29189

Items get stuck in replication queue during initial data load

    XMLWordPrintable

Details

    Description

      Build 5.5.0-2432.

      Setup:

      • 2 nodes
      • 20GB memory quota
      • 1 bucket, full ejection

      Steps:

      /opt/couchbase/bin/cbbackupmgr restore --archive /backups --repo bigfun20M  --threads 16 --host http://172.23.96.5 --username Administrator --password password

      cbbackupmgr finished successfully, but 188,367 items pending replication for hours.

      From the test output:

      2018-04-07T13:11:37 [INFO] ep_dcp_replica_items_remaining = 197,661
      		 
      2018-04-07T13:11:37 [INFO] ep_dcp_other_items_remaining reached 0
      		 
      2018-04-07T13:11:39 [INFO] ep_dcp_replica_items_remaining = 197,661
      		 
      2018-04-07T13:11:41 [INFO] ep_dcp_replica_items_remaining = 197,760
      		 
      2018-04-07T13:11:43 [INFO] ep_dcp_replica_items_remaining = 197,760
      		 
      2018-04-07T13:11:45 [INFO] ep_dcp_replica_items_remaining = 187,495
      		 
      2018-04-07T13:11:47 [INFO] ep_dcp_replica_items_remaining = 190,984
      		 
      2018-04-07T13:11:49 [INFO] ep_dcp_replica_items_remaining = 186,694
      		 
      2018-04-07T13:11:51 [INFO] ep_dcp_replica_items_remaining = 197,619
      		 
      2018-04-07T13:11:53 [INFO] ep_dcp_replica_items_remaining = 197,619
      		 
      2018-04-07T13:11:55 [INFO] ep_dcp_replica_items_remaining = 188,367
      		 
      2018-04-07T13:11:57 [INFO] ep_dcp_replica_items_remaining = 188,367
      		 
      2018-04-07T13:11:59 [INFO] ep_dcp_replica_items_remaining = 188,367
      		 
      2018-04-07T13:12:01 [INFO] ep_dcp_replica_items_remaining = 188,367
      		 
      ...
      		 
      2018-04-07T22:04:34 [INFO] ep_dcp_replica_items_remaining = 188,367
      		 
      2018-04-07T22:04:36 [INFO] ep_dcp_replica_items_remaining = 188,367
      		 
      2018-04-07T22:04:38 [INFO] ep_dcp_replica_items_remaining = 188,367
      		 
      2018-04-07T22:04:40 [INFO] ep_dcp_replica_items_remaining = 188,367
      		 
      2018-04-07T22:04:42 [INFO] ep_dcp_replica_items_remaining = 188,367
      		 
      2018-04-07T22:04:44 [INFO] ep_dcp_replica_items_remaining = 188,367
      		 
      2018-04-07T22:04:46 [INFO] ep_dcp_replica_items_remaining = 188,367
      		 
      2018-04-07T22:04:48 [INFO] ep_dcp_replica_items_remaining = 188,367

      Attachments

        Issue Links

          duplicates

          Bug - A problem which impairs or prevents the functions of the product. MB-27933 Couchbase doesn't accept writes from Sync Gateway due to bucket OOM

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed
          is triggering

          Bug - A problem which impairs or prevents the functions of the product. MB-30017 High vb_replica_checkpoint_memory when handling slow streams on the Producer during data ingestion

          • Major - Major loss of function.
          • Closed
          relates to

          Bug - A problem which impairs or prevents the functions of the product. MB-29188 cbbackupmgr restore makes almost no progress due to a large replication backlog on one of the nodes

          • Major - Major loss of function.
          • Closed
          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              pavelpaulau Pavel Paulau (Inactive)
              pavelpaulau Pavel Paulau (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty