Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-48919

eq_dcpq:xdcr:dcp_backfill_b67b789bcfd95e6b97c8af1e8fa5c7cb: zp3EZ30z9hs-XalilQLKUQ== - (vb:271) Stream request failed because the snap start seqno (0) <= start seqno (31608) <= snap end seqno (0) is required

    XMLWordPrintable

Details

    • Bug
    • Resolution: Fixed
    • Critical
    • 7.1.0
    • 7.1.0
    • XDCR
    • 7.1.0-1480-enterprise

    Description

      1. Step 1: Create a 9 node cluster
      2. Step 1.1: Create a 5 node XDCR remote cluster
      3. Step 2: Create required buckets and 20 collections.
      4. Step 2.1: Create required buckets and 10 collections on XDCR remote.
      5. Step 3: Create 1000000 items sequentially
      6. Step 4: Update 1000000 RandonKey keys to create 50 percent fragmentation
      7. Step 5: Create 1000000 items sequentially
      8. Step 6: Update 1000000 RandonKey keys to create 50 percent fragmentation
      9. Step 7: Create full bucket to bucket replications for 10 collections
      10. Step 8: Start an async load: CRUD all 25% at a rate of total 20k ops.
      11. Step 9: Rebalance in with Loading of docs. Kill and resume rebalance at 20%, 40%
      12. Found the mentioned WARNING in memcahced logs

        172.23.121.126

        		 
        Found 'Stream request failed because the snap start seqno' logs - ['2021-10-14T04:23:20.860344-07:00 WARNING 6760: (GleamBookUsers0) DCP (Producer) eq_dcpq:xdcr:dcp_backfill_b67b789bcfd95e6b97c8af1e8fa5c7cb/GleamBookUsers0/GleamBookUsers0_172.23.121.126:11210_0:zp3EZ30z9hs-XalilQLKUQ== - (vb:271) Stream request failed because the snap start seqno (0) <= start seqno (31608) <= snap end seqno (0) is required\n']

      Another observations is that on an idle state cluster the RAM usage is quite high:

      On checking one of the node on Source cluster both goxdcr and memcahced seems to be using pretty high RAM:

      NOTE: After the test finished/failed at the last step, total Items in source cluster are 37,326,445 while items in destination is 20,507,898. While there are constants ops seen on the destinations cluster the item counts is not changing in the destination cluster.

      Attachments

        Issue Links

          is caused by

          Bug - A problem which impairs or prevents the functions of the product. MB-48938 XDCR - Backfill Pipeline non-Zero Starting Timestamp is set with snapStart+snapEnd of 0

          • Major - Major loss of function.
          • Closed
          relates to

          Bug - A problem which impairs or prevents the functions of the product. MB-48936 XDCR - Checkpoint writes obsolete checkpoints back after backfill tasks have been cleaned up

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MB-48935 XDCR - BackfillPipeline resuming from checkpoint always gets the latest checkpoint

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MB-48937 XDCR - P2P - Merging peer node can introduce empty checkpoint

          • Major - Major loss of function.
          • Closed
          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              ritesh.agarwal Ritesh Agarwal
              ritesh.agarwal Ritesh Agarwal
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty