Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-39503

Receiving interlaced vBucket mutations when using sequential backfill



    • Bug
    • Resolution: Not a Bug
    • Major
    • None
    • 6.6.0
    • couchbase-bucket
    • None


      What's the problem?
      When setting the new 'backfill_order' control flag to 'sequential' as expected I'm seeing the majority of mutations in sequential vBucket order, however, during the switch from one vBucket to the next I'm receiving interlaced mutations.

      What am I expecting to see?
      If I was stream two vBuckets, I would expected to see all mutations and the stream end for the first vBucket before receiving any mutations for the next vBucket.

      Steps to reproduce
      1) Use the TLM with the 'mad-hatter' manifest to sync all the required repositories
      2) Change directory to '$SOURCE/godeps/src/github.com/couchbase/gocbcore/v9' and apply this patch to enable the control flag in gocbcore.
      3) Change directory to '$SOURCE/goproj/src/github.com/couchbase/backup' and apply this patch to disable synchronous vBucket streaming and to use the new 'backfill_order' control flag.
      4) Rebuild
      5) Spin up a new Couchbase Server node
      6) To enable reproducing with less data change the number of vBuckets from 64/1024 to 16 with 'curl -X POST -u Administrator:password http://$SERVER_IP/diag/eval -d "ns_config:set(couchbase_num_vbuckets_default, 16)."'.
      7) Load 10,000 1KB items using cbbackupmgr/cbworkloadgen
      8) Configure an archive using 'cbbackupmgr config -a $ARCHIVE -r $REPO'
      9) Perform a backup using a single thread (meaning we should only create a single DCP worker/connection) with 'cbbackupmgr backup -a $ARCHIVE -r $REPO -c $SERVER_IP -u Administrator -p password --threads 1'.

      If a packet capture is taken with 'tcpdump -i any -C 1024 -w $FILENAME -s 0 port 11210' we should see interlaced mutations. I've attached a packet capture which contains interlaced mutations between packets 374-1087 when filtered using 'couchbase.opcode == 0x57' in Wireshark.


        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.



              owend Daniel Owen
              james.lee James Lee
              0 Vote for this issue
              3 Start watching this issue



                Gerrit Reviews

                  There are no open Gerrit changes