Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-39529

Modifiy the DCP client so that we can utilise sequential backfilling



    • Improvement
    • Resolution: Done
    • Major
    • 6.6.0
    • 6.6.0
    • tools


      A new control flag was added into KV Engine called backfill_order which allows you to request either round-robin or sequential backfilling. This control flag was added to facilitate native cloud backup so support for using the new flag should be added to our DCP client. As explained in the tools cloud design document this flag was requested to allow cbbackupmgr to stream 'n' vBuckets at a time instead of having to stream all 64/1024 at once. This greatly reduces the resource requirements for cbbackupmgr backup to cloud and without this change cbbackupmgr backup to cloud will not be able to create "consistent" backups.

      What needs to be changed?
      1) We need to add support to gocbcore for specifying this control flag.
      2) We need to update the cbbackupmgr DCP client to utilize this setting in gocbcore when backing up clusters with a version greater than 6.6.0.

      Are there going to be any issues?
      In theory this should be be a relatively minor change in both cbbackupmgr and gocbcore, however, there are a few things to think about:
      1) We want to avoid opening all the vBucket files for the duration of the backup
      2) By design we persist the failover logs immediately after receiving them, this causes a slight issue because we have to open the vBucket files to write the failover log. Basically this behavior needs to be decoupled.


        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.



              james.lee James Lee
              james.lee James Lee
              0 Vote for this issue
              1 Start watching this issue



                Gerrit Reviews

                  There are no open Gerrit changes