Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-39529

Modifiy the DCP client so that we can utilise sequential backfilling

    XMLWordPrintable

Details

    • Improvement
    • Resolution: Done
    • Major
    • 6.6.0
    • 6.6.0
    • tools

    Description

      A new control flag was added into KV Engine called backfill_order which allows you to request either round-robin or sequential backfilling. This control flag was added to facilitate native cloud backup so support for using the new flag should be added to our DCP client. As explained in the tools cloud design document this flag was requested to allow cbbackupmgr to stream 'n' vBuckets at a time instead of having to stream all 64/1024 at once. This greatly reduces the resource requirements for cbbackupmgr backup to cloud and without this change cbbackupmgr backup to cloud will not be able to create "consistent" backups.

      What needs to be changed?
      1) We need to add support to gocbcore for specifying this control flag.
      2) We need to update the cbbackupmgr DCP client to utilize this setting in gocbcore when backing up clusters with a version greater than 6.6.0.

      Are there going to be any issues?
      In theory this should be be a relatively minor change in both cbbackupmgr and gocbcore, however, there are a few things to think about:
      1) We want to avoid opening all the vBucket files for the duration of the backup
      2) By design we persist the failover logs immediately after receiving them, this causes a slight issue because we have to open the vBucket files to write the failover log. Basically this behavior needs to be decoupled.

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              james.lee James Lee
              james.lee James Lee
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty