Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-37673

cbbackupmgr '--purge' previous failed backup in cloud

    XMLWordPrintable

Details

    • Improvement
    • Resolution: Done
    • Major
    • 6.6.0
    • 6.6.0
    • tools

    Description

      cbbackupmgr's '--purge' argument should retain the same behavior as it does locally when backing up to the cloud.

      At the moment, Rift data stores will be uploaded during the backup, however, if the backup never completes (and the user deletes their local staging directory) there is no meta data in the cloud. This means that the data stores would become orphaned (we don't know that they exist). We should have a mechanism to ensure we can cleanly purge previous failed cloud backups.

      Possible solution:
      Before we start a backup, we should push the 'plan.json' to the cloud (in the correct directory), when we try to perform another backup, it will be cached and detected as a failed backup. We can then ensure we remove any complete/incomplete multipart uploads.

      Along with uploading the plan we would have to upload all of the upload manifest (once and only once; at the time the multipart upload is started for that vBucket). This is because we need to have access to the mulipart upload id.

      This solution may not be a simple as first though, we will have to do this in such a manor that either:
      1) The user is forced to purge that previous backup (it can't be resumed since we don't have enough metadata storage locally)
      2) We automatically detect that this situation and abort the multipart uploads (this is slightly more difficult since we would need to ensure that we have the credentials need to abort them).

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              owend Daniel Owen
              james.lee James Lee
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty