Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-37657

Investigate cbbackupmgr backwards merging

    XMLWordPrintable

Details

    • Task
    • Resolution: Duplicate
    • Major
    • None
    • Cheshire-Cat
    • tools
    • None

    Description

      When performing a merge with cbbackupmgr we move sequentially through the backups in the repository in chronological order aggregating the backups into a single merged backup. Doing so in this fashion means that we may be writing mutations for keys multiple times before we complete the merge. We could improve the efficiency of the process by going through the backups in reverse order only adding keys that do not exist in the merged backup to the merged backup (determining if a key has already been merge may well be expensive and might mean this is not an effective way of merging data).

      This way of merging will have several added side effects:
      1) It will mean Rift data stores are automatically compacted (the data will be written sequentially leaving no dead space).
      2) It will be far cheaper (money and time) to merge backups stored in AWS S3 using this method.

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              james.lee James Lee
              james.lee James Lee
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty