Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-31515

Cbbackup accumulative backup mode performs differential backup since last accumulative

    XMLWordPrintable

Details

    • Bug
    • Resolution: Fixed
    • Critical
    • 6.0.3, 6.5.0
    • 3.1.0, 3.1.1, 3.1.2, 3.1.3, 3.1.4, 3.1.5, 3.1.6, 4.0.0, 4.1.0, 4.1.1, 4.1.2, 4.5.0, 4.5.1, 4.6.0, 4.6.1, 4.6.4, 4.6.2, 4.6.3, 4.6.5, 5.0.0, 5.0.1, 5.1.0, 5.1.1, 5.1.2, 5.5.0, 5.5.1
    • tools
    • None
    • Untriaged
    • No

    Description

      It seems that cbbackup when running accumulative backup mode only backs up the difference since the last accumulative backup.

      Based on the design for accumulative backup (https://github.com/couchbase/couchbase-cli/blob/master/docs/design/cbbackup-incr-func-spec.md#use-case-for-cumulative-incremental-backups), all changes since the last full backup should be backed up in the new accumulative backup.
      However, based on empirical evidence and basic code review, this does not seem to be the case.

      This fundamentally breaks the model of performing accumulative backups as users expect to be able to delete older accumulative backups once they have a new one, since that should contain all of the old changes as well as the new changes.
      While restoring these incremental backups does seem to work okay (all docs get restored to the bucket), if you do delete intermediate accumulative backups then you will be missing documents.

      Reproduction
      This was verified on Couchbase Server 3.0.1 and Couchbase Server 4.6.5. Couchbase Server 5.x was attempted but there is another issue related to this preventing accumulative backups from working properly in this context (see MB-31517).

      1. Create the beer-sample bucket
      2. Create an 'accumulative' backup of the beer-sample bucket, this should just be a full backup as the archive does not exist yet

        $ /opt/couchbase/bin/cbbackup http://localhost:8091 /backup -m accu -u Administrator -p password
          [####################] 100.0% (7303/estimated 7303 msgs)
        

      3. Verify the number of items in the backup is 7303 (total docs in the sample bucket)

        $ sqlite3 /backup/2018-10-04T145813Z/2018-10-04T145813Z-full/bucket-beer-sample/node-127.0.0.1%3A8091/data-0000.cbb 'SELECT count(key) FROM cbb_msg WHERE length(val) != 0'
          7303
        

      4. Create a new document, test1 in the Couchbase Server UI (or however you'd like). There are now 7304 items in the bucket.
      5. Take an accumulative backup, re-using the same archive

        $ /opt/couchbase/bin/cbbackup http://localhost:8091 /backup -m accu -u Administrator -p password
          [                    ] 0.0% (1/estimated 7304 msgs)
        

      6. Verify that this contains a single document as expected

        $ sqlite3 /backup/2018-10-04T145813Z/2018-10-04T150110Z-accu/bucket-beer-sample/node-127.0.0.1%3A8091/data-0000.cbb 'SELECT count(key) FROM cbb_msg WHERE length(val) != 0'
        1
        

      7. Create a new document in the same bucket, test2
      8. Perform another accumulative backup

        $ /opt/couchbase/bin/cbbackup http://localhost:8091 /backup -m accu -u Administrator -p password
          [                    ] 0.0% (1/estimated 7305 msgs)
        

      9. Verify that this contains 2 documents as expected (since there's 2 new docs since the last full backup)

        $ sqlite3 /backup/2018-10-04T145813Z/2018-10-04T150649Z-accu/bucket-beer-sample/node-127.0.0.1%3A8091/data-0000.cbb 'SELECT count(key) FROM cbb_msg WHERE length(val) != 0'
        1
        

      As you can see, the last accumulative backup only contains the 1 new doc since the last accumulative backup, rather than the 2 new docs since the full backup.

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              carlos.gonzalez Carlos Gonzalez Betancort (Inactive)
              matt.carabine Matt Carabine (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty