Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-59890

[CBM] We should handle backup version file not existing on cloud storage

    XMLWordPrintable

Details

    • Bug
    • Resolution: Fixed
    • Critical
    • 7.6.0
    • 7.6.0
    • tools
    • Untriaged
    • 0
    • Unknown

    Description

      What is the problem?
      On Capella testing of 7.6 we were seeing errors about directory version:

      Backup repository creation failed: backup '2023-11-29T07_01_08.857350189Z' is an unsupported version, 
      please use an older version of 'cbbackupmgr'. See the 'Supported Backup Versions' section of the restore documentation for more information
      

      When digging into it this appears to be due to a previously failing run of backup. The backup fails due to a network issue at which point it has uploaded the plan but only written the version to disk.

      When the staging directory is kept around this is fine - we will notice the backup exists and make the user either resume or purge it. If the staging directory is deleted - something that happens every time the backup node goes down in Capella - then we repopulate the staging directory without a version file, causing this error.

      What is the solution?
      A couple of ideas come to mind:

      1. Always upload the version file to cloud storage
      2. Be tolerant to missing version files

      I think #1 is the right way forward here. If the version is missing there could be other reasons (e.g. corruption) that means we should treat the backup as broken.

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              gilad.kalchheim Gilad Kalchheim
              Matt.Hall Matt Hall
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty