Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-39540

cbbackupmgr VBucketBackup reader/writer should only open vBuckets that exist

    XMLWordPrintable

Details

    Description

      What's the problem?
      One of the requirements for backup to cloud is being able to write to n vBuckets on disk at a time. This was an inherent limitation of the original DataBackup which was rewritten and is now known as VBucketBackupReader/Writer. We have multiple functions attached to these reader/writer implementations which open all the vBuckets at once (currently these are only used in metadata mode - which is important because in this case we avoid opening any data files). These OpenAll functions work because we know the top end number of vBuckets that exist in this backup since it's stored in the '.storage_meta' file. This however, doesn't correctly account for when users use vBucket filters or for the info subcommand (which gets all of the snapshots from disk to inform users about the state of an existing backup).

      When we are opening vBuckets using the VBucketBackupReader we are possibly opening vBucket files which don't exist on disk; this is the incorrect behavior we should only open the vBucket files if we know that they exist.

      Why hasn't this been spotted until now?
      Due to the nature of our failover/snapshot files (which are lazy opened + return default values) this has gone unnoticed. Nonetheless this issue should be fixed because there may be unforeseen issues related to using default values when we shouldn't be.

      How should it be fixed?
      The storage metadata file should be extended so that we also store the users chosen VBucketFilter. This should be done in a backwards compatible way so that the current behavior remains the same for 6.0.x and 6.5.x archives.

      We will then conditionally use this vBucketFilter to only open the vBuckets that exist in the backup falling back to the current method when the VBucketFilter is disabled/doesn't exist.

      Attachments

        For Gerrit Dashboard: MB-39540
        # Subject Branch Project Status CR V

        Activity

          People

            owend Daniel Owen
            james.lee James Lee
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty