Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-40415

"_local/vbstate" may get out of sync if a crash happens right after compaction

    XMLWordPrintable

Details

    • Bug
    • Resolution: Fixed
    • Major
    • 6.6.1
    • 6.5.1, 6.6.0, 6.5.0
    • couchbase-bucket
    • None
    • Untriaged
    • 1
    • Unknown

    Description

      Compaction happens in multiple steps, and if we crash before all steps are done we might end up with an inconsistent configuration.

      Consider the following scenario:

      1. Compaction runs and rewrites the new database into a new file with a .compact suffix. If we crash at this point nothing bad wil happen as we'll delete this file during startup.
      2. We rename the file and remove the .compact suffix. If we crash after this point we'll pick up this file after a restart
      3. We update the _local/vbstate document (and collection documents) with the result from compaction.

      If the system crash before the commit happens as part of 3 we'll get an inconsistent _local/vbstate and collection documents.

      We could flip the order of 2 and 3 to work around this, but that also introduce another problem (if there is a problem reading the database header off disk we'll still roll back to the header before as part of opening the database). A better approach would probably be to allow for callbacks to happpen before the final commit happens to allow additional updates.

      (it looks like the data currently being updated in _local/vbstate is for optimizations (onDiskPrepares) and information we don't really need to store in a local document as they reflect information in the database header (updateSeqNum and purgeSeqNum which is currently atomically updated as part of writing the header). If we could drop keeping those values in _local/vbstate (and read the two from the header and work around the onDIskPrepares) we wouldn't need updating the document. We'd have to look into the modifications we do for the collection documents.

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              trond Trond Norbye
              trond Trond Norbye
              Votes:
              1 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty