Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-3360

mbrestore tool does not support restoring from WAL'ed database files

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Won't Fix
    • Affects Version/s: 1.7 alpha 2, 1.8.0
    • Fix Version/s: 1.7 alpha 2
    • Component/s: tools
    • Security Level: Public
    • Labels:
      None

      Description

      This error is presented:
      tcarpenter@sports-mb-test02:/opt/membase/1.6.4.1/data/ns_1]$ /tmp/python271/Python-2.7.1/python /opt/membase/1.6.4.1/bin/ep_engine/management/mbrestore -a default default-0.mb default-1.mb default-2.mb default-3.mb
      Traceback (most recent call last):
      File "/opt/membase/1.6.4.1/bin/ep_engine/management/mbrestore", line 147, in <module>
      main()
      File "/opt/membase/1.6.4.1/bin/ep_engine/management/mbrestore", line 105, in main
      db.executemany("attach ? as ?", zip(db_filenames, attached_dbs))
      sqlite3.OperationalError: file is encrypted or is not a database

      According to Dustin, this is because the mbrestore tool does not currently support WAL db files and the workaround is to run: /opt/membase/bin/ep_engine/management/sqlite3 <filename> 'pragma journal_mode=delete' on all the db files you are trying to restore.

      Please add the above workaround to the mbrestore script itself.

      No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

        Hide
        steve Steve Yen added a comment -

        Related to Perry's April 13, 2012 comment, another workaround to handle the "backup files need to be upgrade to version 2" issue...

        IF the user can determine that the files definitely came from the right version (e.g., backup of 1.8.1 and trying to restore to 1.8.1), then perhaps the user_version pragma information did not transfer correctly.

        For example, use sqlite3 to run...

        pragma user_version;

        The result should be '2'. If not then a workaround fix would be, in sqlite3 to use the following across all the sqlite files...

        pragma user_version=2;

        Show
        steve Steve Yen added a comment - Related to Perry's April 13, 2012 comment, another workaround to handle the "backup files need to be upgrade to version 2" issue... IF the user can determine that the files definitely came from the right version (e.g., backup of 1.8.1 and trying to restore to 1.8.1), then perhaps the user_version pragma information did not transfer correctly. For example, use sqlite3 to run... pragma user_version; The result should be '2'. If not then a workaround fix would be, in sqlite3 to use the following across all the sqlite files... pragma user_version=2;
        Hide
        steve Steve Yen added a comment -

        Additionally, as part of the workaround, the user should also run "pragma wal_checkpoint(FULL)" to incorporate any WAL files before changing the journal_mode.

        So...

        pragma wal_checkpoint(FULL);
        pragma journal_mode=delete;

        See...

        http://www.sqlite.org/pragma.html#pragma_wal_checkpoint

        One last workaround, possibly already mentioned, is to run cbrestore on a box with updated python and python sqlite3 binding.

        Show
        steve Steve Yen added a comment - Additionally, as part of the workaround, the user should also run "pragma wal_checkpoint(FULL)" to incorporate any WAL files before changing the journal_mode. So... pragma wal_checkpoint(FULL); pragma journal_mode=delete; See... http://www.sqlite.org/pragma.html#pragma_wal_checkpoint One last workaround, possibly already mentioned, is to run cbrestore on a box with updated python and python sqlite3 binding.
        Hide
        peter peter added a comment -

        Steve, since the workaround did the trick, can this issue be closed?

        Show
        peter peter added a comment - Steve, since the workaround did the trick, can this issue be closed?
        Hide
        perry Perry Krug added a comment -

        This has been reported on 1.8:
        "When trying to restore from snapshot with cbrestore, it fails with:

        "Either the metadata db file is not specified
        or the backup files need to be upgraded to version 2.
        Please use cbdbupgrade for upgrade or contact support."

        But using cbbackup on snapshot data and restoring afterwards works."

        Show
        perry Perry Krug added a comment - This has been reported on 1.8: "When trying to restore from snapshot with cbrestore, it fails with: "Either the metadata db file is not specified or the backup files need to be upgraded to version 2. Please use cbdbupgrade for upgrade or contact support." But using cbbackup on snapshot data and restoring afterwards works."
        Hide
        perry Perry Krug added a comment -

        There are a few strategies for converting the data. They're not directly compatible, but you can either upgrade the database (http://www.couchbase.org/wiki/display/membase/Membase+Server+1.7) or use our restore tool to pull the data out and replay it into a new cluster (http://www.couchbase.org/wiki/display/membase/Backup+and+Restore+with+Membase). The latter step is where this bug came from, and if you use the latest mbrestore script (provided with the 1.7 installation) you won't need this workaround.

        Show
        perry Perry Krug added a comment - There are a few strategies for converting the data. They're not directly compatible, but you can either upgrade the database ( http://www.couchbase.org/wiki/display/membase/Membase+Server+1.7 ) or use our restore tool to pull the data out and replay it into a new cluster ( http://www.couchbase.org/wiki/display/membase/Backup+and+Restore+with+Membase ). The latter step is where this bug came from, and if you use the latest mbrestore script (provided with the 1.7 installation) you won't need this workaround.

          People

          • Assignee:
            steve Steve Yen
            Reporter:
            perry Perry Krug
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Gerrit Reviews

              There are no open Gerrit changes