Critical Description
I have a Mac running couchbase server enterprise 4.5.1. It collects data in Couchbase and runs cbbackmgr each night:
/Users/eben/Downloads/couchbase-server-enterprise_4.5.1-2845/Couchbase\ Server.app/Contents/Resources/couchbase-core/bin/cbbackupmgr backup --archive ~/backup-archive/ --repo cluster --host couchbase://localhost --username Administrator --password research
|
On October 21, I tried upgrading from CE (which I'd installed by accident) to EE. I just quit one version and launched the other. The server came up, but the buckets never became available (they appeared but were in the 'yellow' state in the UI). When cbbackupmgr ran that night, it failed with the error:
Warning: Progess bar disabled because terminal width is less than 80 characters
|
Error backing up cluster: The operation has timed out.
|
The log entry in the backup was as follows:
2016-10-21T01:20:01.221-07:00 (Cmd) backup --archive /Users/eben/backup-archive/ --repo cluster --host couchbase://localhost --username Administrator --password research
|
2016-10-21T01:20:01.230-07:00 (Rest) GET http://localhost:8091/pools 200
|
2016-10-21T01:20:01.242-07:00 (Rest) GET http://localhost:8091/pools/default/buckets 200
|
2016-10-21T01:20:01.251-07:00 (Plan) Executing transfer plan
|
2016-10-21T01:20:01.252-07:00 (Plan) Transfering bucket configuration for ForumSubscribers
|
2016-10-21T01:20:01.267-07:00 (Rest) GET http://localhost:8091/pools/default/buckets 200
|
[FDB INFO] Forestdb blockcache size 2147483648 initialized in 72495 us
|
|
[FDB INFO] Forestdb opened database file /Users/eben/backup-archive/cluster/2016-10-21T01_20_01.24834218-07_00/ForumSubscribers-2000c956b7aa76baab3556e1a5d6fab2/data/shard_0.fdb
|
[FDB INFO] Forestdb closed database file /Users/eben/backup-archive/cluster/2016-10-21T01_20_01.24834218-07_00/ForumSubscribers-2000c956b7aa76baab3556e1a5d6fab2/data/shard_0.fdb
|
2016-10-21T01:20:01.355-07:00 (Plan) Transfering views definitions for ForumSubscribers
|
2016-10-21T01:20:01.360-07:00 (Rest) GET http://localhost:8091/pools/default/nodeServices 200
|
2016-10-21T01:20:01.365-07:00 (Rest) GET http://localhost:8091/pools/default/buckets/ForumSubscribers/ddocs 200
|
2016-10-21T01:20:01.367-07:00 (Plan) Transfering GSI index definitions for ForumSubscribers
|
2016-10-21T01:20:01.367-07:00 (Rest) GET http://localhost:8091/pools/default/nodeServices 200
|
2016-10-21T01:20:01.392-07:00 (Rest) GET http://localhost:9102/getIndexMetadata?bucket=ForumSubscribers 200
|
2016-10-21T01:20:01.393-07:00 (Plan) Executing queries against ForumSubscribers
|
2016-10-21T01:20:01.394-07:00 (Plan) Transfering full text index definitions for ForumSubscribers
|
2016-10-21T01:20:01.395-07:00 (Rest) GET http://localhost:8091/pools/default/nodeServices 200
|
2016-10-21T01:20:01.396-07:00 (Plan) Deciding which key value data to transfer for ForumSubscribers
|
2016-10-21T01:20:01.407-07:00 (Rest) GET http://localhost:8091/pools/default/buckets 200
|
2016-10-21T01:20:26.434-07:00 ^[[31mERROR: The operation has timed out.^[[0m^[[2m -- plan.(*dataRange).execute() at range.go:49^[[0m
|
2016-10-21T01:20:26.434-07:00 (Plan) Transfer plan failed due to error The operation has timed out.
|
2016-10-21T01:20:26.434-07:00 (Cmd) Error backing up cluster: The operation has timed out.
|
Subsequent to that, any time I try to use that archive to do an additional backup or restore, I get errors like:
Restoring backup 2016-10-21T01_20_01.24834218-07_00
|
Copying at 0B (estimating time remaining) 0 items / 0B
|
Error restoring cluster: Backup Meta `/Users/eben/backup-archive/cluster/2016-10-21T01_20_01.24834218-07_00/ForumSubscribers-2000c956b7aa76baab3556e1a5d6fab2/range.json` not found
|
with log errors like:
[FDB INFO] Forestdb opened database file /Users/eben/backup-archive/cluster/2016-10-21T01_20_01.24834218-07_00/ForumSubscribers-2000c956b7aa76baab3556e1a5d6fab2/data/shard_0.fdb
|
[FDB INFO] Forestdb closed database file /Users/eben/backup-archive/cluster/2016-10-21T01_20_01.24834218-07_00/ForumSubscribers-2000c956b7aa76baab3556e1a5d6fab2/data/shard_0.fdb
|
2016-10-21T12:41:30.521-07:00 ^[[31mERROR: Backup Meta `/Users/eben/backup-archive/cluster/2016-10-21T01_20_01.24834218-07_00/ForumSubscribers-2000c956b7aa76baab3556e1a5d6fab2/range.json` not found^[[0m^[[2m -- plan.(*dataRange).execute() at range.go:49^[[0m
|
2016-10-21T12:41:30.521-07:00 (Plan) Transfer plan failed due to error Backup Meta `/Users/eben/backup-archive/cluster/2016-10-21T01_20_01.24834218-07_00/ForumSubscribers-2000c956b7aa76baab3556e1a5d6fab2/range.json` not found
|
2016-10-21T12:41:30.521-07:00 (Cmd) Error restoring cluster: Backup Meta `/Users/eben/backup-archive/cluster/2016-10-21T01_20_01.24834218-07_00/ForumSubscribers-2000c956b7aa76baab3556e1a5d6fab2/range.json` not found
|
The full backup.log file is attached.