Details
-
Bug
-
Resolution: Fixed
-
Major
-
7.1.0, 7.1.1, 7.1.2, 7.1.3
-
Untriaged
-
0
-
Yes
Description
What's the issue?
When merging an empty backup created before 7.1.0 cbbackupmgr will create an empty data range.
{"version":1,"source":{},"sink":{"initial":{},"current":{}}}
|
If merged later on, a panic will occur trying to access the source.failover_log attribute which is empty.
panic: runtime error: index out of range [-1]
|
|
goroutine 1 [running]:
|
github.com/couchbase/backup/archive.(*repoDir).shouldRollback(0x0?, 0x0?, 0x0?)
|
/home/james/Projects/couchbase-build/backup/archive/dir_repo.go:761 +0xf0
|
github.com/couchbase/backup/archive.(*repoDir).updateSourceRanges(_, {0xc00097a6f0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...}, ...)
|
/home/james/Projects/couchbase-build/backup/archive/dir_repo.go:751 +0x7a
|
github.com/couchbase/backup/archive.(*repoDir).getBackupSeqNoRange(_, {_, _}, {_, _}, {_, _}, _)
|
/home/james/Projects/couchbase-build/backup/archive/dir_repo.go:709 +0x4f3
|
github.com/couchbase/backup/archive.(*Source).GetDataRanges(_, {_, _}, {_, _})
|
/home/james/Projects/couchbase-build/backup/archive/source.go:166 +0x8c
|
github.com/couchbase/backup/plan/services/data.(*dataRange).Execute(0xc0000bc5a0, {0x7f44540a73d8, 0xc00054c000}, {0x7f44540a7420, 0xc000152000}, 0xc0009be3b0)
|
/home/james/Projects/couchbase-build/backup/plan/services/data/range.go:45 +0x169
|
github.com/couchbase/backup/plan/services/data.(*Data).Execute(0xc0000138a8, {0x7f44540a73d8, 0xc00054c000}, {0x7f44540a7420, 0xc000152000}, 0xc0009be3b0?)
|
/home/james/Projects/couchbase-build/backup/plan/services/data/data.go:53 +0xca
|
github.com/couchbase/backup/plan/cluster/bucket.(*Bucket).Execute(0xc00054a3f0, {0x7f44540a5968?, 0xc00054c000}, {0x7f44540a5a08?, 0xc000152000}, 0x144aca0?)
|
/home/james/Projects/couchbase-build/backup/plan/cluster/bucket/bucket.go:205 +0xb5a
|
github.com/couchbase/backup/plan/cluster.(*Cluster).Execute.func1(0x0?)
|
/home/james/Projects/couchbase-build/backup/plan/cluster/cluster.go:91 +0x125
|
github.com/couchbase/backup/plan/cluster.(*Cluster).Execute(0xc0000bc550, {0x7f44540a5680?, 0xc00054c000}, {0x7f44540a5730?, 0xc000152000}, 0xc0009be3b0)
|
/home/james/Projects/couchbase-build/backup/plan/cluster/cluster.go:130 +0x431
|
github.com/couchbase/backup/plan.(*Plan).execute(0xc00041b760, {0x1ee7ac8?, 0xc00054c000}, {0x1eeaf08?, 0xc000152000}, {0x1eddb40, 0xc00050f050})
|
/home/james/Projects/couchbase-build/backup/plan/plan.go:126 +0x13a
|
github.com/couchbase/backup/plan.(*Plan).Execute(0x1a96081?, {0x1ee7ac8?, 0xc00054c000?}, {0x1eeaf08?, 0xc000152000?}, {0x1eddb40?, 0xc00050f050?})
|
/home/james/Projects/couchbase-build/backup/plan/plan.go:60 +0x51
|
github.com/couchbase/backup/archive.(*Archive).MergeIncrementalBackups(0xc00041a500, {{0x7fff045b0f75, 0x4}, {0x7fff045b0f82, 0x5}, {0x7fff045b0f8e, 0x3}, 0x3, {0x1eddb40, 0xc00050f050}, ...})
|
/home/james/Projects/couchbase-build/backup/archive/archive.go:1068 +0xde5
|
main.(*MergeContext).Run(0xc000116400)
|
/home/james/Projects/couchbase-build/backup/cmd/cbbackupmgr/merge.go:147 +0xac5
|
github.com/couchbase/cbflag.(*Command).parseFlags(0xc000945f10, 0xc0009bf218, {0xc000000200, 0x8, 0xc000014638?})
|
/home/james/.local/lib/golang/pkg/mod/github.com/couchbase/cbflag@v0.0.0-20220325172535-f0f642f857ed/command.go:283 +0x111e
|
github.com/couchbase/cbflag.(*Command).parse(0xc000945f10, 0xc0009bf218, {0xc000000200?, 0x8, 0x8})
|
/home/james/.local/lib/golang/pkg/mod/github.com/couchbase/cbflag@v0.0.0-20220325172535-f0f642f857ed/command.go:111 +0x12f
|
github.com/couchbase/cbflag.(*Command).parseCommands(0xc00079a0e0, 0xc0009bf218, {0xc0000001f0, 0x9, 0x9})
|
/home/james/.local/lib/golang/pkg/mod/github.com/couchbase/cbflag@v0.0.0-20220325172535-f0f642f857ed/command.go:124 +0x1aa
|
github.com/couchbase/cbflag.(*Command).parse(0xc00079a0e0, 0xc0009bf218, {0xc0000001f0?, 0x9, 0x9})
|
/home/james/.local/lib/golang/pkg/mod/github.com/couchbase/cbflag@v0.0.0-20220325172535-f0f642f857ed/command.go:109 +0x146
|
github.com/couchbase/cbflag.(*CLI).Parse(0xc0003dab00, {0xc0000001e0, 0xa, 0xa})
|
/home/james/.local/lib/golang/pkg/mod/github.com/couchbase/cbflag@v0.0.0-20220325172535-f0f642f857ed/cli.go:70 +0x199
|
main.main()
|
/home/james/Projects/couchbase-build/backup/cmd/cbbackupmgr/main.go:4081 +0xdc1c
|
The reason for cbbackupmgr creating an empty data range is due to backwards compatibility handling (hence the issue initially being seen when moving from 7.0.3 to 7.1.3).
In theory, this issue will affect reading any backup created prior to 7.1.0 using a 7.1.x version of cbbackupmgr.
What's the fix?
We should disregard the failover log stored in range.json and use the one stored in the data directory, which should always be persisted, whether or not any data is transferred.
Is there a workaround?
No, however, this failure case is non-destructive; cbbackupmgr will fail part way through the merge when trying to read the source data range.
- Don't include the affected backups in the merge
- Create a new archive/repository with 7.1.x, future backups/merges should work as expected
Attachments
Issue Links
- relates to
-
MB-49379 [System Test][Backup Service] merge task failed with error "exit status 2" - panic observed in backup service logs
- Closed