Details
-
Bug
-
Resolution: Won't Fix
-
Major
-
None
-
7.1.0
-
None
-
Untriaged
-
1
-
No
Description
What's the issue?
Whilst playing around with PiTR, I noticed that CBM was streaming the same snapshot continuously in a loop eventually breaking out after >6K attempts.
Creating DCP Streams |
$ rg --fixed-strings 'Creating DCP stream | {"uuid":250230374546733,"start_seqno":375000,"end_seqno":377760,"snap_start":0,"snap_end":375000,"retries":0}' ~/Projects/couchbase-archive/logs/backup-0.log -c
|
6717
|
Explanation
When streaming PiTR snapshots the simplified process is as follows:
- Get some stats from the server, including the high seqno for the vbucket
- Ask the server for a stream from disk with a start seqno of the last seen seqno and end seqno of the high seqno of the vbucket
- Persist the resulting snapshot
- If the snapshot ends at or beyond the high seqno then stop, otherwise goto 2.
Consider what happens if disk persistence is a little bit behind in memory. In that case we might ask for a start-end seqno range that hasn't been persisted to disk yet. In that case the server will just not give us a snapshot and, as there is no new information, the check at 4 will goto 2 and we'll retry until some more data is flushed.
Solution
We believe the best way to solve this may be with MB-46854 (stream all snapshot). Then once the server has read everything it has on disk it can close the stream and we'll terminate. Alternatively we may be able to work around this in cbbackupmgr
Attachments
Issue Links
- relates to
-
MB-46854 PiTR: Stream all snapshots
- Closed