Details
-
Improvement
-
Resolution: Duplicate
-
Major
-
2.0, 2.0.1
-
Security Level: Public
Description
Example idea...
1) During a full backup (TAP dump), the cbbackup tool can be enhanced to also track (per-vbucket) the largest seq-num's that it receives on the TAP stream. Periodically (and at the end), cbbackup records those max seq-num's (and possibly the associated CAS's) into its backup files.
2) Later, when the user starts an incremental backup, cbbackup sends its max saved seq-num's at TAP connect, saying "please send me only the changes that came after these per-vbucket seq-num's".
3) The server then sends the changes that came after those seq-num's. If seq-num's are not known by the server, the server should return errors during the TAP connect. In this case, for simplicity, perhaps the client has to instead do a full, non-incremental TAP-dump backup (e.g., step 1).
The protocol enhancements needed for incremental backup would likely also be useful for other needs like search-engine integration, Hadoop integration, OLAP integration, and other integrations where shipping "deltas" of data is required.