Automated File-Based Recovery Tool

We need a way to automated way of being able to restore a bucket based on the method of copying a set of couchstore files to the correct location.

This is necessary for 2 reasons:

(1) The "traditional" methods of using cbbackup / cbrestore / cbtransfer can take a reasonable amount of time to restore when there are billions of documents in play.
(2) This technique is occasionally relied upon by the support team when a cluster is in a significant state of disrepair and there is an urgent need to get the cluster online using a filesystem backup.

The most user-friendly we could make this would be if the user provided a target cluster name (ii) a bucket name and (iii) a directory containing a number of couchstore files and the script brought up that bucket on the cluster using the specified files.

At the highest level the script should be concerned with:

• Identifying the vbucket topology for the bucket
• Establishing which vbucket files (both active and replica) are available
• Copying the files into the correct location
• Ensuring the bucket is appropriately warmed-up to load the data in these files into cache.

And some questions for consideration:

• Do we need to provide a solution that helps take a copy of the files? (i.e. backup as well as restore)
• Does this need to concern both active and replica couchstore files? (active only == quicker file copy but requires reinstatement of replicas)
• Can this be accomplished without bringing down the entire target cluster?
• Should this operate on a single node basis (i.e. run once on each node)
• Should this run on a single bucket basis or be capable of handling multiple buckets?