Provide more fault-tolerant batch mutations

Thanks for the suggestions , I've added some documentation to the patch that details this, along with some suggestions on what tunables the app can use.

Hello Graham,

Thanks for your help in this, I think that is a good starting point for the users to extends and adapt the loader to their requirements.

As we discussed via Slack, it will be great to document the relation between executors and maxconcurrent operations. This will help our users to forecast the write load and fix the loader to their cluster sizing.

Thanks again

I haven't gone as far as you've suggested, for this initial patch.  Here I've just added application control over the size of the batching, but it's a decent start and I think will actually cover most of the cases where users don't need to push the envelope.

For those users that truly want to maximise performance, more advanced flow control will be required.  But at that point I think apps may be better off doing the saveToCouchbase logic themselves - if you look at the saveToCouchbase code, you'll see it's very simple.  Because it's very tricky to write a truly generic doc loader like this, e.g. the app could also want its own logging, profiling and so on in there.  So my feeling is that saveToCouchbase should try to hit 80% of the cases, and those apps that want to push things further will want to do it themselves. 

What are your thoughts?

Example of loader:

https://github.com/rfmvlc/spark-loader

We are able to control the timeouts and retry delay, something that would be great is to have the ability of:

• Throttling

• Fallback mechanism

• # of retries configuration