Investigate better utilisation of ActiveStreamProcessorTask

History.

back in approx Couchbase 3.x the worker thread associated with a DCP producer would do the following for in-memory streaming.

• call into ep-engine "dcp step"
• find a stream
• call into checkpoint-manager and copy queued_item (shared-pointers) into the readyQueue to form snapshots
• transfer readyQueue to network

The call into checkpoint-manager was problematic as the amount of data to copy could be large, and that caused issues by tieing up the worker thread (also maybe contention with/waiting for checkpoint-manager lock). At that time the solution was to add a NonIO task that was responsible for populating the readyQueue - ActiveStreamCheckpointProcessor.

With that in-place, the worker threads now just needed to move the readyQueue data out to the network.

However re-thinking this, there is no consideration for small checkpoints and it seems there's a lot of moving parts/complexity/context-switching just to form small snapshots.

This ticket exist to investigate if we can be a little smarter and only use the ActiveStreamCheckpointProcessor in the case that there could be a lot of data - and test/measure doing say 5 items or less inline in the worker thread and 5 or more in the task.

This could be beneficial in multi-bucket scenarios keeping the NonIO wakeups and context switching "down" .