Increase the default number of vBuckets which can run flushed concurrently

Summary

The number of vBuckets which can be written to (flushed) concurrently is limited to the minimum of (num_writer_threads, num_shards). We should increase the defaults so larger capacity systems can better utilise the available disk resources.

Details

The out-of-the-box defaults for those config parameters are num_writer_threads:4 and num_shards:4, so only 4 vBuckets can be flushed at once.

Experiments (see MB-36657) suggest that increasing the number of concurrent vBuckets which can be flushed has a significant increase in SyncWrite throughput - as we can better utilise the available disk bandwidth (note X axis is log scale):

To address this, we should change the default values for num_writer_threads and num_shards to scale based on capacity of the environment. The proposed new default values are:

• num_writer_threads - Equal to the number of available CPU cores.
• num_shards - Equals to the closest power-of-2 less than or equal to the number of available CPU cores.