ep_num_locks in fixed, doesn't expand upon resize

The number of locks used for each HashTable (ep_num_locks) it set to a fixed value (originally 5, 47 since 3.1.2 / 4.0.0) during initialisation:

$ /opt/couchbase/bin/cbstats localhost:11210 all|grep " curr_items:|ep_ht_locks:"
ep_ht_locks: 5

Furthermore, it doesn't increase as part of the HashTable resize. This means that as bucket size and/or node count increases, there will be increased lock contention.

For example, a 42 node cluster with 7B items will have ~24 vBuckets per node, and 6.8M docs per vBucket. Hence each lock will be owning 6.8M/5 == 1.3M items.

We should consider increasing the default value of ep_ht_locks and/or dynamically expanding the lock count along with the HashBucket count when a HashTable is resized.