Details
-
Improvement
-
Resolution: Won't Do
-
Major
-
None
-
2.5.1, 2.5.0, 3.0
-
Security Level: Public
-
None
-
Debian 7
Description
The number of locks used for each HashTable (ep_num_locks) it set to a fixed value (originally 5, 47 since 3.1.2 / 4.0.0) during initialisation:
$ /opt/couchbase/bin/cbstats localhost:11210 all|grep " curr_items:|ep_ht_locks:"
ep_ht_locks: 5
Furthermore, it doesn't increase as part of the HashTable resize. This means that as bucket size and/or node count increases, there will be increased lock contention.
For example, a 42 node cluster with 7B items will have ~24 vBuckets per node, and 6.8M docs per vBucket. Hence each lock will be owning 6.8M/5 == 1.3M items.
We should consider increasing the default value of ep_ht_locks and/or dynamically expanding the lock count along with the HashBucket count when a HashTable is resized.