Details
-
Improvement
-
Resolution: Fixed
-
Major
-
6.5.0
Description
As seen in investigating MB-36973, there is a limit on how many pthread_key (thread local) objects can be created by a process, defined by PTHREAD_KEYS_MAX. On Linux this seems to normally be 1024, on macOS is seems to be 512.
In the aforementioned MB, we hit this limit when running multi-bucket tests with 32 Buckets; as we were using one pthread_key inside each Shard and the node had 24 shards configured. 32 * 24 = 768 pthread_keys, plus other misc uses made us hit the limit.
To avoid these kinds of problem in future we should attempt to minimise our usage of pthread_key. Per-bucket is probably ok (only 30 buckets supported); but anything which is per shard / or other object which we have many of per bucket may result in hitting the limit again.