Details
-
Improvement
-
Resolution: Unresolved
-
Major
-
Cheshire-Cat
-
None
Description
See MB-36379. We have an MVP solution in which we set file descriptor limits at bucket initialization to some limit that will never increase/decrease based on the remaining file descriptors available. One issue here is that each backen takes an equal proportion of the file descriptors which is non-optimal and means that we pay a cost for the existence of couchstore/magma Buckets even if we don't use them.
Would like to improve this so that the storage backends can take the remaining file descriptors after the front end takes the amount required for configured connections. We could then utilize all file descriptors if we have only a single storage backend present.
This requires a bunch of thought in regards to what we do if we have to decrease file descriptor limits. We may have to things such as wait for flushes to complete, cancel/wait for backfills to complete (this could take hours-days for massive use cases).
Attachments
Issue Links
- relates to
-
MB-42015 CouchKVStoreFileCache limit on cluster_run likely to prevent full bucket of DCP streams
- Closed
-
MB-36572 Keep couchstore DB instances open across writes
- Reopened
-
MB-36379 Provide an API to set storage engine file descriptor limit
- Closed
-
MB-37783 Increase default file descriptor limit for couchbase-server service
- Closed