Details
-
Improvement
-
Resolution: Fixed
-
Major
-
feature-backlog
-
Security Level: Public
-
None
-
ForestDB: Oct 26 - Nov 13
Description
In some use cases, applications need to create lots of KV instances and read/write items from/to those instances. If we create a separate database file for each KV instance, there will be lots of files to be opened, which causes too many file descriptors to be used, or we should open / close each file repeatedly, which incurs lots of additional overhead (e.g.,open / close system call, reading a database header, reading btree nodes again, etc.).
To address this issue, we need to support multiple KV instances in a single database file. This will allow us to manage our own buffer cache more efficiently and increase a write batch size significantly, which improve the write throughput a lot.
In addition, this will help us improve the ep-engine flusher write throughput when we switch from Couchstore to ForestDB, because we can put mulitple VBuckets in a single ForestDB file (i.e., a single shard file).