Details
-
Improvement
-
Resolution: Done
-
Major
-
None
-
4.5.0
Description
During investigation of MB-16750, it was observed that couchstore as used by ep-engine uses much smaller B-Tree node sizes (1279 bytes) compared to map-reduce.
Having spoken to Volker about the difference, it appears to originate from a patch by Damian in 2008 from the original Erlang CouchDB implementation: https://github.com/apache/couchdb/commit/96aaee3666626459ec091ee7e1a65e32b3b9afa6
This has a relatively opaque commit message:
Adjusted btree chunk size to give a 25% performance increase in random testing.
Subsequent changes in the view engine (https://github.com/couchbase/couchdb/commit/c62a4b242d40bd6270a17c5f7e6e5f3fd61d3837 amongst others) have increased the size significantly, it is currently at 6144 for key-pointer nodes and 7168 for key-value nodes.
Since the current value (1279) was selected many things have changed:
a) Snappy compression was introduced (so actual on-disk size is typically even smaller)
b) The implementation has changed from Erlang to C (so computational vs. disk read tradeoff are likely very different)
c) The predominant performance metrics are now disk write (persistTo) and disk read (BG fetch in DGM situations), with a managed cache in front (compared to CouchDB).
I believe it makes sense to re-evaluate the current selected value.
See also: MB-19995.
Attachments
Issue Links
- relates to
-
MB-19995 couchstore: Investigate effect of 8K buffers for iobuffer file ops
- Closed