Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-20003

couchstore: Investigate effect of B-Tree node size

    XMLWordPrintable

Details

    • Improvement
    • Resolution: Done
    • Major
    • None
    • 4.5.0
    • couchbase-bucket

    Description

      During investigation of MB-16750, it was observed that couchstore as used by ep-engine uses much smaller B-Tree node sizes (1279 bytes) compared to map-reduce.

      Having spoken to Volker about the difference, it appears to originate from a patch by Damian in 2008 from the original Erlang CouchDB implementation: https://github.com/apache/couchdb/commit/96aaee3666626459ec091ee7e1a65e32b3b9afa6

      This has a relatively opaque commit message:

      Adjusted btree chunk size to give a 25% performance increase in random testing.

      Subsequent changes in the view engine (https://github.com/couchbase/couchdb/commit/c62a4b242d40bd6270a17c5f7e6e5f3fd61d3837 amongst others) have increased the size significantly, it is currently at 6144 for key-pointer nodes and 7168 for key-value nodes.

      Since the current value (1279) was selected many things have changed:

      a) Snappy compression was introduced (so actual on-disk size is typically even smaller)
      b) The implementation has changed from Erlang to C (so computational vs. disk read tradeoff are likely very different)
      c) The predominant performance metrics are now disk write (persistTo) and disk read (BG fetch in DGM situations), with a managed cache in front (compared to CouchDB).

      I believe it makes sense to re-evaluate the current selected value.

      See also: MB-19995.

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              jung-sang Jung-Sang Ahn (Inactive)
              drigby Dave Rigby (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty