Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-25662

Investigate effect of hinting to OS that couchstore file access is random

    XMLWordPrintable

Details

    Description

      Couchstore files are essentially accessed in random order - the B-Tree is distributed throughout the file; and documents are scatted through it.

      In DGM scenarios where the fsCache is a precious resource, it may be worthwhile to tell the OS to not prefetch blocks after the accessed range.

      I suspect this it most likely to help during compaction (when reading the old file), as we want to minimise the amount of fscache we "steal" from other, still in existence files. It may also be advantageous for point-lookup accesses; as we don't want to pollute the fscache for adjacent blocks we never use.

      Having said all this, the OS (Linux) may be smart enough to detect that we don't use much of the pre-fetched blocks, and hence won't keep for long in the active set of the fscache. As such, I think we need to perform some experiments to test if hinting is beneficial or not.

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          drigby Dave Rigby added a comment -

          I've uploaded a patch which enables fadvise(RANDOM) every time a couchstore file is opened (i.e. irrespective of the type of access): http://review.couchbase.org/#/c/82209/

          drigby Dave Rigby added a comment - I've uploaded a patch which enables fadvise(RANDOM) every time a couchstore file is opened (i.e. irrespective of the type of access): http://review.couchbase.org/#/c/82209/
          wayne Wayne Siu added a comment -

          Dave RigbyTai Tran
          Do we still plan to include this ticket in 4.6.4?

          wayne Wayne Siu added a comment - Dave Rigby Tai Tran Do we still plan to include this ticket in 4.6.4?

          People

            tai.tran Tai Tran (Inactive)
            drigby Dave Rigby
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty