Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-11560

Compaction: Don't write to /tmp

    XMLWordPrintable

Details

    • Improvement
    • Resolution: Fixed
    • Critical
    • techdebt-backlog
    • 2.5.1, 2.5.0, 3.0
    • storage-engine
    • Security Level: Public
    • None

    Description

      As part of investigating MB-11557, I discovered that couch_compact actually writes to files outside the data directory. Specifically it creates 3 temporary files (using tmpfile() ), which from an initial glance appear to be used to build the new, compacted b-tree (please correct me if I'm wrong).

      This is *bad* for a number of reasons:

      1) Nowhere in our documentation or install guide do we mention that /tmp is needed by Couchbase.
      2) If /tmp isn't writable, then compaction fails (with an opaque error message) - see MB-11557. Note nothing else seems to require /tmp, so the system works perfectly apart from your disks slowly running out of space.
      3) Related to (1), while users typically configure a fast IO subsystem for the data and index directories they have no reason to do so for /tmp - and so this may be on a slow subsystem, creating a bottleneck for IO

      Ideally we wouldn't use additional temporary disk space at all (and just use an mmap()ed file / memory), but we should at least write to the same directory as the output file, which is already in the data / index directory and can be assumed to be fast.

      Additional info:

      Attached strace file shows the writes/reads made to the three temporary files while compacting an example couchdb file:

      /opt/couchbase/bin/couch_dbinfo 9.couch.2
      DB Info (9.couch.2)
      file format version: 11
      update_seq: 35961
      doc count: 19946
      deleted doc count: 0
      data size: 2.59 MB
      B-tree size: 1.11 MB
      total disk size: 10.18 MB

      Approximately 3.2MB was written to temporary files (rough grep of writes to FDs 5 and 6).

      Output (compacted) file:

      [vagrant@localhost vagrant]$ /opt/couchbase/bin/couch_dbinfo couch.compacted
      DB Info (couch.compacted)
      file format version: 11
      update_seq: 35961
      doc count: 19946
      deleted doc count: 0
      data size: 2.64 MB
      B-tree size: 1.16 MB
      total disk size: 2.64 MB

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              drigby Dave Rigby (Inactive)
              drigby Dave Rigby (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty