Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-38428

Reduce persistMajority tail latency - optimize couchstore compaction

    XMLWordPrintable

Details

    Description

      Allow compaction to be partially concurrent with Flushing, by allowing the Flusher to continue to run while compacting the previous file. When compaction gets to the the end of the snapshot of the file when it opened, then block the Flusher and check if any more mutations have been written to the (old) couchstore file, if so copy just that (hopefully small) number over to the new file.

      This allows the bulk (99+%) of the compaction to occur without blocking the flusher, minimising the period when exclusive access is required.

      Attachments

        Issue Links

          has to be done after

          Bug - A problem which impairs or prevents the functions of the product. MB-41747 Couchstore: Memcached crashed in Warmup::loadKVPairsforShard(unsigned short) () from /opt/couchbase/bin/../lib/libep.so

          • Test Blocker - QE-related tests can't proceed
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MB-40415 "_local/vbstate" may get out of sync if a crash happens right after compaction

          • Major - Major loss of function.
          • Closed
          relates to

          Improvement - An improvement or enhancement to an existing feature or task. MB-38426 Reduce SyncWrite latency

          • Major - Major loss of function.
          • Closed
          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              owend Daniel Owen
              owend Daniel Owen
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                PagerDuty