Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-25994

Duration of compaction becomes impractical at large scale (vbucket size > 100GB)

    XMLWordPrintable

Details

    • Untriaged
    • Centos 64-bit
    • No

    Description

      A spin-off of MB-25982.

      Even if there is enough memory to mitigate impact of MB-25982, it still takes about 1 hour to finish compaction of 1 vbucket.

      When there are 20-40 vbuckets per server and there are one or multiple replica copies, sequential compaction of all vbuckets will easily take a day or two to finish. There is a good chance that compaction will start falling behind the fragmentation. persistTo latency is going to be miserable.

      Although we can address MB-25982 and improve persistTo latency by increasing the number of vbuckets, the total duration is unlikely to change (if not get worse).

      Other side effects are well known: no page cache, high disk IO utilization, high read latency, poor write and read amplification.

      Attachments

        Issue Links

          Activity

            People

              srinath.duvuru Srinath Duvuru
              pavelpaulau Pavel Paulau (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

                Created:
                Updated:

                PagerDuty