Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-59832

Default dcp_backfill_byte_limit value (20MB) might cause readyQ spikes to above the Bucket Quota

    XMLWordPrintable

Details

    • Bug
    • Resolution: Unresolved
    • Major
    • Morpheus
    • 7.6.0
    • couchbase-bucket
    • None
    • Triaged
    • 0
    • Unknown

    Description

      The work done in MB-46740 resolves most of the issues observes in CBSEs with regard to huge allocations in the DCP readyQ. That is, in the presence of slow DCP components the DCP might easily move a lot of data from Checkpoint to readyQ (in-memory phase). The work completed in MB-46740 prevents that.

      Differently, the DCP backfill phase already has a control parameter for limiting the allocation in the readyQ. That is dcp_backfill_byte_limit in EP configuration.
      That is a per-connection limit on how much a single DCP Producer is allowed to allocate into the Stream readyQs at backfill, defaulted to 20MB.
      Thus, allocation there is linear in the number of DCP connections. Eg, 100 Producers might end up allocating 2GB in the readyQs.
      We have tried to lower that value to 2MB in MB-46740 but we hit some perf regression by that (MB-58742). Param reset to 20MB for now.

      In practice I've never observed problematic readyQ spikes by backfill, not in internal QE/Perf tests and not in CBSEs either. So that doesn't feel as critical as the in-memory phase fixed in MB-46740.

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              paolo.cocchi Paolo Cocchi
              paolo.cocchi Paolo Cocchi
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty