Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-49685

Seqno-ordered collection filtered backfills monopolise task/thread

    XMLWordPrintable

Details

    • Bug
    • Resolution: Unresolved
    • Major
    • Morpheus
    • 7.0.0, 7.0.1, 7.0.2, 7.1.0
    • couchbase-bucket
    • None
    • Untriaged
    • 1
    • No

    Description

      The amount of time a backfill may run for before yielding (allowing other backfills to run) is indirectly limited by:

      • a per-scan byte limit
      • outstanding bytes limit (bytes read by backfill but not yet sent by the stream)

      However, for a collection-filtered stream, these limits are only applied to items which actually match the filter. A single collection may only be a small fraction of the data on disk - a seqno-ordered backfill may read a significant amount of data (taking a significant amount of time) before enough items for the desired collection have been read.

      bool ActiveStream::backfillReceived(std::unique_ptr<Item> itm,
                                          backfill_source_t backfill_source) {
      ...
          // Is the item accepted by the stream filter (e.g matching collection?)
          if (!filter.checkAndUpdate(*itm)) {
              // Skip this item, but continue backfill at next item.
              return true;
          }
      ...
          if (!producer->recordBackfillManagerBytesRead(
                      resp->getApproximateSize())) {
              return false;
          }
      

      This may adversely affect other backfills for the same backfill manager, and other task running on AuxIO threads.

      investigate altering the scan limit to account for bytes read even if they do not match the filter, or an additional limit solely tracking bytes or items read.

      Uncovered during investigation for MB-48569

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              owend Daniel Owen
              james.harrison James Harrison (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty