Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-63345

DCP seqno backfill for Magma is inefficient for collection streams or high RR

    XMLWordPrintable

Details

    • Improvement
    • Resolution: Unresolved
    • Major
    • Morpheus
    • 7.6.0, 7.1.0, 7.2.0, 7.1.6, 7.6.2, 7.2.6, 7.6.1, 7.6.4
    • couchbase-bucket
    • None
    • 0

    Description

      A seqno backfill iterates the documents on disk in sequence order.

      Collection streams only need to see the portion of the documents in that collection(s).

      The seqno backfill code for Couchstore checks whether the document key is matched by the collection filter, and conditionally reads the value of the document.

      Even if the document is required, the seqno backfill code for Couchstore checks whether the document is resident and avoids reading the value from disk needlessly.

      The seqno backfill code for Magma always reads the value from disk. This inefficiency has apparently always been there https://review.couchbase.org/c/kv_engine/+/108698 but was noticed recently by Hareen Kancharla.

      The MagmaKVStore uses the Magma::SeqIterator iterator to read the values on disk. From what I can see, that interface does not provide a way to read the value conditionally, so fix this, we will need changes in Magma.

      The impact of this is potentially large, depending on the proportion of the collection and the value size of the documents that get discarded.

      Attachments

        Activity

          People

            owend Daniel Owen
            vesko.karaganev Vesko Karaganev
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:

              PagerDuty