Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-32231

Collections: Filtered Disk Snapshot end can be incorrect

    XMLWordPrintable

Details

    • Bug
    • Resolution: Fixed
    • Major
    • 7.0.0
    • master
    • couchbase-bucket
    • None

    Description

      A filtered DCP stream can set the initial 'disk' snapshot end to an item which will not replicate.

      The problem occurs because the backfill phase occurs in two phases.

      1) Ask the disk for the range seqnos (start, end) this is done for the entire VB (no filtering), a snapshot is pushed to the DCP ready queue with those values.
      2) scan the seqno index, DCP only includes items which match the filter

      The issue is that the end seqno can be a mutation which will not match the filter, this the client sees

      • snap {0, 10}
      • mutation {1}
      • ... 2 to 10 are filtered

      and no more

      in-memory snapshots are ok because the range is calculated after filtering.

      For filtering, we may need to change the way the snapshot end is notified to the client, we cannot for example hold off sending the marker until we've applied filtering as that could consume a lot of memory. Possible we need to send an explicit snap start and snap end message.

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              richard.demellow Richard deMellow
              jwalker Jim Walker
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty