Details
-
Bug
-
Resolution: Fixed
-
Major
-
master
-
None
-
Untriaged
-
No
Description
A filtered DCP stream can set the initial 'disk' snapshot end to an item which will not replicate.
The problem occurs because the backfill phase occurs in two phases.
1) Ask the disk for the range seqnos (start, end) this is done for the entire VB (no filtering), a snapshot is pushed to the DCP ready queue with those values.
2) scan the seqno index, DCP only includes items which match the filter
The issue is that the end seqno can be a mutation which will not match the filter, this the client sees
- snap {0, 10}
- mutation {1}
- ... 2 to 10 are filtered
and no more
in-memory snapshots are ok because the range is calculated after filtering.
For filtering, we may need to change the way the snapshot end is notified to the client, we cannot for example hold off sending the marker until we've applied filtering as that could consume a lot of memory. Possible we need to send an explicit snap start and snap end message.
Attachments
Issue Links
- relates to
-
MB-37402 Collections: Handling max-visible seqno
- Resolved