Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-37873

A collection with 100% items purged sends a snapshot marker but not the end

    XMLWordPrintable

Details

    Description

      Summary

      • Scenario results in DCP sending a snapshot marker and no data.
      • Indexer likely to become stuck (similar issue to MB-37209)
      • Note that buckets (and anything prior collections) avoid this because we will never purge the item @ high-seqno, thus DCP would always find a tombstone to represent the end of the snapshot.
      • Issue only affects collection streams and as of now we would purge the item @ collection high-seqno.

      Steps to reproduce.

      1. create 1 nodes
      2. Create a number of collections and add documents to all collections (i used enough so all vbuckets had data)
        1. Do this in sequence though. E.g if you have collections A, B and C. Add all documents to A then all to B and finally all to C.
      3. Delete documents from either collection B (or A), can be done by giving a small expiry at the write phase and forcing compaction.
      4. Purge all tombstones - I had a developer build so was able to trick the purger into thinking everything was valid for purging regardless of the 3 day purge interval.

      Now that the node is set-up, we can observe that collection B has no items, but a DCP stream which requests collection B only will yield the following.

      • DCP snapshot, flags=disk, start 0, end collection high-seqno (or high-seqno if MB-32231 is not fixed)
      • DCP system-event (collection B created @ seqno x)
      • No more data, stream will not send anything for the end seqno the marker mentioned

      Ideas to resolve:

      • Make the purger collection aware, it should never purge the collection's item at its high-seqno
      • More DCP messages, we could use DCP_SEQNO_ADVANCED in place of the hidden missing end item?

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              richard.demellow Richard deMellow
              jwalker Jim Walker
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty