Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-50444

Collections: GetAllVBSeqs returns max-visible seqno to collection aware client for a bucket request

    XMLWordPrintable

Details

    • Bug
    • Resolution: Fixed
    • Major
    • 7.1.0
    • 7.0.0, 7.0.1, 7.0.2, 7.0.3
    • couchbase-bucket
    • None
    • Untriaged
    • 1
    • Unknown

    Description

      Discovered when testing changes for MB-50413 (and loosely related).

      In mad-hatter (durability release) KV-engine tracked the max-visible seqno, this is the highest committed item of a vbucket. For clients that use DCP and GetAllVBSeqnos and do not support sync-replication, this value tracks the highest item DCP will send them, i.e. we won't send a prepare/abort. This fixed a number of issues where DCP clients would hang waiting for DCP to send them a seqno that GetAllVBSeqnos told them about.

      In cheshire-cat (collections release) KV-engine made lots of changes to DCP to support collections, included in this was a new DCP message called SeqnoAdvacned. A message that DCP uses in-place of items it wouldn't send. This is used to ensure a snapshot end is reached in the case where a snapshot ends on something the client doesn't know about, e.g. another collection or prepare/abort.

      A client can do a bucket DCP stream, and see all collections, but still not enable sync-writes, so they don't see prepare/abort. However when they call GetAllVBSeqs for the bucket high-seqno, we still send them the max-visible seqno, when we technically should send them the high-seqno (because they have stated they support collections ... SeqnoAdvance)

      Example:

      If the bucket stores (and is quiesced)

      • seqno:1 committed mutation key1
      • seqno:3 abort key2

      A client that is enabling collections (yet does not enable sync-writes, i.e. everyone except KV-engine itself)

      • GetAllVBSeqs(collection=c1) returns 1
      • DCP stream for c1 returns
      • DCP snapshot start=1, end=3
      • DCP mutation key1 seqno1
      • DCP SeqnoAdvance seqno3

      the proposed change will mean in this case GetAllVBSeqs returns 3

      Note there is also another discrepancy here as well, if for example the data in the bucket is all in collection c1 and the client targets c1 for all operations.

      • GetAllVBSeqs returns 3
      • DCP returns
      • DCP snapshot start=1, end=3
      • DCP mutation key1 seqno1
      • DCP SeqnoAdvance seqno3

      GetAllVBSeqs when targetting a collection is already returning the high-seqno

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            jwalker Jim Walker
            jwalker Jim Walker
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty