Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-27800

Do not indicate rollback if purger has not run between successive DCP stream requests from a particular client



    • Improvement
    • Resolution: Unresolved
    • Critical
    • Morpheus
    • 5.0.1, 5.5.0
    • couchbase-bucket
    • None



      Say a DCP client first connects after a long time the server is started (somebody decides to build a secondary index after a few days), and at that time we could have numbers like high_seqno = 1,000,000; purge_seqno = 600,000.

      And, after sending upto seqno 500,000 items, the client disconnects and reconnects back immediately asking the server to resume from 500,001.

      Today, the server would ask the client to rollback to 0 in both cases as start_seqno < purge_seqno (500,001 < 600,000).

      (Additionally, like in CBSE-4662, the problem can be aggravated with consistent failures in completing the large snapshot  0 to 1,000,000).


      Proposed Solution:

      Theoretically, if there is no purger runs between the 2 requests of a DCP client with same snapshot numbers {snap_start_seqno, snap_end_seqno}, we should not ask the client to rollback. This is hard to implement as it is not easy to maintain the state (at the Producer) of when and with what snapshot numbers a client had previously opened the stream.

      However, we might be able to avoid the unnecessary rollback with the following addition to the DCP protocol.

      1. Stream request: clients sends (among other params) the quartet {snap_startseqno_x, snapendseqno_x, start_seqno, purgeseqno_x}
      2. Stream_requestresponse : DCP producer sends the current values for {snapstartseqno_y, snapendseqno_y, purgeseqno_y} followed by a few mutations.
      3. Now say the stream/or connection drops and reconnects back and prefers to start from the point where it was disconnected the client would send new stream_request as {snap_startseqno_y, snapendseqno_y, purgeseqno_y}
      4. The producer will not ask for rollback if there is no more purge, that is it uses the purge_seqno_y to find that out.

       A detailed explanation of the problem at hand and the proposed solution can be found at 


        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.



              jwalker Jim Walker
              manu Manu Dhundi (Inactive)
              0 Vote for this issue
              18 Start watching this issue



                Gerrit Reviews

                  There are no open Gerrit changes