Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-34951

'send_stream_end_on_client_close_stream' for DcpConsumer causes spurious KEY_ENOENT warnings

    XMLWordPrintable

Details

    • Untriaged
    • No
    • KV-Engine Mad-Hatter Beta

    Description

      The DCP option 'send_stream_end_on_client_close_stream' was added via MB-26363 to have the Producer send a STREAM_END message even when the Consumer initiated the close. This was requested by an external DCP client (Go SDK), however the ep-engine DcpConsumer also requested this feature.

      However this can cause (benign) warnings during rebalance when the initial (backfill) Stream is closed and the takeover stream created. The problem is the following sequence of messages (all for the same vbid):

              DcpConsumer            ns_server proxy                DcpProducer
          
              <--- 1. CloseStream Req ------
              ---- 2. CloseStream Resp ---->
                                                 ---- 3. CloseStream Req --->
                                                 <--- 4. CloseStream Resp ---
          
              <--- 5. AddStream(Takeover) --
              <--- 6. StreamEnd Request -------------------------------------
              ---- 7. StreamEnd Response [KEY_ENOENT] ---------------------->
          
              ---- 8. StreamRequest ---------------------------------------->
      

      ns_server is not aware of `send_stream_end_on_client_close_stream`; so it doesn't wait for `6. StreamEnd Request` from the Producer before sending the AddStream request to the Consumer. As such, the Consumer has already removed the entry in it's streams map for the given vbucket, so when it receives the StreamEnd request from the Producer it returns KEY_ENOENT error as the (old) opaque doesn't match the (new) opaque for the takeover stream.

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            drigby Dave Rigby (Inactive)
            drigby Dave Rigby (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty