Details
-
Bug
-
Resolution: Fixed
-
Major
-
6.5.0
-
Untriaged
-
No
-
KV-Engine Mad-Hatter Beta
Description
The DCP option 'send_stream_end_on_client_close_stream' was added via MB-26363 to have the Producer send a STREAM_END message even when the Consumer initiated the close. This was requested by an external DCP client (Go SDK), however the ep-engine DcpConsumer also requested this feature.
However this can cause (benign) warnings during rebalance when the initial (backfill) Stream is closed and the takeover stream created. The problem is the following sequence of messages (all for the same vbid):
DcpConsumer ns_server proxy DcpProducer
|
|
<--- 1. CloseStream Req ------
|
---- 2. CloseStream Resp ---->
|
---- 3. CloseStream Req --->
|
<--- 4. CloseStream Resp ---
|
|
<--- 5. AddStream(Takeover) --
|
<--- 6. StreamEnd Request -------------------------------------
|
---- 7. StreamEnd Response [KEY_ENOENT] ---------------------->
|
|
---- 8. StreamRequest ---------------------------------------->
|
ns_server is not aware of `send_stream_end_on_client_close_stream`; so it doesn't wait for `6. StreamEnd Request` from the Producer before sending the AddStream request to the Consumer. As such, the Consumer has already removed the entry in it's streams map for the given vbucket, so when it receives the StreamEnd request from the Producer it returns KEY_ENOENT error as the (old) opaque doesn't match the (new) opaque for the takeover stream.