Details
-
Bug
-
Resolution: Fixed
-
Major
-
7.6.0, 7.6.1
-
Triaged
-
0
-
Yes
-
March-June 24
Description
With MB-31869 we have removed the dedicated DCP Consumer Buffer (7.6.0).
By that, when a Consumer node enters a OOM phase, DCP Consumer start buffering incoming messaged into Checkpoints.
For those messages we don't (and we can't) enforce the CheckpointQuota by design, as Consumers must be always allowed to process and buffer DCP messages up to limits defined by the dcp_consumer_buffer_ratio config param.
What described so far isn't an issue by itself, but things might appear confusing when looking at cbcollects for 7.6.x.
Eg, we might observe allocations in the Checkpoints that cross the CheckpointQuota. Is that an error in the Checkpoint memory management or we have just buffered DCP messages in the checkpoits?
Possible alternative fixes:
- Account/reserve dcp_consumer_buffer_ratio into checkpoint_memory_ratio
- Expose the real checkpoint mem-usage limit, which is (checkpoint_memory_ratio + dcp_consumer_buffer_ratio)
Problem with (1) is that it would lead to sub-utilization of the checkpoint quota for scenarios where there's no Consumer buffering at all.
We are doing (2).