Details
-
Bug
-
Resolution: Unresolved
-
Major
-
7.6.0
-
None
-
Triaged
-
0
-
Unknown
Description
The work done in MB-46740 resolves most of the issues observes in CBSEs with regard to huge allocations in the DCP readyQ. That is, in the presence of slow DCP components the DCP might easily move a lot of data from Checkpoint to readyQ (in-memory phase). The work completed in MB-46740 prevents that.
Differently, the DCP backfill phase already has a control parameter for limiting the allocation in the readyQ. That is dcp_backfill_byte_limit in EP configuration.
That is a per-connection limit on how much a single DCP Producer is allowed to allocate into the Stream readyQs at backfill, defaulted to 20MB.
Thus, allocation there is linear in the number of DCP connections. Eg, 100 Producers might end up allocating 2GB in the readyQs.
We have tried to lower that value to 2MB in MB-46740 but we hit some perf regression by that (MB-58742). Param reset to 20MB for now.
In practice I've never observed problematic readyQ spikes by backfill, not in internal QE/Perf tests and not in CBSEs either. So that doesn't feel as critical as the in-memory phase fixed in MB-46740.
Attachments
Issue Links
- causes
-
MB-58742 Avg. ingestion rate of 999 collections drops from 195K items/s to 57K items/s on build 7.6.0-1516
- Closed