Skip to:

  1. Skip to Jira Navigation
  2. Skip to Side Navigation
  3. Skip to Main Content

Excessive flushes may kill performance on complicated pipelines

Description

Observed in , Analytics can get overwhelmed with frequent flushes combined with complicated pipelines. In this case, there were 59 secondary indexes & three datasets all defined on a single collection. The excess flushes were caused by:

  • flushes forced due to incorrectly perceived uuid updates

  • regression introduced with fix for , resulting in perpetual flushes on an overloaded system (one likely overloaded due to flushes)

Components

Affects versions

Fix versions

Labels

Environment

None

Link to Log File, atop/blg, CBCollectInfo, Core dump

None

Release Notes Description

None

Activity

Show:

CB robot September 22, 2023 at 10:22 AM

Build couchbase-server-8.0.0-1413 contains cbas-core commit fe417e8 with commit message:
: connector flush fixes

CB robot September 22, 2023 at 5:25 AM

Build capella-analytics-1.0.0-1027 contains cbas-core commit fe417e8 with commit message:
: connector flush fixes

Murtadha Al Hubail September 21, 2023 at 10:43 PM
Edited

Release note:

Description:
Some DCP messages were unnecessarily processed multiple times. In addition, if the Analytics Service has high CPU usage, unnecessary flushes of DCP state to storage may occur, which will additionally contribute to the high CPU usage.

Resolution:
DCP messages are now processed once regardless of the configured dcpIdleQueueFlushThresholdMillis, and high CPU usage will not trigger unnecessary flushes of DCP state.

CB robot September 21, 2023 at 3:57 PM

Build couchbase-server-7.6.0-1534 contains cbas-core commit fe417e8 with commit message:
: connector flush fixes

CB robot September 21, 2023 at 3:04 PM

Build couchbase-server-7.2.3-6603 contains cbas-core commit fe417e8 with commit message:
: connector flush fixes

Fixed
Pinned fields
Click on the next to a field label to start pinning.

Details

Assignee

Reporter

Is this a Regression?

Yes

Triage

Untriaged

Issue Impact

external

Story Points

Priority

Instabug

Open Instabug

PagerDuty

Sentry

Zendesk Support

Created September 15, 2023 at 1:58 PM
Updated March 21, 2025 at 2:49 AM
Resolved September 21, 2023 at 2:00 AM
Instabug