Skip to:

  1. Skip to Jira Navigation
  2. Skip to Side Navigation
  3. Skip to Main Content

OSO/magma backfill might send duplicate items when resuming after pause

Description

Issue introduced by in 7.1.2 (support for ById Scan added to MagmaKVStore).

For OSO backfill (ByKey Scan in KVStore) we miss to set the resume point to the proper key when the backfill is paused. The consequence is that at resume the backfill might resend the last processed key over the stream. By that:

  • DCP clients might see key duplicates in OSO snapshots

  • Slow DCP clients that force the stream to continuous pausing might stall/loop over receiving the same key multiple times in OSO snapshots

 

Issue

Resolution

DCP clients streamed in out-of-sequence-order [OSO] backfill snapshots under Magma observed duplicate documents received in the disk snapshot. This happened where the stream was paused and resumed when the resume point was wrongly set to a key already processed in the stream.

OSO backfill in Magma now sets the correct resume point after a pause.

Components

Affects versions

Fix versions

Labels

Environment

None

Link to Log File, atop/blg, CBCollectInfo, Core dump

None

Release Notes Description

None

Activity

Show:

CB robot September 1, 2023 at 1:29 PM

Build couchbase-server-8.0.0-1392 contains kv_engine commit 0d62c05 with commit message:
: Replace MagmaScanResult::Status::Next with existing Success

CB robot September 1, 2023 at 1:29 PM

Build couchbase-server-8.0.0-1392 contains kv_engine commit 374483c with commit message:
: Remove MagmaScanResult

CB robot September 1, 2023 at 8:37 AM

Build couchbase-server-7.6.0-1446 contains kv_engine commit 0d62c05 with commit message:
: Replace MagmaScanResult::Status::Next with existing Success

CB robot September 1, 2023 at 8:37 AM

Build couchbase-server-7.6.0-1446 contains kv_engine commit 374483c with commit message:
: Remove MagmaScanResult

CB robot September 1, 2023 at 8:34 AM

Build capella-analytics-1.0.0-1007 contains kv_engine commit 0d62c05 with commit message:
: Replace MagmaScanResult::Status::Next with existing Success

Fixed
Pinned fields
Click on the next to a field label to start pinning.

Details

Assignee

Reporter

Is this a Regression?

Unknown

Triage

Triaged

Issue Impact

external

Story Points

Priority

Instabug

Open Instabug

PagerDuty

Sentry

Zendesk Support

Created May 25, 2023 at 11:21 AM
Updated March 21, 2025 at 2:50 AM
Resolved May 26, 2023 at 8:31 PM
Instabug