SINK: In rare circumstances, documents with same ID can be written to Couchbase in wrong order

Description

In CouchbaseSinkTask.put, the flapMap operation processes the upserts in parallel in non-deterministic order. If the batch of records contains duplicate IDs, it's possible for them to be written out of order.

This issue surfaced as a DocumentConcurrentlyModified exception when using persist_to=REPLICA

Caused by: com.couchbase.client.core.DocumentConcurrentlyModifiedException: The CAS on the active node changed for ID “t_hist:9202:2018:20190209:10:10:EUR:9202”, indicating it has been modified in the meantime.
       at com.couchbase.client.core.message.observe.ObserveViaCAS$ObserveItem.<init>(ObserveViaCAS.java:295)
       at com.couchbase.client.core.message.observe.ObserveViaCAS$5.call(ObserveViaCAS.java:80)
       at com.couchbase.client.core.message.observe.ObserveViaCAS$5.call(ObserveViaCAS.java:77)
       at rx.internal.operators.OnSubscribeMap$MapSubscriber.onNext(OnSubscribeMap.java:69)
       ... 25 more
Caused by: rx.exceptions.OnErrorThrowable$OnNextValue: OnError while emitting onNext value: com.couchbase.client.core.message.kv.ObserveResponse.class
       at rx.exceptions.OnErrorThrowable.addValueAsLastCause(OnErrorThrowable.java:118)
       at rx.internal.operators.OnSubscribeMap$MapSubscriber.onNext(OnSubscribeMap.java:73)
       ... 25 more

Environment

None

Gerrit Reviews

None

Release Notes Description

None

Activity

Show:

David Nault March 27, 2020 at 4:41 PM

Note to self: enable mutation tokens

Fixed

Pinned fields

Click on the next to a field label to start pinning.

Details

Assignee

David Nault

Reporter

David Nault

Story Points

Fix versions

4.0.0-dp.3

Priority

Major

Instabug

Open Instabug

PagerDuty

Sentry

Zendesk Support

Created March 26, 2020 at 5:46 PM

Updated June 24, 2020 at 4:21 PM

Resolved June 24, 2020 at 4:21 PM

Configure

Instabug