Uploaded image for project: 'Couchbase Kafka Connector'
  1. Couchbase Kafka Connector
  2. KAFKAC-262

Sink: Missed updates when using N1QL mode

    XMLWordPrintable

Details

    • Bug
    • Resolution: Fixed
    • Major
    • 4.1.0
    • None
    • None
    • 1

    Description

      A connector configured to use N1QL mode could mistakenly ignore updates if Kafka records targeting the same document(s) arrive in rapid succession.

      This happened because the connector was using an event deduplication algorithm based on the document ID / Kafka record key: if two records with the key/ID arrive in the same batch, the first record was dropped, and the connecter never applied the updates from the dropped record.

      Even if two Kafka records have different IDs, the updates could be applied out of order since N1QL queries are executed concurrently. As a result of this race condition, it was theoretically possible for the updates from the first record to be applied after the updates from the second record.

      Attachments

        Issue Links

          Activity

            People

              david.nault David Nault
              david.nault David Nault
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                PagerDuty