Uploaded image for project: 'Couchbase Kafka Connector'
  1. Couchbase Kafka Connector
  2. KAFKAC-144

A malformed message crashes the connector

    XMLWordPrintable

Details

    • Bug
    • Resolution: Not a Bug
    • Minor
    • None
    • 3.4.3
    • Windows 10
    • 1

    Description

      Summary: When following the [kafka connect quickstart|https://docs.couchbase.com/kafka-connector/3.4/quickstart.html], everything works okay until an improperly formatted message is put into the kafka topic.

      Repro:

      1. Follow the quick start for using Couchbase as a sink.
      2. The 'mvn compile exec:java" step should work correctly, and you should end up with 20 documents in a Couchbase bucket.
      3. Send a message to the topic that is not{{ JSON. For example: bin\windows\kafka-console-producer.bat --broker-list localhost:9092 --topic test-default and then enter a message like "foobar"}}

      Actual: An error will occur like: [2019-04-04 10:43:57,908] ERROR WorkerSinkTask{id=home-delivery-sink-1} Task is being killed and will not recover until manually restarted (org.apache.kafka.connect.runtime.WorkerTask)

      Expected: Almost anything other than crashing and not continuing to process other messages. For instance, it could save the message as a binary document in Couchbase. It could report an error but then skip/ignore the message. In a situation where only well-formed messages get into Kafka, everything is fine. But right now, even one malformed/mistake message shuts the whole connector down.

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            david.nault David Nault
            matthew.groves Matthew Groves
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty