Uploaded image for project: 'Couchbase Kafka Connector'
  1. Couchbase Kafka Connector
  2. KAFKAC-209

Source: Couchbase partition assignment creates unbalanced workloads

    XMLWordPrintable

Details

    • Improvement
    • Resolution: Fixed
    • Major
    • 4.0.0-dp.3
    • None
    • None
    • 1

    Description

      The partition assignment algorithm in `com.couchbase.connect.kafka.util.Config` tries to minimize the number of DCP connections to Couchbase, but it doesn't matter much in the end since the DCP client connects to all nodes anyway.

      A deployment where the number of connector tasks is not a multiple of the number of Couchbase KV nodes will have an uneven partition assignment, with some of the tasks handling more partitions than others.

      It's probably better to evenly distribute the partitions among the connector tasks.

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            david.nault David Nault
            david.nault David Nault
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty