Uploaded image for project: 'Couchbase Kafka Connector'
  1. Couchbase Kafka Connector
  2. KAFKAC-209

Source: Couchbase partition assignment creates unbalanced workloads

    XMLWordPrintable

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 4.0.0-dp.3
    • Labels:
      None
    • Story Points:
      1

      Description

      The partition assignment algorithm inĀ `com.couchbase.connect.kafka.util.Config` tries to minimize the number of DCP connections to Couchbase, but it doesn't matter much in the end since the DCP client connects to all nodes anyway.

      A deployment where the number of connector tasks is not a multiple of the number of Couchbase KV nodes will have an uneven partition assignment, with some of the tasks handling more partitions than others.

      It's probably better to evenly distribute the partitions among the connector tasks.

        Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

            Assignee:
            david.nault David Nault
            Reporter:
            david.nault David Nault
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved:

                Gerrit Reviews

                There are no open Gerrit changes

                  PagerDuty