Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-62782

Investigate Kafka ingestion time results from volume test run on 2216 build

    XMLWordPrintable

Details

    • 0

    Description

      CASE A

      1. Load 50 million documents in Mongo collection.
      2. Create Confluent Kafka topic against the Mongo collection created in previous step.
      3. Deploy a 4 node Columnar instance with 8vCPUs and 64GB memory quota.
      4. Create pair of Kafka links and 5 collections on each of the 2 links.
      5. Connect both links.
      6. While data ingestion is happening perform multiple scaling operations in a loop.
        Logs - https://supportal.couchbase.com/snapshot/0bd616af280a305a6bac473e47024bd3::0

      CASE B

      1. Load 50 million documents in Mongo collection.
      2. Create Confluent Kafka topic against the Mongo collection created in previous step.
      3. Deploy a 4 node Columnar instance with 8vCPUs and 64GB memory quota.
      4. Create 4 Kafka links and 10 collections on each of the 4 links.
      5. Connect all 4 links.
      6. While data ingestion is happening perform multiple scaling operations in a loop.
        Logs - https://supportal.couchbase.com/snapshot/1d7b4c4370cbb58bd180968aeab5cb04::0

      Observation

      Data ingestion completed faster in second case which is weird. Raising this ticket to analyse this descrepancy.

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            sujay.gad Sujay Gad
            sujay.gad Sujay Gad
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty