Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-37888

pillowfight throughput drop in build 7.0.0-1214

    XMLWordPrintable

Details

    • Untriaged
    • Yes
    • KV Sprint 2020-February

    Description

      Pillowfight throughput test with replicate_to=1, persist_to=0. This test is being run with libcouchbase 3.0.0-alpha.5 and python sdk 3.0.0-alpha.5. Up to build 1213 we are seeing consistent throughput but in build 1214 throughput drops to 0-6k. I am not sure of the exact throughput drop since the slow down is so much, the test times out. I have logged into the machine and watched the stats for a bit and witnessed both 0 ops and ~6k ops. Throughput for this test is normally around 200k ops. 

      1226 - http://perf.jenkins.couchbase.com/job/ares/12287/ - Hang

      1214 - http://perf.jenkins.couchbase.com/job/ares/12289/ - Hang

      1213 - http://perf.jenkins.couchbase.com/job/ares/12291/ - 215601

      1210 - http://perf.jenkins.couchbase.com/job/ares/12286/ - 215308

      1180 - http://perf.jenkins.couchbase.com/job/ares/12285/ - 213250

       

      Graphs are unavailable for the bad tests. Changelog doesnt indicate any obvious culprit: http://172.23.123.43:8000/getchangelog?product=couchbase-server&fromb=7.0.0-1213&tob=7.0.0-1215

       

      On top of that, I have runs some experiments with the latest versions of libcouchbase (3.0.0) and python sdk (3.0.0-beta.3). These tests show similar slowdown but on builds that the original test performs fine:

      1214 - http://perf.jenkins.couchbase.com/job/ares/12292/ - Hang

      1213 - http://perf.jenkins.couchbase.com/job/ares/12293/ - Hang

      1210 - http://perf.jenkins.couchbase.com/job/ares/12294/ - Hang

      1180 - http://perf.jenkins.couchbase.com/job/ares/12295/ - Hang 

      Pillowfight command:

      cbc-pillowfight --password password --batch-size 1 --num-items 20000000 --num-threads 128 --min-size 512 --max-size 512 --persist-to 0 --replicate-to 1 --json --spec "couchbase://172.23.133.13/bucket-1?durability_interval=0.02&enable_tracing=false&ipv6=allow" --set-pct 100 --num-cycles 2000000 --no-population

       

       

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            korrigan.clark Korrigan Clark (Inactive)
            korrigan.clark Korrigan Clark (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty