Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-37520

DCP throughput drop (~15%) in CC build 7.0.0-4170 vs 6.6.0 (compression enabled)

    XMLWordPrintable

Details

    Description

      Edit: Updated with newer, post-warmup tests

      In 7.0.0-4170 I see a ~15% reduction in throughput compared with 6.6.0-7909 - 230k op/s vs 197k op/s for DCP Throughput, 1 node, 100M x 1KB, Java client, 100% Resident, compression enabled, post warmup:

      Details

      I have added these new tests to showfast here - 

      http://showfast.sc.couchbase.com/#/timeline/Linux/kv/dcp/all

      DCP Throughput, 1 node, 100M x 1KB, Java client, 100% Resident, compression enabled, post warmup

      DCP Throughput, 1 node, 100M x 1KB, Java client, 50% Resident, compression enabled, post warmup

      DCP Throughput, 1 node, 200M x 1KB, Java client, 50% Resident, compression enabled, post warmup

      These are three tests with different data volume and resident ratio with below test flow - 

      1. setup cluster  [1 node cluster, ~110 GB mem quota , eviction_policy=full eviction]
      2. load data
      3. compact bucket
      4. warmup -> [stop cb server, Drop memory Cache, start cb server back, monitor  warmup status by monitoring ep_warmup_state stat]
      5. run DCP using java-dcp client 
        perf/run.sh couchbase://bucket-1:password@cen-sc34.perf.couchbase.com/bucket-1 100000000 perf/config/compression-enabled.properties > java_dcp.log

      —

      Original description
      Observing ~60% drop DCP throughput in CC build .

      Attachments

        1. 309003c17e91f432d761f9cdfae1dec9.png
          309003c17e91f432d761f9cdfae1dec9.png
          34 kB
        2. 827a76b6dec4d782dc22f59ae9461329.png
          827a76b6dec4d782dc22f59ae9461329.png
          29 kB
        3. DCP drain throughput.png
          DCP drain throughput.png
          11 kB
        4. disk-queue-6.6.0.png
          disk-queue-6.6.0.png
          115 kB
        5. disk-queue-7.0.0.png
          disk-queue-7.0.0.png
          115 kB
        6. image-2020-01-14-17-40-23-196.png
          image-2020-01-14-17-40-23-196.png
          103 kB
        7. image-2020-12-21-20-29-07-911.png
          image-2020-12-21-20-29-07-911.png
          192 kB
        8. image-2020-12-21-20-34-03-617.png
          image-2020-12-21-20-34-03-617.png
          208 kB
        9. image-2021-02-05-09-53-44-710.png
          image-2021-02-05-09-53-44-710.png
          336 kB
        10. image-2021-02-05-10-06-04-947.png
          image-2021-02-05-10-06-04-947.png
          27 kB
        11. image-2021-03-22-10-23-07-516.png
          image-2021-03-22-10-23-07-516.png
          100 kB
        12. memory-6.6.png
          memory-6.6.png
          147 kB
        13. memory-7.0.0.png
          memory-7.0.0.png
          135 kB
        14. pager_runs-6.6.0.png
          pager_runs-6.6.0.png
          59 kB
        15. pager_runs-7.0.0.png
          pager_runs-7.0.0.png
          54 kB
        16. perf.dwarf.png
          perf.dwarf.png
          59 kB
        17. perf.lbr.png
          perf.lbr.png
          103 kB
        18. Screenshot 2020-03-04 at 08.42.20.png
          Screenshot 2020-03-04 at 08.42.20.png
          40 kB
        19. Screenshot 2021-01-19 at 16.48.09.png
          Screenshot 2021-01-19 at 16.48.09.png
          33 kB
        20. Screenshot 2021-02-08 at 16.12.10.png
          Screenshot 2021-02-08 at 16.12.10.png
          34 kB

        Issue Links

          relates to

          Bug - A problem which impairs or prevents the functions of the product. MB-37580 multiple regressions from same build 7.0.0-1023

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MB-38009 drop in xdcr rate (~20%) in initial + rebalance test case

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MB-45071 DCP throughput drop ~20% between 7.0.0-4689 and 7.0.0-4690

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MB-38983 XDCR Performance Investigation for post 7.0.0-1697

          • Major - Major loss of function.
          • Closed
          For Gerrit Dashboard: MB-37520
          # Subject Branch Project Status CR V
          147476,3 MB-37520: Don't use AtomicSharedPtr for DcpProducer::backfillMgr master kv_engine Status: MERGED +2 +1
          147606,3 MB-37520: Change DcpProducer::StreamsMap to AtomicHashArray size 1024 master kv_engine Status: MERGED +2 +1
          147926,4 MB-37520: Reduce lock scope in ActiveStream::backfillReceived master kv_engine Status: MERGED +2 +1
          147931,3 MB-37520: DcpProducer: Set isSnappyEnabled once in ctor master kv_engine Status: MERGED +2 +1
          147932,8 MB-37520: Use dynamic_bitset for VBReadyQueue master kv_engine Status: MERGED +2 +1
          148048,2 MB-37520: Add individual accessors to ItemIface master kv_engine Status: MERGED +2 +1
          148049,2 MB-37520: Use individual Item accessors in Connection DCP methods master kv_engine Status: MERGED +2 +1
          148979,2 MB-37520: Fix incorrect VBReadyQueue map_size stat master kv_engine Status: MERGED +2 +1

          Activity

            People

              sharath.sulochana Sharath Sulochana (Inactive)
              sharath.sulochana Sharath Sulochana (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                PagerDuty