Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-32158

Q1 latency and throughput degradation after MB-31848 change

    XMLWordPrintable

Details

    • Bug
    • Resolution: Not a Bug
    • Major
    • 6.5.0
    • 6.5.0
    • performance
    • Triton cluster
    • Untriaged
    • Centos 64-bit
    • No

    Description

      Despite the statement in MB-31848 the throughput is actually degraded after that change.

      I'm getting 80k  q/sec versus 100k in Daily test for Q1

       

      The latency profile is interesting. There are no more slow requests in higher percentiles but overall latency got way higher. Which also caused throughput degradation:

      Full comparison:

      http://cbmonitor.sc.couchbase.com/reports/html/?snapshot=triton_650-1717_access_fb82&label=6.5.0-1717&snapshot=triton_650-1719_access_8626&label=6.5.0-1719

       

      Logs from 1719:

      https://s3-us-west-2.amazonaws.com/perf-artifacts/jenkins-triton-20785/triton-srv-01-ip6.perf.couchbase.com.zip

      https://s3-us-west-2.amazonaws.com/perf-artifacts/jenkins-triton-20785/triton-srv-02-ip6.perf.couchbase.com.zip

      https://s3-us-west-2.amazonaws.com/perf-artifacts/jenkins-triton-20785/triton-srv-03-ip6.perf.couchbase.com.zip

      https://s3-us-west-2.amazonaws.com/perf-artifacts/jenkins-triton-20785/triton-srv-04-ip6.perf.couchbase.com.zip

      https://s3-us-west-2.amazonaws.com/perf-artifacts/jenkins-triton-20785/triton-srv-05-ip6.perf.couchbase.com.zip

      https://s3-us-west-2.amazonaws.com/perf-artifacts/jenkins-triton-20785/triton-srv-06-ip6.perf.couchbase.com.zip

       

      services:
      triton-srv-01-ip6.perf.couchbase.com:kv
      triton-srv-02-ip6.perf.couchbase.com:kv
      triton-srv-03-ip6.perf.couchbase.com:kv
      triton-srv-04-ip6.perf.couchbase.com:kv
      triton-srv-05-ip6.perf.couchbase.com:index
      triton-srv-06-ip6.perf.couchbase.com:n1ql
       

       

      Q1 performance in Daily:

      http://showfast.sc.couchbase.com/daily/#/history/N1QL%7CQ1,%20Key-Value%20Lookup,%205M%20docs%7CAvg%20Query%20Throughput%20(queries/sec)

       

       

       

      Attachments

        1. cpu_1.pprof
          152 kB
        2. goroutine_1
          1007 kB
        3. heap_1
          12 kB
        4. profile023.svg
          121 kB
        5. Screen Shot 2018-11-27 at 2.10.47 PM.png
          Screen Shot 2018-11-27 at 2.10.47 PM.png
          447 kB

        Issue Links

          For Gerrit Dashboard: MB-32158
          # Subject Branch Project Status CR V

          Activity

            People

              oleksandr.gyryk Alex Gyryk (Inactive)
              oleksandr.gyryk Alex Gyryk (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty