Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-49301

Index monitor slowing down index scans

    XMLWordPrintable

Details

    • Untriaged
    • Centos 64-bit
    • 1
    • Unknown

    Description

      Avg. Query Throughput (queries/sec), CI19, Composite Filters (1000 -> 1), Covered, MOI, not_bounded

      1593 - https://perf.jenkins.couchbase.com/job/iris-multi-client/12799/ - 59501.0
      1594 - https://perf.jenkins.couchbase.com/job/iris-multi-client/12802/ - 37752.0

      http://changelog.build.couchbase.com/?product=couchbase-server&fromVersion=7.1.0&fromBuild=1593&toVersion=7.1.0&toBuild=1594&f_goxdcr=on&f_ns_server=on&f_product-metadata=on

      http://cbmonitor.sc.couchbase.com/reports/html/?snapshot=iris_710-1593_access_4fff&snapshot=iris_710-1594_access_5f5e

      http://showfast.sc.couchbase.com/#/timeline/Linux/n1ql/API2/MOI

      Many tests that push indexer cpu utilization to the max have regressed in 1594. It seems the only related commits are from the introduction of index monitor and indexer autofailover. It looks like default timeout for the monitor request is 2 seconds which may be too low for heavy indexer usage. In the logs you can see lots of these monitor requests failing

      https://s3-us-west-2.amazonaws.com/perf-artifacts/jenkins-iris-multi-client-12802/172.23.100.45.zip <---indexer
      https://s3-us-west-2.amazonaws.com/perf-artifacts/jenkins-iris-multi-client-12802/172.23.100.55.zip
      https://s3-us-west-2.amazonaws.com/perf-artifacts/jenkins-iris-multi-client-12802/172.23.100.70.zip
      https://s3-us-west-2.amazonaws.com/perf-artifacts/jenkins-iris-multi-client-12802/172.23.100.71.zip
      https://s3-us-west-2.amazonaws.com/perf-artifacts/jenkins-iris-multi-client-12802/172.23.100.72.zip
      https://s3-us-west-2.amazonaws.com/perf-artifacts/jenkins-iris-multi-client-12802/172.23.100.73.zip
      https://s3-us-west-2.amazonaws.com/perf-artifacts/jenkins-iris-multi-client-12802/172.23.100.9.zip

      If there is a way to increase the timeout, I can try that to see if there is less slow down. Also would like to disable indexer autofailover if possible. Can you provide the endpoints and requests to do so?

      Attachments

        Issue Links

          Activity

            People

              kevin.cherkauer Kevin Cherkauer (Inactive)
              korrigan.clark Korrigan Clark (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                PagerDuty