Details
-
Bug
-
Status: Closed
-
Major
-
Resolution: Not a Bug
-
7.1.0
-
Untriaged
-
Centos 64-bit
-
1
-
Unknown
Description
Avg. Query Throughput (queries/sec), CI19, Composite Filters (1000 -> 1), Covered, MOI, not_bounded
1593 - https://perf.jenkins.couchbase.com/job/iris-multi-client/12799/ - 59501.0
1594 - https://perf.jenkins.couchbase.com/job/iris-multi-client/12802/ - 37752.0
http://showfast.sc.couchbase.com/#/timeline/Linux/n1ql/API2/MOI
Many tests that push indexer cpu utilization to the max have regressed in 1594. It seems the only related commits are from the introduction of index monitor and indexer autofailover. It looks like default timeout for the monitor request is 2 seconds which may be too low for heavy indexer usage. In the logs you can see lots of these monitor requests failing
https://s3-us-west-2.amazonaws.com/perf-artifacts/jenkins-iris-multi-client-12802/172.23.100.45.zip <---indexer
https://s3-us-west-2.amazonaws.com/perf-artifacts/jenkins-iris-multi-client-12802/172.23.100.55.zip
https://s3-us-west-2.amazonaws.com/perf-artifacts/jenkins-iris-multi-client-12802/172.23.100.70.zip
https://s3-us-west-2.amazonaws.com/perf-artifacts/jenkins-iris-multi-client-12802/172.23.100.71.zip
https://s3-us-west-2.amazonaws.com/perf-artifacts/jenkins-iris-multi-client-12802/172.23.100.72.zip
https://s3-us-west-2.amazonaws.com/perf-artifacts/jenkins-iris-multi-client-12802/172.23.100.73.zip
https://s3-us-west-2.amazonaws.com/perf-artifacts/jenkins-iris-multi-client-12802/172.23.100.9.zip
If there is a way to increase the timeout, I can try that to see if there is less slow down. Also would like to disable indexer autofailover if possible. Can you provide the endpoints and requests to do so?
Attachments
Issue Links
- relates to
-
MB-49501 Indexer cpu throttle cuts query throughput by 47%
-
- Closed
-
Korrigan Clark I will take a look.
Request from GSI team: Can you please also try a run with:
and report whether this run triggered Autofailover to happen. This is to get a data point as to whether CPU throttling is actually needed to prevent false failovers in this case, where the index nodes are heavily loaded. Thank you!