There was a previous issue opened https://issues.couchbase.com/browse/MB-49301 but resolved as not a bug. Maybe more details of the CPU throttle algorithm could help understand the result better and how its intended to work. When I set the throttle level to 0.95, query throughput gets reduced by 40%. The performance test pushes indexer cpu normally close to 100%. If the throttle level is set to 0.95 why does cpu usage get throttled down to 60% instead of right below 95%. It seems there is a lot of wasted cpu cycles here.
Comparing these two test on 7.1.0-1650:
Avg. Query Throughput (queries/sec), CI6, Group By Query (1K matches), MOI, not_bounded, s=1 c=1 i=1
Avg. Query Throughput (queries/sec), CI6, Group By Query (1K matches), MOI, not_bounded, Indexer CPU Throttle 0.95, s=1 c=1 i=1
Indexer cpu drops from 4800% to 3000%
Logs from throttle run (.45 is the index node):
Is this really the intended behaviour? This seems like it could have negative side effects when upgrading a cluster if this is the default setting.
|For Gerrit Dashboard: MB-49501|
|166026,2||MB-49662 Part 1 (7.1.0 1650): Autofailover: Make CPU throttling safer||unstable||indexing||Status: MERGED||+2||+1|
|166259,2||MB-49501 Part 2 (7.1.0 1695): Autofailover: Make CPU throttling safer||unstable||indexing||Status: MERGED||+2||+1|
|166270,2||MB-49501 Part 3 (7.1.0 1695): Autofailover: Make CPU throttling safer||unstable||indexing||Status: MERGED||+2||+1|
|167155,1||MB-49501 Part 4 (7.1.0 1861): Temp disable CPU throttling for perf tests||unstable||indexing||Status: MERGED||+2||+1|
|167275,2||MB-49501 Part 5 (7.1.0 1861): Log long heartbeats; reduce dump frequency||unstable||indexing||Status: MERGED||+2||+1|
|168569,1||MB-49501 Part 6 (7.1.0 2037): Shorten HealthCheck "Slow call" threshold||unstable||indexing||Status: MERGED||+2||+1|