Details
-
Bug
-
Resolution: Unresolved
-
Minor
-
7.1.0
-
None
-
Untriaged
-
1
-
Unknown
Description
GSI functional test set03_planner_test.go TestPlanner occasionally fails with the error:
set03_planner_test.go:411: validation fails: cpu usage of indexer does not match sum of index cpu use
initialPlacementTest() logs this here:
if err := planner.ValidateSolution(p.GetResult()); err != nil {
t.Fatal(err)
}
ValidateSolution() is in planner/util.go and is the source of the message (which also needs to be improved).
This failure indicates that the CPU usage tracked at the top level of the final plan does not equal the sum of the CPU usages tracked in each individual indexer node in the plan, both taken as the nearest math.Floor() integer.
It happens very rarely. In 50 local runs it failed only once for me, on the GSI unstable branch based on Neo 2012 build of 2022-01-06, though I might have pulled newer code onto unstable in that repo at some point.
It also failed once recently in CI Bot pegged to Neo build 2043 of 2022-01-11:
http://ci2i-unstable.northscale.in/gsi-12.01.2022-16.14.fail.html
I will attach the logs from that CI run and TestPlanner outputs from it, my local failing run, and one local passing run.
(Note this appears unrelated to the plan.CpuQuota calculation change staged in https://review.couchbase.org/c/indexing/+/168343/8..9)