Details
-
Improvement
-
Resolution: Fixed
-
Major
-
5.5.0
Description
When a N1QL node has more goroutines ready to run than a certain multiplier of the available cores, the golang scheduler struggles to have them running in a timely manner.
This impacts requests service time, in particular when consumers with available incoming documents struggle to consume them because of lack of execution slots.
This MB tracks reducing the number of active goroutines, in one of two ways
- coalesce operators into one, when a single goroutine can readily perform the work of two or more (e.g. limit and/or offset and/or initial projection)
- have execution goroutines terminate as early as possible, rather than waiting for the request to complete: certain parent goroutines (e.g. authorise, sequence) float around for the whole duration of the request even though their useful work has finished because they are being used to notify children operators of the stop message, by monitoring the stop channel, and trickling any incoming message down.
The net result is that when a request terminates, all of the goroutines in the execution tree wake up and start telling each other to go away, taking valuable execution time away from other requests.
We should have a mechanism to trickle down stop messages without relying on completed goroutines to forward messages, so that this flurry of wakes can be avoided.
Attachments
For Gerrit Dashboard: MB-26266 | ||||||
---|---|---|---|---|---|---|
# | Subject | Branch | Project | Status | CR | V |
85322,10 | MB-26266 lower load on golang kernel | master | query | Status: MERGED | +2 | +1 |
85398,2 | MB-26266 lower load on golang kernel | master | query | Status: MERGED | +2 | +1 |
87018,6 | MB-26266 serialize certain low cost operators | master | query | Status: MERGED | +2 | +1 |