Details
-
Bug
-
Resolution: Fixed
-
Critical
-
7.0.0
-
Untriaged
-
1
-
No
Description
Backfill request handler's cooldown mechanism is flawed such that it could cause the run() routine to run the persist case when there is no operations to persist.
This can lead to the handler stuck forever waiting for an operation that will never come, and all backfill operations will be unresponsive... such as handling VB done events, or raising future backfills, etc
A typical symptom would be a backfill pipeline that hangs and doesn't go away (potentially with changes_left staying at 0)
The stack trace would show a bunch of go-routines doing HandleVBTaskDone(), (each one per VB), and one go routine stuck at this location:
// No more incoming requests - done bursting handling, do a single metakv operation
|
select {
|
case persistType := <-b.persistenceNeededCh:
|
err := b.metaKvOp(persistType)
|
Attachments
Issue Links
- is a backport of
-
MB-47773 XDCR - backfill_request_handler could hang forever
- Closed