Details
-
Bug
-
Resolution: Fixed
-
Critical
-
7.6.0, 6.6.6, 7.0.0-Beta1, 7.0.0, 7.0.1, 7.0.2, 7.0.3, 7.0.4, 7.1.4, 7.0.5, 7.1.0, 7.1.1, 7.1.2, 7.2.0, 7.1.3, 7.2.1, 7.1.5, 7.2.4, 7.2.2, 7.1.6, 7.2.3, 7.2.5, 7.6.1
-
Untriaged
-
0
-
Unknown
Description
Various phases of Warmup are driven by the class WarmupVbucketVisitor this class owns a ScanContext and supports the pause/resume pattern. Pause occurs when the scanning of a vbucket hits a time threshold, e.g. 100ms.
The problem is that the ScanContext is owned by the WarmupVBucketVisitor and the management of which vbucket (file) that object refers to is done by the WarmupVBucketVisitor, e.g. once the scan reaches the end of the file or an error occurs (reset the context).
The bug here is what happens when a VBucket is deleted (set dead etc...) whilst paused, essentially the pause/resume loop which is owned by KVBucket will try to resume warmup of a vbucket, which was deleted. The next vbucket is instead selected and we call back into WarmupVBucketVisitor::visit, yet the scan context is still pointing to the previous vbucket.
E.g. assume vb:1 and vb:2 exist and warmup begins a WarmupVbucketVisitor phase, we start at vb:1
- KVBucket::pauseResume starts with vb:1
- Call WarmupVbucketVisitor::visit for vb:1
- scanContext = initScanContext(vb:1)
- scan(scanContext)
- 100ms time-up, pause
- setDead(vb:1)
- KVBucket::pauseResume tries to continue with vb:1. vb:1 dead, move to vb:2
- Call WarmupVbucketVisitor::visit for vb:2
- scan(scanContext) !!! scanContext is from vb:1
- The code in LoadStorageKVPairCallback::callback gets called back for all items in vb:1, yet it too fails to find the vb:1 object and no data is loaded
- scan reaches end and returns success
- Warmup thinks vb:2 was scanned....