Details
-
Task
-
Resolution: Done
-
Major
-
7.6.0
-
None
-
0
Description
A number of bugs have been found in the usage of GlobalTask to manage both per-bucket (ep-engine) tasks and 'global' tasks not tied to a single bucket.
Specifically, GlobalTask has two different ctors, taking either a Taskable, or an EpEngine instance. The former (Taskable) ctor is intended when the Task is not associated with a specific bucket, and hence per-bucket memory tracking should not be performed. The latter (EpEngine) ctor is intended to be used for per-bucket tasks, and sets GlobalTask::engine to a non-null value.
However, it is relatively easy to call the "wrong" ctor - there's been two instances so far (MB-58576 and MB-59022) where the Tasable ctor was called for bucket-specific tasks and hence we did failed to track memory correctly.
Moreover, GlobalTask - while now part of the "server" part of KV-Engine, still includes ep-engine specific headers, as it needs to perform memory tracking operations for ep-engine instances.
For these reasons, we should restructure GlobalTask, moving the ep-engine specific functionality into a new EpTask subclass. This includes moving GlobalTask::engine member variable to EpTask, along with the corresponding ctor, making it more difficult to accidently call the wrong ctor. This also removes all of the "engines/ep" includes from GlobalTask, creating a cleaner separation of logic.
Attachments
Issue Links
- causes
-
MB-59806 Non-bucket specific stats (e.g. ExecutorPool::doTaskQStat) do not correctly account freed memory
- Closed
- relates to
-
MB-58576 StrictQuotaItemPager does not correctly account memory (de)allocations
- Closed
-
MB-59022 VBucketSyncWriteTimeoutTask does not correctly switch engine
- Closed
-
MB-59285 ItemFreqDecayerTaskManager incorrectly accounted to first created bucket
- Closed
-
MB-59286 Warmup::warmedUpVBuckets incorrectly accounts singleton memory to current bucket
- Closed
For Gerrit Dashboard: MB-59062 | ||||||
---|---|---|---|---|---|---|
# | Subject | Branch | Project | Status | CR | V |
198771,2 | MB-59062 [1/7]: Refactor ItemPager in preparation for EpTask | master | kv_engine | Status: MERGED | +2 | +1 |
198772,3 | MB-59062 [2/7]: Introduce EpTask & EpNotifiableTask, re-parent ItemPagers | master | kv_engine | Status: MERGED | +2 | +1 |
198774,3 | MB-59062 [3/7]: Migrate ep-engine notifiable tasks to EpNotifiableTask | master | kv_engine | Status: MERGED | +2 | +1 |
198781,2 | MB-59062 [4/7]: Introduce EpLimitedConcurrencyTask, migrate ep-engine tasks | master | kv_engine | Status: MERGED | +2 | +1 |
198803,2 | MB-59062 [5/7]: Introduce EpLambdaTask | master | kv_engine | Status: MERGED | +2 | +1 |
198804,2 | MB-59062 [6/7]: Migrate remaining ep-engine tasks to EpTask | master | kv_engine | Status: MERGED | +2 | +1 |
198805,5 | MB-59062 [7/7]: Move ep-engine logic from GlobalTask to EpTask | master | kv_engine | Status: MERGED | +2 | +1 |
199574,3 | MB-59287: Restore missing periodic wakeup for ItemPager | master | kv_engine | Status: MERGED | +2 | +1 |