Details
-
Improvement
-
Resolution: Fixed
-
Major
-
Cheshire-Cat
-
1
-
KV-Engine 2021-Feb
Description
Related to MB-43818 / MB-43373, with the introduction of FollyExecutorPool we no longer have a minidump of where a fatal exception was thrown from, only where it was caught.
This is due to the fact that folly::ThreadPoolExecutor::runTask() has a catch(...) statement inside it, meaning uncaught exceptions are caught (and logged). This is in contrast to CB3ExecutorPool, which didn't have any catch blocks and hence such uncaught exceptions would trigger std::terminate, with a backtrace of where the exception was thrown.
To partly address this, in MB-43373 we added our own catch(...) inside GlobalTask::execute to avoid the Folly one which calls std::terminate. This restores the fatal behaviour; but leaves the backtrace pointing at where std::terminate was called - the frames relating to where the exception was thrown are lost.
Additionally, MB-43818 was raised to explicitly record the backtrace of select thrown exceptions via a new throwWithTrace function; which are printed (if present) as part of our terminate_handler. This restores a textual backtrace of where the exception was thrown from.
However, a couple of recent bugs (and their investigation) has highlighted the deficiencies in this approach - while we can see where the (fatal) exception was thrown from in terms of code lines, we have lost all additional context which was previous present in the minidump - local variables, stack variables, and some globals. This is a not-insignificant loss of information; particularly when considering customer-encountered bugs where we really need as much information as possible.