Details
-
Bug
-
Resolution: Fixed
-
Critical
-
4.1.0
-
None
-
Untriaged
-
Unknown
Description
Seen in a customer environment (v4.1.0) during shutdown of a bucket.
Backtrace of crashing thread from minidump:
#0 0x000000308f432625 in *__GI_raise (sig=<optimized out>) at ../nptl/sysdeps/unix/sysv/linux/raise.c:64
|
#1 0x000000308f433e05 in *__GI_abort () at abort.c:92
|
#2 0x00007fddb1679d16 in cb_assert_die (expression=<optimized out>, file=<optimized out>, line=<optimized out>) at platform/src/cbassert.c:48
|
#3 0x00007fdda19ab119 in verifyEngine (engine=0x0) at ep-engine/src/objectregistry.cc:58
|
#4 ObjectRegistry::onDeleteItem (pItem=pItem@entry=0x7fbbe573acb0) at ep-engine/src/objectregistry.cc:154
|
#5 0x00007fdda18e7676 in ~Item (this=0x7fbbe573acb0, __in_chrg=<optimized out>) at ep-engine/src/item.h:412
|
#6 ~SingleThreadedRCPtr (this=0x7fbd22308730, __in_chrg=<optimized out>) at ep-engine/src/atomic.h:271
|
#7 ~_List_node (this=0x7fbd22308720, __in_chrg=<optimized out>) at /usr/include/c++/4.8.2/bits/stl_list.h:106
|
#8 destroy<std::_List_node<SingleThreadedRCPtr<Item> > > (this=<optimized out>, __p=0x7fbd22308720) at /usr/include/c++/4.8.2/ext/new_allocator.h:124
|
#9 _M_clear (this=0x7fbd2f074908) at /usr/include/c++/4.8.2/bits/list.tcc:75
|
#10 ~_List_base (this=0x7fbd2f074908, __in_chrg=<optimized out>) at /usr/include/c++/4.8.2/bits/stl_list.h:378
|
#11 ~list (this=0x7fbd2f074908, __in_chrg=<optimized out>) at /usr/include/c++/4.8.2/bits/stl_list.h:438
|
#12 Checkpoint::~Checkpoint (this=0x7fbd2f0748a0, __in_chrg=<optimized out>) at ep-engine/src/checkpoint.cc:81
|
#13 0x00007fdda18ecd19 in CheckpointManager::~CheckpointManager (this=0x7fcd9c84d298, __in_chrg=<optimized out>) at ep-engine/src/checkpoint.cc:263
|
#14 0x00007fdda19827b9 in VBucket::~VBucket (this=0x7fcd9c84d200, __in_chrg=<optimized out>) at ep-engine/src/vbucket.cc:118
|
#15 0x00007fdda18e2d34 in RCPtr<VBucket>::~RCPtr (this=0x7fbbe3bccdc8, __in_chrg=<optimized out>) at ep-engine/src/atomic.h:189
|
#16 0x00007fdda1921f88 in ~VBucketMemoryDeletionTask (this=0x7fbbe3bccd80, __in_chrg=<optimized out>) at ep-engine/src/ep.cc:152
|
#17 VBucketMemoryDeletionTask::~VBucketMemoryDeletionTask (this=0x7fbbe3bccd80, __in_chrg=<optimized out>) at ep-engine/src/ep.cc:152
|
#18 0x00007fdda19427a9 in ~SingleThreadedRCPtr (this=0x7fbbd57fcc40, __in_chrg=<optimized out>) at ep-engine/src/atomic.h:271
|
#19 ExecutorPool::_stopTaskGroup (this=0x7fdda1edaa00, e=0x7fdae3bb9e00, taskType=<optimized out>, force=<optimized out>) at ep-engine/src/executorpool.cc:536
|
#20 0x00007fdda19428c5 in ExecutorPool::stopTaskGroup (this=0x7fdda1edaa00, e=e@entry=0x7fdae3bb9e00, taskType=taskType@entry=NONIO_TASK_IDX, force=<optimized out>) at ep-engine/src/executorpool.cc:546
|
#21 0x00007fdda191b813 in EventuallyPersistentStore::~EventuallyPersistentStore (this=0x7fdb5cd81780, __in_chrg=<optimized out>) at ep-engine/src/ep.cc:389
|
#22 0x00007fdda193bebd in EventuallyPersistentEngine::~EventuallyPersistentEngine (this=0x7fdae3bb9e00, __in_chrg=<optimized out>) at ep-engine/src/ep_engine.cc:6162
|
#23 0x00007fdda193c061 in EvpDestroy (handle=0x7fdae3bb9e00, force=<optimized out>) at ep-engine/src/ep_engine.cc:141
|
#24 0x00007fddad1f2b9f in engine_shutdown_thread (arg=0x7fdadb462be0) at memcached/engines/bucket_engine/bucket_engine.c:1567
|
#25 0x00007fddb1678f68 in platform_thread_wrap (arg=0x7fc0c47a19a0) at platform/src/cb_pthreads.c:26
|
#26 0x000000308f807a51 in start_thread (arg=0x7fbbd57fd700) at pthread_create.c:301
|
#27 0x000000308f4e893d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:115
|
The crash occurred during shutdown of the data bucket. The issue looks similar (but not exactly the same) as MB-20054.
In the case of MB-20054, during ExecutorPool::unregisterBucket we deleted an Item which triggered an assert in ObjectRegistry::onDeleteItem. In this instance, the same assert was hit, but via a different method - ExecutorPool::stopTaskGroup.
