Details
-
Bug
-
Resolution: Fixed
-
Critical
-
4.1.2
-
None
-
Untriaged
-
Unknown
Description
Issue originally detected in unit tests and fix raised (http://review.couchbase.org/58619), but no MB was created at the time.
Issue has since been seen in a full system test, where it caused memcached process to crash when attempting to dereference invalid memory.
Instance from unit test:
WARNING: ThreadSanitizer: heap-use-after-free (pid=8561)
|
Read of size 8 at 0x7d240000d6a8 by thread T15:
|
#0 BackfillManager::backfill() ep-engine/src/dcp/backfill-manager.cc:250 (ep.so+0x00000004f35a)
|
#1 BackfillManagerTask::run() ep-engine/src/dcp/backfill-manager.cc:43 (ep.so+0x00000004ee6f)
|
#2 ExecutorThread::run() ep-engine/src/executorthread.cc:115 (ep.so+0x0000000f1736)
|
#3 launch_executor_thread(void*) ep-engine/src/executorthread.cc:33 (ep.so+0x0000000f12e5)
|
#4 platform_thread_wrap(void*) platform/src/cb_pthreads.cc:54 (libplatform.so.0.1.0+0x00000000551b)
|
|
|
Previous write of size 8 at 0x7d240000d6a8 by thread T15:
|
#0 operator delete(void*) <null> (engine_testapp+0x00000046485b)
|
#1 DcpProducer::~DcpProducer() ep-engine/src/dcp/producer.cc:167 (ep.so+0x00000006377b)
|
#2 DcpProducer::~DcpProducer() ep-engine/src/dcp/producer.cc:165 (ep.so+0x000000063a45)
|
#3 ActiveStream::~ActiveStream() ep-engine/src/atomic.h:272 (ep.so+0x00000006ed6d)
|
#4 ActiveStream::~ActiveStream() ep-engine/src/dcp/stream.cc:200 (ep.so+0x00000006f8b5)
|
#5 BackfillManager::backfill() ep-engine/src/atomic.h:272 (ep.so+0x00000004f345)
|
#6 BackfillManagerTask::run() ep-engine/src/dcp/backfill-manager.cc:43 (ep.so+0x00000004ee6f)
|
#7 ExecutorThread::run() ep-engine/src/executorthread.cc:115 (ep.so+0x0000000f1736)
|
#8 launch_executor_thread(void*) ep-engine/src/executorthread.cc:33 (ep.so+0x0000000f12e5)
|
#9 platform_thread_wrap(void*) platform/src/cb_pthreads.cc:54 (libplatform.so.0.1.0+0x00000000551b)
|
Instance from full system (backtrace of crashing thread) - note the this ptr for DcpConnMap in frame 5 (0x2130217020201c3) is an address which doesn't occur in the processes' memory map:
#0 0x00007f92ad4ac003 in test_and_set (__m=std::memory_order_acquire, this=0x213021702021283) at /usr/include/c++/4.8.2/bits/atomic_base.h:287
|
#1 tryAcquire (this=0x213021702021283) at /ep-engine/src/atomic.cc:30
|
#2 SpinLock::acquire (this=this@entry=0x213021702021283) at /ep-engine/src/atomic.cc:36
|
#3 0x00007f92ad53b4d8 in lock (this=<synthetic pointer>) at /ep-engine/src/atomic.h:133
|
#4 SpinLockHolder (theLock=0x213021702021283, this=<synthetic pointer>) at /ep-engine/src/atomic.h:125
|
#5 DcpConnMap::decrNumActiveSnoozingBackfills (this=0x2130217020201c3) at /ep-engine/src/connmap.cc:1280
|
#6 0x00007f92ad4bd960 in BackfillManager::backfill (this=0x7f7fce16e340) at /ep-engine/src/dcp-backfill-manager.cc:247
|
#7 0x00007f92ad4bda0d in BackfillManagerTask::run (this=0x7f6709eed080) at /ep-engine/src/dcp-backfill-manager.cc:45
|
#8 0x00007f92ad523b4e in ExecutorThread::run (this=this@entry=0x7f92c3b75800) at /ep-engine/src/executorthread.cc:118
|
#9 0x00007f92ad5241b9 in launch_executor_thread (arg=0x7f92c3b75800) at /ep-engine/src/executorthread.cc:34
|
#10 0x00007f92c585ef68 in platform_thread_wrap (arg=0x7f92c27e64c0) at /platform/src/cb_pthreads.c:26
|
#11 0x00000035540079d1 in start_thread () from /lib64/libpthread.so.0
|
#12 0x0000003553ce88fd in clone () from /lib64/libc.so.6
|