Details
-
Bug
-
Resolution: Fixed
-
Major
-
4.5.0
-
None
-
Untriaged
-
No
Description
Spotted on CV, I've not tracked this back to anything in the code, the stack-trace on thread 9 would be useful and also DEBUG=1.
Issue occured on CV running ep-engine_basic_tests for the following commit - dc1efdba7f565c0bbdf2b4d023465117a19c8090 (http://review.couchbase.org/#/c/60787/)
Full log - http://cv.jenkins.couchbase.com/job/ep-engine-threadsanitizer-watson/11/console
05:37:10 Running [0044/0058]: set/delete (invalid cas) (couchstore)...(1 sec) OK
|
05:37:10 Running [0045/0058]: delete/set/delete (couchstore)...(0 sec) OK
|
05:37:10 Running [0046/0058]: get/delete with missing db file (couchstore)...(1 sec) OK
|
05:37:10 Running [0047/0058]: retain rowid over a soft delete (couchstore)...(4 sec) OK
|
05:37:10 Memory tracker not enabled ...==================
|
05:37:10 WARNING: ThreadSanitizer: data race (pid=89769)
|
05:37:10 Write of size 8 at 0x7d10000b4510 by thread T13 (mutexes: write M15394):
|
05:37:10 #0 operator delete(void*) <null> (engine_testapp+0x0000004648db)
|
05:37:10 #1 <null> <null> (libstdc++.so.6+0x0000000c1ac7)
|
05:37:10 #2 _ZN12CouchKVStore15snapshotVBucketEtR13vbucket_stateP8CallbackIJ10KVStatsCtxEEb /home/couchbase/cvjenkins/workspace/ep-engine-threadsanitizer-watson/ep-engine/src/couch-kvstore/couch-kvstore.cc:980 (ep.so+0x00000016f867)
|
05:37:10 #3 EventuallyPersistentStore::persistVBState(Priority const&, unsigned short) /home/couchbase/cvjenkins/workspace/ep-engine-threadsanitizer-watson/ep-engine/src/ep.cc:1271 (ep.so+0x00000008d62f)
|
05:37:10 #4 VBStatePersistTask::run() /home/couchbase/cvjenkins/workspace/ep-engine-threadsanitizer-watson/ep-engine/src/tasks.cc:88 (ep.so+0x000000137dd6)
|
05:37:10 #5 ExecutorThread::run() /home/couchbase/cvjenkins/workspace/ep-engine-threadsanitizer-watson/ep-engine/src/executorthread.cc:115 (ep.so+0x0000000f7c36)
|
05:37:10 #6 launch_executor_thread(void*) /home/couchbase/cvjenkins/workspace/ep-engine-threadsanitizer-watson/ep-engine/src/executorthread.cc:33 (ep.so+0x0000000f77e5)
|
05:37:10 #7 platform_thread_wrap(void*) /home/couchbase/cvjenkins/workspace/ep-engine-threadsanitizer-watson/platform/src/cb_pthreads.cc:54 (libplatform.so.0.1.0+0x00000000568b)
|
05:37:10
|
05:37:10 Previous atomic write of size 4 at 0x7d10000b4510 by thread T9:
|
05:37:10 [failed to restore the stack]
|
05:37:10
|
05:37:10 Mutex M15394 (0x7db40000a000) created at:
|
05:37:10 #0 pthread_mutex_trylock <null> (engine_testapp+0x000000468f80)
|
05:37:10 #1 EventuallyPersistentStore::persistVBState(Priority const&, unsigned short) /usr/bin/../lib/gcc/x86_64-linux-gnu/4.9/../../../../include/x86_64-linux-gnu/c++/4.9/bits/gthr-default.h:757 (ep.so+0x00000008d2b8)
|
05:37:10 #2 VBStatePersistTask::run() /home/couchbase/cvjenkins/workspace/ep-engine-threadsanitizer-watson/ep-engine/src/tasks.cc:88 (ep.so+0x000000137dd6)
|
05:37:10 #3 ExecutorThread::run() /home/couchbase/cvjenkins/workspace/ep-engine-threadsanitizer-watson/ep-engine/src/executorthread.cc:115 (ep.so+0x0000000f7c36)
|
05:37:10 #4 launch_executor_thread(void*) /home/couchbase/cvjenkins/workspace/ep-engine-threadsanitizer-watson/ep-engine/src/executorthread.cc:33 (ep.so+0x0000000f77e5)
|
05:37:10 #5 platform_thread_wrap(void*) /home/couchbase/cvjenkins/workspace/ep-engine-threadsanitizer-watson/platform/src/cb_pthreads.cc:54 (libplatform.so.0.1.0+0x00000000568b)
|
05:37:10
|
05:37:10 Thread T13 'mc:writer_12' (tid=89785, running) created by main thread at:
|
05:37:10 #0 pthread_create <null> (engine_testapp+0x0000004677d1)
|
05:37:10 #1 cb_create_named_thread /home/couchbase/cvjenkins/workspace/ep-engine-threadsanitizer-watson/platform/src/cb_pthreads.cc:104 (libplatform.so.0.1.0+0x0000000054e5)
|
05:37:10 #2 ExecutorThread::start() /home/couchbase/cvjenkins/workspace/ep-engine-threadsanitizer-watson/ep-engine/src/executorthread.cc:46 (ep.so+0x0000000f7187)
|
05:37:10 #3 ExecutorPool::_startWorkers() /home/couchbase/cvjenkins/workspace/ep-engine-threadsanitizer-watson/ep-engine/src/executorpool.cc:515 (ep.so+0x0000000f0aea)
|
05:37:10 #4 ExecutorPool::_registerTaskable(Taskable&) /home/couchbase/cvjenkins/workspace/ep-engine-threadsanitizer-watson/ep-engine/src/executorpool.cc:479 (ep.so+0x0000000f0159)
|
05:37:10 #5 ExecutorPool::registerTaskable(Taskable&) /home/couchbase/cvjenkins/workspace/ep-engine-threadsanitizer-watson/ep-engine/src/executorpool.cc:484 (ep.so+0x0000000f238e)
|
05:37:10 #6 EventuallyPersistentStore::EventuallyPersistentStore(EventuallyPersistentEngine&) /home/couchbase/cvjenkins/workspace/ep-engine-threadsanitizer-watson/ep-engine/src/ep.cc:301 (ep.so+0x00000008332a)
|
05:37:10 #7 EventuallyPersistentEngine::initialize(char const*) /home/couchbase/cvjenkins/workspace/ep-engine-threadsanitizer-watson/ep-engine/src/ep_engine.cc:2132 (ep.so+0x0000000b828e)
|
05:37:10 #8 EvpInitialize(engine_interface*, char const*) /home/couchbase/cvjenkins/workspace/ep-engine-threadsanitizer-watson/ep-engine/src/ep_engine.cc:143 (ep.so+0x0000000b5bc8)
|
05:37:10 #9 init_engine_instance <null> (libmcd_util.so.1.0.0+0x00000000588b)
|
05:37:10 #10 create_bucket(bool, char const*) /home/couchbase/cvjenkins/workspace/ep-engine-threadsanitizer-watson/memcached/programs/engine_testapp/engine_testapp.cc:982 (engine_testapp+0x0000004cd4e7)
|
05:37:10 #11 execute_test(test, char const*, char const*) /home/couchbase/cvjenkins/workspace/ep-engine-threadsanitizer-watson/memcached/programs/engine_testapp/engine_testapp.cc:1128 (engine_testapp+0x0000004cc5c8)
|
05:37:10 #12 __libc_start_main /build/buildd/eglibc-2.15/csu/libc-start.c:226 (libc.so.6+0x00000002176c)
|
05:37:10
|
05:37:10 Thread T9 'mc:writer_8' (tid=89781, running) created by main thread at:
|
05:37:10 #0 pthread_create <null> (engine_testapp+0x0000004677d1)
|
05:37:10 #1 cb_create_named_thread /home/couchbase/cvjenkins/workspace/ep-engine-threadsanitizer-watson/platform/src/cb_pthreads.cc:104 (libplatform.so.0.1.0+0x0000000054e5)
|
05:37:10 #2 ExecutorThread::start() /home/couchbase/cvjenkins/workspace/ep-engine-threadsanitizer-watson/ep-engine/src/executorthread.cc:46 (ep.so+0x0000000f7187)
|
05:37:10 #3 ExecutorPool::_startWorkers() /home/couchbase/cvjenkins/workspace/ep-engine-threadsanitizer-watson/ep-engine/src/executorpool.cc:515 (ep.so+0x0000000f0aea)
|
05:37:10 #4 ExecutorPool::_registerTaskable(Taskable&) /home/couchbase/cvjenkins/workspace/ep-engine-threadsanitizer-watson/ep-engine/src/executorpool.cc:479 (ep.so+0x0000000f0159)
|
05:37:10 #5 ExecutorPool::registerTaskable(Taskable&) /home/couchbase/cvjenkins/workspace/ep-engine-threadsanitizer-watson/ep-engine/src/executorpool.cc:484 (ep.so+0x0000000f238e)
|
05:37:10 #6 EventuallyPersistentStore::EventuallyPersistentStore(EventuallyPersistentEngine&) /home/couchbase/cvjenkins/workspace/ep-engine-threadsanitizer-watson/ep-engine/src/ep.cc:301 (ep.so+0x00000008332a)
|
05:37:10 #7 EventuallyPersistentEngine::initialize(char const*) /home/couchbase/cvjenkins/workspace/ep-engine-threadsanitizer-watson/ep-engine/src/ep_engine.cc:2132 (ep.so+0x0000000b828e)
|
05:37:10 #8 EvpInitialize(engine_interface*, char const*) /home/couchbase/cvjenkins/workspace/ep-engine-threadsanitizer-watson/ep-engine/src/ep_engine.cc:143 (ep.so+0x0000000b5bc8)
|
05:37:10 #9 init_engine_instance <null> (libmcd_util.so.1.0.0+0x00000000588b)
|
05:37:10 #10 create_bucket(bool, char const*) /home/couchbase/cvjenkins/workspace/ep-engine-threadsanitizer-watson/memcached/programs/engine_testapp/engine_testapp.cc:982 (engine_testapp+0x0000004cd4e7)
|
05:37:10 #11 execute_test(test, char const*, char const*) /home/couchbase/cvjenkins/workspace/ep-engine-threadsanitizer-watson/memcached/programs/engine_testapp/engine_testapp.cc:1128 (engine_testapp+0x0000004cc5c8)
|
05:37:10 #12 __libc_start_main /build/buildd/eglibc-2.15/csu/libc-start.c:226 (libc.so.6+0x00000002176c)
|
05:37:10
|
05:37:10 SUMMARY: ThreadSanitizer: data race ??:0 operator delete(void*)
|
05:37:10 ==================
|
05:37:10 Running [0048/0058]: vbucket deletion doesn't affect new data (couchstore)...(7 sec) UNKNOWN
|
05:37:10 ThreadSanitizer: reported 1 warnings
|
05:37:10 Running [0049/0058]: non-resident decrementers (couchstore)...(1 sec) OK
|
05:37:10 Running [0050/0058]: resident ratio after warmup (couchstore)...(0 sec) OK
|