Details
-
Bug
-
Resolution: Duplicate
-
Major
-
None
-
None
-
Untriaged
-
Unknown
Description
Identified by TSAN here. Issue appears when running the memcached_testapp.default.HelloTest unit test.
03:12:39 WARNING: ThreadSanitizer: data race (pid=20440)
|
03:12:39 Read of size 8 at 0x7b6000040150 by thread T6 (mutexes: write M2761, write M2706):
|
03:12:39 #0 std::chrono::duration<long, std::ratio<1l, 1000000000l> >::count() const /usr/local/include/c++/7.3.0/chrono:341 (memcached+0x000000476ecc)
|
03:12:39 #1 Connection::toJSON() const /home/couchbase/jenkins/workspace/kv_engine-threadsanitizer-master-gcc7/kv_engine/daemon/connection.cc:199 (memcached+0x000000476ecc)
|
03:12:39 #2 connection_stats(void (*)(char const*, unsigned short, char const*, unsigned int, gsl::not_null<void const*>), void const*, long) /home/couchbase/jenkins/workspace/kv_engine-threadsanitizer-master-gcc7/kv_engine/daemon/connections.cc:255 (memcached+0x00000048b1eb)
|
03:12:39 #3 stat_connections_executor /home/couchbase/jenkins/workspace/kv_engine-threadsanitizer-master-gcc7/kv_engine/daemon/protocol/mcbp/stats_context.cc:628 (memcached+0x0000004f8960)
|
03:12:39 #4 StatsCommandContext::step() /home/couchbase/jenkins/workspace/kv_engine-threadsanitizer-master-gcc7/kv_engine/daemon/protocol/mcbp/stats_context.cc:889 (memcached+0x0000004feb3c)
|
03:12:39 #5 SteppableCommandContext::drive() /home/couchbase/jenkins/workspace/kv_engine-threadsanitizer-master-gcc7/kv_engine/daemon/protocol/mcbp/steppable_command_context.cc:35 (memcached+0x000000500a72)
|
03:12:39 #6 stat_executor /home/couchbase/jenkins/workspace/kv_engine-threadsanitizer-master-gcc7/kv_engine/daemon/mcbp_executors.cc:199 (memcached+0x0000004a6b9d)
|
03:12:39 #7 std::_Function_handler<void (Cookie&), void (*)(Cookie&)>::_M_invoke(std::_Any_data const&, Cookie&) /usr/local/include/c++/7.3.0/bits/std_function.h:316 (memcached+0x0000004ad6ee)
|
03:12:39 #8 std::function<void (Cookie&)>::operator()(Cookie&) const /usr/local/include/c++/7.3.0/bits/std_function.h:706 (memcached+0x0000004adccf)
|
03:12:39 #9 execute_request_packet /home/couchbase/jenkins/workspace/kv_engine-threadsanitizer-master-gcc7/kv_engine/daemon/mcbp_executors.cc:744 (memcached+0x0000004ac0da)
|
03:12:39 #10 mcbp_execute_packet(Cookie&) /home/couchbase/jenkins/workspace/kv_engine-threadsanitizer-master-gcc7/kv_engine/daemon/mcbp_executors.cc:825 (memcached+0x0000004ac2d5)
|
03:12:39 #11 conn_execute /home/couchbase/jenkins/workspace/kv_engine-threadsanitizer-master-gcc7/kv_engine/daemon/statemachine_mcbp.cc:354 (memcached+0x00000050836c)
|
03:12:39 #12 McbpStateMachine::execute() /home/couchbase/jenkins/workspace/kv_engine-threadsanitizer-master-gcc7/kv_engine/daemon/statemachine_mcbp.cc:154 (memcached+0x000000508253)
|
03:12:39 #13 Connection::runStateMachinery() /home/couchbase/jenkins/workspace/kv_engine-threadsanitizer-master-gcc7/kv_engine/daemon/connection.cc:1351 (memcached+0x000000480a7d)
|
03:12:39 #14 Connection::runEventLoop(short) /home/couchbase/jenkins/workspace/kv_engine-threadsanitizer-master-gcc7/kv_engine/daemon/connection.cc:1406 (memcached+0x000000480c24)
|
03:12:39 #15 run_event_loop(Connection*, short) /home/couchbase/jenkins/workspace/kv_engine-threadsanitizer-master-gcc7/kv_engine/daemon/connections.cc:150 (memcached+0x00000048b4b8)
|
03:12:39 #16 event_handler(int, short, void*) /home/couchbase/jenkins/workspace/kv_engine-threadsanitizer-master-gcc7/kv_engine/daemon/memcached.cc:946 (memcached+0x000000418e5b)
|
03:12:39 #17 event_process_active_single_queue.isra.26 <null> (libevent_core.so.2.1.8+0x00000001aa23)
|
03:12:39 #18 CouchbaseThread::run() /home/couchbase/jenkins/workspace/kv_engine-threadsanitizer-master-gcc7/platform/src/cb_pthreads.cc:59 (libplatform_so.so.0.1.0+0x000000009655)
|
03:12:39 #19 platform_thread_wrap /home/couchbase/jenkins/workspace/kv_engine-threadsanitizer-master-gcc7/platform/src/cb_pthreads.cc:72 (libplatform_so.so.0.1.0+0x000000009655)
|
03:12:39 #20 <null> <null> (libtsan.so.0+0x000000024feb)
|
03:12:39
|
03:12:39 Previous write of size 8 at 0x7b6000040150 by thread T5 (mutexes: write M2751):
|
03:12:39 #0 std::chrono::duration<long, std::ratio<1l, 1000000000l> >::operator+=(std::chrono::duration<long, std::ratio<1l, 1000000000l> > const&) /usr/local/include/c++/7.3.0/chrono:377 (memcached+0x000000475cf3)
|
03:12:39 #1 Connection::addCpuTime(std::chrono::duration<long, std::ratio<1l, 1000000000l> >) /home/couchbase/jenkins/workspace/kv_engine-threadsanitizer-master-gcc7/kv_engine/daemon/connection.cc:566 (memcached+0x000000475cf3)
|
03:12:39 #2 run_event_loop(Connection*, short) /home/couchbase/jenkins/workspace/kv_engine-threadsanitizer-master-gcc7/kv_engine/daemon/connections.cc:155 (memcached+0x00000048b4ce)
|
03:12:39 #3 event_handler(int, short, void*) /home/couchbase/jenkins/workspace/kv_engine-threadsanitizer-master-gcc7/kv_engine/daemon/memcached.cc:946 (memcached+0x000000418e5b)
|
03:12:39 #4 event_process_active_single_queue.isra.26 <null> (libevent_core.so.2.1.8+0x00000001aa23)
|
03:12:39 #5 CouchbaseThread::run() /home/couchbase/jenkins/workspace/kv_engine-threadsanitizer-master-gcc7/platform/src/cb_pthreads.cc:59 (libplatform_so.so.0.1.0+0x000000009655)
|
03:12:39 #6 platform_thread_wrap /home/couchbase/jenkins/workspace/kv_engine-threadsanitizer-master-gcc7/platform/src/cb_pthreads.cc:72 (libplatform_so.so.0.1.0+0x000000009655)
|
03:12:39 #7 <null> <null> (libtsan.so.0+0x000000024feb)
|
03:12:39
|
03:12:39 Location is heap block of size 952 at 0x7b6000040000 allocated by thread T5:
|
03:12:39 #0 operator new(unsigned long) <null> (libtsan.so.0+0x00000006a4d6)
|
03:12:39 #1 allocate_connection /home/couchbase/jenkins/workspace/kv_engine-threadsanitizer-master-gcc7/kv_engine/daemon/connections.cc:358 (memcached+0x00000048c00c)
|
03:12:39 #2 conn_new(int, unsigned short, event_base*, LIBEVENT_THREAD*) /home/couchbase/jenkins/workspace/kv_engine-threadsanitizer-master-gcc7/kv_engine/daemon/connections.cc:182 (memcached+0x00000048c3c5)
|
03:12:39 #3 dispatch_new_connections /home/couchbase/jenkins/workspace/kv_engine-threadsanitizer-master-gcc7/kv_engine/daemon/thread.cc:247 (memcached+0x00000045ad21)
|
03:12:39 #4 thread_libevent_process /home/couchbase/jenkins/workspace/kv_engine-threadsanitizer-master-gcc7/kv_engine/daemon/thread.cc:285 (memcached+0x00000045aefb)
|
03:12:39 #5 event_process_active_single_queue.isra.26 <null> (libevent_core.so.2.1.8+0x00000001aa23)
|
03:12:39 #6 CouchbaseThread::run() /home/couchbase/jenkins/workspace/kv_engine-threadsanitizer-master-gcc7/platform/src/cb_pthreads.cc:59 (libplatform_so.so.0.1.0+0x000000009655)
|
03:12:39 #7 platform_thread_wrap /home/couchbase/jenkins/workspace/kv_engine-threadsanitizer-master-gcc7/platform/src/cb_pthreads.cc:72 (libplatform_so.so.0.1.0+0x000000009655)
|
03:12:39 #8 <null> <null> (libtsan.so.0+0x000000024feb)
|
03:12:39
|
03:12:39 Mutex M2761 (0x7b78000014a8) created at:
|
03:12:39 #0 pthread_mutex_lock <null> (libtsan.so.0+0x00000003876f)
|
03:12:39 #1 __gthread_mutex_lock /usr/local/include/c++/7.3.0/x86_64-pc-linux-gnu/bits/gthr-default.h:748 (memcached+0x00000045afa2)
|
03:12:39 #2 std::mutex::lock() /usr/local/include/c++/7.3.0/bits/std_mutex.h:103 (memcached+0x00000045afa2)
|
03:12:39 #3 std::lock_guard<std::mutex>::lock_guard(std::mutex&) /usr/local/include/c++/7.3.0/bits/std_mutex.h:162 (memcached+0x00000045afa2)
|
03:12:39 #4 thread_libevent_process /home/couchbase/jenkins/workspace/kv_engine-threadsanitizer-master-gcc7/kv_engine/daemon/thread.cc:287 (memcached+0x00000045afa2)
|
03:12:39 #5 event_process_active_single_queue.isra.26 <null> (libevent_core.so.2.1.8+0x00000001aa23)
|
03:12:39 #6 CouchbaseThread::run() /home/couchbase/jenkins/workspace/kv_engine-threadsanitizer-master-gcc7/platform/src/cb_pthreads.cc:59 (libplatform_so.so.0.1.0+0x000000009655)
|
03:12:39 #7 platform_thread_wrap /home/couchbase/jenkins/workspace/kv_engine-threadsanitizer-master-gcc7/platform/src/cb_pthreads.cc:72 (libplatform_so.so.0.1.0+0x000000009655)
|
03:12:39 #8 <null> <null> (libtsan.so.0+0x000000024feb)
|
03:12:39
|
03:12:39 Mutex M2706 (0x0000019b49e0) created at:
|
03:12:39 #0 pthread_mutex_lock <null> (libtsan.so.0+0x00000003876f)
|
03:12:39 #1 __gthread_mutex_lock /usr/local/include/c++/7.3.0/x86_64-pc-linux-gnu/bits/gthr-default.h:748 (memcached+0x00000048c063)
|
03:12:39 #2 std::mutex::lock() /usr/local/include/c++/7.3.0/bits/std_mutex.h:103 (memcached+0x00000048c063)
|
03:12:39 #3 std::lock_guard<std::mutex>::lock_guard(std::mutex&) /usr/local/include/c++/7.3.0/bits/std_mutex.h:162 (memcached+0x00000048c063)
|
03:12:39 #4 allocate_connection /home/couchbase/jenkins/workspace/kv_engine-threadsanitizer-master-gcc7/kv_engine/daemon/connections.cc:359 (memcached+0x00000048c063)
|
03:12:39 #5 conn_new(int, unsigned short, event_base*, LIBEVENT_THREAD*) /home/couchbase/jenkins/workspace/kv_engine-threadsanitizer-master-gcc7/kv_engine/daemon/connections.cc:182 (memcached+0x00000048c3c5)
|
03:12:39 #6 dispatch_new_connections /home/couchbase/jenkins/workspace/kv_engine-threadsanitizer-master-gcc7/kv_engine/daemon/thread.cc:247 (memcached+0x00000045ad21)
|
03:12:39 #7 thread_libevent_process /home/couchbase/jenkins/workspace/kv_engine-threadsanitizer-master-gcc7/kv_engine/daemon/thread.cc:285 (memcached+0x00000045aefb)
|
03:12:39 #8 event_process_active_single_queue.isra.26 <null> (libevent_core.so.2.1.8+0x00000001aa23)
|
03:12:39 #9 CouchbaseThread::run() /home/couchbase/jenkins/workspace/kv_engine-threadsanitizer-master-gcc7/platform/src/cb_pthreads.cc:59 (libplatform_so.so.0.1.0+0x000000009655)
|
03:12:39 #10 platform_thread_wrap /home/couchbase/jenkins/workspace/kv_engine-threadsanitizer-master-gcc7/platform/src/cb_pthreads.cc:72 (libplatform_so.so.0.1.0+0x000000009655)
|
03:12:39 #11 <null> <null> (libtsan.so.0+0x000000024feb)
|
03:12:39
|
03:12:39 Mutex M2751 (0x7b7800001220) created at:
|
03:12:39 #0 pthread_mutex_lock <null> (libtsan.so.0+0x00000003876f)
|
03:12:39 #1 __gthread_mutex_lock /usr/local/include/c++/7.3.0/x86_64-pc-linux-gnu/bits/gthr-default.h:748 (memcached+0x00000045afa2)
|
03:12:39 #2 std::mutex::lock() /usr/local/include/c++/7.3.0/bits/std_mutex.h:103 (memcached+0x00000045afa2)
|
03:12:39 #3 std::lock_guard<std::mutex>::lock_guard(std::mutex&) /usr/local/include/c++/7.3.0/bits/std_mutex.h:162 (memcached+0x00000045afa2)
|
03:12:39 #4 thread_libevent_process /home/couchbase/jenkins/workspace/kv_engine-threadsanitizer-master-gcc7/kv_engine/daemon/thread.cc:287 (memcached+0x00000045afa2)
|
03:12:39 #5 event_process_active_single_queue.isra.26 <null> (libevent_core.so.2.1.8+0x00000001aa23)
|
03:12:39 #6 CouchbaseThread::run() /home/couchbase/jenkins/workspace/kv_engine-threadsanitizer-master-gcc7/platform/src/cb_pthreads.cc:59 (libplatform_so.so.0.1.0+0x000000009655)
|
03:12:39 #7 platform_thread_wrap /home/couchbase/jenkins/workspace/kv_engine-threadsanitizer-master-gcc7/platform/src/cb_pthreads.cc:72 (libplatform_so.so.0.1.0+0x000000009655)
|
03:12:39 #8 <null> <null> (libtsan.so.0+0x000000024feb)
|
03:12:39
|
03:12:39 Thread T6 'mc:worker_3' (tid=20448, running) created by main thread at:
|
03:12:39 #0 pthread_create <null> (libtsan.so.0+0x0000000282a0)
|
03:12:39 #1 cb_create_named_thread /home/couchbase/jenkins/workspace/kv_engine-threadsanitizer-master-gcc7/platform/src/cb_pthreads.cc:110 (libplatform_so.so.0.1.0+0x0000000093fe)
|
03:12:39 #2 create_worker /home/couchbase/jenkins/workspace/kv_engine-threadsanitizer-master-gcc7/kv_engine/daemon/thread.cc:90 (memcached+0x00000045b2c2)
|
03:12:39 #3 thread_init(unsigned long, event_base*, void (*)(int, short, void*)) /home/couchbase/jenkins/workspace/kv_engine-threadsanitizer-master-gcc7/kv_engine/daemon/thread.cc:448 (memcached+0x00000045be11)
|
03:12:39 #4 memcached_main /home/couchbase/jenkins/workspace/kv_engine-threadsanitizer-master-gcc7/kv_engine/daemon/memcached.cc:2453 (memcached+0x0000004222df)
|
03:12:39 #5 main /home/couchbase/jenkins/workspace/kv_engine-threadsanitizer-master-gcc7/kv_engine/daemon/main.cc:33 (memcached+0x0000004123c5)
|
03:12:39
|
03:12:39 Thread T5 'mc:worker_2' (tid=20447, running) created by main thread at:
|
03:12:39 #0 pthread_create <null> (libtsan.so.0+0x0000000282a0)
|
03:12:39 #1 cb_create_named_thread /home/couchbase/jenkins/workspace/kv_engine-threadsanitizer-master-gcc7/platform/src/cb_pthreads.cc:110 (libplatform_so.so.0.1.0+0x0000000093fe)
|
03:12:39 #2 create_worker /home/couchbase/jenkins/workspace/kv_engine-threadsanitizer-master-gcc7/kv_engine/daemon/thread.cc:90 (memcached+0x00000045b2c2)
|
03:12:39 #3 thread_init(unsigned long, event_base*, void (*)(int, short, void*)) /home/couchbase/jenkins/workspace/kv_engine-threadsanitizer-master-gcc7/kv_engine/daemon/thread.cc:448 (memcached+0x00000045be11)
|
03:12:39 #4 memcached_main /home/couchbase/jenkins/workspace/kv_engine-threadsanitizer-master-gcc7/kv_engine/daemon/memcached.cc:2453 (memcached+0x0000004222df)
|
03:12:39 #5 main /home/couchbase/jenkins/workspace/kv_engine-threadsanitizer-master-gcc7/kv_engine/daemon/main.cc:33 (memcached+0x0000004123c5)
|
03:12:39
|
03:12:39
|
SUMMARY: ThreadSanitizer: data race /usr/local/include/c++/7.3.0/chrono:341 in std::chrono::duration<long, std::ratio<1l, 1000000000l> >::count() const
|
There are 3 issues which are all reporting the same issue, the block of code this is coming from is when referencing time based variables to add them to a cJSON object:
|
connection.cc |
cJSON_AddStringToObject(obj,
|
"total_cpu_time",
|
std::to_string(total_cpu_time.count()).c_str());
|
cJSON_AddStringToObject(obj,
|
"min_sched_time",
|
std::to_string(min_sched_time.count()).c_str());
|
cJSON_AddStringToObject(obj,
|
"max_sched_time",
|
std::to_string(max_sched_time.count()).c_str());
|