Details
-
Bug
-
Resolution: Fixed
-
Major
-
6.5.0
-
Untriaged
-
Unknown
Description
Summary
ThreadSanitizer reports data races in goxdcr - using the sigar library in a non-thread-safe manner.
I haven't looked in detail, but it appears that sigar isn't thread-safe, and hence any calls into it should be guarded with external locking. However, goxdcr doesn't appear to be doing this; it's calling into sigar from multiple different threads.
Steps to Reproduce
- Build with ThreadSanitizer:
make -j8 EXTRA_CMAKE_OPTIONS="-D CB_THREADSANITIZER=1" - Run mats (or any other workload which spins up a cluster:
TSAN_OPTIONS="log_path=sanitizers.log second_deadlock_stack=1" make run-mats
Expected Results
Empty tsan.log files (no TSan issues reported).
Actual Results
Multiple TSan warnings reported from goxdcr - for example:
ThreadSanitizer: data race/home/couchbase/jenkins/workspace/kv_engine-master-post-commit-TSan/sigar/src/os/linux/linux_sigar.c:569proc_stat_read
|
|
|
Read of size 4 at 0x7b58000001a0 by thread T9:
|
#0 proc_stat_read /home/couchbase/jenkins/workspace/kv_engine-master-post-commit-TSan/sigar/src/os/linux/linux_sigar.c:569 (libsigar.so+0x0000000032b3)
|
#1 sigar_proc_time_get /home/couchbase/jenkins/workspace/kv_engine-master-post-commit-TSan/sigar/src/os/linux/linux_sigar.c:685 (libsigar.so+0x0000000074f5)
|
#2 sigar_proc_cpu_get /home/couchbase/jenkins/workspace/kv_engine-master-post-commit-TSan/sigar/src/sigar.c:148 (libsigar.so+0x000000008449)
|
#3 _cgo_7fb665c3d3e9_Cfunc_sigar_proc_cpu_get (goxdcr+0x0000009678ad)
|
|
|
Previous write of size 8 at 0x7b58000001a0 by thread T8:
|
#0 malloc (libtsan.so.0+0x0000000274e1)
|
#1 sigar_os_open /home/couchbase/jenkins/workspace/kv_engine-master-post-commit-TSan/sigar/src/os/linux/linux_sigar.c:162 (libsigar.so+0x000000005bf6)
|
#2 sigar_open /home/couchbase/jenkins/workspace/kv_engine-master-post-commit-TSan/sigar/src/sigar.c:42 (libsigar.so+0x0000000081de)
|
#3 _cgo_7fb665c3d3e9_Cfunc_sigar_open (goxdcr+0x000000967846)
|
|
|
Location is heap block of size 672 at 0x7b5800000000 allocated by thread T8:
|
#0 malloc (libtsan.so.0+0x0000000274e1)
|
#1 sigar_os_open /home/couchbase/jenkins/workspace/kv_engine-master-post-commit-TSan/sigar/src/os/linux/linux_sigar.c:162 (libsigar.so+0x000000005bf6)
|
#2 sigar_open /home/couchbase/jenkins/workspace/kv_engine-master-post-commit-TSan/sigar/src/sigar.c:42 (libsigar.so+0x0000000081de)
|
#3 _cgo_7fb665c3d3e9_Cfunc_sigar_open (goxdcr+0x000000967846)
|
|
|
Thread T9 (tid=5750, running) created by thread T7 at:
|
#0 pthread_create (libtsan.so.0+0x0000000282a0)
|
#1 _cgo_try_pthread_create (goxdcr+0x000000967b8f)
|
|
|
Thread T8 (tid=5749, running) created by thread T6 at:
|
#0 pthread_create (libtsan.so.0+0x0000000282a0)
|
#1 _cgo_try_pthread_create (goxdcr+0x000000967b8f)
|
Additional examples can be seen at: http://cv.jenkins.couchbase.com/view/Commit%20Validation/view/kv_engine/job/kv_engine-master-post-commit-TSan/38/ThreadSanitizer/
Attachments
| For Gerrit Dashboard: MB-35492 | ||||||
|---|---|---|---|---|---|---|
| # | Subject | Branch | Project | Status | CR | V |
| 113109,1 | MB-35492: goxdcr use of sigar is non-thread-safe | master | goxdcr | Status: ABANDONED | 0 | 0 |
| 113422,3 | MB-35492: TSan: Pass TSAN_FLAG to Golang CGO code | master | tlm | Status: MERGED | +2 | +1 |