Details
-
Bug
-
Resolution: Fixed
-
Critical
-
master
-
Untriaged
-
Unknown
Description
As shown in
http://cv.jenkins.couchbase.com/job/kv_engine.ASan-UBSan/job/master/8804/console
There are repeated failures when executing the "test vbucket compact no purge" test
20:03:36 Running [0147/0154]: test vbucket compact no purge...=================================================================
|
20:03:36 ==14500==ERROR: AddressSanitizer: heap-use-after-free on address 0x6070003a2a89 at pc 0x7f216e722d32 bp 0x7f2146947070 sp 0x7f2146946820
|
20:03:36 READ of size 3 at 0x6070003a2a89 thread T4661 (mc:reader_0)
|
20:03:36 #0 0x7f216e722d31 (/home/couchbase/jenkins/workspace/kv_engine.ASan-UBSan_master/install/lib/libasan.so.4+0x75d31)
|
20:03:36 #1 0x5c5857 in std::char_traits<char>::copy(char*, char const*, unsigned long) /usr/local/include/c++/7.3.0/bits/char_traits.h:350
|
20:03:36 #2 0x5c5857 in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_S_copy(char*, char const*, unsigned long) /usr/local/include/c++/7.3.0/bits/basic_string.h:340
|
20:03:36 #3 0x5c5857 in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_S_copy_chars(char*, char const*, char const*) /usr/local/include/c++/7.3.0/bits/basic_string.h:387
|
20:03:36 #4 0x5c5857 in void std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_construct<char const*>(char const*, char const*, std::forward_iterator_tag) /usr/local/include/c++/7.3.0/bits/basic_string.tcc:225
|
20:03:36 #5 0x7f21690e5d32 in void std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_construct_aux<char const*>(char const*, char const*, std::__false_type) /usr/local/include/c++/7.3.0/bits/basic_string.h:236
|
20:03:36 #6 0x7f21690e5d32 in void std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_construct<char const*>(char const*, char const*) /usr/local/include/c++/7.3.0/bits/basic_string.h:255
|
20:03:36 #7 0x7f21690e5d32 in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::basic_string(char const*, unsigned long, std::allocator<char> const&) /usr/local/include/c++/7.3.0/bits/basic_string.h:502
|
20:03:36 #8 0x7f21690e5d32 in DiskDocKey::DiskDocKey(char const*, unsigned long) /home/couchbase/jenkins/workspace/kv_engine.ASan-UBSan_master/kv_engine/engines/ep/src/diskdockey.cc:48
|
20:03:36 #9 0x7f21699d319e in makeDiskDocKey /home/couchbase/jenkins/workspace/kv_engine.ASan-UBSan_master/kv_engine/engines/ep/src/magma-kvstore/magma-kvstore.cc:374
|
20:03:36 #10 0x7f21699fe6ec in MagmaKVStore::scan(ScanContext*) /home/couchbase/jenkins/workspace/kv_engine.ASan-UBSan_master/kv_engine/engines/ep/src/magma-kvstore/magma-kvstore.cc:1634
|
20:03:36 #11 0x7f21697adab4 in Warmup::keyDumpforShard(unsigned short) /home/couchbase/jenkins/workspace/kv_engine.ASan-UBSan_master/kv_engine/engines/ep/src/warmup.cc:1298
|
20:03:36 #12 0x7f21697f893f in WarmupKeyDump::run() /home/couchbase/jenkins/workspace/kv_engine.ASan-UBSan_master/kv_engine/engines/ep/src/warmup.cc:369
|
20:03:36 #13 0x7f216943f0ba in GlobalTask::execute() /home/couchbase/jenkins/workspace/kv_engine.ASan-UBSan_master/kv_engine/engines/ep/src/globaltask.cc:73
|
20:03:36 #14 0x7f2169407cbd in ExecutorThread::run() /home/couchbase/jenkins/workspace/kv_engine.ASan-UBSan_master/kv_engine/engines/ep/src/executorthread.cc:188
|
20:03:36 #15 0x7f216940a403 in launch_executor_thread /home/couchbase/jenkins/workspace/kv_engine.ASan-UBSan_master/kv_engine/engines/ep/src/executorthread.cc:36
|
20:03:36 #16 0x7f21620c7708 in CouchbaseThread::run() /home/couchbase/jenkins/workspace/kv_engine.ASan-UBSan_master/platform/src/cb_pthreads.cc:58
|
20:03:36 #17 0x7f21620c7708 in platform_thread_wrap /home/couchbase/jenkins/workspace/kv_engine.ASan-UBSan_master/platform/src/cb_pthreads.cc:71
|
20:03:36 #18 0x7f21602ab6b9 in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x76b9)
|
20:03:36 #19 0x7f215ffe141c in clone (/lib/x86_64-linux-gnu/libc.so.6+0x10741c)
|
20:03:36
|
20:03:36 0x6070003a2a89 is located 25 bytes inside of 76-byte region [0x6070003a2a70,0x6070003a2abc)
|
20:03:36 freed by thread T4661 (mc:reader_0) here:
|
20:03:36 #0 0x7f216e7886b0 in operator delete(void*) (/home/couchbase/jenkins/workspace/kv_engine.ASan-UBSan_master/install/lib/libasan.so.4+0xdb6b0)
|
20:03:36 #1 0x7f216181b14e in __gnu_cxx::new_allocator<char>::deallocate(char*, unsigned long) /tmp/deploy/objdir/x86_64-pc-linux-gnu/libstdc++-v3/include/ext/new_allocator.h:125
|
20:03:36 #2 0x7f216181b14e in std::allocator_traits<std::allocator<char> >::deallocate(std::allocator<char>&, char*, unsigned long) /tmp/deploy/objdir/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/alloc_traits.h:462
|
20:03:36 #3 0x7f216181b14e in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_destroy(unsigned long) /tmp/deploy/objdir/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/basic_string.h:226
|
20:03:36 #4 0x7f216181b14e in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_dispose() /tmp/deploy/objdir/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/basic_string.h:221
|
20:03:36 #5 0x7f216181b14e in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_mutate(unsigned long, unsigned long, char const*, unsigned long) /tmp/deploy/objdir/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/basic_string.tcc:327
|
20:03:36
|
20:03:36 previously allocated by thread T4661 (mc:reader_0) here:
|
20:03:36 #0 0x7f216e7879b0 in operator new(unsigned long) (/home/couchbase/jenkins/workspace/kv_engine.ASan-UBSan_master/install/lib/libasan.so.4+0xda9b0)
|
20:03:36 #1 0x7f216181b0c9 in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_mutate(unsigned long, unsigned long, char const*, unsigned long) /tmp/deploy/objdir/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/basic_string.tcc:317
|
20:03:36
|
20:03:36 Thread T4661 (mc:reader_0) created by T0 here:
|
20:03:36 #0 0x7f216e6e3a00 in __interceptor_pthread_create (/home/couchbase/jenkins/workspace/kv_engine.ASan-UBSan_master/install/lib/libasan.so.4+0x36a00)
|
20:03:36 #1 0x7f21620c71fa in cb_create_named_thread(unsigned long*, void (*)(void*), void*, int, char const*) /home/couchbase/jenkins/workspace/kv_engine.ASan-UBSan_master/platform/src/cb_pthreads.cc:109
|
20:03:36 #2 0x7f2169403afb in ExecutorThread::start() /home/couchbase/jenkins/workspace/kv_engine.ASan-UBSan_master/kv_engine/engines/ep/src/executorthread.cc:53
|
20:03:36 #3 0x7f21693cd5b5 in ExecutorPool::_adjustWorkers(task_type_t, unsigned long) /home/couchbase/jenkins/workspace/kv_engine.ASan-UBSan_master/kv_engine/engines/ep/src/executorpool.cc:500
|
20:03:36 #4 0x7f21693cf865 in ExecutorPool::_startWorkers() /home/couchbase/jenkins/workspace/kv_engine.ASan-UBSan_master/kv_engine/engines/ep/src/executorpool.cc:563
|
20:03:36 #5 0x7f21693cb263 in ExecutorPool::_registerTaskable(Taskable&) /home/couchbase/jenkins/workspace/kv_engine.ASan-UBSan_master/kv_engine/engines/ep/src/executorpool.cc:456
|
20:03:36 #6 0x7f21693cb58f in ExecutorPool::registerTaskable(Taskable&) /home/couchbase/jenkins/workspace/kv_engine.ASan-UBSan_master/kv_engine/engines/ep/src/executorpool.cc:461
|
20:03:36 #7 0x7f2169508e6c in KVBucket::KVBucket(EventuallyPersistentEngine&) /home/couchbase/jenkins/workspace/kv_engine.ASan-UBSan_master/kv_engine/engines/ep/src/kv_bucket.cc:306
|
20:03:36 #8 0x7f21691806e9 in EPBucket::EPBucket(EventuallyPersistentEngine&) /home/couchbase/jenkins/workspace/kv_engine.ASan-UBSan_master/kv_engine/engines/ep/src/ep_bucket.cc:252
|
20:03:36 #9 0x7f216924d39e in std::_MakeUniq<EPBucket>::__single_object std::make_unique<EPBucket, EventuallyPersistentEngine&>(EventuallyPersistentEngine&) /usr/local/include/c++/7.3.0/bits/unique_ptr.h:825
|
20:03:36 #10 0x7f216924d39e in EventuallyPersistentEngine::makeBucket(Configuration&) /home/couchbase/jenkins/workspace/kv_engine.ASan-UBSan_master/kv_engine/engines/ep/src/ep_engine.cc:6546
|
20:03:36 #11 0x7f216929483c in EventuallyPersistentEngine::initialize(char const*) /home/couchbase/jenkins/workspace/kv_engine.ASan-UBSan_master/kv_engine/engines/ep/src/ep_engine.cc:2137
|
20:03:36 #12 0x710aa9 in MockTestHarness::create_bucket(bool, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) /home/couchbase/jenkins/workspace/kv_engine.ASan-UBSan_master/kv_engine/programs/engine_testapp/engine_testapp.cc:217
|
20:03:36 #13 0x70ba2b in MockTestHarness::reload_engine(EngineIface**, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool, bool) /home/couchbase/jenkins/workspace/kv_engine.ASan-UBSan_master/kv_engine/programs/engine_testapp/engine_testapp.cc:241
|
20:03:36 #14 0x4bac13 in test_vbucket_compact_no_purge /home/couchbase/jenkins/workspace/kv_engine.ASan-UBSan_master/kv_engine/engines/ep/tests/ep_testsuite.cc:7906
|
20:03:36 #15 0x70678d in execute_test /home/couchbase/jenkins/workspace/kv_engine.ASan-UBSan_master/kv_engine/programs/engine_testapp/engine_testapp.cc:356
|
20:03:36 #16 0x709fb5 in main /home/couchbase/jenkins/workspace/kv_engine.ASan-UBSan_master/kv_engine/programs/engine_testapp/engine_testapp.cc:609
|
20:03:36 #17 0x7f215fefa82f in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2082f)
|
20:03:36
|
[2020-01-04T20:03:36.731Z] SUMMARY: AddressSanitizer: heap-use-after-free (/home/couchbase/jenkins/workspace/kv_engine.ASan-UBSan_master/install/lib/libasan.so.4+0x75d31)
|
20:03:36 Shadow bytes around the buggy address:
|
20:03:36 0x0c0e8006c500: fd fd fd fd fa fa fa fa fd fd fd fd fd fd fd fd
|
20:03:36 0x0c0e8006c510: fd fd fa fa fa fa fd fd fd fd fd fd fd fd fd fd
|
20:03:36 0x0c0e8006c520: fa fa fa fa 00 00 00 00 00 00 00 00 00 fa fa fa
|
20:03:36 0x0c0e8006c530: fa fa 00 00 00 00 00 00 00 00 00 00 fa fa fa fa
|
20:03:36 0x0c0e8006c540: fd fd fd fd fd fd fd fd fd fa fa fa fa fa fd fd
|
20:03:36 =>0x0c0e8006c550: fd[fd]fd fd fd fd fd fd fa fa fa fa fd fd fd fd
|
20:03:36 0x0c0e8006c560: fd fd fd fd fd fa fa fa fa fa 00 00 00 00 00 00
|
20:03:36 0x0c0e8006c570: 00 00 00 00 fa fa fa fa fd fd fd fd fd fd fd fd
|
20:03:36 0x0c0e8006c580: fd fa fa fa fa fa 00 00 00 00 00 00 00 00 00 00
|
20:03:36 0x0c0e8006c590: fa fa fa fa 00 00 00 00 00 00 00 00 00 03 fa fa
|
20:03:36 0x0c0e8006c5a0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
|
20:03:36 Shadow byte legend (one shadow byte represents 8 application bytes):
|
20:03:36 Addressable: 00
|
20:03:36 Partially addressable: 01 02 03 04 05 06 07
|
20:03:36 Heap left redzone: fa
|
20:03:36 Freed heap region: fd
|
20:03:36 Stack left redzone: f1
|
20:03:36 Stack mid redzone: f2
|
20:03:36 Stack right redzone: f3
|
20:03:36 Stack after return: f5
|
20:03:36 Stack use after scope: f8
|
20:03:36 Global redzone: f9
|
20:03:36 Global init order: f6
|
20:03:36 Poisoned by user: f7
|
20:03:36 Container overflow: fc
|
20:03:36 Array cookie: ac
|
20:03:36 Intra object redzone: bb
|
20:03:36 ASan internal: fe
|
20:03:36 Left alloca redzone: ca
|
20:03:36 Right alloca redzone: cb
|