Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-41916

TSAN data race between EventuallyPersistentEngine::initialize & Environment::getMaxBackendFileDescriptors

    XMLWordPrintable

Details

    • Triaged
    • 1
    • Unknown

    Description

      15:08:42  Running [0053/0158]: seqno stats...==================
      15:08:42  WARNING: ThreadSanitizer: data race (pid=4759)
      15:08:42    Write of size 8 at 0x7f111c636f68 by main thread:
      15:08:42      #0 EventuallyPersistentEngine::initialize(char const*) /home/couchbase/jenkins/workspace/kv_engine.threadsanitizer_master/build/../kv_engine/engines/ep/src/ep_engine.cc:2163:31 (libep.so+0x2e7e68)
      15:08:42      #1 MockTestHarness::create_bucket(bool, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) /home/couchbase/jenkins/workspace/kv_engine.threadsanitizer_master/build/../kv_engine/programs/engine_testapp/engine_testapp.cc:212:52 (ep_testsuite+0x5741f7)
      15:08:42      #2 execute_test(test, char const*, char const*) /home/couchbase/jenkins/workspace/kv_engine.threadsanitizer_master/build/../kv_engine/programs/engine_testapp/engine_testapp.cc:366:43 (ep_testsuite+0x573195)
      15:08:42      #3 main /home/couchbase/jenkins/workspace/kv_engine.threadsanitizer_master/build/../kv_engine/programs/engine_testapp/engine_testapp.cc:629:37 (ep_testsuite+0x57261b)
      15:08:42  
      15:08:42    Previous read of size 8 at 0x7f111c636f68 by thread T24 (mutexes: write M733):
      15:08:42      #0 Environment::getMaxBackendFileDescriptors() const /home/couchbase/jenkins/workspace/kv_engine.threadsanitizer_master/build/../kv_engine/engines/ep/src/environment.cc:31:31 (libep.so+0x34cae8)
      15:08:42      #1 CouchKVStoreFileCache::Handle::resize(unsigned long) /home/couchbase/jenkins/workspace/kv_engine.threadsanitizer_master/build/../kv_engine/engines/ep/src/couch-kvstore/couch-kvstore-file-cache.cc:73:40 (libep.so+0x4c61cb)
      15:08:42      #2 CouchKVFileHandle::~CouchKVFileHandle() /home/couchbase/jenkins/workspace/kv_engine.threadsanitizer_master/build/../kv_engine/engines/ep/src/couch-kvstore/couch-kvstore.h:132:17 (libep.so+0x4b9518)
      15:08:42      #3 CouchKVFileHandle::~CouchKVFileHandle() /home/couchbase/jenkins/workspace/kv_engine.threadsanitizer_master/build/../kv_engine/engines/ep/src/couch-kvstore/couch-kvstore.h:129:35 (libep.so+0x4b9585)
      15:08:42      #4 std::default_delete<KVFileHandle>::operator()(KVFileHandle*) const /usr/bin/../lib/gcc/x86_64-linux-gnu/7.3.0/../../../../include/c++/7.3.0/bits/unique_ptr.h:78:2 (libep.so+0x2c05c3)
      15:08:42      #5 std::unique_ptr<KVFileHandle, std::default_delete<KVFileHandle> >::~unique_ptr() /usr/bin/../lib/gcc/x86_64-linux-gnu/7.3.0/../../../../include/c++/7.3.0/bits/unique_ptr.h:268:4 (libep.so+0x2c0517)
      15:08:42      #6 ScanContext::~ScanContext() /home/couchbase/jenkins/workspace/kv_engine.threadsanitizer_master/build/../kv_engine/engines/ep/src/kvstore.h:229:5 (libep.so+0x38e1bc)
      15:08:42      #7 BySeqnoScanContext::~BySeqnoScanContext() /home/couchbase/jenkins/workspace/kv_engine.threadsanitizer_master/build/../kv_engine/engines/ep/src/kvstore.h:259:7 (libep.so+0x38e1e5)
      15:08:42      #8 std::default_delete<BySeqnoScanContext>::operator()(BySeqnoScanContext*) const /usr/bin/../lib/gcc/x86_64-linux-gnu/7.3.0/../../../../include/c++/7.3.0/bits/unique_ptr.h:78:2 (libep.so+0x1f7873)
      15:08:42      #9 std::unique_ptr<BySeqnoScanContext, std::default_delete<BySeqnoScanContext> >::~unique_ptr() /usr/bin/../lib/gcc/x86_64-linux-gnu/7.3.0/../../../../include/c++/7.3.0/bits/unique_ptr.h:268:4 (libep.so+0x1f6777)
      15:08:42      #10 Warmup::loadKVPairsforShard(unsigned short) /home/couchbase/jenkins/workspace/kv_engine.threadsanitizer_master/build/../kv_engine/engines/ep/src/warmup.cc:1531:5 (libep.so+0x44127f)
      15:08:42      #11 WarmupLoadingKVPairs::run() /home/couchbase/jenkins/workspace/kv_engine.threadsanitizer_master/build/../kv_engine/engines/ep/src/warmup.cc:476:18 (libep.so+0x45927d)
      15:08:42      #12 GlobalTask::execute() /home/couchbase/jenkins/workspace/kv_engine.threadsanitizer_master/build/../kv_engine/engines/ep/src/globaltask.cc:73:12 (libep.so+0x3708e9)
      15:08:42      #13 CB3ExecutorThread::run() /home/couchbase/jenkins/workspace/kv_engine.threadsanitizer_master/build/../kv_engine/engines/ep/src/cb3_executorthread.cc:174:39 (libep.so+0x190834)
      15:08:42      #14 launch_executor_thread(void*) /home/couchbase/jenkins/workspace/kv_engine.threadsanitizer_master/build/../kv_engine/engines/ep/src/cb3_executorthread.cc:34:15 (libep.so+0x190205)
      15:08:42      #15 CouchbaseThread::run() /home/couchbase/jenkins/workspace/kv_engine.threadsanitizer_master/build/../platform/src/cb_pthreads.cc:58:9 (libplatform_so.so.0.1.0+0x2c05a)
      15:08:42      #16 platform_thread_wrap(void*) /home/couchbase/jenkins/workspace/kv_engine.threadsanitizer_master/build/../platform/src/cb_pthreads.cc:71:14 (libplatform_so.so.0.1.0+0x2a99e)
      15:08:42  
      15:08:42    Location is global 'Environment::get()::env' of size 8 at 0x7f111c636f68 (libep.so+0x000000bd8f68)
      15:08:42  
      15:08:42    Mutex M733 (0x7f111c637e88) created at:
      15:08:42      #0 pthread_mutex_lock <null> (ep_testsuite+0x4d6a76)
      15:08:42      #1 __gthread_mutex_lock(pthread_mutex_t*) /usr/bin/../lib/gcc/x86_64-linux-gnu/7.3.0/../../../../include/x86_64-linux-gnu/c++/7.3.0/bits/gthr-default.h:748:12 (ep_testsuite+0x55fc03)
      15:08:42      #2 std::mutex::lock() /usr/bin/../lib/gcc/x86_64-linux-gnu/7.3.0/../../../../include/c++/7.3.0/bits/std_mutex.h:103:17 (ep_testsuite+0x565095)
      15:08:42      #3 folly::detail::LockTraitsImpl<std::mutex, (folly::detail::MutexLevel)0, false>::lock(std::mutex&) /home/couchbase/jenkins/workspace/kv_engine.threadsanitizer_master/build/tlm/deps/folly.exploded/include/folly/LockTraits.h:124:11 (libep.so+0x37b675)
      15:08:42      #4 std::integral_constant<bool, true> folly::LockPolicyExclusive::lock<std::mutex>(std::mutex&) /home/couchbase/jenkins/workspace/kv_engine.threadsanitizer_master/build/tlm/deps/folly.exploded/include/folly/LockTraits.h:479:5 (libep.so+0x37b5f5)
      15:08:42      #5 folly::LockedPtrBase<folly::Synchronized<CouchKVStoreFileCache::Handle, std::mutex>, std::mutex, folly::LockPolicyExclusive>::LockedPtrBase(folly::Synchronized<CouchKVStoreFileCache::Handle, std::mutex>*) /home/couchbase/jenkins/workspace/kv_engine.threadsanitizer_master/build/tlm/deps/folly.exploded/include/folly/Synchronized.h:1283:10 (libep.so+0x4a34c9)
      15:08:42      #6 folly::LockedPtr<folly::Synchronized<CouchKVStoreFileCache::Handle, std::mutex>, folly::LockPolicyExclusive>::LockedPtr(folly::Synchronized<CouchKVStoreFileCache::Handle, std::mutex>*) /home/couchbase/jenkins/workspace/kv_engine.threadsanitizer_master/build/tlm/deps/folly.exploded/include/folly/Synchronized.h:1402:50 (libep.so+0x4a345e)
      15:08:42      #7 folly::SynchronizedBase<folly::Synchronized<CouchKVStoreFileCache::Handle, std::mutex>, (folly::detail::MutexLevel)0>::lock() /home/couchbase/jenkins/workspace/kv_engine.threadsanitizer_master/build/tlm/deps/folly.exploded/include/folly/Synchronized.h:384:12 (libep.so+0x4a341e)
      15:08:42      #8 CouchKVStoreFileCache::getHandle() /home/couchbase/jenkins/workspace/kv_engine.threadsanitizer_master/build/../kv_engine/engines/ep/src/couch-kvstore/couch-kvstore-file-cache.h:78:22 (libep.so+0x49cc3e)
      15:08:42      #9 CouchKVStoreConfig::setCouchstoreFileCacheMaxSize(unsigned long) /home/couchbase/jenkins/workspace/kv_engine.threadsanitizer_master/build/../kv_engine/engines/ep/src/couch-kvstore/couch-kvstore-config.cc:86:34 (libep.so+0x4c5043)
      15:08:42      #10 CouchKVStoreConfig::CouchKVStoreConfig(Configuration&, unsigned short, unsigned short) /home/couchbase/jenkins/workspace/kv_engine.threadsanitizer_master/build/../kv_engine/engines/ep/src/couch-kvstore/couch-kvstore-config.cc:67:5 (libep.so+0x4c4f06)
      15:08:42      #11 std::_MakeUniq<CouchKVStoreConfig>::__single_object std::make_unique<CouchKVStoreConfig, Configuration&, unsigned long const&, unsigned short&>(Configuration&, unsigned long const&, unsigned short&) /usr/bin/../lib/gcc/x86_64-linux-gnu/7.3.0/../../../../include/c++/7.3.0/bits/unique_ptr.h:825:34 (libep.so+0x3c3532)
      15:08:42      #12 KVShard::KVShard(EventuallyPersistentEngine&, unsigned short) /home/couchbase/jenkins/workspace/kv_engine.threadsanitizer_master/build/../kv_engine/engines/ep/src/kvshard.cc:48:20 (libep.so+0x3c1d73)
      15:08:42      #13 std::_MakeUniq<KVShard>::__single_object std::make_unique<KVShard, EventuallyPersistentEngine&, unsigned long&>(EventuallyPersistentEngine&, unsigned long&) /usr/bin/../lib/gcc/x86_64-linux-gnu/7.3.0/../../../../include/c++/7.3.0/bits/unique_ptr.h:825:34 (libep.so+0x435a04)
      15:08:42      #14 VBucketMap::VBucketMap(Configuration&, KVBucket&) /home/couchbase/jenkins/workspace/kv_engine.threadsanitizer_master/build/../kv_engine/engines/ep/src/vbucketmap.cc:30:26 (libep.so+0x43435d)
      15:08:42      #15 KVBucket::KVBucket(EventuallyPersistentEngine&) /home/couchbase/jenkins/workspace/kv_engine.threadsanitizer_master/build/../kv_engine/engines/ep/src/kv_bucket.cc:292:7 (libep.so+0x393ea6)
      15:08:42      #16 EPBucket::EPBucket(EventuallyPersistentEngine&) /home/couchbase/jenkins/workspace/kv_engine.threadsanitizer_master/build/../kv_engine/engines/ep/src/ep_bucket.cc:257:7 (libep.so+0x2a0e1b)
      15:08:42      #17 std::_MakeUniq<EPBucket>::__single_object std::make_unique<EPBucket, EventuallyPersistentEngine&>(EventuallyPersistentEngine&) /usr/bin/../lib/gcc/x86_64-linux-gnu/7.3.0/../../../../include/c++/7.3.0/bits/unique_ptr.h:825:34 (libep.so+0x2ff504)
      15:08:42      #18 EventuallyPersistentEngine::makeBucket(Configuration&) /home/couchbase/jenkins/workspace/kv_engine.threadsanitizer_master/build/../kv_engine/engines/ep/src/ep_engine.cc:6531:16 (libep.so+0x2e8a6b)
      15:08:42      #19 EventuallyPersistentEngine::initialize(char const*) /home/couchbase/jenkins/workspace/kv_engine.threadsanitizer_master/build/../kv_engine/engines/ep/src/ep_engine.cc:2242:16 (libep.so+0x2e8529)
      15:08:42      #20 MockTestHarness::create_bucket(bool, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) /home/couchbase/jenkins/workspace/kv_engine.threadsanitizer_master/build/../kv_engine/programs/engine_testapp/engine_testapp.cc:212:52 (ep_testsuite+0x5741f7)
      15:08:42      #21 execute_test(test, char const*, char const*) /home/couchbase/jenkins/workspace/kv_engine.threadsanitizer_master/build/../kv_engine/programs/engine_testapp/engine_testapp.cc:366:43 (ep_testsuite+0x573195)
      15:08:42      #22 main /home/couchbase/jenkins/workspace/kv_engine.threadsanitizer_master/build/../kv_engine/programs/engine_testapp/engine_testapp.cc:629:37 (ep_testsuite+0x57261b)
      15:08:42  
      15:08:42    Thread T24 'mc:Reader_3' (tid=6751, running) created by main thread at:
      15:08:42      #0 pthread_create <null> (ep_testsuite+0x4b881b)
      15:08:42      #1 cb_create_named_thread(unsigned long*, void (*)(void*), void*, int, char const*) /home/couchbase/jenkins/workspace/kv_engine.threadsanitizer_master/build/../platform/src/cb_pthreads.cc:109:11 (libplatform_so.so.0.1.0+0x2a8ee)
      15:08:42      #2 CB3ExecutorThread::start() /home/couchbase/jenkins/workspace/kv_engine.threadsanitizer_master/build/../kv_engine/engines/ep/src/cb3_executorthread.cc:51:9 (libep.so+0x19007c)
      15:08:42      #3 CB3ExecutorPool::_adjustWorkers(task_type_t, unsigned long) /home/couchbase/jenkins/workspace/kv_engine.threadsanitizer_master/build/../kv_engine/engines/ep/src/cb3_executorpool.cc:465:33 (libep.so+0x16c26b)
      15:08:42      #4 CB3ExecutorPool::_startWorkers() /home/couchbase/jenkins/workspace/kv_engine.threadsanitizer_master/build/../kv_engine/engines/ep/src/cb3_executorpool.cc:525:5 (libep.so+0x16c8b5)
      15:08:42      #5 CB3ExecutorPool::_registerTaskable(Taskable&) /home/couchbase/jenkins/workspace/kv_engine.threadsanitizer_master/build/../kv_engine/engines/ep/src/cb3_executorpool.cc:419:5 (libep.so+0x16bf96)
      15:08:42      #6 CB3ExecutorPool::registerTaskable(Taskable&) /home/couchbase/jenkins/workspace/kv_engine.threadsanitizer_master/build/../kv_engine/engines/ep/src/cb3_executorpool.cc:424:5 (libep.so+0x16c006)
      15:08:42      #7 KVBucket::KVBucket(EventuallyPersistentEngine&) /home/couchbase/jenkins/workspace/kv_engine.threadsanitizer_master/build/../kv_engine/engines/ep/src/kv_bucket.cc:323:26 (libep.so+0x39430d)
      15:08:42      #8 EPBucket::EPBucket(EventuallyPersistentEngine&) /home/couchbase/jenkins/workspace/kv_engine.threadsanitizer_master/build/../kv_engine/engines/ep/src/ep_bucket.cc:257:7 (libep.so+0x2a0e1b)
      15:08:42      #9 std::_MakeUniq<EPBucket>::__single_object std::make_unique<EPBucket, EventuallyPersistentEngine&>(EventuallyPersistentEngine&) /usr/bin/../lib/gcc/x86_64-linux-gnu/7.3.0/../../../../include/c++/7.3.0/bits/unique_ptr.h:825:34 (libep.so+0x2ff504)
      15:08:42      #10 EventuallyPersistentEngine::makeBucket(Configuration&) /home/couchbase/jenkins/workspace/kv_engine.threadsanitizer_master/build/../kv_engine/engines/ep/src/ep_engine.cc:6531:16 (libep.so+0x2e8a6b)
      15:08:42      #11 EventuallyPersistentEngine::initialize(char const*) /home/couchbase/jenkins/workspace/kv_engine.threadsanitizer_master/build/../kv_engine/engines/ep/src/ep_engine.cc:2242:16 (libep.so+0x2e8529)
      15:08:42      #12 MockTestHarness::create_bucket(bool, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) /home/couchbase/jenkins/workspace/kv_engine.threadsanitizer_master/build/../kv_engine/programs/engine_testapp/engine_testapp.cc:212:52 (ep_testsuite+0x5741f7)
      15:08:42      #13 MockTestHarness::reload_engine(EngineIface**, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool, bool) /home/couchbase/jenkins/workspace/kv_engine.threadsanitizer_master/build/../kv_engine/programs/engine_testapp/engine_testapp.cc:250:36 (ep_testsuite+0x57439b)
      15:08:42      #14 test_warmup_with_threshold(EngineIface*) /home/couchbase/jenkins/workspace/kv_engine.threadsanitizer_master/build/../kv_engine/engines/ep/tests/ep_testsuite.cc:3675:18 (ep_testsuite+0x5c1219)
      15:08:42      #15 execute_test(test, char const*, char const*) /home/couchbase/jenkins/workspace/kv_engine.threadsanitizer_master/build/../kv_engine/programs/engine_testapp/engine_testapp.cc:378:19 (ep_testsuite+0x57321a)
      15:08:42      #16 main /home/couchbase/jenkins/workspace/kv_engine.threadsanitizer_master/build/../kv_engine/programs/engine_testapp/engine_testapp.cc:629:37 (ep_testsuite+0x57261b)
      15:08:42  
      15:08:42  SUMMARY: ThreadSanitizer: data race /home/couchbase/jenkins/workspace/kv_engine.threadsanitizer_master/build/../kv_engine/engines/ep/src/ep_engine.cc:2163:31 in EventuallyPersistentEngine::initialize(char const*)
      

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            richard.demellow Richard deMellow
            richard.demellow Richard deMellow
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty