Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-27539

Data race observed in RocksDB PersistToTest.ConsistentStateAfterShutdown

    XMLWordPrintable

Details

    • Bug
    • Resolution: Fixed
    • Major
    • master
    • 5.5.0
    • couchbase-bucket
    • None
    • Untriaged
    • Unknown

    Description

      Data race observed in the following location: http://cv.jenkins.couchbase.com/job/kv_engine-threadsanitizer-master/5497/console

      WARNING: ThreadSanitizer: data race (pid=5300)
        Write of size 8 at 0x7d7800025ac8 by thread T16 (mutexes: write M2335):
          #0 operator delete(void*) <null> (memcached+0x0000004c9a31)
          #1 rocksdb::Version::Unref() /home/couchbase/jenkins/workspace/cbdeps-platform-build/deps/packages/build/rocksdb/rocksdb-prefix/src/rocksdb/db/version_set.cc:1702 (librocksdb.so.5.8+0x00000033e25b)
       
        Previous read of size 8 at 0x7d7800025ac8 by thread T11:
          #0 SeqnoComparator::Compare(rocksdb::Slice const&, rocksdb::Slice const&) const /home/couchbase/jenkins/workspace/kv_engine-threadsanitizer-master/kv_engine/engines/ep/src/rocksdb-kvstore/rocksdb-kvstore.h:73:29 (ep.so+0x0000001dc89d)
          #1 GetNextFile /home/couchbase/jenkins/workspace/cbdeps-platform-build/deps/packages/build/rocksdb/rocksdb-prefix/src/rocksdb/db/version_set.cc:155 (librocksdb.so.5.8+0x00000033cd73)
          #2 rocksdb::Version::Get(rocksdb::ReadOptions const&, rocksdb::LookupKey const&, rocksdb::PinnableSlice*, rocksdb::Status*, rocksdb::MergeContext*, rocksdb::RangeDelAggregator*, bool*, bool*, unsigned long*) /home/couchbase/jenkins/workspace/cbdeps-platform-build/deps/packages/build/rocksdb/rocksdb-prefix/src/rocksdb/db/version_set.cc:995 (librocksdb.so.5.8+0x00000033cd73)
          #3 RocksDBKVStore::RocksDBKVStore(KVStoreConfig&) /home/couchbase/jenkins/workspace/kv_engine-threadsanitizer-master/kv_engine/engines/ep/src/rocksdb-kvstore/rocksdb-kvstore.cc:276:13 (ep.so+0x0000001ce23b)
          #4 __unique_if<RocksDBKVStore>::__unique_single std::make_unique<RocksDBKVStore, KVStoreConfig&>(KVStoreConfig&) /home/couchbase/jenkins/workspace/kv_engine-threadsanitizer-master/platform/include/platform/make_unique.h:45:37 (ep.so+0x00000010d343)
          #5 KVStoreFactory::create(KVStoreConfig&) /home/couchbase/jenkins/workspace/kv_engine-threadsanitizer-master/kv_engine/engines/ep/src/kvstore.cc:88 (ep.so+0x00000010d343)
          #6 KVShard::KVShard(unsigned short, KVBucket&) /home/couchbase/jenkins/workspace/kv_engine-threadsanitizer-master/kv_engine/engines/ep/src/kvshard.cc:43:23 (ep.so+0x000000134c80)
          #7 __unique_if<KVShard>::__unique_single std::make_unique<KVShard, unsigned long&, KVBucket&>(unsigned long&, KVBucket&) /home/couchbase/jenkins/workspace/kv_engine-threadsanitizer-master/platform/include/platform/make_unique.h:45:37 (ep.so+0x000000172a4b)
          #8 VBucketMap::VBucketMap(Configuration&, KVBucket&) /home/couchbase/jenkins/workspace/kv_engine-threadsanitizer-master/kv_engine/engines/ep/src/vbucketmap.cc:32 (ep.so+0x000000172a4b)
          #9 KVBucket::KVBucket(EventuallyPersistentEngine&) /home/couchbase/jenkins/workspace/kv_engine-threadsanitizer-master/kv_engine/engines/ep/src/kv_bucket.cc:226:7 (ep.so+0x0000001193f3)
          #10 EPBucket::EPBucket(EventuallyPersistentEngine&) /home/couchbase/jenkins/workspace/kv_engine-threadsanitizer-master/kv_engine/engines/ep/src/ep_bucket.cc:169:7 (ep.so+0x0000000a4085)
          #11 __unique_if<EPBucket>::__unique_single std::make_unique<EPBucket, EventuallyPersistentEngine&>(EventuallyPersistentEngine&) /home/couchbase/jenkins/workspace/kv_engine-threadsanitizer-master/platform/include/platform/make_unique.h:45:37 (ep.so+0x0000000c39bf)
          #12 EventuallyPersistentEngine::makeBucket(Configuration&) /home/couchbase/jenkins/workspace/kv_engine-threadsanitizer-master/kv_engine/engines/ep/src/ep_engine.cc:5390 (ep.so+0x0000000c39bf)
          #13 EventuallyPersistentEngine::initialize(char const*) /home/couchbase/jenkins/workspace/kv_engine-threadsanitizer-master/kv_engine/engines/ep/src/ep_engine.cc:1950:16 (ep.so+0x0000000c3533)
          #14 EvpInitialize(gsl::not_null<engine_interface*>, char const*) /home/couchbase/jenkins/workspace/kv_engine-threadsanitizer-master/kv_engine/engines/ep/src/ep_engine.cc:149:35 (ep.so+0x0000000bcf3d)
          #15 EWB_Engine::initialize(gsl::not_null<engine_interface*>, char const*) /home/couchbase/jenkins/workspace/kv_engine-threadsanitizer-master/kv_engine/engines/ewouldblock_engine/ewouldblock_engine.cc:301:33 (ewouldblock_engine.so+0x000000009fd6)
          #16 CreateBucketThread::create() /home/couchbase/jenkins/workspace/kv_engine-threadsanitizer-master/kv_engine/daemon/memcached.cc:2019:22 (memcached+0x0000004d0c5b)
          #17 CreateBucketThread::run() /home/couchbase/jenkins/workspace/kv_engine-threadsanitizer-master/kv_engine/daemon/memcached.cc:2078:5 (memcached+0x0000004d1351)
          #18 Couchbase::Thread::thread_entry() /home/couchbase/jenkins/workspace/kv_engine-threadsanitizer-master/platform/src/thread.cc:45:5 (libplatform_so.so.0.1.0+0x000000018882)
          #19 Couchbase::StartThreadDelegator::run(Couchbase::Thread&) /home/couchbase/jenkins/workspace/kv_engine-threadsanitizer-master/platform/src/thread.cc:59:16 (libplatform_so.so.0.1.0+0x000000018b05)
          #20 task_thread_main(void*) /home/couchbase/jenkins/workspace/kv_engine-threadsanitizer-master/platform/src/thread.cc:65 (libplatform_so.so.0.1.0+0x000000018b05)
          #21 CouchbaseThread::run() /home/couchbase/jenkins/workspace/kv_engine-threadsanitizer-master/platform/src/cb_pthreads.cc:59:9 (libplatform_so.so.0.1.0+0x000000009532)
          #22 platform_thread_wrap(void*) /home/couchbase/jenkins/workspace/kv_engine-threadsanitizer-master/platform/src/cb_pthreads.cc:72:14 (libplatform_so.so.0.1.0+0x0000000086a8)
       
        Mutex M2335 (0x7d7c0000f2b0) created at:
          #0 pthread_mutex_init <null> (memcached+0x00000043d39a)
          #1 rocksdb::port::Mutex::Mutex(bool) /home/couchbase/jenkins/workspace/cbdeps-platform-build/deps/packages/build/rocksdb/rocksdb-prefix/src/rocksdb/port/port_posix.cc:41 (librocksdb.so.5.8+0x0000003b8078)
          #2 RocksDBKVStore::RocksDBKVStore(KVStoreConfig&) /home/couchbase/jenkins/workspace/kv_engine-threadsanitizer-master/kv_engine/engines/ep/src/rocksdb-kvstore/rocksdb-kvstore.cc:271:5 (ep.so+0x0000001ce178)
          #3 __unique_if<RocksDBKVStore>::__unique_single std::make_unique<RocksDBKVStore, KVStoreConfig&>(KVStoreConfig&) /home/couchbase/jenkins/workspace/kv_engine-threadsanitizer-master/platform/include/platform/make_unique.h:45:37 (ep.so+0x00000010d343)
          #4 KVStoreFactory::create(KVStoreConfig&) /home/couchbase/jenkins/workspace/kv_engine-threadsanitizer-master/kv_engine/engines/ep/src/kvstore.cc:88 (ep.so+0x00000010d343)
          #5 KVShard::KVShard(unsigned short, KVBucket&) /home/couchbase/jenkins/workspace/kv_engine-threadsanitizer-master/kv_engine/engines/ep/src/kvshard.cc:43:23 (ep.so+0x000000134c80)
          #6 __unique_if<KVShard>::__unique_single std::make_unique<KVShard, unsigned long&, KVBucket&>(unsigned long&, KVBucket&) /home/couchbase/jenkins/workspace/kv_engine-threadsanitizer-master/platform/include/platform/make_unique.h:45:37 (ep.so+0x000000172a4b)
          #7 VBucketMap::VBucketMap(Configuration&, KVBucket&) /home/couchbase/jenkins/workspace/kv_engine-threadsanitizer-master/kv_engine/engines/ep/src/vbucketmap.cc:32 (ep.so+0x000000172a4b)
          #8 KVBucket::KVBucket(EventuallyPersistentEngine&) /home/couchbase/jenkins/workspace/kv_engine-threadsanitizer-master/kv_engine/engines/ep/src/kv_bucket.cc:226:7 (ep.so+0x0000001193f3)
          #9 EPBucket::EPBucket(EventuallyPersistentEngine&) /home/couchbase/jenkins/workspace/kv_engine-threadsanitizer-master/kv_engine/engines/ep/src/ep_bucket.cc:169:7 (ep.so+0x0000000a4085)
          #10 __unique_if<EPBucket>::__unique_single std::make_unique<EPBucket, EventuallyPersistentEngine&>(EventuallyPersistentEngine&) /home/couchbase/jenkins/workspace/kv_engine-threadsanitizer-master/platform/include/platform/make_unique.h:45:37 (ep.so+0x0000000c39bf)
          #11 EventuallyPersistentEngine::makeBucket(Configuration&) /home/couchbase/jenkins/workspace/kv_engine-threadsanitizer-master/kv_engine/engines/ep/src/ep_engine.cc:5390 (ep.so+0x0000000c39bf)
          #12 EventuallyPersistentEngine::initialize(char const*) /home/couchbase/jenkins/workspace/kv_engine-threadsanitizer-master/kv_engine/engines/ep/src/ep_engine.cc:1950:16 (ep.so+0x0000000c3533)
          #13 EvpInitialize(gsl::not_null<engine_interface*>, char const*) /home/couchbase/jenkins/workspace/kv_engine-threadsanitizer-master/kv_engine/engines/ep/src/ep_engine.cc:149:35 (ep.so+0x0000000bcf3d)
          #14 EWB_Engine::initialize(gsl::not_null<engine_interface*>, char const*) /home/couchbase/jenkins/workspace/kv_engine-threadsanitizer-master/kv_engine/engines/ewouldblock_engine/ewouldblock_engine.cc:301:33 (ewouldblock_engine.so+0x000000009fd6)
          #15 CreateBucketThread::create() /home/couchbase/jenkins/workspace/kv_engine-threadsanitizer-master/kv_engine/daemon/memcached.cc:2019:22 (memcached+0x0000004d0c5b)
          #16 CreateBucketThread::run() /home/couchbase/jenkins/workspace/kv_engine-threadsanitizer-master/kv_engine/daemon/memcached.cc:2078:5 (memcached+0x0000004d1351)
          #17 Couchbase::Thread::thread_entry() /home/couchbase/jenkins/workspace/kv_engine-threadsanitizer-master/platform/src/thread.cc:45:5 (libplatform_so.so.0.1.0+0x000000018882)
          #18 Couchbase::StartThreadDelegator::run(Couchbase::Thread&) /home/couchbase/jenkins/workspace/kv_engine-threadsanitizer-master/platform/src/thread.cc:59:16 (libplatform_so.so.0.1.0+0x000000018b05)
          #19 task_thread_main(void*) /home/couchbase/jenkins/workspace/kv_engine-threadsanitizer-master/platform/src/thread.cc:65 (libplatform_so.so.0.1.0+0x000000018b05)
          #20 CouchbaseThread::run() /home/couchbase/jenkins/workspace/kv_engine-threadsanitizer-master/platform/src/cb_pthreads.cc:59:9 (libplatform_so.so.0.1.0+0x000000009532)
          #21 platform_thread_wrap(void*) /home/couchbase/jenkins/workspace/kv_engine-threadsanitizer-master/platform/src/cb_pthreads.cc:72:14 (libplatform_so.so.0.1.0+0x0000000086a8)
      

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            paolo.cocchi Paolo Cocchi
            sriram Sriram Ganesan (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty