Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-26654

[RocksDB] Fix data races in RocksDBKVStore::delVBucket

    XMLWordPrintable

Details

    • Bug
    • Resolution: Fixed
    • Major
    • master
    • master
    • couchbase-bucket
    • None
    • Untriaged
    • Unknown

    Description

      ThreadSanitizer shows a data race in RocksDBKVStore::delVBucket and RocksDBKVStore::getStat:

      WARNING: ThreadSanitizer: data race (pid=26106)
        Read of size 8 at 0x7d9000008008 by main thread (mutexes: write M6352, write M3354):
          #0 std::unique_ptr<KVRocksDB, std::default_delete<KVRocksDB> >::get() const /usr/bin/../lib/gcc/x86_64-linux-gnu/5.4.0/../../../../include/c++/5.4.0/bits/unique_ptr.h:305:16 (ep.so+0x0000001c62d7)
          #1 std::unique_ptr<KVRocksDB, std::default_delete<KVRocksDB> >::operator bool() const /usr/bin/../lib/gcc/x86_64-linux-gnu/5.4.0/../../../../include/c++/5.4.0/bits/unique_ptr.h:319 (ep.so+0x0000001c62d7)
          #2 RocksDBKVStore::getStat(char const*, unsigned long&) /home/couchbase/jenkins/workspace/kv_engine-threadsanitizer-master/kv_engine/engines/ep/src/rocksdb-kvstore/rocksdb-kvstore.cc:611 (ep.so+0x0000001c62d7)
          #3 KVBucket::getKVStoreStat(char const*, unsigned long&, KVBucketIface::KVSOption) /home/couchbase/jenkins/workspace/kv_engine-threadsanitizer-master/kv_engine/engines/ep/src/kv_bucket.cc:2162:50 (ep.so+0x000000123579)
          #4 EventuallyPersistentEngine::doEngineStats(void const*, void (*)(char const*, unsigned short, char const*, unsigned int, void const*)) /home/couchbase/jenkins/workspace/kv_engine-threadsanitizer-master/kv_engine/engines/ep/src/ep_engine.cc:2642:19 (ep.so+0x0000000b7f6e)
          #5 EventuallyPersistentEngine::getStats(void const*, char const*, int, void (*)(char const*, unsigned short, char const*, unsigned int, void const*)) /home/couchbase/jenkins/workspace/kv_engine-threadsanitizer-master/kv_engine/engines/ep/src/ep_engine.cc:3577:14 (ep.so+0x0000000bcbe3)
          #6 EvpGetStats(engine_interface*, void const*, char const*, int, void (*)(char const*, unsigned short, char const*, unsigned int, void const*)) /home/couchbase/jenkins/workspace/kv_engine-threadsanitizer-master/kv_engine/engines/ep/src/ep_engine.cc:316:35 (ep.so+0x0000000b061e)
       
        Previous write of size 8 at 0x7d9000008008 by thread T9 (mutexes: write M6361):
          #0 void std::swap<KVRocksDB*>(KVRocksDB*&, KVRocksDB*&) /usr/bin/../lib/gcc/x86_64-linux-gnu/5.4.0/../../../../include/c++/5.4.0/bits/move.h:186:11 (ep.so+0x0000001c5632)
          #1 std::unique_ptr<KVRocksDB, std::default_delete<KVRocksDB> >::reset(KVRocksDB*) /usr/bin/../lib/gcc/x86_64-linux-gnu/5.4.0/../../../../include/c++/5.4.0/bits/unique_ptr.h:342 (ep.so+0x0000001c5632)
          #2 RocksDBKVStore::delVBucket(unsigned short, unsigned long) /home/couchbase/jenkins/workspace/kv_engine-threadsanitizer-master/kv_engine/engines/ep/src/rocksdb-kvstore/rocksdb-kvstore.cc:551 (ep.so+0x0000001c5632)
          #3 VBucketMemoryAndDiskDeletionTask::run() /home/couchbase/jenkins/workspace/kv_engine-threadsanitizer-master/kv_engine/engines/ep/src/vbucketdeletiontask.cc:87:30 (ep.so+0x000000167681)
          #4 ExecutorThread::run() /home/couchbase/jenkins/workspace/kv_engine-threadsanitizer-master/kv_engine/engines/ep/src/executorthread.cc:122:39 (ep.so+0x0000000f0480)
          #5 launch_executor_thread(void*) /home/couchbase/jenkins/workspace/kv_engine-threadsanitizer-master/kv_engine/engines/ep/src/executorthread.cc:35:19 (ep.so+0x0000000efdf5)
          #6 CouchbaseThread::run() /home/couchbase/jenkins/workspace/kv_engine-threadsanitizer-master/platform/src/cb_pthreads.cc:59:9 (libplatform_so.so.0.1.0+0x000000007d32)
          #7 platform_thread_wrap(void*) /home/couchbase/jenkins/workspace/kv_engine-threadsanitizer-master/platform/src/cb_pthreads.cc:72:14 (libplatform_so.so.0.1.0+0x000000006ea8)
      

      Another data race in RocksDBKVStore::delVBucket and RocksDBKVStore::getCachePointers:

      WARNING: ThreadSanitizer: data race (pid=26106)
        Write of size 8 at 0x7d9000008008 by thread T9 (mutexes: write M6361):
          #0 void std::swap<KVRocksDB*>(KVRocksDB*&, KVRocksDB*&) /usr/bin/../lib/gcc/x86_64-linux-gnu/5.4.0/../../../../include/c++/5.4.0/bits/move.h:186:11 (ep.so+0x0000001c5632)
          #1 std::unique_ptr<KVRocksDB, std::default_delete<KVRocksDB> >::reset(KVRocksDB*) /usr/bin/../lib/gcc/x86_64-linux-gnu/5.4.0/../../../../include/c++/5.4.0/bits/unique_ptr.h:342 (ep.so+0x0000001c5632)
          #2 RocksDBKVStore::delVBucket(unsigned short, unsigned long) /home/couchbase/jenkins/workspace/kv_engine-threadsanitizer-master/kv_engine/engines/ep/src/rocksdb-kvstore/rocksdb-kvstore.cc:551 (ep.so+0x0000001c5632)
          #3 VBucketMemoryAndDiskDeletionTask::run() /home/couchbase/jenkins/workspace/kv_engine-threadsanitizer-master/kv_engine/engines/ep/src/vbucketdeletiontask.cc:87:30 (ep.so+0x000000167681)
          #4 ExecutorThread::run() /home/couchbase/jenkins/workspace/kv_engine-threadsanitizer-master/kv_engine/engines/ep/src/executorthread.cc:122:39 (ep.so+0x0000000f0480)
          #5 launch_executor_thread(void*) /home/couchbase/jenkins/workspace/kv_engine-threadsanitizer-master/kv_engine/engines/ep/src/executorthread.cc:35:19 (ep.so+0x0000000efdf5)
          #6 CouchbaseThread::run() /home/couchbase/jenkins/workspace/kv_engine-threadsanitizer-master/platform/src/cb_pthreads.cc:59:9 (libplatform_so.so.0.1.0+0x000000007d32)
          #7 platform_thread_wrap(void*) /home/couchbase/jenkins/workspace/kv_engine-threadsanitizer-master/platform/src/cb_pthreads.cc:72:14 (libplatform_so.so.0.1.0+0x000000006ea8)
       
        Previous read of size 8 at 0x7d9000008008 by main thread (mutexes: write M894102188324867984):
          #0 std::unique_ptr<KVRocksDB, std::default_delete<KVRocksDB> >::get() const /usr/bin/../lib/gcc/x86_64-linux-gnu/5.4.0/../../../../include/c++/5.4.0/bits/unique_ptr.h:305:16 (ep.so+0x0000001c6fb7)
          #1 std::unique_ptr<KVRocksDB, std::default_delete<KVRocksDB> >::operator bool() const /usr/bin/../lib/gcc/x86_64-linux-gnu/5.4.0/../../../../include/c++/5.4.0/bits/unique_ptr.h:319 (ep.so+0x0000001c6fb7)
          #2 RocksDBKVStore::getCachePointers() /home/couchbase/jenkins/workspace/kv_engine-threadsanitizer-master/kv_engine/engines/ep/src/rocksdb-kvstore/rocksdb-kvstore.cc:666 (ep.so+0x0000001c6fb7)
          #3 RocksDBKVStore::getStat(char const*, unsigned long&) /home/couchbase/jenkins/workspace/kv_engine-threadsanitizer-master/kv_engine/engines/ep/src/rocksdb-kvstore/rocksdb-kvstore.cc:619:22 (ep.so+0x0000001c6448)
          #4 KVBucket::getKVStoreStat(char const*, unsigned long&, KVBucketIface::KVSOption) /home/couchbase/jenkins/workspace/kv_engine-threadsanitizer-master/kv_engine/engines/ep/src/kv_bucket.cc:2157:50 (ep.so+0x000000123512)
          #5 EventuallyPersistentEngine::doEngineStats(void const*, void (*)(char const*, unsigned short, char const*, unsigned int, void const*)) /home/couchbase/jenkins/workspace/kv_engine-threadsanitizer-master/kv_engine/engines/ep/src/ep_engine.cc:2642:19 (ep.so+0x0000000b7f6e)
          #6 EventuallyPersistentEngine::getStats(void const*, char const*, int, void (*)(char const*, unsigned short, char const*, unsigned int, void const*)) /home/couchbase/jenkins/workspace/kv_engine-threadsanitizer-master/kv_engine/engines/ep/src/ep_engine.cc:3577:14 (ep.so+0x0000000bcbe3)
          #7 EvpGetStats(engine_interface*, void const*, char const*, int, void (*)(char const*, unsigned short, char const*, unsigned int, void const*)) /home/couchbase/jenkins/workspace/kv_engine-threadsanitizer-master/kv_engine/engines/ep/src/ep_engine.cc:316:35 (ep.so+0x0000000b061e)
      

      Full log at http://cv.jenkins.couchbase.com/job/kv_engine-threadsanitizer-master/3528/consoleFull#198641714c5751633-7aa5-473a-91e8-426e0afa5a7b

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            paolo.cocchi Paolo Cocchi
            paolo.cocchi Paolo Cocchi
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty