Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-41856

Magma crashed in KVBucket::deleteExpiredItem -> MagmaKVStore::compactionCallBack -> magma::DocSeqGCContext::Transform

    XMLWordPrintable

Details

    • Bug
    • Resolution: Fixed
    • Test Blocker
    • 7.0.0
    • Cheshire-Cat
    • couchbase-bucket
    • 7.0.0-3291
    • Untriaged
    • 1
    • Unknown

    Description

      On Node 172.23.120.241

      Core was generated by `/opt/couchbase/bin/memcached -C /opt/couchbase/var/lib/couchbase/config/memcach'.
      #0  0x00007fd3cb822387 in raise () from /lib64/libc.so.6
      #0  0x00007fd3cb822387 in raise () from /lib64/libc.so.6
      #1  0x00007fd3cb823a78 in abort () from /lib64/libc.so.6
      #2  0x00007fd3cc380195 in __gnu_cxx::__verbose_terminate_handler() () from /opt/couchbase/bin/../lib/libstdc++.so.6
      #3  0x00000000005a64a2 in backtrace_terminate_handler() ()
      #4  0x00007fd3cc37df86 in __cxxabiv1::__terminate(void (*)()) () from /opt/couchbase/bin/../lib/libstdc++.so.6
      #5  0x00007fd3cc37dfd1 in std::terminate() () from /opt/couchbase/bin/../lib/libstdc++.so.6
      #6  0x00007fd3cc37e266 in __cxa_rethrow () from /opt/couchbase/bin/../lib/libstdc++.so.6
      #7  0x00000000005a64c9 in backtrace_terminate_handler() ()
      #8  0x00007fd3cc37df86 in __cxxabiv1::__terminate(void (*)()) () from /opt/couchbase/bin/../lib/libstdc++.so.6
      #9  0x00007fd3cc37dfd1 in std::terminate() () from /opt/couchbase/bin/../lib/libstdc++.so.6
      #10 0x00007fd3cc37ed0f in __cxa_pure_virtual () from /opt/couchbase/bin/../lib/libstdc++.so.6
      #11 0x00007fd3cfa916a8 in VBucket::deleteExpiredItem(Item const&, long, ExpireBy) () from /opt/couchbase/bin/../lib/libep.so
      #12 0x00007fd3cfa4ff56 in KVBucket::deleteExpiredItem(Item&, long, ExpireBy) () from /opt/couchbase/bin/../lib/libep.so
      #13 0x00007fd3cfb1150f in MagmaKVStore::compactionCallBack(MagmaKVStore::MagmaCompactionCB&, magma::Slice const&, magma::Slice const&, magma::Slice const&) () from /opt/couchbase/bin/../lib/libep.so
      #14 0x00007fd3cf37d01a in magma::DocSeqGCContext::Transform(magma::Slice const&, magma::Slice const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&, magma::Slice*) () from /opt/couchbase/bin/../lib/libmagma.so
      #15 0x00007fd3cf324efc in magma::ItemGCFilter::Process(std::vector<std::pair<magma::Slice, magma::Slice>, std::allocator<std::pair<magma::Slice, magma::Slice> > >&, std::vector<std::pair<magma::Slice, magma::Slice>, std::allocator<std::pair<magma::Slice, magma::Slice> > >*) () from /opt/couchbase/bin/../lib/libmagma.so
      #16 0x00007fd3cf392de4 in magma::LSMTree::mergeSSTables(std::vector<std::shared_ptr<magma::Table>, std::allocator<std::shared_ptr<magma::Table> > >&, magma::ItemGCFilter&, unsigned long, double, std::vector<std::shared_ptr<magma::Table>, std::allocator<std::shared_ptr<magma::Table> > >*) () from /opt/couchbase/bin/../lib/libmagma.so
      #17 0x00007fd3cf31d8ea in magma::LSMTree::compactLevelForInputs(int, int, std::unique_lock<std::mutex>&, std::vector<std::shared_ptr<magma::Table>, std::allocator<std::shared_ptr<magma::Table> > >&, std::vector<std::shared_ptr<magma::Table>, std::allocator<std::shared_ptr<magma::Table> > >&, bool, std::function<std::unique_ptr<magma::GCContext, std::default_delete<magma::GCContext> > (bool)>) () from /opt/couchbase/bin/../lib/libmagma.so
      #18 0x00007fd3cf31edcd in magma::LSMTree::compactLevel(int, std::unique_lock<std::mutex>&, bool, std::function<std::unique_ptr<magma::GCContext, std::default_delete<magma::GCContext> > (bool)>) () from /opt/couchbase/bin/../lib/libmagma.so
      #19 0x00007fd3cf31f0a8 in magma::LSMTree::compact(std::unique_lock<std::mutex>&) () from /opt/couchbase/bin/../lib/libmagma.so
      #20 0x00007fd3cf3899b7 in std::_Function_handler<void (), magma::LSMTree::queueCompaction()::{lambda()#1}>::_M_invoke(std::_Any_data const&) () from /opt/couchbase/bin/../lib/libmagma.so
      #21 0x00007fd3cf3de0c5 in magma::TaskWorker::loop(void*) () from /opt/couchbase/bin/../lib/libmagma.so
      #22 0x00007fd3ce382947 in platform_thread_wrap(void*) () from /opt/couchbase/bin/../lib/libplatform_so.so.0.1.0
      #23 0x00007fd3cbbc1ea5 in start_thread () from /lib64/libpthread.so.0
      #24 0x00007fd3cb8ea8dd in clone () from /lib64/libc.so.6
      

      QE Test:
      /data/workspace/centos-p0-magma-vset00-00-crash_recovery_graceful_7.0_P1/job_logs/testrunner-20-Oct-02_23-11-21/test_6

      magma.magma_crash_recovery.MagmaCrashTests.magma.magma_crash_recovery.MagmaCrashTests.test_crash_during_ops,doc_size:256,conf_file:conf/magma/crash_recovery.conf,upgrade_version:7.0.0-3291,graceful:True,spec:crash_recovery,rerun:false,num_nodes:4,GROUP:P1;graceful,doc_ops:delete:expiry,sdk_timeout:60,case_number:13,cluster_name:testexec.18915,ini:/tmp/testexec.18915.ini,get-cbcollect-info:True,replicas:0,multiplier:5,bucket_storage:magma,log_level:error,bucket_eviction_policy:fullEviction,logs_folder:/data/workspace/centos-p0-magma-vset00-00-crash_recovery_graceful_7.0_P1/logs/testrunner-20-Oct-02_23-11-21/test_13,nodes_init:4,num_items:10000000,randomize_value:true,infra_log_level:critical
      

      Seems like MB-41430 is not completely fixed.

      Attachments

        Activity

          People

            ritesh.agarwal Ritesh Agarwal
            ritesh.agarwal Ritesh Agarwal
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              PagerDuty