Details
-
Bug
-
Resolution: Fixed
-
Test Blocker
-
Cheshire-Cat
-
7.0.0-2208
-
Untriaged
-
Centos 64-bit
-
1
-
Yes
-
KV Sprint 2020-June
Description
Steps to repro:
* Had a four node cluster
|
* Created 7 million docs(of size 2048 bytes)
|
* Updated all 7 million docs such that docs moves from seq Tree to key Tree
|
* Repeate the above step twice
|
* Observed Magma crashed with MagmaKVStore::makeCompactionContext(Vbid)
|
QE test:
./testrunner -i /tmp/testexec.95080.ini bucket_storage=magma,GROUP=P0;SET2,bucket_eviction_policy=fullEviction,randomize_value=True,rerun=false,disable_magma_commit_points=True,fragmentation=50,replicas=0,test_itr=2,get-cbcollect-info=True,infra_log_level=critical,log_level=error -t magma.magma_basic_crud.BasicCrudTests.test_move_val_btwn_key_and_seq_trees,num_items=7000000,nodes_init=4,sdk_timeout=60,vbuckets=128,randomize_doc_size=True,GROUP=P0;SET2
|
BackTrace:
(gdb) bt
|
#0 0x00007fac481ca337 in raise () from /lib64/libc.so.6
|
#1 0x00007fac481cba28 in abort () from /lib64/libc.so.6
|
#2 0x00007fac48d28195 in __gnu_cxx::__verbose_terminate_handler() () from /opt/couchbase/bin/../lib/libstdc++.so.6
|
#3 0x000000000054a9f2 in backtrace_terminate_handler() ()
|
#4 0x00007fac48d25f86 in __cxxabiv1::__terminate(void (*)()) () from /opt/couchbase/bin/../lib/libstdc++.so.6
|
#5 0x00007fac48d25fd1 in std::terminate() () from /opt/couchbase/bin/../lib/libstdc++.so.6
|
#6 0x00007fac48d26213 in __cxa_throw () from /opt/couchbase/bin/../lib/libstdc++.so.6
|
#7 0x00007fac4c5fabcf in MagmaKVStore::makeCompactionContext(Vbid) () from /opt/couchbase/bin/../lib/libep.so
|
#8 0x00007fac4c5fb446 in MagmaKVStore::compactionCallBack(MagmaKVStore::MagmaCompactionCB&, magma::Slice const&, magma::Slice const&, magma::Slice const&) ()
|
from /opt/couchbase/bin/../lib/libep.so
|
#9 0x00007fac4bee6c8a in magma::DocSeqGCContext::Transform(magma::Slice const&, magma::Slice const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&, magma::Slice*) () from /opt/couchbase/bin/../lib/libmagma.so
|
#10 0x00007fac4be9eb7e in magma::ItemGCFilter::Process(std::vector<std::pair<magma::Slice, magma::Slice>, std::allocator<std::pair<magma::Slice, magma::Slice> > >&, std::vector<std::pair<magma::Slice, magma::Slice>, std::allocator<std::pair<magma::Slice, magma::Slice> > >*) () from /opt/couchbase/bin/../lib/libmagma.so
|
#11 0x00007fac4bef6960 in magma::LSMTree::mergeSSTables(std::vector<std::shared_ptr<magma::Table>, std::allocator<std::shared_ptr<magma::Table> > >&, magma::ItemGCFilter&, unsigned long, double, std::vector<std::shared_ptr<magma::Table>, std::allocator<std::shared_ptr<magma::Table> > >*) ()
|
from /opt/couchbase/bin/../lib/libmagma.so
|
#12 0x00007fac4be97ac6 in magma::LSMTree::compactLevelForInputs(int, int, std::unique_lock<std::mutex>&, std::vector<std::shared_ptr<magma::Table>, std::allocator<std::shared_ptr<magma::Table> > >&, std::vector<std::shared_ptr<magma::Table>, std::allocator<std::shared_ptr<magma::Table> > >&, bool, std::function<std::unique_ptr<magma::GCContext, std::default_delete<magma::GCContext> > (bool)>) () from /opt/couchbase/bin/../lib/libmagma.so
|
#13 0x00007fac4be98eed in magma::LSMTree::compactLevel(int, std::unique_lock<std::mutex>&, bool, std::function<std::unique_ptr<magma::GCContext, std::default_delete<magma::GCContext> > (bool)>) () from /opt/couchbase/bin/../lib/libmagma.so
|
#14 0x00007fac4be9a2e6 in magma::LSMTree::compact(std::unique_lock<std::mutex>&) () from /opt/couchbase/bin/../lib/libmagma.so
|
#15 0x00007fac4beee367 in std::_Function_handler<void (), magma::LSMTree::queueCompaction()::{lambda()#1}>::_M_invoke(std::_Any_data const&) ()
|
from /opt/couchbase/bin/../lib/libmagma.so
|
#16 0x00007fac4bf3dd39 in magma::TaskWorker::loop(void*) () from /opt/couchbase/bin/../lib/libmagma.so
|
#17 0x00007fac4af2c777 in platform_thread_wrap(void*) () from /opt/couchbase/bin/../lib/libplatform_so.so.0.1.0
|
#18 0x00007fac48569e65 in start_thread () from /lib64/libpthread.so.0
|
#19 0x00007fac4829288d in clone () from /lib64/libc.so.6
|
(gdb)
|
Attachments
Issue Links
- is duplicated by
-
MB-40081 CV intermittent failure DelVBucketConcurrentOperationsTest
- Resolved
-
MB-40179 Failed to write tree state, Error occured during memtable flush. Files/vBuckets delete are not is sync with data flush during rebalance.
- Closed
-
MB-40178 [Magma] Rebalance failed with mover crashed:"badmatch"
- Closed
For Gerrit Dashboard: MB-39669 | ||||||
---|---|---|---|---|---|---|
# | Subject | Branch | Project | Status | CR | V |
131368,3 | MB-39669: Handle Magma implicit compactions before warmup | master | kv_engine | Status: MERGED | +2 | +1 |
132234,2 | MB-37159: Enable DelVBucketConcurrentOperationsTest | master | kv_engine | Status: MERGED | +2 | +1 |