Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-55935

[Magma-CDC] Memcached crashed in magma::LSMTree::verifyTreeOnCompact

    XMLWordPrintable

Details

    Description

      Steps To Recreate:

      1. Create a 4 node cluster
      2. Create a magma buckets with (bucket_history_retention_seconds=3600,bucket_history_retention_bytes=20000000000)
      3. Create 2 scopes and create 1 non default collections in each of the scope
      4. After creating collection update the collection history setting to true
      5. Create 12500000 items in each collection
      6. Upsert all the document twice
      7. Drop non default collection
      8. Observed Memcached crashed in magma::LSMTree::verifyTreeOnCompact

      Below Core Dump was found on node 172.23.107.155

      Logs:

      2023-03-12T15:33:42.956596-07:00 CRITICAL [(default) magma_7/kvstore-23/rev-000000001/seqIndex]TableID: 147 out of order
      2023-03-12T15:33:42.956617-07:00 CRITICAL [(default) magma_7/kvstore-23/rev-000000001/seqIndex]LSMTree
      2023-03-12T15:33:42.957173-07:00 CRITICAL [(default) magma_7/kvstore-23/rev-000000001/seqIndex]Tree Before Compaction : TreeSnapshot : (default) magma_7/kvstore-23/rev-000000001/seqIndex
      2023-03-12T15:33:42.957663-07:00 CRITICAL [(default) magma_7/kvstore-23/rev-000000001/seqIndex]Tree After Compaction : TreeSnapshot : (default) magma_7/kvstore-23/rev-000000001/seqIndex
      2023-03-12T15:33:42.957784-07:00 CRITICAL [(default) magma_7/kvstore-23/rev-000000001/seqIndex]Source Level List :  | ID:/data/default/magma.7/kvstore-23/rev-000000001/seqIndex/sstable.125.data Range:({"Seqno":5,"op":"delta","sn":68072}, {"Seqno":18109,"op":"delta","sn":68078}) Size:16384 LogicalSize:2098088 DataBlocksSize:22139 DataBlocksCompressedSize:8339 IndexBlocksCompressedSize:155 AvgDataBlockSize:1389.83 CompressionRatio:2.65487 FileSize:16384 ItemCount:1015 State:Deleted HistoryLowTimestamp:0 HistoryHighTimestamp:0 |  | ID:/data/default/magma.7/kvstore-23/rev-000000001/seqIndex/sstable.126.data Range:({"Seqno":18110,"op":"delta","sn":68078}, {"Seqno":19123,"op":"delta","sn":68078}) Size:16384 LogicalSize:2097966 DataBlocksSize:22118 DataBlocksCompressedSize:8326 IndexBlocksCompressedSize:150 AvgDataBlockSize:1387.67 CompressionRatio:2.6565 FileSize:16384 ItemCount:1014 State:Deleted HistoryLowTimestamp:0 HistoryHighTimestamp:0 |  | ID:/data/default/magma.7/kvstore-23/rev-000000001/seqIndex/sstable.133.data Range:({"Seqno":23180,"op":"delta","sn":68078}, {"Seqno":24193,"op":"delta","sn":68078}) Size:16384 LogicalSize:2097966 DataBlocksSize:22119 DataBlocksCompressedSize:8316 IndexBlocksCompressedSize:153 AvgDataBlockSize:1386 CompressionRatio:2.65981 FileSize:16384 ItemCount:1014 State:Deleted HistoryLowTimestamp:0 HistoryHighTimestamp:0 |  | ID:/data/default/magma.7/kvstore-23/rev-000000001/seqIndex/sstable.134.data Range:({"Seqno":24194,"op":"delta","sn":68078}, {"Seqno":25207,"op":"delta","sn":68078}) Size:16384 LogicalSize:2097966 DataBlocksSize:22118 DataBlocksCompressedSize:8317 IndexBlocksCompressedSize:152 AvgDataBlockSize:1386.17 CompressionRatio:2.65937 FileSize:16384 ItemCount:1014 State:Deleted HistoryLowTimestamp:0 HistoryHighTimestamp:0 | 
      2023-03-12T15:33:42.957838-07:00 CRITICAL [(default) magma_7/kvstore-23/rev-000000001/seqIndex]Destination Level List :  | ID:/data/default/magma.7/kvstore-23/rev-000000001/seqIndex/sstable.120.data Range:({"Seqno":4,"op":"update","sn":22}, {"Seqno":5,"op":"update","sn":25}) Size:8192 LogicalSize:223 DataBlocksSize:276 DataBlocksCompressedSize:208 IndexBlocksCompressedSize:33 AvgDataBlockSize:208 CompressionRatio:1.32692 FileSize:8192 ItemCount:2 State:Deleted HistoryLowTimestamp:1678659708 HistoryHighTimestamp:1678659708 |  | ID:/data/default/magma.7/kvstore-23/rev-000000001/seqIndex/sstable.62.data Range:({"Seqno":23328,"op":"update","sn":38159}, {"Seqno":25868,"op":"update","sn":43949}) Size:3928064 LogicalSize:5257329 DataBlocksSize:5324676 DataBlocksCompressedSize:3844068 IndexBlocksCompressedSize:27879 AvgDataBlockSize:3022.07 CompressionRatio:1.38517 FileSize:3928064 ItemCount:2541 State:Deleted HistoryLowTimestamp:1678659963 HistoryHighTimestamp:1678660110 | 
      

      BackTrace:

      (gdb) bt full
      #0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
              set = {__val = {0, 3977018461437964602, 0, 0, 0, 0, 0, 0, 0, 0, 16776662, 16777050, 0, 0, 0, 0}}
              pid = <optimized out>
              tid = <optimized out>
              ret = <optimized out>
      #1  0x00007f54688fe535 in __GI_abort () at abort.c:79
              save_stage = 1
              act = {__sigaction_handler = {sa_handler = 0xb, sa_sigaction = 0xb}, sa_mask = {__val = {140000509916800, 3432, 140000509899424,
                    140000509916931, 140000508602223, 27, 1, 10, 139988486327232, 0, 139997269082672, 12948240, 140000508609817, 140000509916800,
                    140000508610819, 140000509916800}}, sa_flags = 1760414736, sa_restorer = 0x1091c50 <stderr>}
              sigs = {__val = {32, 0 <repeats 15 times>}}
      #2  0x00007f5468ed263c in __gnu_cxx::__verbose_terminate_handler () at /tmp/deploy/objdir/../gcc-10.2.0/libstdc++-v3/libsupc++/vterminate.cc:95
              terminating = false
              t = <optimized out>
      #3  0x0000000000b490eb in backtrace_terminate_handler ()
          at /home/couchbase/jenkins/workspace/couchbase-server-unix/kv_engine/utilities/terminate_handler.cc:88
      No locals.
      #4  0x00007f5468edd8f6 in __cxxabiv1::__terminate (handler=<optimized out>)
          at /tmp/deploy/objdir/../gcc-10.2.0/libstdc++-v3/libsupc++/eh_terminate.cc:48
      No locals.
      #5  0x00007f5468edd961 in std::terminate () at /tmp/deploy/objdir/../gcc-10.2.0/libstdc++-v3/libsupc++/eh_terminate.cc:58
      No locals.
      #6  0x00007f5468eddbf4 in __cxxabiv1::__cxa_throw (obj=obj@entry=0x7f519c000ba0, tinfo=0x1092060 <typeinfo for std::runtime_error>,
          dest=0x444be0 <std::runtime_error::~runtime_error()@plt>) at /tmp/deploy/objdir/../gcc-10.2.0/libstdc++-v3/libsupc++/eh_throw.cc:95
              globals = <optimized out>
              header = 0x7f519c000b20
      #7  0x0000000000510abc in magma::LSMTree::verifyTreeOnCompact (this=0x7f543c79a010, tree=..., srcLevelList=..., dstLevelList=...)
          at /opt/gcc-10.2.0/include/c++/10.2.0/bits/std_function.h:245
      No locals.
      #8  0x00000000009f91e9 in magma::LSMTree::compactLevelForInputs(int, int, std::unique_lock<std::mutex>&, magma::LSMTree::CompactionTrigger, std::vector<std::shared_ptr<magma::Table>, std::allocator<std::shared_ptr<magma::Table> > >&, std::vector<std::shared_ptr<magma::Table>, std::allocator<std::shared_ptr<magma::Table> > >&, std::function<std::unique_ptr<magma::GCContext, std::default_delete<magma::GCContext> > (bool)>) (this=<optimized out>,
          level=<optimized out>, dstLevel=<optimized out>, lock=..., trigger=..., srcLevelList=..., dstLevelList=..., makeGCContext=...)
          at /opt/gcc-10.2.0/include/c++/10.2.0/ext/atomicity.h:100
              bt = {dest = 0x7f54542b13f8, start = {__d = {__r = 3889367595981737}}, name = 0x0, out = 0x0}
              status = {s = {_M_t = {<std::__uniq_ptr_impl<magma::Status::state, std::default_delete<magma::Status::state> >> = {
                      _M_t = {<std::_Tuple_impl<0, magma::Status::state*, std::default_delete<magma::Status::state> >> = {<std::_Tuple_impl<1, std::default_delete<magma::Status::state> >> = {<std::_Head_base<1, std::default_delete<magma::Status::state>, true>> = {<std::default_delete<magma::Status::state>> = {<No data fields>}, <No data fields>}, <No data fields>}, <std::_Head_base<0, magma::Status::state*, false>> = {
      --Type <RET> for more, q to quit, c to continue without paging--
                            _M_head_impl = 0x0}, <No data fields>}, <No data fields>}}, <No data fields>}}}
              compactID = 58
              _c = {fn = {<std::_Maybe_unary_or_binary_function<void>> = {<No data fields>}, <std::_Function_base> = {static _M_max_size = 16,
                    static _M_max_align = 8, _M_functor = {_M_unused = {_M_object = 0x7f5367681dc0, _M_const_object = 0x7f5367681dc0,
                        _M_function_pointer = 0x7f5367681dc0,
                        _M_member_pointer = (void (std::_Undefined_class::*)(std::_Undefined_class * const)) 0x7f5367681dc0},
                      _M_pod_data = "\300\035hgS\177\000\000\000\000\000\000\000\000\000"},
                    _M_manager = 0x9f29d0 <std::_Function_handler<void(), magma::LSMTree::compactLevelForInputs(magma::TreeLevel, magma::TreeLevel, std::unique_lock<std::mutex>&, magma::LSMTree::CompactionTrigger, magma::TableList&, magma::TableList&, magma::GCContextBuilder)::<lambda()> >::_M_manager(std::_Any_data &, const std::_Any_data &, std::_Manager_operation)>},
                  _M_invoker = 0x9f3400 <std::_Function_handler<void(), magma::LSMTree::compactLevelForInputs(magma::TreeLevel, magma::TreeLevel, std::unique_lock<std::mutex>&, magma::LSMTree::CompactionTrigger, magma::TableList&, magma::TableList&, magma::GCContextBuilder)::<lambda()> >::_M_invoke(const std::_Any_data &)>}}
              srcLevelSize = 8439617
              targetSize = 4194304
              isFileCountCompaction = false
              t0 = <optimized out>
              srcDataSize = <optimized out>
              dstDataSize = <optimized out>
              srcLogicalDataSz = 8391986
              dstLogicalDataSz = 5257552
              height = <optimized out>
              isHighestLevel = <optimized out>
              writeAmp = 0.626496732
              optimizeWriteAmp = true
              nonL0LocalCompaction = false
              _restoreState = {fn = {<std::_Maybe_unary_or_binary_function<void>> = {<No data fields>}, <std::_Function_base> = {static _M_max_size = 16,
                    static _M_max_align = 8, _M_functor = {_M_unused = {_M_object = 0x7f543c41f500, _M_const_object = 0x7f543c41f500,
                        _M_function_pointer = 0x7f543c41f500,
                        _M_member_pointer = (void (std::_Undefined_class::*)(std::_Undefined_class * const)) 0x7f543c41f500, this adjustment 17},
                      _M_pod_data = "\000\365A<T\177\000\000\021\000\000\000\000\000\000"},
                    _M_manager = 0x9f2930 <std::_Function_handler<void(), magma::LSMTree::compactLevelForInputs(magma::TreeLevel, magma::TreeLevel, std::unique_lock<std::mutex>&, magma::LSMTree::CompactionTrigger, magma::TableList&, magma::TableList&, magma::GCContextBuilder)::<lambda()> >::_M_manager(std::_Any_data &, const std::_Any_data &, std::_Manager_operation)>},
                  _M_invoker = 0x9f3140 <std::_Function_handler<void(), magma::LSMTree::compactLevelForInputs(magma::TreeLevel, magma::TreeLevel, std::unique_lock<std::mutex>&, magma::LSMTree::CompactionTrigger, magma::TableList&, magma::TableList&, magma::GCContextBuilder)::<lambda()> >::_M_invoke(const std::_Any_data &)>}}
              inputList = {<std::_Vector_base<std::shared_ptr<magma::Table>, std::allocator<std::shared_ptr<magma::Table> > >> = {
                  _M_impl = {<std::allocator<std::shared_ptr<magma::Table> >> = {<__gnu_cxx::new_allocator<std::shared_ptr<magma::Table> >> = {<No data field--Type <RET> for more, q to quit, c to continue without paging--
      s>}, <No data fields>}, <std::_Vector_base<std::shared_ptr<magma::Table>, std::allocator<std::shared_ptr<magma::Table> > >::_Vector_impl_data> = {
                      _M_start = 0x7f53f83cef00, _M_finish = 0x7f53f83cef60, _M_end_of_storage = 0x7f53f83cef80}, <No data fields>}}, <No data fields>}
              outputList = {<std::_Vector_base<std::shared_ptr<magma::Table>, std::allocator<std::shared_ptr<magma::Table> > >> = {
                  _M_impl = {<std::allocator<std::shared_ptr<magma::Table> >> = {<__gnu_cxx::new_allocator<std::shared_ptr<magma::Table> >> = {<No data fields>}, <No data fields>}, <std::_Vector_base<std::shared_ptr<magma::Table>, std::allocator<std::shared_ptr<magma::Table> > >::_Vector_impl_data> = {
                      _M_start = 0x7f526c8273c0, _M_finish = 0x7f526c8273d0, _M_end_of_storage = 0x7f526c8273d0}, <No data fields>}}, <No data fields>}
              needSSTablesMerging = true
              bloomFilterAccuracy = <optimized out>
              levelSize = <optimized out>
              triggerRatio = 1.0009999999999999
              gcFilter = {_M_t = {<std::__uniq_ptr_impl<magma::ItemGCFilter, std::default_delete<magma::ItemGCFilter> >> = {
                    _M_t = {<std::_Tuple_impl<0, magma::ItemGCFilter*, std::default_delete<magma::ItemGCFilter> >> = {<std::_Tuple_impl<1, std::default_delete<magma::ItemGCFilter> >> = {<std::_Head_base<1, std::default_delete<magma::ItemGCFilter>, true>> = {<std::default_delete<magma::ItemGCFilter>> = {<No data fields>}, <No data fields>}, <No data fields>}, <std::_Head_base<0, magma::ItemGCFilter*, false>> = {
                          _M_head_impl = 0x7f545c28a000}, <No data fields>}, <No data fields>}}, <No data fields>}}
              _ = {fn = {<std::_Maybe_unary_or_binary_function<void>> = {<No data fields>}, <std::_Function_base> = {static _M_max_size = 16,
                    static _M_max_align = 8, _M_functor = {_M_unused = {_M_object = 0x7f53a77e4550, _M_const_object = 0x7f53a77e4550,
                        _M_function_pointer = 0x7f53a77e4550,
                        _M_member_pointer = (void (std::_Undefined_class::*)(std::_Undefined_class * const)) 0x7f53a77e4550, this adjustment 139997269083528}, _M_pod_data = "PE~\247S\177\000\000\210E~\247S\177\000"},
                    _M_manager = 0x9f26a0 <std::_Function_handler<void(), magma::LSMTree::compactLevelForInputs(magma::TreeLevel, magma::TreeLevel, std::unique_lock<std::mutex>&, magma::LSMTree::CompactionTrigger, magma::TableList&, magma::TableList&, magma::GCContextBuilder)::<lambda()> >::_M_manager(std::_Any_data &, const std::_Any_data &, std::_Manager_operation)>},
                  _M_invoker = 0x9f2a60 <std::_Function_handler<void(), magma::LSMTree::compactLevelForInputs(magma::TreeLevel, magma::TreeLevel, std::unique_lock<std::mutex>&, magma::LSMTree::CompactionTrigger, magma::TableList&, magma::TableList&, magma::GCContextBuilder)::<lambda()> >::_M_invoke(const std::_Any_data &)>}}
              outDataSize = <optimized out>
              resultState = magma::Table::Deleted
              newSnapshot = {<std::__shared_ptr<magma::TreeSnapshot, (__gnu_cxx::_Lock_policy)2>> = {<std::__shared_ptr_access<magma::TreeSnapshot, (__gnu_cxx::_Lock_policy)2, false, false>> = {<No data fields>}, _M_ptr = 0x7f5454310790, _M_refcount = {_M_pi = 0x7f5454310780}}, <No data fields>}
              dur = 18
              secs = <optimized out>
              MBps = <optimized out>
              rMBps = 6.9167841165965618e-310
              wMBps = 6.9168244377508308e-310
              gcRatio = 6.9167841165965618e-310
              newTriggerRatio = 6.9168244377508308e-310
      #9  0x00000000009fb0ef in magma::LSMTree::compactLevel(int, std::unique_lock<std::mutex>&, magma::LSMTree::CompactionTrigger, std::function<std::unique_ptr<magma::GCContext, std::default_delete<magma::GCContext> > (bool)>) (this=0x7f543c79a010, level=3, lock=..., trigger=..., makeGCContext=...)
      --Type <RET> for more, q to quit, c to continue without paging--
          at /opt/gcc-10.2.0/include/c++/10.2.0/bits/std_function.h:248
              srcLevelList = {<std::_Vector_base<std::shared_ptr<magma::Table>, std::allocator<std::shared_ptr<magma::Table> > >> = {
                  _M_impl = {<std::allocator<std::shared_ptr<magma::Table> >> = {<__gnu_cxx::new_allocator<std::shared_ptr<magma::Table> >> = {<No data fields>}, <No data fields>}, <std::_Vector_base<std::shared_ptr<magma::Table>, std::allocator<std::shared_ptr<magma::Table> > >::_Vector_impl_data> = {
                      _M_start = 0x7f539ac8a600, _M_finish = 0x7f539ac8a640, _M_end_of_storage = 0x7f539ac8a640}, <No data fields>}}, <No data fields>}
              dstLevelList = {<std::_Vector_base<std::shared_ptr<magma::Table>, std::allocator<std::shared_ptr<magma::Table> > >> = {
                  _M_impl = {<std::allocator<std::shared_ptr<magma::Table> >> = {<__gnu_cxx::new_allocator<std::shared_ptr<magma::Table> >> = {<No data fields>}, <No data fields>}, <std::_Vector_base<std::shared_ptr<magma::Table>, std::allocator<std::shared_ptr<magma::Table> > >::_Vector_impl_data> = {
                      _M_start = 0x7f53f87149a0, _M_finish = 0x7f53f87149c0, _M_end_of_storage = 0x7f53f87149c0}, <No data fields>}}, <No data fields>}
              inputList = {<std::_Vector_base<std::shared_ptr<magma::Table>, std::allocator<std::shared_ptr<magma::Table> > >> = {
                  _M_impl = {<std::allocator<std::shared_ptr<magma::Table> >> = {<__gnu_cxx::new_allocator<std::shared_ptr<magma::Table> >> = {<No data fields>}, <No data fields>}, <std::_Vector_base<std::shared_ptr<magma::Table>, std::allocator<std::shared_ptr<magma::Table> > >::_Vector_impl_data> = {
                      _M_start = 0x0, _M_finish = 0x0, _M_end_of_storage = 0x0}, <No data fields>}}, <No data fields>}
              outputList = {<std::_Vector_base<std::shared_ptr<magma::Table>, std::allocator<std::shared_ptr<magma::Table> > >> = {
                  _M_impl = {<std::allocator<std::shared_ptr<magma::Table> >> = {<__gnu_cxx::new_allocator<std::shared_ptr<magma::Table> >> = {<No data fields>}, <No data fields>}, <std::_Vector_base<std::shared_ptr<magma::Table>, std::allocator<std::shared_ptr<magma::Table> > >::_Vector_impl_data> = {
                      _M_start = 0x0, _M_finish = 0x0, _M_end_of_storage = 0x0}, <No data fields>}}, <No data fields>}
              status = {s = {_M_t = {<std::__uniq_ptr_impl<magma::Status::state, std::default_delete<magma::Status::state> >> = {
                      _M_t = {<std::_Tuple_impl<0, magma::Status::state*, std::default_delete<magma::Status::state> >> = {<std::_Tuple_impl<1, std::default_delete<magma::Status::state> >> = {<std::_Head_base<1, std::default_delete<magma::Status::state>, true>> = {<std::default_delete<magma::Status::state>> = {<No data fields>}, <No data fields>}, <No data fields>}, <std::_Head_base<0, magma::Status::state*, false>> = {
                            _M_head_impl = 0x0}, <No data fields>}, <No data fields>}}, <No data fields>}}}
              dstLevel = 4
      #10 0x00000000009fb3c1 in magma::LSMTree::compact (this=this@entry=0x7f543c79a010, lock=...)
          at /home/couchbase/jenkins/workspace/couchbase-server-unix/magma/lsm/compaction.cc:407
              level = <optimized out>
              status = {s = {_M_t = {<std::__uniq_ptr_impl<magma::Status::state, std::default_delete<magma::Status::state> >> = {
                      _M_t = {<std::_Tuple_impl<0, magma::Status::state*, std::default_delete<magma::Status::state> >> = {<std::_Tuple_impl<1, std::default_delete<magma::Status::state> >> = {<std::_Head_base<1, std::default_delete<magma::Status::state>, true>> = {<std::default_delete<magma::Status::state>> = {<No data fields>}, <No data fields>}, <No data fields>}, <std::_Head_base<0, magma::Status::state*, false>> = {
                            _M_head_impl = 0x2}, <No data fields>}, <No data fields>}}, <No data fields>}}}
              trigger = <error reading variable>
              __for_range = @0x7f53a77e48a0: {<std::_Vector_base<std::pair<int, magma::LSMTree::CompactionTrigger>, std::allocator<std::pair<int, magma::LSMTree::CompactionTrigger> > >> = {
                  _M_impl = {<std::allocator<std::pair<int, magma::LSMTree::CompactionTrigger> >> = {<__gnu_cxx::new_allocator<std::pair<int, magma::LSMTree::CompactionTrigger> >> = {<No data fields>}, <No data fields>}, <std::_Vector_base<std::pair<int, magma::LSMTree::CompactionTrigger>, std::allocator<std::pair<int, magma::LSMTree::CompactionTrigger> > >::_Vector_impl_data> = {_M_start = 0x7f5394839360, _M_finish = 0x7f5394839390,
                      _M_end_of_storage = 0x7f5394839390}, <No data fields>}}, <No data fields>}
              __for_begin = <optimized out>
      --Type <RET> for more, q to quit, c to continue without paging--
              __for_end = <optimized out>
              compacted = false
              triggers = {<std::_Vector_base<std::pair<int, magma::LSMTree::CompactionTrigger>, std::allocator<std::pair<int, magma::LSMTree::CompactionTrigger> > >> = {
                  _M_impl = {<std::allocator<std::pair<int, magma::LSMTree::CompactionTrigger> >> = {<__gnu_cxx::new_allocator<std::pair<int, magma::LSMTree::CompactionTrigger> >> = {<No data fields>}, <No data fields>}, <std::_Vector_base<std::pair<int, magma::LSMTree::CompactionTrigger>, std::allocator<std::pair<int, magma::LSMTree::CompactionTrigger> > >::_Vector_impl_data> = {_M_start = 0x7f5394839360, _M_finish = 0x7f5394839390,
                      _M_end_of_storage = 0x7f5394839390}, <No data fields>}}, <No data fields>}
      #11 0x0000000000a1d038 in magma::LSMTree::doCompactionWork (this=0x7f543c79a010)
          at /home/couchbase/jenkins/workspace/couchbase-server-unix/magma/lsm/lsm_tree.cc:417
              lock = {_M_device = 0x7f543c79a100, _M_owns = true}
              lock = <optimized out>
      #12 operator() (__closure=<optimized out>) at /home/couchbase/jenkins/workspace/couchbase-server-unix/magma/lsm/lsm_tree.cc:428
              this = <optimized out>
              this = <optimized out>
      #13 std::__invoke_impl<void, magma::LSMTree::queueCompaction()::<lambda()>&> (__f=...) at /opt/gcc-10.2.0/include/c++/10.2.0/bits/invoke.h:60
      No locals.
      #14 std::__invoke_r<void, magma::LSMTree::queueCompaction()::<lambda()>&> (__fn=...) at /opt/gcc-10.2.0/include/c++/10.2.0/bits/invoke.h:110
      No locals.
      #15 std::_Function_handler<void(), magma::LSMTree::queueCompaction()::<lambda()> >::_M_invoke(const std::_Any_data &) (__functor=...)
          at /opt/gcc-10.2.0/include/c++/10.2.0/bits/std_function.h:291
      No locals.
      #16 0x00000000009e1cd8 in std::function<void ()>::operator()() const (this=0x7f53d8741e90)
          at /opt/gcc-10.2.0/include/c++/10.2.0/bits/std_function.h:248
      No locals.
      #17 magma::TimedTask::Complete (this=0x7f546438a750, skipExecution=<optimized out>)
          at /home/couchbase/jenkins/workspace/couchbase-server-unix/magma/util/worker.cc:158
              wg = 0x7f543c79a368
              envGuard = {previous = {<std::__pair_base<void*, int>> = {<No data fields>}, first = 0x0, second = 2}}
              currTask = {<std::__shared_ptr<magma::Task, (__gnu_cxx::_Lock_policy)2>> = {<std::__shared_ptr_access<magma::Task, (__gnu_cxx::_Lock_policy)2, false, false>> = {<No data fields>}, _M_ptr = 0x7f53d8741e80, _M_refcount = {_M_pi = 0x7f53d8741e70}}, <No data fields>}
      #18 0x00000000009e3651 in magma::TaskWorker::loop (w=0x7f5467423780)
          at /home/couchbase/jenkins/workspace/couchbase-server-unix/magma/util/worker.cc:180
              task = {<std::__shared_ptr<magma::TimedTask, (__gnu_cxx::_Lock_policy)2>> = {<std::__shared_ptr_access<magma::TimedTask, (__gnu_cxx::_Lock_policy)2, false, false>> = {<No data fields>}, _M_ptr = 0x7f546438a750, _M_refcount = {_M_pi = 0x7f546438a740}}, <No data fields>}
              worker = 0x7f5467423780
      #19 0x0000000000ba4dd9 in CouchbaseThread::run (this=0x7f546736f240)
          at /home/couchbase/jenkins/workspace/couchbase-server-unix/platform/src/cb_pthreads.cc:51
      No locals.
      --Type <RET> for more, q to quit, c to continue without paging--
      #20 platform_thread_wrap (arg=0x7f546736f240) at /home/couchbase/jenkins/workspace/couchbase-server-unix/platform/src/cb_pthreads.cc:64
              context = <optimized out>
      #21 0x00007f546ab0cfa3 in start_thread (arg=<optimized out>) at pthread_create.c:486
              ret = <optimized out>
              pd = <optimized out>
              now = <optimized out>
              unwind_buf = {cancel_jmp_buf = {{jmp_buf = {139997269157632, -9148817379358010463, 139998149919614, 139998149919615, 139997269157632,
                      140000486373264, 9200733008426046369, 9196361921599640481}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {
                    prev = 0x0, cleanup = 0x0, canceltype = 0}}}
              not_first_call = <optimized out>
      #22 0x00007f54689d4eff in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
      No locals.
      

      QE-TEST:

      guides/gradlew --refresh-dependencies testrunner -P jython=/opt/jython/bin/jython -P 'args=-i /tmp/testexec.111831.ini bucket_storage=magma,GROUP=P2;SET0,bucket_eviction_policy=fullEviction,randomize_value=True,rerun=false,disable_magma_commit_points=True,fragmentation=50,replicas=0,test_itr=2,init_loading=False,enable_dp=false,get-cbcollect-info=True,autoCompactionDefined=true,bucket_history_retention_seconds=3600,bucket_history_retention_bytes=20000000000,upgrade_version=7.2.0-5242 -t storage.magma.magma_basic_crud.BasicCrudTests.test_drop_collections_after_upserts,num_items=12500000,nodes_init=4,sdk_timeout=60,num_scopes=2,GROUP=P2;SET0'
      

      Attachments

        Issue Links

          For Gerrit Dashboard: MB-55935
          # Subject Branch Project Status CR V

          Activity

            People

              ankush.sharma Ankush Sharma
              ankush.sharma Ankush Sharma
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty