Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-45010

[Magma] Memcached crashed in std::execute_native_thread_routine (__p=0x7f825c2deee0)

    XMLWordPrintable

Details

    Description

      Steps to repro:*

      1. Created a 3 node cluster
      2. Create 10 scopes and 20 collections in each scope
      3. Created 15K items in each collection
      4. Rebalance in 2 nodes (with doc loading going on)
      5. Observed Memcached crashes in std::execute_native_thread_routine

      BackTrace:

      (gdb) bt full
      #0  0x00007f82605e9337 in raise () from /lib64/libc.so.6
      No symbol table info available.
      #1  0x00007f82605eaa28 in abort () from /lib64/libc.so.6
      No symbol table info available.
      #2  0x00007f8261147195 in __gnu_cxx::__verbose_terminate_handler () at /tmp/deploy/gcc-7.3.0/libstdc++-v3/libsupc++/vterminate.cc:95
              terminating = false
              t = <optimized out>
      #3  0x0000000000558162 in backtrace_terminate_handler () at /tmp/workspace/couchbase-server-unix/kv_engine/utilities/terminate_handler.cc:93
      No locals.
      #4  0x00007f8261144f86 in __cxxabiv1::__terminate (handler=<optimized out>) at /tmp/deploy/gcc-7.3.0/libstdc++-v3/libsupc++/eh_terminate.cc:47
      No locals.
      #5  0x00007f8261144fd1 in std::terminate () at /tmp/deploy/gcc-7.3.0/libstdc++-v3/libsupc++/eh_terminate.cc:57
      No locals.
      #6  0x00007f826116fdfe in std::execute_native_thread_routine (__p=0x7f825c2deee0) at /tmp/deploy/gcc-7.3.0/libstdc++-v3/src/c++11/thread.cc:91
              __t = {_M_t = {
                  _M_t = {<std::_Tuple_impl<0, std::thread::_State*, std::default_delete<std::thread::_State> >> = {<std::_Tuple_impl<1, std::default_delete<std::thread::_State> >> = {<std::_Head_base<1, std::default_delete<std::thread::_State>, true>> = {<std::default_delete<std::thread::_State>> = {<No data fields>}, <No data fields>}, <No data fields>}, <std::_Head_base<0, std::thread::_State*, false>> = {
                        _M_head_impl = 0x7f825c2deee0}, <No data fields>}, <No data fields>}}}
      #7  0x00007f8260988e65 in start_thread () from /lib64/libpthread.so.0
      No symbol table info available.
      #8  0x00007f82606b188d in clone () from /lib64/libc.so.6
      No symbol table info available.
      (gdb) f 6
      #6  0x00007f826116fdfe in std::execute_native_thread_routine (__p=0x7f825c2deee0) at /tmp/deploy/gcc-7.3.0/libstdc++-v3/src/c++11/thread.cc:91
      91	/tmp/deploy/gcc-7.3.0/libstdc++-v3/src/c++11/thread.cc: No such file or directory.
      

      QE test:

      guides/gradlew --refresh-dependencies testrunner -P jython=/opt/jython/bin/jython -P 'args=-i /tmp/testexec.31507.ini bucket_storage=magma,get-cbcollect-info=True,bucket_eviction_policy=fullEviction,randomize_value=True,rerun=false,infra_log_level=debug,log_level=debug,get-cbcollect-info=True,infra_log_level=critical,log_level=error,upgrade_version=7.0.0-4678 -t bucket_collections.collections_rebalance.CollectionsRebalance.test_data_load_collections_with_rebalance_in,nodes_init=3,nodes_in=2,bucket_spec=magma_templates.buckets_all_membase_for_rebalance_tests,data_load_stage=before,GROUP=P0_rebalance'
      

      Note: Debug info was installed on the node while collecting backtrace info

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              ankush.sharma Ankush Sharma
              ankush.sharma Ankush Sharma
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty