2021-01-30 19:04:28,681 | test | CRITICAL | MainThread | [basetestcase:check_coredump_exist:728] 172.23.106.122: 1 core dump seen 2021-01-30 19:04:29,487 | test | CRITICAL | MainThread | [basetestcase:check_coredump_exist:729] 172.23.106.122: Stack Trace of first crash - 57b69006-a57c-4a5f-5ab3f88f-e4097e7f.dmp Core was generated by `/opt/couchbase/bin/memcached -C /opt/couchbase/var/lib/couchbase/config/memcach'. #0 0x00007fa8f3740337 in raise () from /lib64/libc.so.6 #0 0x00007fa8f3740337 in raise () from /lib64/libc.so.6 No symbol table info available. #1 0x00007fa8f3741a28 in abort () from /lib64/libc.so.6 No symbol table info available. #2 0x00007fa8f429e195 in __gnu_cxx::__verbose_terminate_handler() () from /opt/couchbase/bin/../lib/libstdc++.so.6 No symbol table info available. #3 0x0000000000555632 in backtrace_terminate_handler() () No symbol table info available. #4 0x00007fa8f429bf86 in __cxxabiv1::__terminate(void (*)()) () from /opt/couchbase/bin/../lib/libstdc++.so.6 No symbol table info available. #5 0x00007fa8f429bfd1 in std::terminate() () from /opt/couchbase/bin/../lib/libstdc++.so.6 No symbol table info available. #6 0x00007fa8f7ca6ba3 in GlobalTask::execute() () from /opt/couchbase/bin/../lib/libep.so No symbol table info available. #7 0x00007fa8f7ca0c02 in void folly::detail::function::FunctionTraits::callSmall(folly::detail::function::Data&) () from /opt/couchbase/bin/../lib/libep.so No symbol table info available. #8 0x00007fa8f7e1eeb6 in folly::ThreadPoolExecutor::runTask(std::shared_ptr const&, folly::ThreadPoolExecutor::Task&&) () from /opt/couchbase/bin/../lib/libep.so No symbol table info available. #9 0x00007fa8f7e073aa in folly::CPUThreadPoolExecutor::threadRun(std::shared_ptr) () from /opt/couchbase/bin/../lib/libep.so No symbol table info available. #10 0x00007fa8f7e21fd9 in void folly::detail::function::FunctionTraits::callBig))(std::shared_ptr)> >(folly::detail::function::Data&) () from /opt/couchbase/bin/../lib/libep.so No symbol table info available. #11 0x00007fa8f7c9f043 in void folly::detail::function::FunctionTraits::callBig&&)::{lambda()#1}>(folly::detail::function::Data&) () from /opt/couchbase/bin/../lib/libep.so No symbol table info available. #12 0x00007fa8f42c6dcf in execute_native_thread_routine () from /opt/couchbase/bin/../lib/libstdc++.so.6 No symbol table info available. #13 0x00007fa8f3adfe65 in start_thread () from /lib64/libpthread.so.0 No symbol table info available. #14 0x00007fa8f380888d in clone () from /lib64/libc.so.6 No symbol table info available. [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib64/libthread_db.so.1". Loaded symbols for /lib64/libpthread.so.0 Reading symbols from /lib64/libc.so.6...(no debugging symbols found)...done. Loaded symbols for /lib64/libc.so.6 Reading symbols from /lib64/ld-linux-x86-64.so.2...(no debugging symbols found)...done. Loaded symbols for /lib64/ld-linux-x86-64.so.2 Reading symbols from /lib64/librt.so.1...(no debugging symbols found)...done. Loaded symbols for /lib64/librt.so.1 0x00007f8292e44e63 in epoll_wait () from /lib64/libc.so.6 Thread 61 (Thread 0x7f82921ff700 (LWP 61546)): #0 0x00007f829311fda2 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f8294bef5a7 in background_thread_entry () from /opt/couchbase/bin/../lib/libjemalloc.so.2 #2 0x00007f829311be65 in start_thread () from /lib64/libpthread.so.0 #3 0x00007f8292e4488d in clone () from /lib64/libc.so.6 Thread 60 (Thread 0x7f829147e700 (LWP 61547)): #0 0x00007f829311fda2 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f82966178cd in spdlog::details::thread_pool::process_next_msg_() () from /opt/couchbase/bin/../lib/libmemcached_logger.so.1.0.0 #2 0x00007f8296617e20 in spdlog::details::thread_pool::worker_loop_() () from /opt/couchbase/bin/../lib/libmemcached_logger.so.1.0.0 #3 0x00007f8293902dcf in execute_native_thread_routine () from /opt/couchbase/bin/../lib/libstdc++.so.6 #4 0x00007f829311be65 in start_thread () from /lib64/libpthread.so.0 #5 0x00007f8292e4488d in clone () from /lib64/libc.so.6 Thread 59 (Thread 0x7f8290c7d700 (LWP 61548)): #0 0x00007f829311fda2 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f8296616f8a in _ZNSt6thread11_State_implINS_8_InvokerISt5tupleIJZN6spdlog7details15periodic_workerC4ERKSt8functionIFvvEENSt6chrono8durationIlSt5ratioILl1ELl1EEEEEUlvE_EEEEE6_M_runEv () from /opt/couchbase/bin/../lib/libmemcached_logger.so.1.0.0 #2 0x00007f8293902dcf in execute_native_thread_routine () from /opt/couchbase/bin/../lib/libstdc++.so.6 #3 0x00007f829311be65 in start_thread () from /lib64/libpthread.so.0 #4 0x00007f8292e4488d in clone () from /lib64/libc.so.6 Thread 58 (Thread 0x7f829047c700 (LWP 61549)): #0 0x00007f8292e39bed in poll () from /lib64/libc.so.6 #1 0x000000000046a1ee in check_stdin_thread(void*) () #2 0x00007f82958de2d7 in platform_thread_wrap(void*) () from /opt/couchbase/bin/../lib/libplatform_so.so.0.1.0 #3 0x00007f829311be65 in start_thread () from /lib64/libpthread.so.0 #4 0x00007f8292e4488d in clone () from /lib64/libc.so.6 Thread 57 (Thread 0x7f828fc7b700 (LWP 61550)): #0 0x00007f829311f9f5 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f8296eafb2b in worker_thread () from /opt/couchbase/bin/../lib/libstatistics.so #2 0x00007f829311be65 in start_thread () from /lib64/libpthread.so.0 #3 0x00007f8292e4488d in clone () from /lib64/libc.so.6 Thread 56 (Thread 0x7f828f47a700 (LWP 61551)): #0 0x00007f829311f9f5 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f8296eafb2b in worker_thread () from /opt/couchbase/bin/../lib/libstatistics.so #2 0x00007f829311be65 in start_thread () from /lib64/libpthread.so.0 #3 0x00007f8292e4488d in clone () from /lib64/libc.so.6 Thread 55 (Thread 0x7f828ec79700 (LWP 61552)): #0 0x00007f8292e39bed in poll () from /lib64/libc.so.6 #1 0x00007f8296ea6ad8 in master_thread () from /opt/couchbase/bin/../lib/libstatistics.so #2 0x00007f829311be65 in start_thread () from /lib64/libpthread.so.0 #3 0x00007f8292e4488d in clone () from /lib64/libc.so.6 Thread 54 (Thread 0x7f828e478700 (LWP 61553)): #0 0x00007f829311fda2 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x0000000000491280 in ExternalAuthManagerThread::run() () #2 0x00007f82958e920f in Couchbase::Thread::thread_entry() () from /opt/couchbase/bin/../lib/libplatform_so.so.0.1.0 #3 0x00007f82958de2d7 in platform_thread_wrap(void*) () from /opt/couchbase/bin/../lib/libplatform_so.so.0.1.0 #4 0x00007f829311be65 in start_thread () from /lib64/libpthread.so.0 #5 0x00007f8292e4488d in clone () from /lib64/libc.so.6 Thread 53 (Thread 0x7f828d8f7700 (LWP 61554)): #0 0x00007f829311fda2 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00000000004ef6a6 in AuditImpl::consume_events() () #2 0x00007f82958de2d7 in platform_thread_wrap(void*) () from /opt/couchbase/bin/../lib/libplatform_so.so.0.1.0 #3 0x00007f829311be65 in start_thread () from /lib64/libpthread.so.0 #4 0x00007f8292e4488d in clone () from /lib64/libc.so.6 Thread 52 (Thread 0x7f828d0f6700 (LWP 61555)): #0 0x00007f8292e44e63 in epoll_wait () from /lib64/libc.so.6 #1 0x00007f82949b63c5 in epoll_dispatch () from /opt/couchbase/bin/../lib/libevent_core-2.1.so.7 #2 0x00007f82949ad376 in event_base_loop () from /opt/couchbase/bin/../lib/libevent_core-2.1.so.7 #3 0x000000000046adf9 in worker_libevent(void*) () #4 0x00007f82958de2d7 in platform_thread_wrap(void*) () from /opt/couchbase/bin/../lib/libplatform_so.so.0.1.0 #5 0x00007f829311be65 in start_thread () from /lib64/libpthread.so.0 #6 0x00007f8292e4488d in clone () from /lib64/libc.so.6 Thread 51 (Thread 0x7f828c8f5700 (LWP 61556)): #0 0x00007f8292e44e63 in epoll_wait () from /lib64/libc.so.6 #1 0x00007f82949b63c5 in epoll_dispatch () from /opt/couchbase/bin/../lib/libevent_core-2.1.so.7 #2 0x00007f82949ad376 in event_base_loop () from /opt/couchbase/bin/../lib/libevent_core-2.1.so.7 #3 0x000000000046adf9 in worker_libevent(void*) () #4 0x00007f82958de2d7 in platform_thread_wrap(void*) () from /opt/couchbase/bin/../lib/libplatform_so.so.0.1.0 #5 0x00007f829311be65 in start_thread () from /lib64/libpthread.so.0 #6 0x00007f8292e4488d in clone () from /lib64/libc.so.6 Thread 50 (Thread 0x7f8287fff700 (LWP 61557)): #0 0x00007f8292e44e63 in epoll_wait () from /lib64/libc.so.6 #1 0x00007f82949b63c5 in epoll_dispatch () from /opt/couchbase/bin/../lib/libevent_core-2.1.so.7 #2 0x00007f82949ad376 in event_base_loop () from /opt/couchbase/bin/../lib/libevent_core-2.1.so.7 #3 0x000000000046adf9 in worker_libevent(void*) () #4 0x00007f82958de2d7 in platform_thread_wrap(void*) () from /opt/couchbase/bin/../lib/libplatform_so.so.0.1.0 #5 0x00007f829311be65 in start_thread () from /lib64/libpthread.so.0 #6 0x00007f8292e4488d in clone () from /lib64/libc.so.6 Thread 49 (Thread 0x7f82877fe700 (LWP 61558)): #0 0x00007f8292e44e63 in epoll_wait () from /lib64/libc.so.6 #1 0x00007f82949b63c5 in epoll_dispatch () from /opt/couchbase/bin/../lib/libevent_core-2.1.so.7 #2 0x00007f82949ad376 in event_base_loop () from /opt/couchbase/bin/../lib/libevent_core-2.1.so.7 #3 0x000000000046adf9 in worker_libevent(void*) () #4 0x00007f82958de2d7 in platform_thread_wrap(void*) () from /opt/couchbase/bin/../lib/libplatform_so.so.0.1.0 #5 0x00007f829311be65 in start_thread () from /lib64/libpthread.so.0 #6 0x00007f8292e4488d in clone () from /lib64/libc.so.6 Thread 48 (Thread 0x7f8286ffd700 (LWP 61559)): #0 0x00007f8292e44e63 in epoll_wait () from /lib64/libc.so.6 #1 0x00007f82949b63c5 in epoll_dispatch () from /opt/couchbase/bin/../lib/libevent_core-2.1.so.7 #2 0x00007f82949ad376 in event_base_loop () from /opt/couchbase/bin/../lib/libevent_core-2.1.so.7 #3 0x000000000046adf9 in worker_libevent(void*) () #4 0x00007f82958de2d7 in platform_thread_wrap(void*) () from /opt/couchbase/bin/../lib/libplatform_so.so.0.1.0 #5 0x00007f829311be65 in start_thread () from /lib64/libpthread.so.0 #6 0x00007f8292e4488d in clone () from /lib64/libc.so.6 Thread 47 (Thread 0x7f82867fc700 (LWP 61560)): #0 0x00007f8292e44e63 in epoll_wait () from /lib64/libc.so.6 #1 0x00007f82949b63c5 in epoll_dispatch () from /opt/couchbase/bin/../lib/libevent_core-2.1.so.7 #2 0x00007f82949ad376 in event_base_loop () from /opt/couchbase/bin/../lib/libevent_core-2.1.so.7 #3 0x000000000046adf9 in worker_libevent(void*) () #4 0x00007f82958de2d7 in platform_thread_wrap(void*) () from /opt/couchbase/bin/../lib/libplatform_so.so.0.1.0 #5 0x00007f829311be65 in start_thread () from /lib64/libpthread.so.0 #6 0x00007f8292e4488d in clone () from /lib64/libc.so.6 Thread 46 (Thread 0x7f8285ffb700 (LWP 61561)): #0 0x00007f8292e44e63 in epoll_wait () from /lib64/libc.so.6 #1 0x00007f82949b63c5 in epoll_dispatch () from /opt/couchbase/bin/../lib/libevent_core-2.1.so.7 #2 0x00007f82949ad376 in event_base_loop () from /opt/couchbase/bin/../lib/libevent_core-2.1.so.7 #3 0x000000000046adf9 in worker_libevent(void*) () #4 0x00007f82958de2d7 in platform_thread_wrap(void*) () from /opt/couchbase/bin/../lib/libplatform_so.so.0.1.0 #5 0x00007f829311be65 in start_thread () from /lib64/libpthread.so.0 #6 0x00007f8292e4488d in clone () from /lib64/libc.so.6 Thread 45 (Thread 0x7f82857fa700 (LWP 61562)): #0 0x00007f829311f9f5 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f82938fd1bc in std::condition_variable::wait(std::unique_lock&) () from /opt/couchbase/bin/../lib/libstdc++.so.6 #2 0x00000000004a1af1 in Executor::run() () #3 0x00007f82958e920f in Couchbase::Thread::thread_entry() () from /opt/couchbase/bin/../lib/libplatform_so.so.0.1.0 #4 0x00007f82958de2d7 in platform_thread_wrap(void*) () from /opt/couchbase/bin/../lib/libplatform_so.so.0.1.0 #5 0x00007f829311be65 in start_thread () from /lib64/libpthread.so.0 #6 0x00007f8292e4488d in clone () from /lib64/libc.so.6 Thread 44 (Thread 0x7f8284ff9700 (LWP 61563)): #0 0x00007f829311f9f5 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f82938fd1bc in std::condition_variable::wait(std::unique_lock&) () from /opt/couchbase/bin/../lib/libstdc++.so.6 #2 0x00000000004a1af1 in Executor::run() () #3 0x00007f82958e920f in Couchbase::Thread::thread_entry() () from /opt/couchbase/bin/../lib/libplatform_so.so.0.1.0 #4 0x00007f82958de2d7 in platform_thread_wrap(void*) () from /opt/couchbase/bin/../lib/libplatform_so.so.0.1.0 #5 0x00007f829311be65 in start_thread () from /lib64/libpthread.so.0 #6 0x00007f8292e4488d in clone () from /lib64/libc.so.6 Thread 43 (Thread 0x7f827ffff700 (LWP 61564)): #0 0x00007f829311f9f5 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f82938fd1bc in std::condition_variable::wait(std::unique_lock&) () from /opt/couchbase/bin/../lib/libstdc++.so.6 #2 0x00000000004a1af1 in Executor::run() () #3 0x00007f82958e920f in Couchbase::Thread::thread_entry() () from /opt/couchbase/bin/../lib/libplatform_so.so.0.1.0 #4 0x00007f82958de2d7 in platform_thread_wrap(void*) () from /opt/couchbase/bin/../lib/libplatform_so.so.0.1.0 #5 0x00007f829311be65 in start_thread () from /lib64/libpthread.so.0 #6 0x00007f8292e4488d in clone () from /lib64/libc.so.6 Thread 42 (Thread 0x7f827f7fe700 (LWP 61565)): #0 0x00007f829311f9f5 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f82938fd1bc in std::condition_variable::wait(std::unique_lock&) () from /opt/couchbase/bin/../lib/libstdc++.so.6 #2 0x00000000004a1af1 in Executor::run() () #3 0x00007f82958e920f in Couchbase::Thread::thread_entry() () from /opt/couchbase/bin/../lib/libplatform_so.so.0.1.0 #4 0x00007f82958de2d7 in platform_thread_wrap(void*) () from /opt/couchbase/bin/../lib/libplatform_so.so.0.1.0 #5 0x00007f829311be65 in start_thread () from /lib64/libpthread.so.0 #6 0x00007f8292e4488d in clone () from /lib64/libc.so.6 Thread 41 (Thread 0x7f827effd700 (LWP 61566)): #0 0x00007f829311f9f5 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f82938fd1bc in std::condition_variable::wait(std::unique_lock&) () from /opt/couchbase/bin/../lib/libstdc++.so.6 #2 0x00000000004a1af1 in Executor::run() () #3 0x00007f82958e920f in Couchbase::Thread::thread_entry() () from /opt/couchbase/bin/../lib/libplatform_so.so.0.1.0 #4 0x00007f82958de2d7 in platform_thread_wrap(void*) () from /opt/couchbase/bin/../lib/libplatform_so.so.0.1.0 #5 0x00007f829311be65 in start_thread () from /lib64/libpthread.so.0 #6 0x00007f8292e4488d in clone () from /lib64/libc.so.6 Thread 40 (Thread 0x7f827e7fc700 (LWP 61567)): #0 0x00007f829311f9f5 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f82938fd1bc in std::condition_variable::wait(std::unique_lock&) () from /opt/couchbase/bin/../lib/libstdc++.so.6 #2 0x00000000004a1af1 in Executor::run() () #3 0x00007f82958e920f in Couchbase::Thread::thread_entry() () from /opt/couchbase/bin/../lib/libplatform_so.so.0.1.0 #4 0x00007f82958de2d7 in platform_thread_wrap(void*) () from /opt/couchbase/bin/../lib/libplatform_so.so.0.1.0 #5 0x00007f829311be65 in start_thread () from /lib64/libpthread.so.0 #6 0x00007f8292e4488d in clone () from /lib64/libc.so.6 Thread 39 (Thread 0x7f827dffb700 (LWP 61568)): #0 0x00007f829311f9f5 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f82938fd1bc in std::condition_variable::wait(std::unique_lock&) () from /opt/couchbase/bin/../lib/libstdc++.so.6 #2 0x00000000004a1af1 in Executor::run() () #3 0x00007f82958e920f in Couchbase::Thread::thread_entry() () from /opt/couchbase/bin/../lib/libplatform_so.so.0.1.0 #4 0x00007f82958de2d7 in platform_thread_wrap(void*) () from /opt/couchbase/bin/../lib/libplatform_so.so.0.1.0 #5 0x00007f829311be65 in start_thread () from /lib64/libpthread.so.0 #6 0x00007f8292e4488d in clone () from /lib64/libc.so.6 Thread 38 (Thread 0x7f827afff700 (LWP 61570)): #0 0x00007f829311fda2 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f8294beee51 in background_thread_entry () from /opt/couchbase/bin/../lib/libjemalloc.so.2 #2 0x00007f829311be65 in start_thread () from /lib64/libpthread.so.0 #3 0x00007f8292e4488d in clone () from /lib64/libc.so.6 Thread 37 (Thread 0x7f827a5fe700 (LWP 61571)): #0 0x00007f8292e3eba9 in syscall () from /lib64/libc.so.6 #1 0x00007f829743ddbd in folly::detail::futexWaitImpl(std::atomic const*, unsigned int, std::chrono::time_point > > const*, std::chrono::time_point > > const*, unsigned int) () from /opt/couchbase/bin/../lib/libep.so #2 0x00007f8297444fca in folly::detail::FutexResult folly::detail::futexWaitUntil, std::chrono::_V2::steady_clock, std::chrono::duration > >(std::atomic const*, unsigned int, std::chrono::time_point > > const&, unsigned int) () from /opt/couchbase/bin/../lib/libep.so #3 0x00007f829744a8dd in bool folly::SaturatingSemaphore::tryWaitSlow > >(std::chrono::time_point > > const&, folly::WaitOptions const&) () from /opt/couchbase/bin/../lib/libep.so #4 0x00007f829744af46 in bool folly::detail::LifoSemBase, std::atomic>::try_wait_until > >(std::chrono::time_point > > const&) () from /opt/couchbase/bin/../lib/libep.so #5 0x00007f829744e6a0 in folly::UnboundedBlockingQueue::try_take_for(std::chrono::duration >) () from /opt/couchbase/bin/../lib/libep.so #6 0x00007f8297443337 in folly::CPUThreadPoolExecutor::threadRun(std::shared_ptr) () from /opt/couchbase/bin/../lib/libep.so #7 0x00007f829745dfd9 in void folly::detail::function::FunctionTraits::callBig))(std::shared_ptr)> >(folly::detail::function::Data&) () from /opt/couchbase/bin/../lib/libep.so #8 0x00007f82972db043 in void folly::detail::function::FunctionTraits::callBig&&)::{lambda()#1}>(folly::detail::function::Data&) () from /opt/couchbase/bin/../lib/libep.so #9 0x00007f8293902dcf in execute_native_thread_routine () from /opt/couchbase/bin/../lib/libstdc++.so.6 #10 0x00007f829311be65 in start_thread () from /lib64/libpthread.so.0 #11 0x00007f8292e4488d in clone () from /lib64/libc.so.6 Thread 36 (Thread 0x7f8279dfd700 (LWP 61572)): #0 0x00007f8292e3eba9 in syscall () from /lib64/libc.so.6 #1 0x00007f829743ddbd in folly::detail::futexWaitImpl(std::atomic const*, unsigned int, std::chrono::time_point > > const*, std::chrono::time_point > > const*, unsigned int) () from /opt/couchbase/bin/../lib/libep.so #2 0x00007f8297444fca in folly::detail::FutexResult folly::detail::futexWaitUntil, std::chrono::_V2::steady_clock, std::chrono::duration > >(std::atomic const*, unsigned int, std::chrono::time_point > > const&, unsigned int) () from /opt/couchbase/bin/../lib/libep.so #3 0x00007f829744a8dd in bool folly::SaturatingSemaphore::tryWaitSlow > >(std::chrono::time_point > > const&, folly::WaitOptions const&) () from /opt/couchbase/bin/../lib/libep.so #4 0x00007f829744af46 in bool folly::detail::LifoSemBase, std::atomic>::try_wait_until > >(std::chrono::time_point > > const&) () from /opt/couchbase/bin/../lib/libep.so #5 0x00007f829744e6a0 in folly::UnboundedBlockingQueue::try_take_for(std::chrono::duration >) () from /opt/couchbase/bin/../lib/libep.so #6 0x00007f8297443337 in folly::CPUThreadPoolExecutor::threadRun(std::shared_ptr) () from /opt/couchbase/bin/../lib/libep.so #7 0x00007f829745dfd9 in void folly::detail::function::FunctionTraits::callBig))(std::shared_ptr)> >(folly::detail::function::Data&) () from /opt/couchbase/bin/../lib/libep.so #8 0x00007f82972db043 in void folly::detail::function::FunctionTraits::callBig&&)::{lambda()#1}>(folly::detail::function::Data&) () from /opt/couchbase/bin/../lib/libep.so #9 0x00007f8293902dcf in execute_native_thread_routine () from /opt/couchbase/bin/../lib/libstdc++.so.6 #10 0x00007f829311be65 in start_thread () from /lib64/libpthread.so.0 #11 0x00007f8292e4488d in clone () from /lib64/libc.so.6 Thread 35 (Thread 0x7f82795fc700 (LWP 61573)): #0 0x00007f8292e3eba9 in syscall () from /lib64/libc.so.6 #1 0x00007f829743ddbd in folly::detail::futexWaitImpl(std::atomic const*, unsigned int, std::chrono::time_point > > const*, std::chrono::time_point > > const*, unsigned int) () from /opt/couchbase/bin/../lib/libep.so #2 0x00007f8297444fca in folly::detail::FutexResult folly::detail::futexWaitUntil, std::chrono::_V2::steady_clock, std::chrono::duration > >(std::atomic const*, unsigned int, std::chrono::time_point > > const&, unsigned int) () from /opt/couchbase/bin/../lib/libep.so #3 0x00007f829744a8dd in bool folly::SaturatingSemaphore::tryWaitSlow > >(std::chrono::time_point > > const&, folly::WaitOptions const&) () from /opt/couchbase/bin/../lib/libep.so #4 0x00007f829744af46 in bool folly::detail::LifoSemBase, std::atomic>::try_wait_until > >(std::chrono::time_point > > const&) () from /opt/couchbase/bin/../lib/libep.so #5 0x00007f829744e6a0 in folly::UnboundedBlockingQueue::try_take_for(std::chrono::duration >) () from /opt/couchbase/bin/../lib/libep.so #6 0x00007f8297443337 in folly::CPUThreadPoolExecutor::threadRun(std::shared_ptr) () from /opt/couchbase/bin/../lib/libep.so #7 0x00007f829745dfd9 in void folly::detail::function::FunctionTraits::callBig))(std::shared_ptr)> >(folly::detail::function::Data&) () from /opt/couchbase/bin/../lib/libep.so #8 0x00007f82972db043 in void folly::detail::function::FunctionTraits::callBig&&)::{lambda()#1}>(folly::detail::function::Data&) () from /opt/couchbase/bin/../lib/libep.so #9 0x00007f8293902dcf in execute_native_thread_routine () from /opt/couchbase/bin/../lib/libstdc++.so.6 #10 0x00007f829311be65 in start_thread () from /lib64/libpthread.so.0 #11 0x00007f8292e4488d in clone () from /lib64/libc.so.6 Thread 34 (Thread 0x7f8278dfb700 (LWP 61574)): #0 0x00007f8292e3eba9 in syscall () from /lib64/libc.so.6 #1 0x00007f829743ddbd in folly::detail::futexWaitImpl(std::atomic const*, unsigned int, std::chrono::time_point > > const*, std::chrono::time_point > > const*, unsigned int) () from /opt/couchbase/bin/../lib/libep.so #2 0x00007f8297444fca in folly::detail::FutexResult folly::detail::futexWaitUntil, std::chrono::_V2::steady_clock, std::chrono::duration > >(std::atomic const*, unsigned int, std::chrono::time_point > > const&, unsigned int) () from /opt/couchbase/bin/../lib/libep.so #3 0x00007f829744a8dd in bool folly::SaturatingSemaphore::tryWaitSlow > >(std::chrono::time_point > > const&, folly::WaitOptions const&) () from /opt/couchbase/bin/../lib/libep.so #4 0x00007f829744af46 in bool folly::detail::LifoSemBase, std::atomic>::try_wait_until > >(std::chrono::time_point > > const&) () from /opt/couchbase/bin/../lib/libep.so #5 0x00007f829744e6a0 in folly::UnboundedBlockingQueue::try_take_for(std::chrono::duration >) () from /opt/couchbase/bin/../lib/libep.so #6 0x00007f8297443337 in folly::CPUThreadPoolExecutor::threadRun(std::shared_ptr) () from /opt/couchbase/bin/../lib/libep.so #7 0x00007f829745dfd9 in void folly::detail::function::FunctionTraits::callBig))(std::shared_ptr)> >(folly::detail::function::Data&) () from /opt/couchbase/bin/../lib/libep.so #8 0x00007f82972db043 in void folly::detail::function::FunctionTraits::callBig&&)::{lambda()#1}>(folly::detail::function::Data&) () from /opt/couchbase/bin/../lib/libep.so #9 0x00007f8293902dcf in execute_native_thread_routine () from /opt/couchbase/bin/../lib/libstdc++.so.6 #10 0x00007f829311be65 in start_thread () from /lib64/libpthread.so.0 #11 0x00007f8292e4488d in clone () from /lib64/libc.so.6 Thread 33 (Thread 0x7f82727da700 (LWP 61575)): #0 0x00007f8295b91d5b in LZ4_decompress_safe () from /opt/couchbase/bin/../lib/liblz4.so.1 #1 0x00007f829693fd83 in magma::Compression::Decompress(magma::Slice, magma::Slice) () from /opt/couchbase/bin/../lib/libmagma.so #2 0x00007f8296888538 in magma::SSTable::BlockDecoder::Decode(magma::Compression*, std::unique_ptr >, unsigned long, std::unique_ptr >&, magma::Slice&) () from /opt/couchbase/bin/../lib/libmagma.so #3 0x00007f829688b2af in magma::SSTable::readBlock(magma::BlockPointer const&, magma::Block*, magma::ReaderStats&, bool, magma::BufferedFileReader*) () from /opt/couchbase/bin/../lib/libmagma.so #4 0x00007f829688f0d3 in magma::SSTableIterator::initIterators(magma::Slice const&) () from /opt/couchbase/bin/../lib/libmagma.so #5 0x00007f829689012c in magma::SSTableIterator::Seek(magma::Slice const&) () from /opt/couchbase/bin/../lib/libmagma.so #6 0x00007f829692814c in magma::LSMTree::Get(magma::Slice const&, std::shared_ptr&, magma::LookupBuffer&, magma::Slice&, bool&, bool) () from /opt/couchbase/bin/../lib/libmagma.so #7 0x00007f8296917b40 in magma::BySeqIterator::doKeyIndexFetch() () from /opt/couchbase/bin/../lib/libmagma.so #8 0x00007f8296918024 in magma::BySeqIterator::doFetch() () from /opt/couchbase/bin/../lib/libmagma.so #9 0x00007f82973c1950 in MagmaKVStore::scan(BySeqnoScanContext&) () from /opt/couchbase/bin/../lib/libep.so #10 0x00007f8297366a57 in Warmup::loadKVPairsforShard(unsigned short) () from /opt/couchbase/bin/../lib/libep.so #11 0x00007f8297372b70 in WarmupLoadingKVPairs::run() () from /opt/couchbase/bin/../lib/libep.so #12 0x00007f82972e2b83 in GlobalTask::execute() () from /opt/couchbase/bin/../lib/libep.so #13 0x00007f82972dcc02 in void folly::detail::function::FunctionTraits::callSmall(folly::detail::function::Data&) () from /opt/couchbase/bin/../lib/libep.so #14 0x00007f829745aeb6 in folly::ThreadPoolExecutor::runTask(std::shared_ptr const&, folly::ThreadPoolExecutor::Task&&) () from /opt/couchbase/bin/../lib/libep.so #15 0x00007f82974433aa in folly::CPUThreadPoolExecutor::threadRun(std::shared_ptr) () from /opt/couchbase/bin/../lib/libep.so #16 0x00007f829745dfd9 in void folly::detail::function::FunctionTraits::callBig))(std::shared_ptr)> >(folly::detail::function::Data&) () from /opt/couchbase/bin/../lib/libep.so #17 0x00007f82972db043 in void folly::detail::function::FunctionTraits::callBig&&)::{lambda()#1}>(folly::detail::function::Data&) () from /opt/couchbase/bin/../lib/libep.so #18 0x00007f8293902dcf in execute_native_thread_routine () from /opt/couchbase/bin/../lib/libstdc++.so.6 #19 0x00007f829311be65 in start_thread () from /lib64/libpthread.so.0 #20 0x00007f8292e4488d in clone () from /lib64/libc.so.6 Thread 32 (Thread 0x7f8271fd9700 (LWP 61576)): #0 0x00007f8292e3eba9 in syscall () from /lib64/libc.so.6 #1 0x00007f829743ddbd in folly::detail::futexWaitImpl(std::atomic const*, unsigned int, std::chrono::time_point > > const*, std::chrono::time_point > > const*, unsigned int) () from /opt/couchbase/bin/../lib/libep.so #2 0x00007f8297444fca in folly::detail::FutexResult folly::detail::futexWaitUntil, std::chrono::_V2::steady_clock, std::chrono::duration > >(std::atomic const*, unsigned int, std::chrono::time_point > > const&, unsigned int) () from /opt/couchbase/bin/../lib/libep.so #3 0x00007f829744a8dd in bool folly::SaturatingSemaphore::tryWaitSlow > >(std::chrono::time_point > > const&, folly::WaitOptions const&) () from /opt/couchbase/bin/../lib/libep.so #4 0x00007f829744af46 in bool folly::detail::LifoSemBase, std::atomic>::try_wait_until > >(std::chrono::time_point > > const&) () from /opt/couchbase/bin/../lib/libep.so #5 0x00007f829744e6a0 in folly::UnboundedBlockingQueue::try_take_for(std::chrono::duration >) () from /opt/couchbase/bin/../lib/libep.so #6 0x00007f8297443337 in folly::CPUThreadPoolExecutor::threadRun(std::shared_ptr) () from /opt/couchbase/bin/../lib/libep.so #7 0x00007f829745dfd9 in void folly::detail::function::FunctionTraits::callBig))(std::shared_ptr)> >(folly::detail::function::Data&) () from /opt/couchbase/bin/../lib/libep.so #8 0x00007f82972db043 in void folly::detail::function::FunctionTraits::callBig&&)::{lambda()#1}>(folly::detail::function::Data&) () from /opt/couchbase/bin/../lib/libep.so #9 0x00007f8293902dcf in execute_native_thread_routine () from /opt/couchbase/bin/../lib/libstdc++.so.6 #10 0x00007f829311be65 in start_thread () from /lib64/libpthread.so.0 #11 0x00007f8292e4488d in clone () from /lib64/libc.so.6 Thread 31 (Thread 0x7f82717d8700 (LWP 61577)): #0 0x00007f8292e3eba9 in syscall () from /lib64/libc.so.6 #1 0x00007f829743ddbd in folly::detail::futexWaitImpl(std::atomic const*, unsigned int, std::chrono::time_point > > const*, std::chrono::time_point > > const*, unsigned int) () from /opt/couchbase/bin/../lib/libep.so #2 0x00007f8297444fca in folly::detail::FutexResult folly::detail::futexWaitUntil, std::chrono::_V2::steady_clock, std::chrono::duration > >(std::atomic const*, unsigned int, std::chrono::time_point > > const&, unsigned int) () from /opt/couchbase/bin/../lib/libep.so #3 0x00007f829744aa17 in bool folly::SaturatingSemaphore::tryWaitSlow > >(std::chrono::time_point > > const&, folly::WaitOptions const&) () from /opt/couchbase/bin/../lib/libep.so #4 0x00007f829744af46 in bool folly::detail::LifoSemBase, std::atomic>::try_wait_until > >(std::chrono::time_point > > const&) () from /opt/couchbase/bin/../lib/libep.so #5 0x00007f829744e6a0 in folly::UnboundedBlockingQueue::try_take_for(std::chrono::duration >) () from /opt/couchbase/bin/../lib/libep.so #6 0x00007f8297443337 in folly::CPUThreadPoolExecutor::threadRun(std::shared_ptr) () from /opt/couchbase/bin/../lib/libep.so #7 0x00007f829745dfd9 in void folly::detail::function::FunctionTraits::callBig))(std::shared_ptr)> >(folly::detail::function::Data&) () from /opt/couchbase/bin/../lib/libep.so #8 0x00007f82972db043 in void folly::detail::function::FunctionTraits::callBig&&)::{lambda()#1}>(folly::detail::function::Data&) () from /opt/couchbase/bin/../lib/libep.so #9 0x00007f8293902dcf in execute_native_thread_routine () from /opt/couchbase/bin/../lib/libstdc++.so.6 #10 0x00007f829311be65 in start_thread () from /lib64/libpthread.so.0 #11 0x00007f8292e4488d in clone () from /lib64/libc.so.6 Thread 30 (Thread 0x7f8270fd7700 (LWP 61578)): #0 0x00007f8292e3eba9 in syscall () from /lib64/libc.so.6 #1 0x00007f829743ddbd in folly::detail::futexWaitImpl(std::atomic const*, unsigned int, std::chrono::time_point > > const*, std::chrono::time_point > > const*, unsigned int) () from /opt/couchbase/bin/../lib/libep.so #2 0x00007f8297444fca in folly::detail::FutexResult folly::detail::futexWaitUntil, std::chrono::_V2::steady_clock, std::chrono::duration > >(std::atomic const*, unsigned int, std::chrono::time_point > > const&, unsigned int) () from /opt/couchbase/bin/../lib/libep.so #3 0x00007f829744aa17 in bool folly::SaturatingSemaphore::tryWaitSlow > >(std::chrono::time_point > > const&, folly::WaitOptions const&) () from /opt/couchbase/bin/../lib/libep.so #4 0x00007f829744af46 in bool folly::detail::LifoSemBase, std::atomic>::try_wait_until > >(std::chrono::time_point > > const&) () from /opt/couchbase/bin/../lib/libep.so #5 0x00007f829744e6a0 in folly::UnboundedBlockingQueue::try_take_for(std::chrono::duration >) () from /opt/couchbase/bin/../lib/libep.so #6 0x00007f8297443337 in folly::CPUThreadPoolExecutor::threadRun(std::shared_ptr) () from /opt/couchbase/bin/../lib/libep.so #7 0x00007f829745dfd9 in void folly::detail::function::FunctionTraits::callBig))(std::shared_ptr)> >(folly::detail::function::Data&) () from /opt/couchbase/bin/../lib/libep.so #8 0x00007f82972db043 in void folly::detail::function::FunctionTraits::callBig&&)::{lambda()#1}>(folly::detail::function::Data&) () from /opt/couchbase/bin/../lib/libep.so #9 0x00007f8293902dcf in execute_native_thread_routine () from /opt/couchbase/bin/../lib/libstdc++.so.6 #10 0x00007f829311be65 in start_thread () from /lib64/libpthread.so.0 #11 0x00007f8292e4488d in clone () from /lib64/libc.so.6 Thread 29 (Thread 0x7f8233fff700 (LWP 61579)): #0 0x00007f8292e3eba9 in syscall () from /lib64/libc.so.6 #1 0x00007f829743ddbd in folly::detail::futexWaitImpl(std::atomic const*, unsigned int, std::chrono::time_point > > const*, std::chrono::time_point > > const*, unsigned int) () from /opt/couchbase/bin/../lib/libep.so #2 0x00007f8297444fca in folly::detail::FutexResult folly::detail::futexWaitUntil, std::chrono::_V2::steady_clock, std::chrono::duration > >(std::atomic const*, unsigned int, std::chrono::time_point > > const&, unsigned int) () from /opt/couchbase/bin/../lib/libep.so #3 0x00007f829744a8dd in bool folly::SaturatingSemaphore::tryWaitSlow > >(std::chrono::time_point > > const&, folly::WaitOptions const&) () from /opt/couchbase/bin/../lib/libep.so #4 0x00007f829744af46 in bool folly::detail::LifoSemBase, std::atomic>::try_wait_until > >(std::chrono::time_point > > const&) () from /opt/couchbase/bin/../lib/libep.so #5 0x00007f829744e6a0 in folly::UnboundedBlockingQueue::try_take_for(std::chrono::duration >) () from /opt/couchbase/bin/../lib/libep.so #6 0x00007f8297443337 in folly::CPUThreadPoolExecutor::threadRun(std::shared_ptr) () from /opt/couchbase/bin/../lib/libep.so #7 0x00007f829745dfd9 in void folly::detail::function::FunctionTraits::callBig))(std::shared_ptr)> >(folly::detail::function::Data&) () from /opt/couchbase/bin/../lib/libep.so #8 0x00007f82972db043 in void folly::detail::function::FunctionTraits::callBig&&)::{lambda()#1}>(folly::detail::function::Data&) () from /opt/couchbase/bin/../lib/libep.so #9 0x00007f8293902dcf in execute_native_thread_routine () from /opt/couchbase/bin/../lib/libstdc++.so.6 #10 0x00007f829311be65 in start_thread () from /lib64/libpthread.so.0 #11 0x00007f8292e4488d in clone () from /lib64/libc.so.6 Thread 28 (Thread 0x7f82337fe700 (LWP 61580)): #0 0x00007f8292e3eba9 in syscall () from /lib64/libc.so.6 #1 0x00007f829743ddbd in folly::detail::futexWaitImpl(std::atomic const*, unsigned int, std::chrono::time_point > > const*, std::chrono::time_point > > const*, unsigned int) () from /opt/couchbase/bin/../lib/libep.so #2 0x00007f8297444fca in folly::detail::FutexResult folly::detail::futexWaitUntil, std::chrono::_V2::steady_clock, std::chrono::duration > >(std::atomic const*, unsigned int, std::chrono::time_point > > const&, unsigned int) () from /opt/couchbase/bin/../lib/libep.so #3 0x00007f829744a8dd in bool folly::SaturatingSemaphore::tryWaitSlow > >(std::chrono::time_point > > const&, folly::WaitOptions const&) () from /opt/couchbase/bin/../lib/libep.so #4 0x00007f829744af46 in bool folly::detail::LifoSemBase, std::atomic>::try_wait_until > >(std::chrono::time_point > > const&) () from /opt/couchbase/bin/../lib/libep.so #5 0x00007f829744e6a0 in folly::UnboundedBlockingQueue::try_take_for(std::chrono::duration >) () from /opt/couchbase/bin/../lib/libep.so #6 0x00007f8297443337 in folly::CPUThreadPoolExecutor::threadRun(std::shared_ptr) () from /opt/couchbase/bin/../lib/libep.so #7 0x00007f829745dfd9 in void folly::detail::function::FunctionTraits::callBig))(std::shared_ptr)> >(folly::detail::function::Data&) () from /opt/couchbase/bin/../lib/libep.so #8 0x00007f82972db043 in void folly::detail::function::FunctionTraits::callBig&&)::{lambda()#1}>(folly::detail::function::Data&) () from /opt/couchbase/bin/../lib/libep.so #9 0x00007f8293902dcf in execute_native_thread_routine () from /opt/couchbase/bin/../lib/libstdc++.so.6 #10 0x00007f829311be65 in start_thread () from /lib64/libpthread.so.0 #11 0x00007f8292e4488d in clone () from /lib64/libc.so.6 Thread 27 (Thread 0x7f8232ffd700 (LWP 61581)): #0 0x00007f8292e3eba9 in syscall () from /lib64/libc.so.6 #1 0x00007f829743ddbd in folly::detail::futexWaitImpl(std::atomic const*, unsigned int, std::chrono::time_point > > const*, std::chrono::time_point > > const*, unsigned int) () from /opt/couchbase/bin/../lib/libep.so #2 0x00007f8297444fca in folly::detail::FutexResult folly::detail::futexWaitUntil, std::chrono::_V2::steady_clock, std::chrono::duration > >(std::atomic const*, unsigned int, std::chrono::time_point > > const&, unsigned int) () from /opt/couchbase/bin/../lib/libep.so #3 0x00007f829744a8dd in bool folly::SaturatingSemaphore::tryWaitSlow > >(std::chrono::time_point > > const&, folly::WaitOptions const&) () from /opt/couchbase/bin/../lib/libep.so #4 0x00007f829744af46 in bool folly::detail::LifoSemBase, std::atomic>::try_wait_until > >(std::chrono::time_point > > const&) () from /opt/couchbase/bin/../lib/libep.so #5 0x00007f829744e6a0 in folly::UnboundedBlockingQueue::try_take_for(std::chrono::duration >) () from /opt/couchbase/bin/../lib/libep.so #6 0x00007f8297443337 in folly::CPUThreadPoolExecutor::threadRun(std::shared_ptr) () from /opt/couchbase/bin/../lib/libep.so #7 0x00007f829745dfd9 in void folly::detail::function::FunctionTraits::callBig))(std::shared_ptr)> >(folly::detail::function::Data&) () from /opt/couchbase/bin/../lib/libep.so #8 0x00007f82972db043 in void folly::detail::function::FunctionTraits::callBig&&)::{lambda()#1}>(folly::detail::function::Data&) () from /opt/couchbase/bin/../lib/libep.so #9 0x00007f8293902dcf in execute_native_thread_routine () from /opt/couchbase/bin/../lib/libstdc++.so.6 #10 0x00007f829311be65 in start_thread () from /lib64/libpthread.so.0 #11 0x00007f8292e4488d in clone () from /lib64/libc.so.6 Thread 26 (Thread 0x7f82327fc700 (LWP 61582)): #0 0x00007f8292e3eba9 in syscall () from /lib64/libc.so.6 #1 0x00007f829743ddbd in folly::detail::futexWaitImpl(std::atomic const*, unsigned int, std::chrono::time_point > > const*, std::chrono::time_point > > const*, unsigned int) () from /opt/couchbase/bin/../lib/libep.so #2 0x00007f8297444fca in folly::detail::FutexResult folly::detail::futexWaitUntil, std::chrono::_V2::steady_clock, std::chrono::duration > >(std::atomic const*, unsigned int, std::chrono::time_point > > const&, unsigned int) () from /opt/couchbase/bin/../lib/libep.so #3 0x00007f829744a8dd in bool folly::SaturatingSemaphore::tryWaitSlow > >(std::chrono::time_point > > const&, folly::WaitOptions const&) () from /opt/couchbase/bin/../lib/libep.so #4 0x00007f829744af46 in bool folly::detail::LifoSemBase, std::atomic>::try_wait_until > >(std::chrono::time_point > > const&) () from /opt/couchbase/bin/../lib/libep.so #5 0x00007f829744e6a0 in folly::UnboundedBlockingQueue::try_take_for(std::chrono::duration >) () from /opt/couchbase/bin/../lib/libep.so #6 0x00007f8297443337 in folly::CPUThreadPoolExecutor::threadRun(std::shared_ptr) () from /opt/couchbase/bin/../lib/libep.so #7 0x00007f829745dfd9 in void folly::detail::function::FunctionTraits::callBig))(std::shared_ptr)> >(folly::detail::function::Data&) () from /opt/couchbase/bin/../lib/libep.so #8 0x00007f82972db043 in void folly::detail::function::FunctionTraits::callBig&&)::{lambda()#1}>(folly::detail::function::Data&) () from /opt/couchbase/bin/../lib/libep.so #9 0x00007f8293902dcf in execute_native_thread_routine () from /opt/couchbase/bin/../lib/libstdc++.so.6 #10 0x00007f829311be65 in start_thread () from /lib64/libpthread.so.0 #11 0x00007f8292e4488d in clone () from /lib64/libc.so.6 Thread 25 (Thread 0x7f8231ffb700 (LWP 61583)): #0 0x00007f8292e3eba9 in syscall () from /lib64/libc.so.6 #1 0x00007f829743ddbd in folly::detail::futexWaitImpl(std::atomic const*, unsigned int, std::chrono::time_point > > const*, std::chrono::time_point > > const*, unsigned int) () from /opt/couchbase/bin/../lib/libep.so #2 0x00007f8297444fca in folly::detail::FutexResult folly::detail::futexWaitUntil, std::chrono::_V2::steady_clock, std::chrono::duration > >(std::atomic const*, unsigned int, std::chrono::time_point > > const&, unsigned int) () from /opt/couchbase/bin/../lib/libep.so #3 0x00007f829744a8dd in bool folly::SaturatingSemaphore::tryWaitSlow > >(std::chrono::time_point > > const&, folly::WaitOptions const&) () from /opt/couchbase/bin/../lib/libep.so #4 0x00007f829744af46 in bool folly::detail::LifoSemBase, std::atomic>::try_wait_until > >(std::chrono::time_point > > const&) () from /opt/couchbase/bin/../lib/libep.so #5 0x00007f829744e6a0 in folly::UnboundedBlockingQueue::try_take_for(std::chrono::duration >) () from /opt/couchbase/bin/../lib/libep.so #6 0x00007f8297443337 in folly::CPUThreadPoolExecutor::threadRun(std::shared_ptr) () from /opt/couchbase/bin/../lib/libep.so #7 0x00007f829745dfd9 in void folly::detail::function::FunctionTraits::callBig))(std::shared_ptr)> >(folly::detail::function::Data&) () from /opt/couchbase/bin/../lib/libep.so #8 0x00007f82972db043 in void folly::detail::function::FunctionTraits::callBig&&)::{lambda()#1}>(folly::detail::function::Data&) () from /opt/couchbase/bin/../lib/libep.so #9 0x00007f8293902dcf in execute_native_thread_routine () from /opt/couchbase/bin/../lib/libstdc++.so.6 #10 0x00007f829311be65 in start_thread () from /lib64/libpthread.so.0 #11 0x00007f8292e4488d in clone () from /lib64/libc.so.6 Thread 24 (Thread 0x7f82317fa700 (LWP 61584)): #0 0x00007f8292e3eba9 in syscall () from /lib64/libc.so.6 #1 0x00007f829743ddbd in folly::detail::futexWaitImpl(std::atomic const*, unsigned int, std::chrono::time_point > > const*, std::chrono::time_point > > const*, unsigned int) () from /opt/couchbase/bin/../lib/libep.so #2 0x00007f8297444fca in folly::detail::FutexResult folly::detail::futexWaitUntil, std::chrono::_V2::steady_clock, std::chrono::duration > >(std::atomic const*, unsigned int, std::chrono::time_point > > const&, unsigned int) () from /opt/couchbase/bin/../lib/libep.so #3 0x00007f829744aa17 in bool folly::SaturatingSemaphore::tryWaitSlow > >(std::chrono::time_point > > const&, folly::WaitOptions const&) () from /opt/couchbase/bin/../lib/libep.so #4 0x00007f829744af46 in bool folly::detail::LifoSemBase, std::atomic>::try_wait_until > >(std::chrono::time_point > > const&) () from /opt/couchbase/bin/../lib/libep.so #5 0x00007f829744e6a0 in folly::UnboundedBlockingQueue::try_take_for(std::chrono::duration >) () from /opt/couchbase/bin/../lib/libep.so #6 0x00007f8297443337 in folly::CPUThreadPoolExecutor::threadRun(std::shared_ptr) () from /opt/couchbase/bin/../lib/libep.so #7 0x00007f829745dfd9 in void folly::detail::function::FunctionTraits::callBig))(std::shared_ptr)> >(folly::detail::function::Data&) () from /opt/couchbase/bin/../lib/libep.so #8 0x00007f82972db043 in void folly::detail::function::FunctionTraits::callBig&&)::{lambda()#1}>(folly::detail::function::Data&) () from /opt/couchbase/bin/../lib/libep.so #9 0x00007f8293902dcf in execute_native_thread_routine () from /opt/couchbase/bin/../lib/libstdc++.so.6 #10 0x00007f829311be65 in start_thread () from /lib64/libpthread.so.0 #11 0x00007f8292e4488d in clone () from /lib64/libc.so.6 Thread 23 (Thread 0x7f8230ff9700 (LWP 61585)): #0 0x00007f8292e3eba9 in syscall () from /lib64/libc.so.6 #1 0x00007f829743ddbd in folly::detail::futexWaitImpl(std::atomic const*, unsigned int, std::chrono::time_point > > const*, std::chrono::time_point > > const*, unsigned int) () from /opt/couchbase/bin/../lib/libep.so #2 0x00007f8297444fca in folly::detail::FutexResult folly::detail::futexWaitUntil, std::chrono::_V2::steady_clock, std::chrono::duration > >(std::atomic const*, unsigned int, std::chrono::time_point > > const&, unsigned int) () from /opt/couchbase/bin/../lib/libep.so #3 0x00007f829744aa17 in bool folly::SaturatingSemaphore::tryWaitSlow > >(std::chrono::time_point > > const&, folly::WaitOptions const&) () from /opt/couchbase/bin/../lib/libep.so #4 0x00007f829744af46 in bool folly::detail::LifoSemBase, std::atomic>::try_wait_until > >(std::chrono::time_point > > const&) () from /opt/couchbase/bin/../lib/libep.so #5 0x00007f829744e6a0 in folly::UnboundedBlockingQueue::try_take_for(std::chrono::duration >) () from /opt/couchbase/bin/../lib/libep.so #6 0x00007f8297443337 in folly::CPUThreadPoolExecutor::threadRun(std::shared_ptr) () from /opt/couchbase/bin/../lib/libep.so #7 0x00007f829745dfd9 in void folly::detail::function::FunctionTraits::callBig))(std::shared_ptr)> >(folly::detail::function::Data&) () from /opt/couchbase/bin/../lib/libep.so #8 0x00007f82972db043 in void folly::detail::function::FunctionTraits::callBig&&)::{lambda()#1}>(folly::detail::function::Data&) () from /opt/couchbase/bin/../lib/libep.so #9 0x00007f8293902dcf in execute_native_thread_routine () from /opt/couchbase/bin/../lib/libstdc++.so.6 #10 0x00007f829311be65 in start_thread () from /lib64/libpthread.so.0 #11 0x00007f8292e4488d in clone () from /lib64/libc.so.6 Thread 22 (Thread 0x7f81f3fff700 (LWP 61586)): #0 0x00007f829311fda2 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f829696c1c9 in magma::TaskQueue::dequeue(std::atomic&) () from /opt/couchbase/bin/../lib/libmagma.so #2 0x00007f829696c603 in magma::TaskWorker::loop(void*) () from /opt/couchbase/bin/../lib/libmagma.so #3 0x00007f82958de2d7 in platform_thread_wrap(void*) () from /opt/couchbase/bin/../lib/libplatform_so.so.0.1.0 #4 0x00007f829311be65 in start_thread () from /lib64/libpthread.so.0 #5 0x00007f8292e4488d in clone () from /lib64/libc.so.6 Thread 21 (Thread 0x7f81f37fe700 (LWP 61587)): #0 0x00007f829311fda2 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f829696c1c9 in magma::TaskQueue::dequeue(std::atomic&) () from /opt/couchbase/bin/../lib/libmagma.so #2 0x00007f829696c603 in magma::TaskWorker::loop(void*) () from /opt/couchbase/bin/../lib/libmagma.so #3 0x00007f82958de2d7 in platform_thread_wrap(void*) () from /opt/couchbase/bin/../lib/libplatform_so.so.0.1.0 #4 0x00007f829311be65 in start_thread () from /lib64/libpthread.so.0 #5 0x00007f8292e4488d in clone () from /lib64/libc.so.6 Thread 20 (Thread 0x7f81f2ffd700 (LWP 61588)): #0 0x00007f829311fda2 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f829696c1c9 in magma::TaskQueue::dequeue(std::atomic&) () from /opt/couchbase/bin/../lib/libmagma.so #2 0x00007f829696c603 in magma::TaskWorker::loop(void*) () from /opt/couchbase/bin/../lib/libmagma.so #3 0x00007f82958de2d7 in platform_thread_wrap(void*) () from /opt/couchbase/bin/../lib/libplatform_so.so.0.1.0 #4 0x00007f829311be65 in start_thread () from /lib64/libpthread.so.0 #5 0x00007f8292e4488d in clone () from /lib64/libc.so.6 Thread 19 (Thread 0x7f81f27fc700 (LWP 61589)): #0 0x00007f829311fda2 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f829696c1c9 in magma::TaskQueue::dequeue(std::atomic&) () from /opt/couchbase/bin/../lib/libmagma.so #2 0x00007f829696c603 in magma::TaskWorker::loop(void*) () from /opt/couchbase/bin/../lib/libmagma.so #3 0x00007f82958de2d7 in platform_thread_wrap(void*) () from /opt/couchbase/bin/../lib/libplatform_so.so.0.1.0 #4 0x00007f829311be65 in start_thread () from /lib64/libpthread.so.0 #5 0x00007f8292e4488d in clone () from /lib64/libc.so.6 Thread 18 (Thread 0x7f81f1ffb700 (LWP 61590)): #0 0x00007f829311f9f5 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f82938fd1bc in std::condition_variable::wait(std::unique_lock&) () from /opt/couchbase/bin/../lib/libstdc++.so.6 #2 0x00007f829696c0fc in magma::TaskQueue::dequeue(std::atomic&) () from /opt/couchbase/bin/../lib/libmagma.so #3 0x00007f829696c603 in magma::TaskWorker::loop(void*) () from /opt/couchbase/bin/../lib/libmagma.so #4 0x00007f82958de2d7 in platform_thread_wrap(void*) () from /opt/couchbase/bin/../lib/libplatform_so.so.0.1.0 #5 0x00007f829311be65 in start_thread () from /lib64/libpthread.so.0 #6 0x00007f8292e4488d in clone () from /lib64/libc.so.6 Thread 17 (Thread 0x7f81f17fa700 (LWP 61591)): #0 0x00007f829311f9f5 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f82938fd1bc in std::condition_variable::wait(std::unique_lock&) () from /opt/couchbase/bin/../lib/libstdc++.so.6 #2 0x00007f829696c0fc in magma::TaskQueue::dequeue(std::atomic&) () from /opt/couchbase/bin/../lib/libmagma.so #3 0x00007f829696c603 in magma::TaskWorker::loop(void*) () from /opt/couchbase/bin/../lib/libmagma.so #4 0x00007f82958de2d7 in platform_thread_wrap(void*) () from /opt/couchbase/bin/../lib/libplatform_so.so.0.1.0 #5 0x00007f829311be65 in start_thread () from /lib64/libpthread.so.0 #6 0x00007f8292e4488d in clone () from /lib64/libc.so.6 Thread 16 (Thread 0x7f81f0ff9700 (LWP 61592)): #0 0x00007f829311f9f5 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f82938fd1bc in std::condition_variable::wait(std::unique_lock&) () from /opt/couchbase/bin/../lib/libstdc++.so.6 #2 0x00007f829696c0fc in magma::TaskQueue::dequeue(std::atomic&) () from /opt/couchbase/bin/../lib/libmagma.so #3 0x00007f829696c603 in magma::TaskWorker::loop(void*) () from /opt/couchbase/bin/../lib/libmagma.so #4 0x00007f82958de2d7 in platform_thread_wrap(void*) () from /opt/couchbase/bin/../lib/libplatform_so.so.0.1.0 #5 0x00007f829311be65 in start_thread () from /lib64/libpthread.so.0 #6 0x00007f8292e4488d in clone () from /lib64/libc.so.6 Thread 15 (Thread 0x7f81f07f8700 (LWP 61593)): #0 0x00007f829311f9f5 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f82938fd1bc in std::condition_variable::wait(std::unique_lock&) () from /opt/couchbase/bin/../lib/libstdc++.so.6 #2 0x00007f829696c0fc in magma::TaskQueue::dequeue(std::atomic&) () from /opt/couchbase/bin/../lib/libmagma.so #3 0x00007f829696c603 in magma::TaskWorker::loop(void*) () from /opt/couchbase/bin/../lib/libmagma.so #4 0x00007f82958de2d7 in platform_thread_wrap(void*) () from /opt/couchbase/bin/../lib/libplatform_so.so.0.1.0 #5 0x00007f829311be65 in start_thread () from /lib64/libpthread.so.0 #6 0x00007f8292e4488d in clone () from /lib64/libc.so.6 Thread 14 (Thread 0x7f81edcff700 (LWP 61595)): #0 0x00007f829311f9f5 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f82938fd1bc in std::condition_variable::wait(std::unique_lock&) () from /opt/couchbase/bin/../lib/libstdc++.so.6 #2 0x00007f829696c0fc in magma::TaskQueue::dequeue(std::atomic&) () from /opt/couchbase/bin/../lib/libmagma.so #3 0x00007f829696c603 in magma::TaskWorker::loop(void*) () from /opt/couchbase/bin/../lib/libmagma.so #4 0x00007f82958de2d7 in platform_thread_wrap(void*) () from /opt/couchbase/bin/../lib/libplatform_so.so.0.1.0 #5 0x00007f829311be65 in start_thread () from /lib64/libpthread.so.0 #6 0x00007f8292e4488d in clone () from /lib64/libc.so.6 Thread 13 (Thread 0x7f81ed4fe700 (LWP 61596)): #0 0x00007f829311f9f5 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f82938fd1bc in std::condition_variable::wait(std::unique_lock&) () from /opt/couchbase/bin/../lib/libstdc++.so.6 #2 0x00007f829696c0fc in magma::TaskQueue::dequeue(std::atomic&) () from /opt/couchbase/bin/../lib/libmagma.so #3 0x00007f829696c603 in magma::TaskWorker::loop(void*) () from /opt/couchbase/bin/../lib/libmagma.so #4 0x00007f82958de2d7 in platform_thread_wrap(void*) () from /opt/couchbase/bin/../lib/libplatform_so.so.0.1.0 #5 0x00007f829311be65 in start_thread () from /lib64/libpthread.so.0 #6 0x00007f8292e4488d in clone () from /lib64/libc.so.6 Thread 12 (Thread 0x7f81eccfd700 (LWP 61597)): #0 0x00007f829311f9f5 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f82938fd1bc in std::condition_variable::wait(std::unique_lock&) () from /opt/couchbase/bin/../lib/libstdc++.so.6 #2 0x00007f829696c0fc in magma::TaskQueue::dequeue(std::atomic&) () from /opt/couchbase/bin/../lib/libmagma.so #3 0x00007f829696c603 in magma::TaskWorker::loop(void*) () from /opt/couchbase/bin/../lib/libmagma.so #4 0x00007f82958de2d7 in platform_thread_wrap(void*) () from /opt/couchbase/bin/../lib/libplatform_so.so.0.1.0 #5 0x00007f829311be65 in start_thread () from /lib64/libpthread.so.0 #6 0x00007f8292e4488d in clone () from /lib64/libc.so.6 Thread 11 (Thread 0x7f81ec4fc700 (LWP 61598)): #0 0x00007f829311f9f5 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f82938fd1bc in std::condition_variable::wait(std::unique_lock&) () from /opt/couchbase/bin/../lib/libstdc++.so.6 #2 0x00007f829696c0fc in magma::TaskQueue::dequeue(std::atomic&) () from /opt/couchbase/bin/../lib/libmagma.so #3 0x00007f829696c603 in magma::TaskWorker::loop(void*) () from /opt/couchbase/bin/../lib/libmagma.so #4 0x00007f82958de2d7 in platform_thread_wrap(void*) () from /opt/couchbase/bin/../lib/libplatform_so.so.0.1.0 #5 0x00007f829311be65 in start_thread () from /lib64/libpthread.so.0 #6 0x00007f8292e4488d in clone () from /lib64/libc.so.6 Thread 10 (Thread 0x7f81ebcfb700 (LWP 61599)): #0 0x00007f829311f9f5 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f82938fd1bc in std::condition_variable::wait(std::unique_lock&) () from /opt/couchbase/bin/../lib/libstdc++.so.6 #2 0x00007f829696c0fc in magma::TaskQueue::dequeue(std::atomic&) () from /opt/couchbase/bin/../lib/libmagma.so #3 0x00007f829696c603 in magma::TaskWorker::loop(void*) () from /opt/couchbase/bin/../lib/libmagma.so #4 0x00007f82958de2d7 in platform_thread_wrap(void*) () from /opt/couchbase/bin/../lib/libplatform_so.so.0.1.0 #5 0x00007f829311be65 in start_thread () from /lib64/libpthread.so.0 #6 0x00007f8292e4488d in clone () from /lib64/libc.so.6 Thread 9 (Thread 0x7f81eb4fa700 (LWP 61600)): #0 0x00007f829311f9f5 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f82938fd1bc in std::condition_variable::wait(std::unique_lock&) () from /opt/couchbase/bin/../lib/libstdc++.so.6 #2 0x00007f829696c0fc in magma::TaskQueue::dequeue(std::atomic&) () from /opt/couchbase/bin/../lib/libmagma.so #3 0x00007f829696c603 in magma::TaskWorker::loop(void*) () from /opt/couchbase/bin/../lib/libmagma.so #4 0x00007f82958de2d7 in platform_thread_wrap(void*) () from /opt/couchbase/bin/../lib/libplatform_so.so.0.1.0 #5 0x00007f829311be65 in start_thread () from /lib64/libpthread.so.0 #6 0x00007f8292e4488d in clone () from /lib64/libc.so.6 Thread 8 (Thread 0x7f81eacf9700 (LWP 61601)): #0 0x00007f829311f9f5 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f82938fd1bc in std::condition_variable::wait(std::unique_lock&) () from /opt/couchbase/bin/../lib/libstdc++.so.6 #2 0x00007f829696c0fc in magma::TaskQueue::dequeue(std::atomic&) () from /opt/couchbase/bin/../lib/libmagma.so #3 0x00007f829696c603 in magma::TaskWorker::loop(void*) () from /opt/couchbase/bin/../lib/libmagma.so #4 0x00007f82958de2d7 in platform_thread_wrap(void*) () from /opt/couchbase/bin/../lib/libplatform_so.so.0.1.0 #5 0x00007f829311be65 in start_thread () from /lib64/libpthread.so.0 #6 0x00007f8292e4488d in clone () from /lib64/libc.so.6 Thread 7 (Thread 0x7f81ea4f8700 (LWP 61602)): #0 0x00007f829311f9f5 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f82938fd1bc in std::condition_variable::wait(std::unique_lock&) () from /opt/couchbase/bin/../lib/libstdc++.so.6 #2 0x00007f829696c0fc in magma::TaskQueue::dequeue(std::atomic&) () from /opt/couchbase/bin/../lib/libmagma.so #3 0x00007f829696c603 in magma::TaskWorker::loop(void*) () from /opt/couchbase/bin/../lib/libmagma.so #4 0x00007f82958de2d7 in platform_thread_wrap(void*) () from /opt/couchbase/bin/../lib/libplatform_so.so.0.1.0 #5 0x00007f829311be65 in start_thread () from /lib64/libpthread.so.0 #6 0x00007f8292e4488d in clone () from /lib64/libc.so.6 Thread 6 (Thread 0x7f81e9cf7700 (LWP 61603)): #0 0x00007f829311f9f5 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f82938fd1bc in std::condition_variable::wait(std::unique_lock&) () from /opt/couchbase/bin/../lib/libstdc++.so.6 #2 0x00007f829696c0fc in magma::TaskQueue::dequeue(std::atomic&) () from /opt/couchbase/bin/../lib/libmagma.so #3 0x00007f829696c603 in magma::TaskWorker::loop(void*) () from /opt/couchbase/bin/../lib/libmagma.so #4 0x00007f82958de2d7 in platform_thread_wrap(void*) () from /opt/couchbase/bin/../lib/libplatform_so.so.0.1.0 #5 0x00007f829311be65 in start_thread () from /lib64/libpthread.so.0 #6 0x00007f8292e4488d in clone () from /lib64/libc.so.6 Thread 5 (Thread 0x7f81e94f6700 (LWP 61604)): #0 0x00007f829311f9f5 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f82938fd1bc in std::condition_variable::wait(std::unique_lock&) () from /opt/couchbase/bin/../lib/libstdc++.so.6 #2 0x00007f829696c0fc in magma::TaskQueue::dequeue(std::atomic&) () from /opt/couchbase/bin/../lib/libmagma.so #3 0x00007f829696c603 in magma::TaskWorker::loop(void*) () from /opt/couchbase/bin/../lib/libmagma.so #4 0x00007f82958de2d7 in platform_thread_wrap(void*) () from /opt/couchbase/bin/../lib/libplatform_so.so.0.1.0 #5 0x00007f829311be65 in start_thread () from /lib64/libpthread.so.0 #6 0x00007f8292e4488d in clone () from /lib64/libc.so.6 Thread 4 (Thread 0x7f81e8cf5700 (LWP 61605)): #0 0x00007f829311f9f5 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f82938fd1bc in std::condition_variable::wait(std::unique_lock&) () from /opt/couchbase/bin/../lib/libstdc++.so.6 #2 0x00007f829696c0fc in magma::TaskQueue::dequeue(std::atomic&) () from /opt/couchbase/bin/../lib/libmagma.so #3 0x00007f829696c603 in magma::TaskWorker::loop(void*) () from /opt/couchbase/bin/../lib/libmagma.so #4 0x00007f82958de2d7 in platform_thread_wrap(void*) () from /opt/couchbase/bin/../lib/libplatform_so.so.0.1.0 #5 0x00007f829311be65 in start_thread () from /lib64/libpthread.so.0 #6 0x00007f8292e4488d in clone () from /lib64/libc.so.6 Thread 3 (Thread 0x7f81e84f4700 (LWP 61606)): #0 0x00007f829311f9f5 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f82938fd1bc in std::condition_variable::wait(std::unique_lock&) () from /opt/couchbase/bin/../lib/libstdc++.so.6 #2 0x00007f829696c0fc in magma::TaskQueue::dequeue(std::atomic&) () from /opt/couchbase/bin/../lib/libmagma.so #3 0x00007f829696c603 in magma::TaskWorker::loop(void*) () from /opt/couchbase/bin/../lib/libmagma.so #4 0x00007f82958de2d7 in platform_thread_wrap(void*) () from /opt/couchbase/bin/../lib/libplatform_so.so.0.1.0 #5 0x00007f829311be65 in start_thread () from /lib64/libpthread.so.0 #6 0x00007f8292e4488d in clone () from /lib64/libc.so.6 Thread 2 (Thread 0x7f81e7cf3700 (LWP 61607)): #0 0x00007f8292e44e63 in epoll_wait () from /lib64/libc.so.6 #1 0x00007f82949b63c5 in epoll_dispatch () from /opt/couchbase/bin/../lib/libevent_core-2.1.so.7 #2 0x00007f82949ad376 in event_base_loop () from /opt/couchbase/bin/../lib/libevent_core-2.1.so.7 #3 0x00007f8297463397 in folly::EventBase::loopBody(int, bool) () from /opt/couchbase/bin/../lib/libep.so #4 0x00007f8297463866 in folly::EventBase::loop() () from /opt/couchbase/bin/../lib/libep.so #5 0x00007f8297465226 in folly::EventBase::loopForever() () from /opt/couchbase/bin/../lib/libep.so #6 0x00007f82974526f9 in folly::IOThreadPoolExecutor::threadRun(std::shared_ptr) () from /opt/couchbase/bin/../lib/libep.so #7 0x00007f829745dfd9 in void folly::detail::function::FunctionTraits::callBig))(std::shared_ptr)> >(folly::detail::function::Data&) () from /opt/couchbase/bin/../lib/libep.so #8 0x00007f8293902dcf in execute_native_thread_routine () from /opt/couchbase/bin/../lib/libstdc++.so.6 #9 0x00007f829311be65 in start_thread () from /lib64/libpthread.so.0 #10 0x00007f8292e4488d in clone () from /lib64/libc.so.6 Thread 1 (Thread 0x7f8298274400 (LWP 61545)): #0 0x00007f8292e44e63 in epoll_wait () from /lib64/libc.so.6 #1 0x00007f82949b63c5 in epoll_dispatch () from /opt/couchbase/bin/../lib/libevent_core-2.1.so.7 #2 0x00007f82949ad376 in event_base_loop () from /opt/couchbase/bin/../lib/libevent_core-2.1.so.7 #3 0x0000000000427763 in memcached_main(int, char**) () #4 0x00007f8292d68505 in __libc_start_main () from /lib64/libc.so.6 #5 0x000000000041fb55 in _start () Detaching from program: /opt/couchbase/bin/memcached, process 61545 2021-01-30 19:04:30,525 | test | CRITICAL | MainThread | [basetestcase:check_coredump_exist:728] 172.23.106.125: 1 core dump seen 2021-01-30 19:04:31,292 | test | CRITICAL | MainThread | [basetestcase:check_coredump_exist:729] 172.23.106.125: Stack Trace of first crash - 70742999-fa1b-46d4-3266c994-d46ca1eb.dmp Core was generated by `/opt/couchbase/bin/memcached -C /opt/couchbase/var/lib/couchbase/config/memcach'. #0 0x00007fa9f8e0d337 in raise () from /lib64/libc.so.6 #0 0x00007fa9f8e0d337 in raise () from /lib64/libc.so.6 No symbol table info available. #1 0x00007fa9f8e0ea28 in abort () from /lib64/libc.so.6 No symbol table info available. #2 0x00007fa9f996b195 in __gnu_cxx::__verbose_terminate_handler() () from /opt/couchbase/bin/../lib/libstdc++.so.6 No symbol table info available. #3 0x0000000000555632 in backtrace_terminate_handler() () No symbol table info available. #4 0x00007fa9f9968f86 in __cxxabiv1::__terminate(void (*)()) () from /opt/couchbase/bin/../lib/libstdc++.so.6 No symbol table info available. #5 0x00007fa9f9968fd1 in std::terminate() () from /opt/couchbase/bin/../lib/libstdc++.so.6 No symbol table info available. #6 0x00007fa9fd373ba3 in GlobalTask::execute() () from /opt/couchbase/bin/../lib/libep.so No symbol table info available. #7 0x00007fa9fd36dc02 in void folly::detail::function::FunctionTraits::callSmall(folly::detail::function::Data&) () from /opt/couchbase/bin/../lib/libep.so No symbol table info available. #8 0x00007fa9fd4ebeb6 in folly::ThreadPoolExecutor::runTask(std::shared_ptr const&, folly::ThreadPoolExecutor::Task&&) () from /opt/couchbase/bin/../lib/libep.so No symbol table info available. #9 0x00007fa9fd4d43aa in folly::CPUThreadPoolExecutor::threadRun(std::shared_ptr) () from /opt/couchbase/bin/../lib/libep.so No symbol table info available. #10 0x00007fa9fd4eefd9 in void folly::detail::function::FunctionTraits::callBig))(std::shared_ptr)> >(folly::detail::function::Data&) () from /opt/couchbase/bin/../lib/libep.so No symbol table info available. #11 0x00007fa9fd36c043 in void folly::detail::function::FunctionTraits::callBig&&)::{lambda()#1}>(folly::detail::function::Data&) () from /opt/couchbase/bin/../lib/libep.so No symbol table info available. #12 0x00007fa9f9993dcf in execute_native_thread_routine () from /opt/couchbase/bin/../lib/libstdc++.so.6 No symbol table info available. #13 0x00007fa9f91ace65 in start_thread () from /lib64/libpthread.so.0 No symbol table info available. #14 0x00007fa9f8ed588d in clone () from /lib64/libc.so.6 No symbol table info available. [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib64/libthread_db.so.1". Loaded symbols for /lib64/libpthread.so.0 Reading symbols from /lib64/libc.so.6...(no debugging symbols found)...done. Loaded symbols for /lib64/libc.so.6 Reading symbols from /lib64/ld-linux-x86-64.so.2...(no debugging symbols found)...done. Loaded symbols for /lib64/ld-linux-x86-64.so.2 Reading symbols from /lib64/librt.so.1...(no debugging symbols found)...done. Loaded symbols for /lib64/librt.so.1 0x00007f75396e9e63 in epoll_wait () from /lib64/libc.so.6 Thread 61 (Thread 0x7f75389ff700 (LWP 29932)): #0 0x00007f75399c4da2 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f753b4945a7 in background_thread_entry () from /opt/couchbase/bin/../lib/libjemalloc.so.2 #2 0x00007f75399c0e65 in start_thread () from /lib64/libpthread.so.0 #3 0x00007f75396e988d in clone () from /lib64/libc.so.6 Thread 60 (Thread 0x7f7537c7e700 (LWP 29933)): #0 0x00007f75399c4da2 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f753cebc8cd in spdlog::details::thread_pool::process_next_msg_() () from /opt/couchbase/bin/../lib/libmemcached_logger.so.1.0.0 #2 0x00007f753cebce20 in spdlog::details::thread_pool::worker_loop_() () from /opt/couchbase/bin/../lib/libmemcached_logger.so.1.0.0 #3 0x00007f753a1a7dcf in execute_native_thread_routine () from /opt/couchbase/bin/../lib/libstdc++.so.6 #4 0x00007f75399c0e65 in start_thread () from /lib64/libpthread.so.0 #5 0x00007f75396e988d in clone () from /lib64/libc.so.6 Thread 59 (Thread 0x7f753747d700 (LWP 29934)): #0 0x00007f75399c4da2 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f753cebbf8a in _ZNSt6thread11_State_implINS_8_InvokerISt5tupleIJZN6spdlog7details15periodic_workerC4ERKSt8functionIFvvEENSt6chrono8durationIlSt5ratioILl1ELl1EEEEEUlvE_EEEEE6_M_runEv () from /opt/couchbase/bin/../lib/libmemcached_logger.so.1.0.0 #2 0x00007f753a1a7dcf in execute_native_thread_routine () from /opt/couchbase/bin/../lib/libstdc++.so.6 #3 0x00007f75399c0e65 in start_thread () from /lib64/libpthread.so.0 #4 0x00007f75396e988d in clone () from /lib64/libc.so.6 Thread 58 (Thread 0x7f7536c7c700 (LWP 29935)): #0 0x00007f75396debed in poll () from /lib64/libc.so.6 #1 0x000000000046a1ee in check_stdin_thread(void*) () #2 0x00007f753c1832d7 in platform_thread_wrap(void*) () from /opt/couchbase/bin/../lib/libplatform_so.so.0.1.0 #3 0x00007f75399c0e65 in start_thread () from /lib64/libpthread.so.0 #4 0x00007f75396e988d in clone () from /lib64/libc.so.6 Thread 57 (Thread 0x7f753647b700 (LWP 29936)): #0 0x00007f75399c49f5 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f753d754b2b in worker_thread () from /opt/couchbase/bin/../lib/libstatistics.so #2 0x00007f75399c0e65 in start_thread () from /lib64/libpthread.so.0 #3 0x00007f75396e988d in clone () from /lib64/libc.so.6 Thread 56 (Thread 0x7f7535c7a700 (LWP 29937)): #0 0x00007f75399c49f5 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f753d754b2b in worker_thread () from /opt/couchbase/bin/../lib/libstatistics.so #2 0x00007f75399c0e65 in start_thread () from /lib64/libpthread.so.0 #3 0x00007f75396e988d in clone () from /lib64/libc.so.6 Thread 55 (Thread 0x7f7535479700 (LWP 29938)): #0 0x00007f75396debed in poll () from /lib64/libc.so.6 #1 0x00007f753d74bad8 in master_thread () from /opt/couchbase/bin/../lib/libstatistics.so #2 0x00007f75399c0e65 in start_thread () from /lib64/libpthread.so.0 #3 0x00007f75396e988d in clone () from /lib64/libc.so.6 Thread 54 (Thread 0x7f7534c78700 (LWP 29939)): #0 0x00007f75399c4da2 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x0000000000491280 in ExternalAuthManagerThread::run() () #2 0x00007f753c18e20f in Couchbase::Thread::thread_entry() () from /opt/couchbase/bin/../lib/libplatform_so.so.0.1.0 #3 0x00007f753c1832d7 in platform_thread_wrap(void*) () from /opt/couchbase/bin/../lib/libplatform_so.so.0.1.0 #4 0x00007f75399c0e65 in start_thread () from /lib64/libpthread.so.0 #5 0x00007f75396e988d in clone () from /lib64/libc.so.6 Thread 53 (Thread 0x7f752ffff700 (LWP 29940)): #0 0x00007f75399c4da2 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00000000004ef6a6 in AuditImpl::consume_events() () #2 0x00007f753c1832d7 in platform_thread_wrap(void*) () from /opt/couchbase/bin/../lib/libplatform_so.so.0.1.0 #3 0x00007f75399c0e65 in start_thread () from /lib64/libpthread.so.0 #4 0x00007f75396e988d in clone () from /lib64/libc.so.6 Thread 52 (Thread 0x7f752f7fe700 (LWP 29941)): #0 0x00007f75396e9e63 in epoll_wait () from /lib64/libc.so.6 #1 0x00007f753b25b3c5 in epoll_dispatch () from /opt/couchbase/bin/../lib/libevent_core-2.1.so.7 #2 0x00007f753b252376 in event_base_loop () from /opt/couchbase/bin/../lib/libevent_core-2.1.so.7 #3 0x000000000046adf9 in worker_libevent(void*) () #4 0x00007f753c1832d7 in platform_thread_wrap(void*) () from /opt/couchbase/bin/../lib/libplatform_so.so.0.1.0 #5 0x00007f75399c0e65 in start_thread () from /lib64/libpthread.so.0 #6 0x00007f75396e988d in clone () from /lib64/libc.so.6 Thread 51 (Thread 0x7f752effd700 (LWP 29942)): #0 0x00007f75396e9e63 in epoll_wait () from /lib64/libc.so.6 #1 0x00007f753b25b3c5 in epoll_dispatch () from /opt/couchbase/bin/../lib/libevent_core-2.1.so.7 #2 0x00007f753b252376 in event_base_loop () from /opt/couchbase/bin/../lib/libevent_core-2.1.so.7 #3 0x000000000046adf9 in worker_libevent(void*) () #4 0x00007f753c1832d7 in platform_thread_wrap(void*) () from /opt/couchbase/bin/../lib/libplatform_so.so.0.1.0 #5 0x00007f75399c0e65 in start_thread () from /lib64/libpthread.so.0 #6 0x00007f75396e988d in clone () from /lib64/libc.so.6 Thread 50 (Thread 0x7f752e7fc700 (LWP 29943)): #0 0x00007f75396e9e63 in epoll_wait () from /lib64/libc.so.6 #1 0x00007f753b25b3c5 in epoll_dispatch () from /opt/couchbase/bin/../lib/libevent_core-2.1.so.7 #2 0x00007f753b252376 in event_base_loop () from /opt/couchbase/bin/../lib/libevent_core-2.1.so.7 #3 0x000000000046adf9 in worker_libevent(void*) () #4 0x00007f753c1832d7 in platform_thread_wrap(void*) () from /opt/couchbase/bin/../lib/libplatform_so.so.0.1.0 #5 0x00007f75399c0e65 in start_thread () from /lib64/libpthread.so.0 #6 0x00007f75396e988d in clone () from /lib64/libc.so.6 Thread 49 (Thread 0x7f752dffb700 (LWP 29944)): #0 0x00007f75396e9e63 in epoll_wait () from /lib64/libc.so.6 #1 0x00007f753b25b3c5 in epoll_dispatch () from /opt/couchbase/bin/../lib/libevent_core-2.1.so.7 #2 0x00007f753b252376 in event_base_loop () from /opt/couchbase/bin/../lib/libevent_core-2.1.so.7 #3 0x000000000046adf9 in worker_libevent(void*) () #4 0x00007f753c1832d7 in platform_thread_wrap(void*) () from /opt/couchbase/bin/../lib/libplatform_so.so.0.1.0 #5 0x00007f75399c0e65 in start_thread () from /lib64/libpthread.so.0 #6 0x00007f75396e988d in clone () from /lib64/libc.so.6 Thread 48 (Thread 0x7f752d7fa700 (LWP 29945)): #0 0x00007f75396e9e63 in epoll_wait () from /lib64/libc.so.6 #1 0x00007f753b25b3c5 in epoll_dispatch () from /opt/couchbase/bin/../lib/libevent_core-2.1.so.7 #2 0x00007f753b252376 in event_base_loop () from /opt/couchbase/bin/../lib/libevent_core-2.1.so.7 #3 0x000000000046adf9 in worker_libevent(void*) () #4 0x00007f753c1832d7 in platform_thread_wrap(void*) () from /opt/couchbase/bin/../lib/libplatform_so.so.0.1.0 #5 0x00007f75399c0e65 in start_thread () from /lib64/libpthread.so.0 #6 0x00007f75396e988d in clone () from /lib64/libc.so.6 Thread 47 (Thread 0x7f752cff9700 (LWP 29946)): #0 0x00007f75396e9e63 in epoll_wait () from /lib64/libc.so.6 #1 0x00007f753b25b3c5 in epoll_dispatch () from /opt/couchbase/bin/../lib/libevent_core-2.1.so.7 #2 0x00007f753b252376 in event_base_loop () from /opt/couchbase/bin/../lib/libevent_core-2.1.so.7 #3 0x000000000046adf9 in worker_libevent(void*) () #4 0x00007f753c1832d7 in platform_thread_wrap(void*) () from /opt/couchbase/bin/../lib/libplatform_so.so.0.1.0 #5 0x00007f75399c0e65 in start_thread () from /lib64/libpthread.so.0 #6 0x00007f75396e988d in clone () from /lib64/libc.so.6 Thread 46 (Thread 0x7f7527fff700 (LWP 29947)): #0 0x00007f75396e9e63 in epoll_wait () from /lib64/libc.so.6 #1 0x00007f753b25b3c5 in epoll_dispatch () from /opt/couchbase/bin/../lib/libevent_core-2.1.so.7 #2 0x00007f753b252376 in event_base_loop () from /opt/couchbase/bin/../lib/libevent_core-2.1.so.7 #3 0x000000000046adf9 in worker_libevent(void*) () #4 0x00007f753c1832d7 in platform_thread_wrap(void*) () from /opt/couchbase/bin/../lib/libplatform_so.so.0.1.0 #5 0x00007f75399c0e65 in start_thread () from /lib64/libpthread.so.0 #6 0x00007f75396e988d in clone () from /lib64/libc.so.6 Thread 45 (Thread 0x7f75277fe700 (LWP 29948)): #0 0x00007f75399c49f5 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f753a1a21bc in std::condition_variable::wait(std::unique_lock&) () from /opt/couchbase/bin/../lib/libstdc++.so.6 #2 0x00000000004a1af1 in Executor::run() () #3 0x00007f753c18e20f in Couchbase::Thread::thread_entry() () from /opt/couchbase/bin/../lib/libplatform_so.so.0.1.0 #4 0x00007f753c1832d7 in platform_thread_wrap(void*) () from /opt/couchbase/bin/../lib/libplatform_so.so.0.1.0 #5 0x00007f75399c0e65 in start_thread () from /lib64/libpthread.so.0 #6 0x00007f75396e988d in clone () from /lib64/libc.so.6 Thread 44 (Thread 0x7f7526ffd700 (LWP 29949)): #0 0x00007f75399c49f5 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f753a1a21bc in std::condition_variable::wait(std::unique_lock&) () from /opt/couchbase/bin/../lib/libstdc++.so.6 #2 0x00000000004a1af1 in Executor::run() () #3 0x00007f753c18e20f in Couchbase::Thread::thread_entry() () from /opt/couchbase/bin/../lib/libplatform_so.so.0.1.0 #4 0x00007f753c1832d7 in platform_thread_wrap(void*) () from /opt/couchbase/bin/../lib/libplatform_so.so.0.1.0 #5 0x00007f75399c0e65 in start_thread () from /lib64/libpthread.so.0 #6 0x00007f75396e988d in clone () from /lib64/libc.so.6 Thread 43 (Thread 0x7f75267fc700 (LWP 29950)): #0 0x00007f75399c49f5 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f753a1a21bc in std::condition_variable::wait(std::unique_lock&) () from /opt/couchbase/bin/../lib/libstdc++.so.6 #2 0x00000000004a1af1 in Executor::run() () #3 0x00007f753c18e20f in Couchbase::Thread::thread_entry() () from /opt/couchbase/bin/../lib/libplatform_so.so.0.1.0 #4 0x00007f753c1832d7 in platform_thread_wrap(void*) () from /opt/couchbase/bin/../lib/libplatform_so.so.0.1.0 #5 0x00007f75399c0e65 in start_thread () from /lib64/libpthread.so.0 #6 0x00007f75396e988d in clone () from /lib64/libc.so.6 Thread 42 (Thread 0x7f7525ffb700 (LWP 29951)): #0 0x00007f75399c49f5 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f753a1a21bc in std::condition_variable::wait(std::unique_lock&) () from /opt/couchbase/bin/../lib/libstdc++.so.6 #2 0x00000000004a1af1 in Executor::run() () #3 0x00007f753c18e20f in Couchbase::Thread::thread_entry() () from /opt/couchbase/bin/../lib/libplatform_so.so.0.1.0 #4 0x00007f753c1832d7 in platform_thread_wrap(void*) () from /opt/couchbase/bin/../lib/libplatform_so.so.0.1.0 #5 0x00007f75399c0e65 in start_thread () from /lib64/libpthread.so.0 #6 0x00007f75396e988d in clone () from /lib64/libc.so.6 Thread 41 (Thread 0x7f75257fa700 (LWP 29952)): #0 0x00007f75399c49f5 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f753a1a21bc in std::condition_variable::wait(std::unique_lock&) () from /opt/couchbase/bin/../lib/libstdc++.so.6 #2 0x00000000004a1af1 in Executor::run() () #3 0x00007f753c18e20f in Couchbase::Thread::thread_entry() () from /opt/couchbase/bin/../lib/libplatform_so.so.0.1.0 #4 0x00007f753c1832d7 in platform_thread_wrap(void*) () from /opt/couchbase/bin/../lib/libplatform_so.so.0.1.0 #5 0x00007f75399c0e65 in start_thread () from /lib64/libpthread.so.0 #6 0x00007f75396e988d in clone () from /lib64/libc.so.6 Thread 40 (Thread 0x7f7524ff9700 (LWP 29953)): #0 0x00007f75399c49f5 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f753a1a21bc in std::condition_variable::wait(std::unique_lock&) () from /opt/couchbase/bin/../lib/libstdc++.so.6 #2 0x00000000004a1af1 in Executor::run() () #3 0x00007f753c18e20f in Couchbase::Thread::thread_entry() () from /opt/couchbase/bin/../lib/libplatform_so.so.0.1.0 #4 0x00007f753c1832d7 in platform_thread_wrap(void*) () from /opt/couchbase/bin/../lib/libplatform_so.so.0.1.0 #5 0x00007f75399c0e65 in start_thread () from /lib64/libpthread.so.0 #6 0x00007f75396e988d in clone () from /lib64/libc.so.6 Thread 39 (Thread 0x7f75247f8700 (LWP 29954)): #0 0x00007f75399c49f5 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f753a1a21bc in std::condition_variable::wait(std::unique_lock&) () from /opt/couchbase/bin/../lib/libstdc++.so.6 #2 0x00000000004a1af1 in Executor::run() () #3 0x00007f753c18e20f in Couchbase::Thread::thread_entry() () from /opt/couchbase/bin/../lib/libplatform_so.so.0.1.0 #4 0x00007f753c1832d7 in platform_thread_wrap(void*) () from /opt/couchbase/bin/../lib/libplatform_so.so.0.1.0 #5 0x00007f75399c0e65 in start_thread () from /lib64/libpthread.so.0 #6 0x00007f75396e988d in clone () from /lib64/libc.so.6 Thread 38 (Thread 0x7f7521fff700 (LWP 29957)): #0 0x00007f75399c4da2 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f753b493e51 in background_thread_entry () from /opt/couchbase/bin/../lib/libjemalloc.so.2 #2 0x00007f75399c0e65 in start_thread () from /lib64/libpthread.so.0 #3 0x00007f75396e988d in clone () from /lib64/libc.so.6 Thread 37 (Thread 0x7f75215fe700 (LWP 29958)): #0 0x00007f75396e3ba9 in syscall () from /lib64/libc.so.6 #1 0x00007f753dce2dbd in folly::detail::futexWaitImpl(std::atomic const*, unsigned int, std::chrono::time_point > > const*, std::chrono::time_point > > const*, unsigned int) () from /opt/couchbase/bin/../lib/libep.so #2 0x00007f753dce9fca in folly::detail::FutexResult folly::detail::futexWaitUntil, std::chrono::_V2::steady_clock, std::chrono::duration > >(std::atomic const*, unsigned int, std::chrono::time_point > > const&, unsigned int) () from /opt/couchbase/bin/../lib/libep.so #3 0x00007f753dcef8dd in bool folly::SaturatingSemaphore::tryWaitSlow > >(std::chrono::time_point > > const&, folly::WaitOptions const&) () from /opt/couchbase/bin/../lib/libep.so #4 0x00007f753dceff46 in bool folly::detail::LifoSemBase, std::atomic>::try_wait_until > >(std::chrono::time_point > > const&) () from /opt/couchbase/bin/../lib/libep.so #5 0x00007f753dcf36a0 in folly::UnboundedBlockingQueue::try_take_for(std::chrono::duration >) () from /opt/couchbase/bin/../lib/libep.so #6 0x00007f753dce8337 in folly::CPUThreadPoolExecutor::threadRun(std::shared_ptr) () from /opt/couchbase/bin/../lib/libep.so #7 0x00007f753dd02fd9 in void folly::detail::function::FunctionTraits::callBig))(std::shared_ptr)> >(folly::detail::function::Data&) () from /opt/couchbase/bin/../lib/libep.so #8 0x00007f753db80043 in void folly::detail::function::FunctionTraits::callBig&&)::{lambda()#1}>(folly::detail::function::Data&) () from /opt/couchbase/bin/../lib/libep.so #9 0x00007f753a1a7dcf in execute_native_thread_routine () from /opt/couchbase/bin/../lib/libstdc++.so.6 #10 0x00007f75399c0e65 in start_thread () from /lib64/libpthread.so.0 #11 0x00007f75396e988d in clone () from /lib64/libc.so.6 Thread 36 (Thread 0x7f7520dfd700 (LWP 29959)): #0 0x00007f75396e3ba9 in syscall () from /lib64/libc.so.6 #1 0x00007f753dce2dbd in folly::detail::futexWaitImpl(std::atomic const*, unsigned int, std::chrono::time_point