2021-02-09 01:02:20,815 | test | CRITICAL | MainThread | [basetestcase:check_coredump_exist:712] 172.23.122.0: Stack Trace of first crash - 759f98da-1201-4e43-25085091-8c09ae74.dmp Core was generated by `/opt/couchbase/bin/memcached -C /opt/couchbase/var/lib/couchbase/config/memcach'. #0 0x00007f8d7c5cd337 in raise () from /lib64/libc.so.6 #0 0x00007f8d7c5cd337 in raise () from /lib64/libc.so.6 No symbol table info available. #1 0x00007f8d7c5cea28 in abort () from /lib64/libc.so.6 No symbol table info available. #2 0x00007f8d7d12b195 in __gnu_cxx::__verbose_terminate_handler() () from /opt/couchbase/bin/../lib/libstdc++.so.6 No symbol table info available. #3 0x0000000000555bf2 in backtrace_terminate_handler() () No symbol table info available. #4 0x00007f8d7d128f86 in __cxxabiv1::__terminate(void (*)()) () from /opt/couchbase/bin/../lib/libstdc++.so.6 No symbol table info available. #5 0x00007f8d7d128fd1 in std::terminate() () from /opt/couchbase/bin/../lib/libstdc++.so.6 No symbol table info available. #6 0x00007f8d80b39e03 in GlobalTask::execute() () from /opt/couchbase/bin/../lib/libep.so No symbol table info available. #7 0x00007f8d80b33e62 in void folly::detail::function::FunctionTraits::callSmall(folly::detail::function::Data&) () from /opt/couchbase/bin/../lib/libep.so No symbol table info available. #8 0x00007f8d80cb47f6 in folly::ThreadPoolExecutor::runTask(std::shared_ptr const&, folly::ThreadPoolExecutor::Task&&) () from /opt/couchbase/bin/../lib/libep.so No symbol table info available. #9 0x00007f8d80c9ccea in folly::CPUThreadPoolExecutor::threadRun(std::shared_ptr) () from /opt/couchbase/bin/../lib/libep.so No symbol table info available. #10 0x00007f8d80cb7919 in void folly::detail::function::FunctionTraits::callBig))(std::shared_ptr)> >(folly::detail::function::Data&) () from /opt/couchbase/bin/../lib/libep.so No symbol table info available. #11 0x00007f8d80b322a3 in void folly::detail::function::FunctionTraits::callBig&&)::{lambda()#1}>(folly::detail::function::Data&) () from /opt/couchbase/bin/../lib/libep.so No symbol table info available. #12 0x00007f8d7d153dcf in execute_native_thread_routine () from /opt/couchbase/bin/../lib/libstdc++.so.6 No symbol table info available. #13 0x00007f8d7c96ce65 in start_thread () from /lib64/libpthread.so.0 No symbol table info available. #14 0x00007f8d7c69588d in clone () from /lib64/libc.so.6 No symbol table info available. [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib64/libthread_db.so.1". Loaded symbols for /lib64/libpthread.so.0 Reading symbols from /lib64/libc.so.6...(no debugging symbols found)...done. Loaded symbols for /lib64/libc.so.6 Reading symbols from /lib64/ld-linux-x86-64.so.2...(no debugging symbols found)...done. Loaded symbols for /lib64/ld-linux-x86-64.so.2 Reading symbols from /lib64/librt.so.1...(no debugging symbols found)...done. Loaded symbols for /lib64/librt.so.1 0x00007fe460d12e63 in epoll_wait () from /lib64/libc.so.6 Thread 31 (Thread 0x7fe4601ff700 (LWP 55001)): #0 0x00007fe460fedda2 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007fe462abd5a7 in background_thread_entry () from /opt/couchbase/bin/../lib/libjemalloc.so.2 #2 0x00007fe460fe9e65 in start_thread () from /lib64/libpthread.so.0 #3 0x00007fe460d1288d in clone () from /lib64/libc.so.6 Thread 30 (Thread 0x7fe45f47e700 (LWP 55002)): #0 0x00007fe460fedda2 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007fe4644e58cd in spdlog::details::thread_pool::process_next_msg_() () from /opt/couchbase/bin/../lib/libmemcached_logger.so.1.0.0 #2 0x00007fe4644e5e20 in spdlog::details::thread_pool::worker_loop_() () from /opt/couchbase/bin/../lib/libmemcached_logger.so.1.0.0 #3 0x00007fe4617d0dcf in execute_native_thread_routine () from /opt/couchbase/bin/../lib/libstdc++.so.6 #4 0x00007fe460fe9e65 in start_thread () from /lib64/libpthread.so.0 #5 0x00007fe460d1288d in clone () from /lib64/libc.so.6 Thread 29 (Thread 0x7fe45ec7d700 (LWP 55003)): #0 0x00007fe460fedda2 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007fe4644e4f8a in _ZNSt6thread11_State_implINS_8_InvokerISt5tupleIJZN6spdlog7details15periodic_workerC4ERKSt8functionIFvvEENSt6chrono8durationIlSt5ratioILl1ELl1EEEEEUlvE_EEEEE6_M_runEv () from /opt/couchbase/bin/../lib/libmemcached_logger.so.1.0.0 #2 0x00007fe4617d0dcf in execute_native_thread_routine () from /opt/couchbase/bin/../lib/libstdc++.so.6 #3 0x00007fe460fe9e65 in start_thread () from /lib64/libpthread.so.0 #4 0x00007fe460d1288d in clone () from /lib64/libc.so.6 Thread 28 (Thread 0x7fe45e47c700 (LWP 55004)): #0 0x00007fe460d07bed in poll () from /lib64/libc.so.6 #1 0x000000000046a0de in check_stdin_thread(void*) () #2 0x00007fe4637ac2d7 in platform_thread_wrap(void*) () from /opt/couchbase/bin/../lib/libplatform_so.so.0.1.0 #3 0x00007fe460fe9e65 in start_thread () from /lib64/libpthread.so.0 #4 0x00007fe460d1288d in clone () from /lib64/libc.so.6 Thread 27 (Thread 0x7fe45dc7b700 (LWP 55005)): #0 0x00007fe460fed9f5 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007fe464d82bfb in worker_thread () from /opt/couchbase/bin/../lib/libstatistics.so #2 0x00007fe460fe9e65 in start_thread () from /lib64/libpthread.so.0 #3 0x00007fe460d1288d in clone () from /lib64/libc.so.6 Thread 26 (Thread 0x7fe45d47a700 (LWP 55006)): #0 0x00007fe460fed9f5 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007fe464d82bfb in worker_thread () from /opt/couchbase/bin/../lib/libstatistics.so #2 0x00007fe460fe9e65 in start_thread () from /lib64/libpthread.so.0 #3 0x00007fe460d1288d in clone () from /lib64/libc.so.6 Thread 25 (Thread 0x7fe45cc79700 (LWP 55007)): #0 0x00007fe460d07bed in poll () from /lib64/libc.so.6 #1 0x00007fe464d79ba8 in master_thread () from /opt/couchbase/bin/../lib/libstatistics.so #2 0x00007fe460fe9e65 in start_thread () from /lib64/libpthread.so.0 #3 0x00007fe460d1288d in clone () from /lib64/libc.so.6 Thread 24 (Thread 0x7fe45c478700 (LWP 55008)): #0 0x00007fe460fedda2 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00000000004917e0 in ExternalAuthManagerThread::run() () #2 0x00007fe4637b720f in Couchbase::Thread::thread_entry() () from /opt/couchbase/bin/../lib/libplatform_so.so.0.1.0 #3 0x00007fe4637ac2d7 in platform_thread_wrap(void*) () from /opt/couchbase/bin/../lib/libplatform_so.so.0.1.0 #4 0x00007fe460fe9e65 in start_thread () from /lib64/libpthread.so.0 #5 0x00007fe460d1288d in clone () from /lib64/libc.so.6 Thread 23 (Thread 0x7fe45bc77700 (LWP 55009)): #0 0x00007fe460fedda2 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00000000004efcb6 in AuditImpl::consume_events() () #2 0x00007fe4637ac2d7 in platform_thread_wrap(void*) () from /opt/couchbase/bin/../lib/libplatform_so.so.0.1.0 #3 0x00007fe460fe9e65 in start_thread () from /lib64/libpthread.so.0 #4 0x00007fe460d1288d in clone () from /lib64/libc.so.6 Thread 22 (Thread 0x7fe45b476700 (LWP 55010)): #0 0x00007fe46514d68f in PassiveDurabilityMonitor::completeSyncWrite(StoredDocKeyT const&, PassiveDurabilityMonitor::Resolution, std::optional) () from /opt/couchbase/bin/../lib/libep.so #1 0x00007fe465218c11 in VBucket::processSoftDelete(HashTable::FindUpdateResult&, StoredValue&, unsigned long, ItemMetaData const&, VBQueueItemCtx const&, bool, unsigned long, DeleteSource) () from /opt/couchbase/bin/../lib/libep.so #2 0x00007fe465227adf in VBucket::deleteWithMeta(unsigned long&, unsigned long*, void const*, EventuallyPersistentEngine&, CheckConflicts, ItemMetaData const&, GenerateBySeqno, GenerateCas, unsigned long, Collections::VB::CachingReadHandle const&, DeleteSource) () from /opt/couchbase/bin/../lib/libep.so #3 0x00007fe4651ddbad in KVBucket::deleteWithMeta(DocKey const&, unsigned long&, unsigned long*, Vbid, void const*, cb::bitset<4ul, vbucket_state_t, PermittedVBStatesMap>, CheckConflicts, ItemMetaData const&, GenerateBySeqno, GenerateCas, unsigned long, ExtendedMetaData*, DeleteSource) () from /opt/couchbase/bin/../lib/libep.so #4 0x00007fe465118390 in PassiveStream::processMessage(MutationConsumerMessage*, PassiveStream::MessageType) () from /opt/couchbase/bin/../lib/libep.so #5 0x00007fe465119708 in PassiveStream::messageReceived(std::unique_ptr >) () from /opt/couchbase/bin/../lib/libep.so #6 0x00007fe4650ffc7e in DcpConsumer::deletion(unsigned int, DocKey const&, cb::sized_buffer, unsigned char, unsigned long, Vbid, unsigned long, unsigned long, cb::sized_buffer, unsigned int, IncludeDeleteTime, DeleteSource) () from /opt/couchbase/bin/../lib/libep.so #7 0x00007fe465100092 in DcpConsumer::toMainDeletion(DcpConsumer::DeleteType, unsigned int, DocKey const&, cb::sized_buffer, unsigned char, unsigned long, Vbid, unsigned long, unsigned long, cb::sized_buffer, unsigned int) () from /opt/couchbase/bin/../lib/libep.so #8 0x00007fe46510022a in DcpConsumer::deletionV2(unsigned int, DocKey const&, cb::sized_buffer, unsigned long, unsigned char, unsigned long, Vbid, unsigned long, unsigned long, unsigned int) () from /opt/couchbase/bin/../lib/libep.so #9 0x00007fe46516b743 in non-virtual thunk to EventuallyPersistentEngine::deletion_v2(gsl::not_null, unsigned int, DocKey const&, cb::sized_buffer, unsigned long, unsigned char, unsigned long, Vbid, unsigned long, unsigned long, unsigned int) () from /opt/couchbase/bin/../lib/libep.so #10 0x000000000044ebcf in dcpDeletionV2(Cookie&, unsigned int, DocKey const&, cb::sized_buffer, unsigned long, unsigned char, unsigned long, Vbid, unsigned long, unsigned long, unsigned int) () #11 0x00000000004c1d26 in dcp_deletion_executor(Cookie&) () #12 0x00000000004adfb6 in execute_client_request_packet(Cookie&, cb::mcbp::Request const&) () #13 0x000000000049cd99 in Cookie::execute() () #14 0x000000000048434d in Connection::executeCommandPipeline() () #15 0x0000000000487b85 in Connection::executeCommandsCallback() () #16 0x00000000004884cf in Connection::rw_callback(bufferevent*, void*) () #17 0x00007fe462871d8e in bufferevent_run_deferred_callbacks_unlocked () from /opt/couchbase/bin/../lib/libevent_core-2.1.so.7 #18 0x00007fe46287ad01 in event_process_active_single_queue.isra.31 () from /opt/couchbase/bin/../lib/libevent_core-2.1.so.7 #19 0x00007fe46287b54f in event_base_loop () from /opt/couchbase/bin/../lib/libevent_core-2.1.so.7 #20 0x000000000046ace9 in worker_libevent(void*) () #21 0x00007fe4637ac2d7 in platform_thread_wrap(void*) () from /opt/couchbase/bin/../lib/libplatform_so.so.0.1.0 #22 0x00007fe460fe9e65 in start_thread () from /lib64/libpthread.so.0 #23 0x00007fe460d1288d in clone () from /lib64/libc.so.6 Thread 21 (Thread 0x7fe45ac75700 (LWP 55011)): #0 0x00007fe460ff050d in __lll_lock_wait () from /lib64/libpthread.so.0 #1 0x00007fe460febe5b in _L_lock_883 () from /lib64/libpthread.so.0 #2 0x00007fe460febd28 in pthread_mutex_lock () from /lib64/libpthread.so.0 #3 0x00007fe4651d1482 in KVBucket::wakeUpExpiryPager() () from /opt/couchbase/bin/../lib/libep.so #4 0x00007fe46519cfd8 in EphemeralBucket::attemptToFreeMemory() () from /opt/couchbase/bin/../lib/libep.so #5 0x00007fe4651d9b7b in KVBucket::prepare(Item&, void const*) () from /opt/couchbase/bin/../lib/libep.so #6 0x00007fe465117f21 in PassiveStream::processMessage(MutationConsumerMessage*, PassiveStream::MessageType) () from /opt/couchbase/bin/../lib/libep.so #7 0x00007fe465118598 in PassiveStream::processPrepare(MutationConsumerMessage*) () from /opt/couchbase/bin/../lib/libep.so #8 0x00007fe465119751 in PassiveStream::messageReceived(std::unique_ptr >) () from /opt/couchbase/bin/../lib/libep.so #9 0x00007fe4650fcd55 in DcpConsumer::lookupStreamAndDispatchMessage(DcpConsumer::UpdateFlowControl&, Vbid, unsigned int, std::unique_ptr >) () from /opt/couchbase/bin/../lib/libep.so #10 0x00007fe4650fdaac in DcpConsumer::processMutationOrPrepare(Vbid, unsigned int, DocKey const&, SingleThreadedRCPtr >, cb::sized_buffer, unsigned long) () from /opt/couchbase/bin/../lib/libep.so #11 0x00007fe46510036b in DcpConsumer::prepare(unsigned int, DocKey const&, cb::sized_buffer, unsigned long, unsigned char, unsigned long, Vbid, unsigned int, unsigned long, unsigned long, unsigned int, unsigned int, unsigned char, DocumentState, cb::durability::Level) () from /opt/couchbase/bin/../lib/libep.so #12 0x00007fe46516a179 in EventuallyPersistentEngine::prepare(gsl::not_null, unsigned int, DocKey const&, cb::sized_buffer, unsigned long, unsigned char, unsigned long, Vbid, unsigned int, unsigned long, unsigned long, unsigned int, unsigned int, unsigned char, DocumentState, cb::durability::Level) () from /opt/couchbase/bin/../lib/libep.so #13 0x000000000044f9d3 in dcpPrepare(Cookie&, unsigned int, DocKey const&, cb::sized_buffer, unsigned long, unsigned char, unsigned long, Vbid, unsigned int, unsigned long, unsigned long, unsigned int, unsigned int, unsigned char, DocumentState, cb::durability::Level) () #14 0x00000000004c3b1c in dcp_prepare_executor(Cookie&) () #15 0x00000000004adfb6 in execute_client_request_packet(Cookie&, cb::mcbp::Request const&) () #16 0x000000000049cd99 in Cookie::execute() () #17 0x000000000048434d in Connection::executeCommandPipeline() () #18 0x0000000000487b85 in Connection::executeCommandsCallback() () #19 0x00000000004884cf in Connection::rw_callback(bufferevent*, void*) () #20 0x00007fe462871d8e in bufferevent_run_deferred_callbacks_unlocked () from /opt/couchbase/bin/../lib/libevent_core-2.1.so.7 #21 0x00007fe46287ad01 in event_process_active_single_queue.isra.31 () from /opt/couchbase/bin/../lib/libevent_core-2.1.so.7 #22 0x00007fe46287b54f in event_base_loop () from /opt/couchbase/bin/../lib/libevent_core-2.1.so.7 #23 0x000000000046ace9 in worker_libevent(void*) () #24 0x00007fe4637ac2d7 in platform_thread_wrap(void*) () from /opt/couchbase/bin/../lib/libplatform_so.so.0.1.0 #25 0x00007fe460fe9e65 in start_thread () from /lib64/libpthread.so.0 #26 0x00007fe460d1288d in clone () from /lib64/libc.so.6 Thread 20 (Thread 0x7fe45a474700 (LWP 55012)): #0 0x00007fe46514d6ba in PassiveDurabilityMonitor::completeSyncWrite(StoredDocKeyT const&, PassiveDurabilityMonitor::Resolution, std::optional) () from /opt/couchbase/bin/../lib/libep.so #1 0x00007fe4652194f5 in VBucket::processSet(HashTable::FindUpdateResult&, StoredValue*&, Item&, unsigned long, bool, bool, VBQueueItemCtx const&, cb::StoreIfStatus, bool) () from /opt/couchbase/bin/../lib/libep.so #2 0x00007fe46521c7db in VBucket::setWithMeta(Item&, unsigned long, unsigned long*, void const*, EventuallyPersistentEngine&, CheckConflicts, bool, GenerateBySeqno, GenerateCas, Collections::VB::CachingReadHandle const&) () from /opt/couchbase/bin/../lib/libep.so #3 0x00007fe4651dd53f in KVBucket::setWithMeta(Item&, unsigned long, unsigned long*, void const*, cb::bitset<4ul, vbucket_state_t, PermittedVBStatesMap>, CheckConflicts, bool, GenerateBySeqno, GenerateCas, ExtendedMetaData*) () from /opt/couchbase/bin/../lib/libep.so #4 0x00007fe465118217 in PassiveStream::processMessage(MutationConsumerMessage*, PassiveStream::MessageType) () from /opt/couchbase/bin/../lib/libep.so #5 0x00007fe465119741 in PassiveStream::messageReceived(std::unique_ptr >) () from /opt/couchbase/bin/../lib/libep.so #6 0x00007fe4650fcd55 in DcpConsumer::lookupStreamAndDispatchMessage(DcpConsumer::UpdateFlowControl&, Vbid, unsigned int, std::unique_ptr >) () from /opt/couchbase/bin/../lib/libep.so #7 0x00007fe4650fdaac in DcpConsumer::processMutationOrPrepare(Vbid, unsigned int, DocKey const&, SingleThreadedRCPtr >, cb::sized_buffer, unsigned long) () from /opt/couchbase/bin/../lib/libep.so #8 0x00007fe4650feef3 in DcpConsumer::mutation(unsigned int, DocKey const&, cb::sized_buffer, unsigned long, unsigned char, unsigned long, Vbid, unsigned int, unsigned long, unsigned long, unsigned int, unsigned int, cb::sized_buffer, unsigned char) () from /opt/couchbase/bin/../lib/libep.so #9 0x00007fe46517d73c in EventuallyPersistentEngine::mutation(gsl::not_null, unsigned int, DocKey const&, cb::sized_buffer, unsigned long, unsigned char, unsigned long, Vbid, unsigned int, unsigned long, unsigned long, unsigned int, unsigned int, cb::sized_buffer, unsigned char) () from /opt/couchbase/bin/../lib/libep.so #10 0x000000000044efdc in dcpMutation(Cookie&, unsigned int, DocKey const&, cb::sized_buffer, unsigned long, unsigned char, unsigned long, Vbid, unsigned int, unsigned long, unsigned long, unsigned int, unsigned int, cb::sized_buffer, unsigned char) () #11 0x00000000004c2c8d in dcp_mutation_executor(Cookie&) () #12 0x00000000004adfb6 in execute_client_request_packet(Cookie&, cb::mcbp::Request const&) () #13 0x000000000049cd99 in Cookie::execute() () #14 0x000000000048434d in Connection::executeCommandPipeline() () #15 0x0000000000487b85 in Connection::executeCommandsCallback() () #16 0x00000000004884cf in Connection::rw_callback(bufferevent*, void*) () #17 0x00007fe462871d8e in bufferevent_run_deferred_callbacks_unlocked () from /opt/couchbase/bin/../lib/libevent_core-2.1.so.7 #18 0x00007fe46287ad01 in event_process_active_single_queue.isra.31 () from /opt/couchbase/bin/../lib/libevent_core-2.1.so.7 #19 0x00007fe46287b54f in event_base_loop () from /opt/couchbase/bin/../lib/libevent_core-2.1.so.7 #20 0x000000000046ace9 in worker_libevent(void*) () #21 0x00007fe4637ac2d7 in platform_thread_wrap(void*) () from /opt/couchbase/bin/../lib/libplatform_so.so.0.1.0 #22 0x00007fe460fe9e65 in start_thread () from /lib64/libpthread.so.0 #23 0x00007fe460d1288d in clone () from /lib64/libc.so.6 Thread 19 (Thread 0x7fe459c73700 (LWP 55013)): #0 0x00007fe46514cf9b in PassiveDurabilityMonitor::addSyncWrite(SingleThreadedRCPtr >, std::optional) () from /opt/couchbase/bin/../lib/libep.so #1 0x00007fe465218543 in VBucket::queueItem(SingleThreadedRCPtr >&, VBQueueItemCtx const&) () from /opt/couchbase/bin/../lib/libep.so #2 0x00007fe465219d77 in VBucket::queueDirty(HashTable::HashBucketLock const&, StoredValue&, VBQueueItemCtx const&) () from /opt/couchbase/bin/../lib/libep.so #3 0x00007fe4651a442e in EphemeralVBucket::addNewStoredValue(HashTable::HashBucketLock const&, Item const&, VBQueueItemCtx const&, GenerateRevSeqno) () from /opt/couchbase/bin/../lib/libep.so #4 0x00007fe46521904d in VBucket::processSetInner(HashTable::FindUpdateResult&, StoredValue*&, Item&, unsigned long, bool, bool, VBQueueItemCtx const&, cb::StoreIfStatus, bool) () from /opt/couchbase/bin/../lib/libep.so #5 0x00007fe46521947a in VBucket::processSet(HashTable::FindUpdateResult&, StoredValue*&, Item&, unsigned long, bool, bool, VBQueueItemCtx const&, cb::StoreIfStatus, bool) () from /opt/couchbase/bin/../lib/libep.so #6 0x00007fe46521ce34 in VBucket::prepare(Item&, unsigned long, unsigned long*, void const*, EventuallyPersistentEngine&, CheckConflicts, bool, GenerateBySeqno, GenerateCas, Collections::VB::CachingReadHandle const&) () from /opt/couchbase/bin/../lib/libep.so #7 0x00007fe4651d9b5b in KVBucket::prepare(Item&, void const*) () from /opt/couchbase/bin/../lib/libep.so #8 0x00007fe465117f21 in PassiveStream::processMessage(MutationConsumerMessage*, PassiveStream::MessageType) () from /opt/couchbase/bin/../lib/libep.so #9 0x00007fe465118598 in PassiveStream::processPrepare(MutationConsumerMessage*) () from /opt/couchbase/bin/../lib/libep.so #10 0x00007fe465119751 in PassiveStream::messageReceived(std::unique_ptr >) () from /opt/couchbase/bin/../lib/libep.so #11 0x00007fe4650fcd55 in DcpConsumer::lookupStreamAndDispatchMessage(DcpConsumer::UpdateFlowControl&, Vbid, unsigned int, std::unique_ptr >) () from /opt/couchbase/bin/../lib/libep.so #12 0x00007fe4650fdaac in DcpConsumer::processMutationOrPrepare(Vbid, unsigned int, DocKey const&, SingleThreadedRCPtr >, cb::sized_buffer, unsigned long) () from /opt/couchbase/bin/../lib/libep.so #13 0x00007fe46510036b in DcpConsumer::prepare(unsigned int, DocKey const&, cb::sized_buffer, unsigned long, unsigned char, unsigned long, Vbid, unsigned int, unsigned long, unsigned long, unsigned int, unsigned int, unsigned char, DocumentState, cb::durability::Level) () from /opt/couchbase/bin/../lib/libep.so #14 0x00007fe46516a179 in EventuallyPersistentEngine::prepare(gsl::not_null, unsigned int, DocKey const&, cb::sized_buffer, unsigned long, unsigned char, unsigned long, Vbid, unsigned int, unsigned long, unsigned long, unsigned int, unsigned int, unsigned char, DocumentState, cb::durability::Level) () from /opt/couchbase/bin/../lib/libep.so #15 0x000000000044f9d3 in dcpPrepare(Cookie&, unsigned int, DocKey const&, cb::sized_buffer, unsigned long, unsigned char, unsigned long, Vbid, unsigned int, unsigned long, unsigned long, unsigned int, unsigned int, unsigned char, DocumentState, cb::durability::Level) () #16 0x00000000004c3b1c in dcp_prepare_executor(Cookie&) () #17 0x00000000004adfb6 in execute_client_request_packet(Cookie&, cb::mcbp::Request const&) () #18 0x000000000049cd99 in Cookie::execute() () #19 0x000000000048434d in Connection::executeCommandPipeline() () #20 0x0000000000487b85 in Connection::executeCommandsCallback() () #21 0x00000000004884cf in Connection::rw_callback(bufferevent*, void*) () #22 0x00007fe462871d8e in bufferevent_run_deferred_callbacks_unlocked () from /opt/couchbase/bin/../lib/libevent_core-2.1.so.7 #23 0x00007fe46287ad01 in event_process_active_single_queue.isra.31 () from /opt/couchbase/bin/../lib/libevent_core-2.1.so.7 #24 0x00007fe46287b54f in event_base_loop () from /opt/couchbase/bin/../lib/libevent_core-2.1.so.7 #25 0x000000000046ace9 in worker_libevent(void*) () #26 0x00007fe4637ac2d7 in platform_thread_wrap(void*) () from /opt/couchbase/bin/../lib/libplatform_so.so.0.1.0 #27 0x00007fe460fe9e65 in start_thread () from /lib64/libpthread.so.0 #28 0x00007fe460d1288d in clone () from /lib64/libc.so.6 Thread 18 (Thread 0x7fe459472700 (LWP 55014)): #0 0x00007fe460fed9f5 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007fe4617cb1bc in std::condition_variable::wait(std::unique_lock&) () from /opt/couchbase/bin/../lib/libstdc++.so.6 #2 0x00000000004a2051 in Executor::run() () #3 0x00007fe4637b720f in Couchbase::Thread::thread_entry() () from /opt/couchbase/bin/../lib/libplatform_so.so.0.1.0 #4 0x00007fe4637ac2d7 in platform_thread_wrap(void*) () from /opt/couchbase/bin/../lib/libplatform_so.so.0.1.0 #5 0x00007fe460fe9e65 in start_thread () from /lib64/libpthread.so.0 #6 0x00007fe460d1288d in clone () from /lib64/libc.so.6 Thread 17 (Thread 0x7fe458c71700 (LWP 55015)): #0 0x00007fe460fed9f5 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007fe4617cb1bc in std::condition_variable::wait(std::unique_lock&) () from /opt/couchbase/bin/../lib/libstdc++.so.6 #2 0x00000000004a2051 in Executor::run() () #3 0x00007fe4637b720f in Couchbase::Thread::thread_entry() () from /opt/couchbase/bin/../lib/libplatform_so.so.0.1.0 #4 0x00007fe4637ac2d7 in platform_thread_wrap(void*) () from /opt/couchbase/bin/../lib/libplatform_so.so.0.1.0 #5 0x00007fe460fe9e65 in start_thread () from /lib64/libpthread.so.0 #6 0x00007fe460d1288d in clone () from /lib64/libc.so.6 Thread 16 (Thread 0x7fe453fff700 (LWP 55016)): #0 0x00007fe460fed9f5 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007fe4617cb1bc in std::condition_variable::wait(std::unique_lock&) () from /opt/couchbase/bin/../lib/libstdc++.so.6 #2 0x00000000004a2051 in Executor::run() () #3 0x00007fe4637b720f in Couchbase::Thread::thread_entry() () from /opt/couchbase/bin/../lib/libplatform_so.so.0.1.0 #4 0x00007fe4637ac2d7 in platform_thread_wrap(void*) () from /opt/couchbase/bin/../lib/libplatform_so.so.0.1.0 #5 0x00007fe460fe9e65 in start_thread () from /lib64/libpthread.so.0 #6 0x00007fe460d1288d in clone () from /lib64/libc.so.6 Thread 15 (Thread 0x7fe4537fe700 (LWP 55017)): #0 0x00007fe460fed9f5 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007fe4617cb1bc in std::condition_variable::wait(std::unique_lock&) () from /opt/couchbase/bin/../lib/libstdc++.so.6 #2 0x00000000004a2051 in Executor::run() () #3 0x00007fe4637b720f in Couchbase::Thread::thread_entry() () from /opt/couchbase/bin/../lib/libplatform_so.so.0.1.0 #4 0x00007fe4637ac2d7 in platform_thread_wrap(void*) () from /opt/couchbase/bin/../lib/libplatform_so.so.0.1.0 #5 0x00007fe460fe9e65 in start_thread () from /lib64/libpthread.so.0 #6 0x00007fe460d1288d in clone () from /lib64/libc.so.6 Thread 14 (Thread 0x7fe45107c700 (LWP 55020)): #0 0x00007fe460fedda2 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007fe462abce51 in background_thread_entry () from /opt/couchbase/bin/../lib/libjemalloc.so.2 #2 0x00007fe460fe9e65 in start_thread () from /lib64/libpthread.so.0 #3 0x00007fe460d1288d in clone () from /lib64/libc.so.6 Thread 13 (Thread 0x7fe427fff700 (LWP 55021)): #0 0x00007fe460d0cba9 in syscall () from /lib64/libc.so.6 #1 0x00007fe4653146fd in folly::detail::futexWaitImpl(std::atomic const*, unsigned int, std::chrono::time_point > > const*, std::chrono::time_point > > const*, unsigned int) () from /opt/couchbase/bin/../lib/libep.so #2 0x00007fe46531b90a in folly::detail::FutexResult folly::detail::futexWaitUntil, std::chrono::_V2::steady_clock, std::chrono::duration > >(std::atomic const*, unsigned int, std::chrono::time_point > > const&, unsigned int) () from /opt/couchbase/bin/../lib/libep.so #3 0x00007fe46532121d in bool folly::SaturatingSemaphore::tryWaitSlow > >(std::chrono::time_point > > const&, folly::WaitOptions const&) () from /opt/couchbase/bin/../lib/libep.so #4 0x00007fe465321886 in bool folly::detail::LifoSemBase, std::atomic>::try_wait_until > >(std::chrono::time_point > > const&) () from /opt/couchbase/bin/../lib/libep.so #5 0x00007fe465324fe0 in folly::UnboundedBlockingQueue::try_take_for(std::chrono::duration >) () from /opt/couchbase/bin/../lib/libep.so #6 0x00007fe465319c77 in folly::CPUThreadPoolExecutor::threadRun(std::shared_ptr) () from /opt/couchbase/bin/../lib/libep.so #7 0x00007fe465334919 in void folly::detail::function::FunctionTraits::callBig))(std::shared_ptr)> >(folly::detail::function::Data&) () from /opt/couchbase/bin/../lib/libep.so #8 0x00007fe4651af2a3 in void folly::detail::function::FunctionTraits::callBig&&)::{lambda()#1}>(folly::detail::function::Data&) () from /opt/couchbase/bin/../lib/libep.so #9 0x00007fe4617d0dcf in execute_native_thread_routine () from /opt/couchbase/bin/../lib/libstdc++.so.6 #10 0x00007fe460fe9e65 in start_thread () from /lib64/libpthread.so.0 #11 0x00007fe460d1288d in clone () from /lib64/libc.so.6 Thread 12 (Thread 0x7fe4277fe700 (LWP 55022)): #0 0x00007fe460d0cba9 in syscall () from /lib64/libc.so.6 #1 0x00007fe4653146fd in folly::detail::futexWaitImpl(std::atomic const*, unsigned int, std::chrono::time_point > > const*, std::chrono::time_point > > const*, unsigned int) () from /opt/couchbase/bin/../lib/libep.so #2 0x00007fe46531b90a in folly::detail::FutexResult folly::detail::futexWaitUntil, std::chrono::_V2::steady_clock, std::chrono::duration > >(std::atomic const*, unsigned int, std::chrono::time_point > > const&, unsigned int) () from /opt/couchbase/bin/../lib/libep.so #3 0x00007fe46532121d in bool folly::SaturatingSemaphore::tryWaitSlow > >(std::chrono::time_point > > const&, folly::WaitOptions const&) () from /opt/couchbase/bin/../lib/libep.so #4 0x00007fe465321886 in bool folly::detail::LifoSemBase, std::atomic>::try_wait_until > >(std::chrono::time_point > > const&) () from /opt/couchbase/bin/../lib/libep.so #5 0x00007fe465324fe0 in folly::UnboundedBlockingQueue::try_take_for(std::chrono::duration >) () from /opt/couchbase/bin/../lib/libep.so #6 0x00007fe465319c77 in folly::CPUThreadPoolExecutor::threadRun(std::shared_ptr) () from /opt/couchbase/bin/../lib/libep.so #7 0x00007fe465334919 in void folly::detail::function::FunctionTraits::callBig))(std::shared_ptr)> >(folly::detail::function::Data&) () from /opt/couchbase/bin/../lib/libep.so #8 0x00007fe4651af2a3 in void folly::detail::function::FunctionTraits::callBig&&)::{lambda()#1}>(folly::detail::function::Data&) () from /opt/couchbase/bin/../lib/libep.so #9 0x00007fe4617d0dcf in execute_native_thread_routine () from /opt/couchbase/bin/../lib/libstdc++.so.6 #10 0x00007fe460fe9e65 in start_thread () from /lib64/libpthread.so.0 #11 0x00007fe460d1288d in clone () from /lib64/libc.so.6 Thread 11 (Thread 0x7fe426ffd700 (LWP 55023)): #0 0x00007fe460d0cba9 in syscall () from /lib64/libc.so.6 #1 0x00007fe4653146fd in folly::detail::futexWaitImpl(std::atomic const*, unsigned int, std::chrono::time_point > > const*, std::chrono::time_point > > const*, unsigned int) () from /opt/couchbase/bin/../lib/libep.so #2 0x00007fe46531b90a in folly::detail::FutexResult folly::detail::futexWaitUntil, std::chrono::_V2::steady_clock, std::chrono::duration > >(std::atomic const*, unsigned int, std::chrono::time_point > > const&, unsigned int) () from /opt/couchbase/bin/../lib/libep.so #3 0x00007fe46532121d in bool folly::SaturatingSemaphore::tryWaitSlow > >(std::chrono::time_point > > const&, folly::WaitOptions const&) () from /opt/couchbase/bin/../lib/libep.so #4 0x00007fe465321886 in bool folly::detail::LifoSemBase, std::atomic>::try_wait_until > >(std::chrono::time_point > > const&) () from /opt/couchbase/bin/../lib/libep.so #5 0x00007fe465324fe0 in folly::UnboundedBlockingQueue::try_take_for(std::chrono::duration >) () from /opt/couchbase/bin/../lib/libep.so #6 0x00007fe465319c77 in folly::CPUThreadPoolExecutor::threadRun(std::shared_ptr) () from /opt/couchbase/bin/../lib/libep.so #7 0x00007fe465334919 in void folly::detail::function::FunctionTraits::callBig))(std::shared_ptr)> >(folly::detail::function::Data&) () from /opt/couchbase/bin/../lib/libep.so #8 0x00007fe4651af2a3 in void folly::detail::function::FunctionTraits::callBig&&)::{lambda()#1}>(folly::detail::function::Data&) () from /opt/couchbase/bin/../lib/libep.so #9 0x00007fe4617d0dcf in execute_native_thread_routine () from /opt/couchbase/bin/../lib/libstdc++.so.6 #10 0x00007fe460fe9e65 in start_thread () from /lib64/libpthread.so.0 #11 0x00007fe460d1288d in clone () from /lib64/libc.so.6 Thread 10 (Thread 0x7fe4267fc700 (LWP 55024)): #0 0x00007fe460d0cba9 in syscall () from /lib64/libc.so.6 #1 0x00007fe4653146fd in folly::detail::futexWaitImpl(std::atomic const*, unsigned int, std::chrono::time_point > > const*, std::chrono::time_point > > const*, unsigned int) () from /opt/couchbase/bin/../lib/libep.so #2 0x00007fe46531b90a in folly::detail::FutexResult folly::detail::futexWaitUntil, std::chrono::_V2::steady_clock, std::chrono::duration > >(std::atomic const*, unsigned int, std::chrono::time_point > > const&, unsigned int) () from /opt/couchbase/bin/../lib/libep.so #3 0x00007fe46532121d in bool folly::SaturatingSemaphore::tryWaitSlow > >(std::chrono::time_point > > const&, folly::WaitOptions const&) () from /opt/couchbase/bin/../lib/libep.so #4 0x00007fe465321886 in bool folly::detail::LifoSemBase, std::atomic>::try_wait_until > >(std::chrono::time_point > > const&) () from /opt/couchbase/bin/../lib/libep.so #5 0x00007fe465324fe0 in folly::UnboundedBlockingQueue::try_take_for(std::chrono::duration >) () from /opt/couchbase/bin/../lib/libep.so #6 0x00007fe465319c77 in folly::CPUThreadPoolExecutor::threadRun(std::shared_ptr) () from /opt/couchbase/bin/../lib/libep.so #7 0x00007fe465334919 in void folly::detail::function::FunctionTraits::callBig))(std::shared_ptr)> >(folly::detail::function::Data&) () from /opt/couchbase/bin/../lib/libep.so #8 0x00007fe4651af2a3 in void folly::detail::function::FunctionTraits::callBig&&)::{lambda()#1}>(folly::detail::function::Data&) () from /opt/couchbase/bin/../lib/libep.so #9 0x00007fe4617d0dcf in execute_native_thread_routine () from /opt/couchbase/bin/../lib/libstdc++.so.6 #10 0x00007fe460fe9e65 in start_thread () from /lib64/libpthread.so.0 #11 0x00007fe460d1288d in clone () from /lib64/libc.so.6 Thread 9 (Thread 0x7fe40ffff700 (LWP 55025)): #0 0x00007fe460d0cba9 in syscall () from /lib64/libc.so.6 #1 0x00007fe4653146fd in folly::detail::futexWaitImpl(std::atomic const*, unsigned int, std::chrono::time_point > > const*, std::chrono::time_point > > const*, unsigned int) () from /opt/couchbase/bin/../lib/libep.so #2 0x00007fe46531b90a in folly::detail::FutexResult folly::detail::futexWaitUntil, std::chrono::_V2::steady_clock, std::chrono::duration > >(std::atomic const*, unsigned int, std::chrono::time_point > > const&, unsigned int) () from /opt/couchbase/bin/../lib/libep.so #3 0x00007fe46532121d in bool folly::SaturatingSemaphore::tryWaitSlow > >(std::chrono::time_point > > const&, folly::WaitOptions const&) () from /opt/couchbase/bin/../lib/libep.so #4 0x00007fe465321886 in bool folly::detail::LifoSemBase, std::atomic>::try_wait_until > >(std::chrono::time_point > > const&) () from /opt/couchbase/bin/../lib/libep.so #5 0x00007fe465324fe0 in folly::UnboundedBlockingQueue::try_take_for(std::chrono::duration >) () from /opt/couchbase/bin/../lib/libep.so #6 0x00007fe465319c77 in folly::CPUThreadPoolExecutor::threadRun(std::shared_ptr) () from /opt/couchbase/bin/../lib/libep.so #7 0x00007fe465334919 in void folly::detail::function::FunctionTraits::callBig))(std::shared_ptr)> >(folly::detail::function::Data&) () from /opt/couchbase/bin/../lib/libep.so #8 0x00007fe4651af2a3 in void folly::detail::function::FunctionTraits::callBig&&)::{lambda()#1}>(folly::detail::function::Data&) () from /opt/couchbase/bin/../lib/libep.so #9 0x00007fe4617d0dcf in execute_native_thread_routine () from /opt/couchbase/bin/../lib/libstdc++.so.6 #10 0x00007fe460fe9e65 in start_thread () from /lib64/libpthread.so.0 #11 0x00007fe460d1288d in clone () from /lib64/libc.so.6 Thread 8 (Thread 0x7fe40f7fe700 (LWP 55026)): #0 0x00007fe460d0cba9 in syscall () from /lib64/libc.so.6 #1 0x00007fe4653146fd in folly::detail::futexWaitImpl(std::atomic const*, unsigned int, std::chrono::time_point > > const*, std::chrono::time_point > > const*, unsigned int) () from /opt/couchbase/bin/../lib/libep.so #2 0x00007fe46531b90a in folly::detail::FutexResult folly::detail::futexWaitUntil, std::chrono::_V2::steady_clock, std::chrono::duration > >(std::atomic const*, unsigned int, std::chrono::time_point > > const&, unsigned int) () from /opt/couchbase/bin/../lib/libep.so #3 0x00007fe46532121d in bool folly::SaturatingSemaphore::tryWaitSlow > >(std::chrono::time_point > > const&, folly::WaitOptions const&) () from /opt/couchbase/bin/../lib/libep.so #4 0x00007fe465321886 in bool folly::detail::LifoSemBase, std::atomic>::try_wait_until > >(std::chrono::time_point > > const&) () from /opt/couchbase/bin/../lib/libep.so #5 0x00007fe465324fe0 in folly::UnboundedBlockingQueue::try_take_for(std::chrono::duration >) () from /opt/couchbase/bin/../lib/libep.so #6 0x00007fe465319c77 in folly::CPUThreadPoolExecutor::threadRun(std::shared_ptr) () from /opt/couchbase/bin/../lib/libep.so #7 0x00007fe465334919 in void folly::detail::function::FunctionTraits::callBig))(std::shared_ptr)> >(folly::detail::function::Data&) () from /opt/couchbase/bin/../lib/libep.so #8 0x00007fe4651af2a3 in void folly::detail::function::FunctionTraits::callBig&&)::{lambda()#1}>(folly::detail::function::Data&) () from /opt/couchbase/bin/../lib/libep.so #9 0x00007fe4617d0dcf in execute_native_thread_routine () from /opt/couchbase/bin/../lib/libstdc++.so.6 #10 0x00007fe460fe9e65 in start_thread () from /lib64/libpthread.so.0 #11 0x00007fe460d1288d in clone () from /lib64/libc.so.6 Thread 7 (Thread 0x7fe40effd700 (LWP 55027)): #0 0x00007fe460d0cba9 in syscall () from /lib64/libc.so.6 #1 0x00007fe4653146fd in folly::detail::futexWaitImpl(std::atomic const*, unsigned int, std::chrono::time_point > > const*, std::chrono::time_point > > const*, unsigned int) () from /opt/couchbase/bin/../lib/libep.so #2 0x00007fe46531b90a in folly::detail::FutexResult folly::detail::futexWaitUntil, std::chrono::_V2::steady_clock, std::chrono::duration > >(std::atomic const*, unsigned int, std::chrono::time_point > > const&, unsigned int) () from /opt/couchbase/bin/../lib/libep.so #3 0x00007fe46532121d in bool folly::SaturatingSemaphore::tryWaitSlow > >(std::chrono::time_point > > const&, folly::WaitOptions const&) () from /opt/couchbase/bin/../lib/libep.so #4 0x00007fe465321886 in bool folly::detail::LifoSemBase, std::atomic>::try_wait_until > >(std::chrono::time_point > > const&) () from /opt/couchbase/bin/../lib/libep.so #5 0x00007fe465324fe0 in folly::UnboundedBlockingQueue::try_take_for(std::chrono::duration >) () from /opt/couchbase/bin/../lib/libep.so #6 0x00007fe465319c77 in folly::CPUThreadPoolExecutor::threadRun(std::shared_ptr) () from /opt/couchbase/bin/../lib/libep.so #7 0x00007fe465334919 in void folly::detail::function::FunctionTraits::callBig))(std::shared_ptr)> >(folly::detail::function::Data&) () from /opt/couchbase/bin/../lib/libep.so #8 0x00007fe4651af2a3 in void folly::detail::function::FunctionTraits::callBig&&)::{lambda()#1}>(folly::detail::function::Data&) () from /opt/couchbase/bin/../lib/libep.so #9 0x00007fe4617d0dcf in execute_native_thread_routine () from /opt/couchbase/bin/../lib/libstdc++.so.6 #10 0x00007fe460fe9e65 in start_thread () from /lib64/libpthread.so.0 #11 0x00007fe460d1288d in clone () from /lib64/libc.so.6 Thread 6 (Thread 0x7fe40e7fc700 (LWP 55028)): #0 0x00007fe460d0cba9 in syscall () from /lib64/libc.so.6 #1 0x00007fe4653146fd in folly::detail::futexWaitImpl(std::atomic const*, unsigned int, std::chrono::time_point > > const*, std::chrono::time_point > > const*, unsigned int) () from /opt/couchbase/bin/../lib/libep.so #2 0x00007fe46531b90a in folly::detail::FutexResult folly::detail::futexWaitUntil, std::chrono::_V2::steady_clock, std::chrono::duration > >(std::atomic const*, unsigned int, std::chrono::time_point > > const&, unsigned int) () from /opt/couchbase/bin/../lib/libep.so #3 0x00007fe46532121d in bool folly::SaturatingSemaphore::tryWaitSlow > >(std::chrono::time_point > > const&, folly::WaitOptions const&) () from /opt/couchbase/bin/../lib/libep.so #4 0x00007fe465321886 in bool folly::detail::LifoSemBase, std::atomic>::try_wait_until > >(std::chrono::time_point > > const&) () from /opt/couchbase/bin/../lib/libep.so #5 0x00007fe465324fe0 in folly::UnboundedBlockingQueue::try_take_for(std::chrono::duration >) () from /opt/couchbase/bin/../lib/libep.so #6 0x00007fe465319c77 in folly::CPUThreadPoolExecutor::threadRun(std::shared_ptr) () from /opt/couchbase/bin/../lib/libep.so #7 0x00007fe465334919 in void folly::detail::function::FunctionTraits::callBig))(std::shared_ptr)> >(folly::detail::function::Data&) () from /opt/couchbase/bin/../lib/libep.so #8 0x00007fe4651af2a3 in void folly::detail::function::FunctionTraits::callBig&&)::{lambda()#1}>(folly::detail::function::Data&) () from /opt/couchbase/bin/../lib/libep.so #9 0x00007fe4617d0dcf in execute_native_thread_routine () from /opt/couchbase/bin/../lib/libstdc++.so.6 #10 0x00007fe460fe9e65 in start_thread () from /lib64/libpthread.so.0 #11 0x00007fe460d1288d in clone () from /lib64/libc.so.6 Thread 5 (Thread 0x7fe40dffb700 (LWP 55029)): #0 0x00007fe460d0cba9 in syscall () from /lib64/libc.so.6 #1 0x00007fe4653146fd in folly::detail::futexWaitImpl(std::atomic const*, unsigned int, std::chrono::time_point > > const*, std::chrono::time_point > > const*, unsigned int) () from /opt/couchbase/bin/../lib/libep.so #2 0x00007fe46531b90a in folly::detail::FutexResult folly::detail::futexWaitUntil, std::chrono::_V2::steady_clock, std::chrono::duration > >(std::atomic const*, unsigned int, std::chrono::time_point > > const&, unsigned int) () from /opt/couchbase/bin/../lib/libep.so #3 0x00007fe46532121d in bool folly::SaturatingSemaphore::tryWaitSlow > >(std::chrono::time_point > > const&, folly::WaitOptions const&) () from /opt/couchbase/bin/../lib/libep.so #4 0x00007fe465321886 in bool folly::detail::LifoSemBase, std::atomic>::try_wait_until > >(std::chrono::time_point > > const&) () from /opt/couchbase/bin/../lib/libep.so #5 0x00007fe465324fe0 in folly::UnboundedBlockingQueue::try_take_for(std::chrono::duration >) () from /opt/couchbase/bin/../lib/libep.so #6 0x00007fe465319c77 in folly::CPUThreadPoolExecutor::threadRun(std::shared_ptr) () from /opt/couchbase/bin/../lib/libep.so #7 0x00007fe465334919 in void folly::detail::function::FunctionTraits::callBig))(std::shared_ptr)> >(folly::detail::function::Data&) () from /opt/couchbase/bin/../lib/libep.so #8 0x00007fe4651af2a3 in void folly::detail::function::FunctionTraits::callBig&&)::{lambda()#1}>(folly::detail::function::Data&) () from /opt/couchbase/bin/../lib/libep.so #9 0x00007fe4617d0dcf in execute_native_thread_routine () from /opt/couchbase/bin/../lib/libstdc++.so.6 #10 0x00007fe460fe9e65 in start_thread () from /lib64/libpthread.so.0 #11 0x00007fe460d1288d in clone () from /lib64/libc.so.6 Thread 4 (Thread 0x7fe40d7fa700 (LWP 55030)): #0 0x00007fe4651badb4 in HashTable::pauseResumeVisit(HashTableVisitor&, HashTable::Position&) () from /opt/couchbase/bin/../lib/libep.so #1 0x00007fe4651bb0b1 in HashTable::visit(HashTableVisitor&) () from /opt/couchbase/bin/../lib/libep.so #2 0x00007fe4651d242b in VBCBAdaptor::run() () from /opt/couchbase/bin/../lib/libep.so #3 0x00007fe4651b6de3 in GlobalTask::execute() () from /opt/couchbase/bin/../lib/libep.so #4 0x00007fe4651b0e62 in void folly::detail::function::FunctionTraits::callSmall(folly::detail::function::Data&) () from /opt/couchbase/bin/../lib/libep.so #5 0x00007fe4653317f6 in folly::ThreadPoolExecutor::runTask(std::shared_ptr const&, folly::ThreadPoolExecutor::Task&&) () from /opt/couchbase/bin/../lib/libep.so #6 0x00007fe465319cea in folly::CPUThreadPoolExecutor::threadRun(std::shared_ptr) () from /opt/couchbase/bin/../lib/libep.so #7 0x00007fe465334919 in void folly::detail::function::FunctionTraits::callBig))(std::shared_ptr)> >(folly::detail::function::Data&) () from /opt/couchbase/bin/../lib/libep.so #8 0x00007fe4651af2a3 in void folly::detail::function::FunctionTraits::callBig&&)::{lambda()#1}>(folly::detail::function::Data&) () from /opt/couchbase/bin/../lib/libep.so #9 0x00007fe4617d0dcf in execute_native_thread_routine () from /opt/couchbase/bin/../lib/libstdc++.so.6 #10 0x00007fe460fe9e65 in start_thread () from /lib64/libpthread.so.0 #11 0x00007fe460d1288d in clone () from /lib64/libc.so.6 Thread 3 (Thread 0x7fe40cff9700 (LWP 55031)): #0 0x00007fe460d0cba9 in syscall () from /lib64/libc.so.6 #1 0x00007fe4653146fd in folly::detail::futexWaitImpl(std::atomic const*, unsigned int, std::chrono::time_point > > const*, std::chrono::time_point > > const*, unsigned int) () from /opt/couchbase/bin/../lib/libep.so #2 0x00007fe46531b90a in folly::detail::FutexResult folly::detail::futexWaitUntil, std::chrono::_V2::steady_clock, std::chrono::duration > >(std::atomic const*, unsigned int, std::chrono::time_point > > const&, unsigned int) () from /opt/couchbase/bin/../lib/libep.so #3 0x00007fe46533553f in bool folly::Baton::tryWaitSlow > >(std::chrono::time_point > > const&, folly::WaitOptions const&) () from /opt/couchbase/bin/../lib/libep.so #4 0x00007fe465338e0f in folly::EventBase::runInEventBaseThreadAndWait(folly::Function) () from /opt/couchbase/bin/../lib/libep.so #5 0x00007fe4651ac7ac in FollyExecutorPool::schedule(std::shared_ptr) () from /opt/couchbase/bin/../lib/libep.so #6 0x00007fe4651dc5ee in KVBucket::visitAsync(std::unique_ptr >, char const*, TaskId, std::chrono::duration >) () from /opt/couchbase/bin/../lib/libep.so #7 0x00007fe465150301 in DurabilityTimeoutTask::run() () from /opt/couchbase/bin/../lib/libep.so #8 0x00007fe4651b6de3 in GlobalTask::execute() () from /opt/couchbase/bin/../lib/libep.so #9 0x00007fe4651b0e62 in void folly::detail::function::FunctionTraits::callSmall(folly::detail::function::Data&) () from /opt/couchbase/bin/../lib/libep.so #10 0x00007fe4653317f6 in folly::ThreadPoolExecutor::runTask(std::shared_ptr const&, folly::ThreadPoolExecutor::Task&&) () from /opt/couchbase/bin/../lib/libep.so #11 0x00007fe465319cea in folly::CPUThreadPoolExecutor::threadRun(std::shared_ptr) () from /opt/couchbase/bin/../lib/libep.so #12 0x00007fe465334919 in void folly::detail::function::FunctionTraits::callBig))(std::shared_ptr)> >(folly::detail::function::Data&) () from /opt/couchbase/bin/../lib/libep.so #13 0x00007fe4651af2a3 in void folly::detail::function::FunctionTraits::callBig&&)::{lambda()#1}>(folly::detail::function::Data&) () from /opt/couchbase/bin/../lib/libep.so #14 0x00007fe4617d0dcf in execute_native_thread_routine () from /opt/couchbase/bin/../lib/libstdc++.so.6 #15 0x00007fe460fe9e65 in start_thread () from /lib64/libpthread.so.0 #16 0x00007fe460d1288d in clone () from /lib64/libc.so.6 Thread 2 (Thread 0x7fe3effff700 (LWP 55032)): #0 0x00007fe460d12e63 in epoll_wait () from /lib64/libc.so.6 #1 0x00007fe4628843c5 in epoll_dispatch () from /opt/couchbase/bin/../lib/libevent_core-2.1.so.7 #2 0x00007fe46287b376 in event_base_loop () from /opt/couchbase/bin/../lib/libevent_core-2.1.so.7 #3 0x00007fe465339cd7 in folly::EventBase::loopBody(int, bool) () from /opt/couchbase/bin/../lib/libep.so #4 0x00007fe46533a1a6 in folly::EventBase::loop() () from /opt/couchbase/bin/../lib/libep.so #5 0x00007fe46533bb66 in folly::EventBase::loopForever() () from /opt/couchbase/bin/../lib/libep.so #6 0x00007fe465329039 in folly::IOThreadPoolExecutor::threadRun(std::shared_ptr) () from /opt/couchbase/bin/../lib/libep.so #7 0x00007fe465334919 in void folly::detail::function::FunctionTraits::callBig))(std::shared_ptr)> >(folly::detail::function::Data&) () from /opt/couchbase/bin/../lib/libep.so #8 0x00007fe4617d0dcf in execute_native_thread_routine () from /opt/couchbase/bin/../lib/libstdc++.so.6 #9 0x00007fe460fe9e65 in start_thread () from /lib64/libpthread.so.0 #10 0x00007fe460d1288d in clone () from /lib64/libc.so.6 Thread 1 (Thread 0x7fe46614b400 (LWP 55000)): #0 0x00007fe460d12e63 in epoll_wait () from /lib64/libc.so.6 #1 0x00007fe4628843c5 in epoll_dispatch () from /opt/couchbase/bin/../lib/libevent_core-2.1.so.7 #2 0x00007fe46287b376 in event_base_loop () from /opt/couchbase/bin/../lib/libevent_core-2.1.so.7 #3 0x00000000004277a2 in memcached_main(int, char**) () #4 0x00007fe460c36505 in __libc_start_main () from /lib64/libc.so.6 #5 0x000000000041fb75 in _start () Detaching from program: /opt/couchbase/bin/memcached, process 55000