GNU gdb (GDB) Red Hat Enterprise Linux 7.6.1-119.el7 Copyright (C) 2013 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-redhat-linux-gnu". For bug reporting instructions, please see: ... Reading symbols from /opt/couchbase/bin/memcached...Reading symbols from /usr/lib/debug/opt/couchbase/bin/memcached.debug...done. done. [New LWP 37980] [New LWP 37948] [New LWP 37949] [New LWP 37950] [New LWP 37951] [New LWP 37952] [New LWP 37956] [New LWP 37957] [New LWP 37958] [New LWP 37959] [New LWP 37960] [New LWP 37961] [New LWP 37962] [New LWP 37963] [New LWP 37964] [New LWP 37965] [New LWP 37966] [New LWP 37967] [New LWP 37968] [New LWP 37969] [New LWP 37970] [New LWP 37971] [New LWP 37972] [New LWP 37973] [New LWP 37974] [New LWP 37975] [New LWP 37976] [New LWP 37977] [New LWP 37978] [New LWP 37979] [New LWP 37981] [New LWP 37982] [New LWP 37983] [New LWP 40784] [New LWP 40786] [New LWP 40787] [New LWP 40788] [New LWP 40789] [New LWP 40790] [New LWP 40791] [New LWP 40792] [New LWP 40793] [New LWP 40795] [New LWP 40796] [New LWP 40797] [New LWP 40798] [New LWP 40799] [New LWP 40800] [New LWP 40801] [New LWP 40802] [New LWP 40803] [New LWP 40804] [New LWP 40805] [New LWP 40806] [New LWP 40812] [New LWP 40817] To enable execution of this file add add-auto-load-safe-path /usr/lib64/libthread_db-1.0.so line to your configuration file "/root/.gdbinit". To completely disable this security protection add set auto-load safe-path / line to your configuration file "/root/.gdbinit". For more information about this security protection see the "Auto-loading safe path" section in the GDB manual. E.g., run from the shell: info "(gdb)Auto-loading safe path" Core was generated by `/opt/couchbase/bin/memcached -C /opt/couchbase/var/lib/couchbase/config/memcach'. #0 0x00007f66e420b387 in raise () from /lib64/libc.so.6 Thread 56 (LWP 40817): #0 0x00007f66e69aade2 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f66e5eaee51 in background_thread_sleep (tsdn=, interval=10000000000, info=) at src/background_thread.c:255 #2 background_work_sleep_once (ind=, info=, tsdn=) at src/background_thread.c:307 #3 background_work (ind=, tsd=0x7f6652fedf30) at src/background_thread.c:497 #4 background_thread_entry (ind_arg=) at src/background_thread.c:522 #5 0x00007f66e69a6ea5 in start_thread () from /lib64/libpthread.so.0 #6 0x00007f66e42d38dd in clone () from /lib64/libc.so.6 Thread 55 (LWP 40812): #0 0x00007f66e69aade2 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f66e5eaee51 in background_thread_sleep (tsdn=, interval=7525000000, info=) at src/background_thread.c:255 #2 background_work_sleep_once (ind=, info=, tsdn=) at src/background_thread.c:307 #3 background_work (ind=, tsd=0x7f6655fedf30) at src/background_thread.c:497 #4 background_thread_entry (ind_arg=) at src/background_thread.c:522 #5 0x00007f66e69a6ea5 in start_thread () from /lib64/libpthread.so.0 #6 0x00007f66e42d38dd in clone () from /lib64/libc.so.6 Thread 54 (LWP 40806): #0 0x00007f66e69aaa35 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f66e4b858bc in __gthread_cond_wait (__mutex=, __cond=) at /tmp/deploy/objdir/x86_64-pc-linux-gnu/libstdc++-v3/include/x86_64-pc-linux-gnu/bits/gthr-default.h:865 #2 std::condition_variable::wait (this=, __lock=...) at /tmp/deploy/objdir/../gcc-10.2.0/libstdc++-v3/src/c++11/condition_variable.cc:53 #3 0x0000000000950728 in magma::TaskQueue::dequeue(std::atomic&) () at /home/couchbase/jenkins/workspace/couchbase-server-unix/magma/util/worker.cc:101 #4 0x0000000000950b11 in magma::TaskWorker::loop(void*) () at /home/couchbase/jenkins/workspace/couchbase-server-unix/magma/util/worker.cc:160 #5 0x0000000000af2889 in run (this=0x7f66e2b75b50) at /home/couchbase/jenkins/workspace/couchbase-server-unix/platform/src/cb_pthreads.cc:51 #6 platform_thread_wrap(void*) () at /home/couchbase/jenkins/workspace/couchbase-server-unix/platform/src/cb_pthreads.cc:64 #7 0x00007f66e69a6ea5 in start_thread () from /lib64/libpthread.so.0 #8 0x00007f66e42d38dd in clone () from /lib64/libc.so.6 Thread 53 (LWP 40805): #0 0x00007f66e69aaa35 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f66e4b858bc in __gthread_cond_wait (__mutex=, __cond=) at /tmp/deploy/objdir/x86_64-pc-linux-gnu/libstdc++-v3/include/x86_64-pc-linux-gnu/bits/gthr-default.h:865 #2 std::condition_variable::wait (this=, __lock=...) at /tmp/deploy/objdir/../gcc-10.2.0/libstdc++-v3/src/c++11/condition_variable.cc:53 #3 0x0000000000950728 in magma::TaskQueue::dequeue(std::atomic&) () at /home/couchbase/jenkins/workspace/couchbase-server-unix/magma/util/worker.cc:101 #4 0x0000000000950b11 in magma::TaskWorker::loop(void*) () at /home/couchbase/jenkins/workspace/couchbase-server-unix/magma/util/worker.cc:160 #5 0x0000000000af2889 in run (this=0x7f66e2b75b20) at /home/couchbase/jenkins/workspace/couchbase-server-unix/platform/src/cb_pthreads.cc:51 #6 platform_thread_wrap(void*) () at /home/couchbase/jenkins/workspace/couchbase-server-unix/platform/src/cb_pthreads.cc:64 #7 0x00007f66e69a6ea5 in start_thread () from /lib64/libpthread.so.0 #8 0x00007f66e42d38dd in clone () from /lib64/libc.so.6 Thread 52 (LWP 40804): #0 0x00007f66e69aaa35 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f66e4b858bc in __gthread_cond_wait (__mutex=, __cond=) at /tmp/deploy/objdir/x86_64-pc-linux-gnu/libstdc++-v3/include/x86_64-pc-linux-gnu/bits/gthr-default.h:865 #2 std::condition_variable::wait (this=, __lock=...) at /tmp/deploy/objdir/../gcc-10.2.0/libstdc++-v3/src/c++11/condition_variable.cc:53 #3 0x0000000000950728 in magma::TaskQueue::dequeue(std::atomic&) () at /home/couchbase/jenkins/workspace/couchbase-server-unix/magma/util/worker.cc:101 #4 0x0000000000950b11 in magma::TaskWorker::loop(void*) () at /home/couchbase/jenkins/workspace/couchbase-server-unix/magma/util/worker.cc:160 #5 0x0000000000af2889 in run (this=0x7f66e2b75af0) at /home/couchbase/jenkins/workspace/couchbase-server-unix/platform/src/cb_pthreads.cc:51 #6 platform_thread_wrap(void*) () at /home/couchbase/jenkins/workspace/couchbase-server-unix/platform/src/cb_pthreads.cc:64 #7 0x00007f66e69a6ea5 in start_thread () from /lib64/libpthread.so.0 #8 0x00007f66e42d38dd in clone () from /lib64/libc.so.6 Thread 51 (LWP 40803): #0 0x00007f66e69aaa35 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f66e4b858bc in __gthread_cond_wait (__mutex=, __cond=) at /tmp/deploy/objdir/x86_64-pc-linux-gnu/libstdc++-v3/include/x86_64-pc-linux-gnu/bits/gthr-default.h:865 #2 std::condition_variable::wait (this=, __lock=...) at /tmp/deploy/objdir/../gcc-10.2.0/libstdc++-v3/src/c++11/condition_variable.cc:53 #3 0x0000000000950728 in magma::TaskQueue::dequeue(std::atomic&) () at /home/couchbase/jenkins/workspace/couchbase-server-unix/magma/util/worker.cc:101 #4 0x0000000000950b11 in magma::TaskWorker::loop(void*) () at /home/couchbase/jenkins/workspace/couchbase-server-unix/magma/util/worker.cc:160 #5 0x0000000000af2889 in run (this=0x7f66e2b75ac0) at /home/couchbase/jenkins/workspace/couchbase-server-unix/platform/src/cb_pthreads.cc:51 #6 platform_thread_wrap(void*) () at /home/couchbase/jenkins/workspace/couchbase-server-unix/platform/src/cb_pthreads.cc:64 #7 0x00007f66e69a6ea5 in start_thread () from /lib64/libpthread.so.0 #8 0x00007f66e42d38dd in clone () from /lib64/libc.so.6 Thread 50 (LWP 40802): #0 0x00007f66e69aaa35 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f66e4b858bc in __gthread_cond_wait (__mutex=, __cond=) at /tmp/deploy/objdir/x86_64-pc-linux-gnu/libstdc++-v3/include/x86_64-pc-linux-gnu/bits/gthr-default.h:865 #2 std::condition_variable::wait (this=, __lock=...) at /tmp/deploy/objdir/../gcc-10.2.0/libstdc++-v3/src/c++11/condition_variable.cc:53 #3 0x0000000000950728 in magma::TaskQueue::dequeue(std::atomic&) () at /home/couchbase/jenkins/workspace/couchbase-server-unix/magma/util/worker.cc:101 #4 0x0000000000950b11 in magma::TaskWorker::loop(void*) () at /home/couchbase/jenkins/workspace/couchbase-server-unix/magma/util/worker.cc:160 #5 0x0000000000af2889 in run (this=0x7f66e2b75a90) at /home/couchbase/jenkins/workspace/couchbase-server-unix/platform/src/cb_pthreads.cc:51 #6 platform_thread_wrap(void*) () at /home/couchbase/jenkins/workspace/couchbase-server-unix/platform/src/cb_pthreads.cc:64 #7 0x00007f66e69a6ea5 in start_thread () from /lib64/libpthread.so.0 #8 0x00007f66e42d38dd in clone () from /lib64/libc.so.6 Thread 49 (LWP 40801): #0 0x00007f66e69aaa35 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f66e4b858bc in __gthread_cond_wait (__mutex=, __cond=) at /tmp/deploy/objdir/x86_64-pc-linux-gnu/libstdc++-v3/include/x86_64-pc-linux-gnu/bits/gthr-default.h:865 #2 std::condition_variable::wait (this=, __lock=...) at /tmp/deploy/objdir/../gcc-10.2.0/libstdc++-v3/src/c++11/condition_variable.cc:53 #3 0x0000000000950728 in magma::TaskQueue::dequeue(std::atomic&) () at /home/couchbase/jenkins/workspace/couchbase-server-unix/magma/util/worker.cc:101 #4 0x0000000000950b11 in magma::TaskWorker::loop(void*) () at /home/couchbase/jenkins/workspace/couchbase-server-unix/magma/util/worker.cc:160 #5 0x0000000000af2889 in run (this=0x7f66e2b75a60) at /home/couchbase/jenkins/workspace/couchbase-server-unix/platform/src/cb_pthreads.cc:51 #6 platform_thread_wrap(void*) () at /home/couchbase/jenkins/workspace/couchbase-server-unix/platform/src/cb_pthreads.cc:64 #7 0x00007f66e69a6ea5 in start_thread () from /lib64/libpthread.so.0 #8 0x00007f66e42d38dd in clone () from /lib64/libc.so.6 Thread 48 (LWP 40800): #0 0x00007f66e69aaa35 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f66e4b858bc in __gthread_cond_wait (__mutex=, __cond=) at /tmp/deploy/objdir/x86_64-pc-linux-gnu/libstdc++-v3/include/x86_64-pc-linux-gnu/bits/gthr-default.h:865 #2 std::condition_variable::wait (this=, __lock=...) at /tmp/deploy/objdir/../gcc-10.2.0/libstdc++-v3/src/c++11/condition_variable.cc:53 #3 0x0000000000950728 in magma::TaskQueue::dequeue(std::atomic&) () at /home/couchbase/jenkins/workspace/couchbase-server-unix/magma/util/worker.cc:101 #4 0x0000000000950b11 in magma::TaskWorker::loop(void*) () at /home/couchbase/jenkins/workspace/couchbase-server-unix/magma/util/worker.cc:160 #5 0x0000000000af2889 in run (this=0x7f66e2b75a30) at /home/couchbase/jenkins/workspace/couchbase-server-unix/platform/src/cb_pthreads.cc:51 #6 platform_thread_wrap(void*) () at /home/couchbase/jenkins/workspace/couchbase-server-unix/platform/src/cb_pthreads.cc:64 #7 0x00007f66e69a6ea5 in start_thread () from /lib64/libpthread.so.0 #8 0x00007f66e42d38dd in clone () from /lib64/libc.so.6 Thread 47 (LWP 40799): #0 0x00007f66e69aaa35 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f66e4b858bc in __gthread_cond_wait (__mutex=, __cond=) at /tmp/deploy/objdir/x86_64-pc-linux-gnu/libstdc++-v3/include/x86_64-pc-linux-gnu/bits/gthr-default.h:865 #2 std::condition_variable::wait (this=, __lock=...) at /tmp/deploy/objdir/../gcc-10.2.0/libstdc++-v3/src/c++11/condition_variable.cc:53 #3 0x0000000000950728 in magma::TaskQueue::dequeue(std::atomic&) () at /home/couchbase/jenkins/workspace/couchbase-server-unix/magma/util/worker.cc:101 #4 0x0000000000950b11 in magma::TaskWorker::loop(void*) () at /home/couchbase/jenkins/workspace/couchbase-server-unix/magma/util/worker.cc:160 #5 0x0000000000af2889 in run (this=0x7f66e2b75a00) at /home/couchbase/jenkins/workspace/couchbase-server-unix/platform/src/cb_pthreads.cc:51 #6 platform_thread_wrap(void*) () at /home/couchbase/jenkins/workspace/couchbase-server-unix/platform/src/cb_pthreads.cc:64 #7 0x00007f66e69a6ea5 in start_thread () from /lib64/libpthread.so.0 #8 0x00007f66e42d38dd in clone () from /lib64/libc.so.6 Thread 46 (LWP 40798): #0 0x00007f66e69aaa35 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f66e4b858bc in __gthread_cond_wait (__mutex=, __cond=) at /tmp/deploy/objdir/x86_64-pc-linux-gnu/libstdc++-v3/include/x86_64-pc-linux-gnu/bits/gthr-default.h:865 #2 std::condition_variable::wait (this=, __lock=...) at /tmp/deploy/objdir/../gcc-10.2.0/libstdc++-v3/src/c++11/condition_variable.cc:53 #3 0x0000000000950728 in magma::TaskQueue::dequeue(std::atomic&) () at /home/couchbase/jenkins/workspace/couchbase-server-unix/magma/util/worker.cc:101 #4 0x0000000000950b11 in magma::TaskWorker::loop(void*) () at /home/couchbase/jenkins/workspace/couchbase-server-unix/magma/util/worker.cc:160 #5 0x0000000000af2889 in run (this=0x7f66e2b759d0) at /home/couchbase/jenkins/workspace/couchbase-server-unix/platform/src/cb_pthreads.cc:51 #6 platform_thread_wrap(void*) () at /home/couchbase/jenkins/workspace/couchbase-server-unix/platform/src/cb_pthreads.cc:64 #7 0x00007f66e69a6ea5 in start_thread () from /lib64/libpthread.so.0 #8 0x00007f66e42d38dd in clone () from /lib64/libc.so.6 Thread 45 (LWP 40797): #0 0x00007f66e69aaa35 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f66e4b858bc in __gthread_cond_wait (__mutex=, __cond=) at /tmp/deploy/objdir/x86_64-pc-linux-gnu/libstdc++-v3/include/x86_64-pc-linux-gnu/bits/gthr-default.h:865 #2 std::condition_variable::wait (this=, __lock=...) at /tmp/deploy/objdir/../gcc-10.2.0/libstdc++-v3/src/c++11/condition_variable.cc:53 #3 0x0000000000950728 in magma::TaskQueue::dequeue(std::atomic&) () at /home/couchbase/jenkins/workspace/couchbase-server-unix/magma/util/worker.cc:101 #4 0x0000000000950b11 in magma::TaskWorker::loop(void*) () at /home/couchbase/jenkins/workspace/couchbase-server-unix/magma/util/worker.cc:160 #5 0x0000000000af2889 in run (this=0x7f66e2b759a0) at /home/couchbase/jenkins/workspace/couchbase-server-unix/platform/src/cb_pthreads.cc:51 #6 platform_thread_wrap(void*) () at /home/couchbase/jenkins/workspace/couchbase-server-unix/platform/src/cb_pthreads.cc:64 #7 0x00007f66e69a6ea5 in start_thread () from /lib64/libpthread.so.0 #8 0x00007f66e42d38dd in clone () from /lib64/libc.so.6 Thread 44 (LWP 40796): #0 0x00007f66e69aaa35 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f66e4b858bc in __gthread_cond_wait (__mutex=, __cond=) at /tmp/deploy/objdir/x86_64-pc-linux-gnu/libstdc++-v3/include/x86_64-pc-linux-gnu/bits/gthr-default.h:865 #2 std::condition_variable::wait (this=, __lock=...) at /tmp/deploy/objdir/../gcc-10.2.0/libstdc++-v3/src/c++11/condition_variable.cc:53 #3 0x0000000000950728 in magma::TaskQueue::dequeue(std::atomic&) () at /home/couchbase/jenkins/workspace/couchbase-server-unix/magma/util/worker.cc:101 #4 0x0000000000950b11 in magma::TaskWorker::loop(void*) () at /home/couchbase/jenkins/workspace/couchbase-server-unix/magma/util/worker.cc:160 #5 0x0000000000af2889 in run (this=0x7f66e2b75970) at /home/couchbase/jenkins/workspace/couchbase-server-unix/platform/src/cb_pthreads.cc:51 #6 platform_thread_wrap(void*) () at /home/couchbase/jenkins/workspace/couchbase-server-unix/platform/src/cb_pthreads.cc:64 #7 0x00007f66e69a6ea5 in start_thread () from /lib64/libpthread.so.0 #8 0x00007f66e42d38dd in clone () from /lib64/libc.so.6 Thread 43 (LWP 40795): #0 0x00007f66e69aaa35 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f66e4b858bc in __gthread_cond_wait (__mutex=, __cond=) at /tmp/deploy/objdir/x86_64-pc-linux-gnu/libstdc++-v3/include/x86_64-pc-linux-gnu/bits/gthr-default.h:865 #2 std::condition_variable::wait (this=, __lock=...) at /tmp/deploy/objdir/../gcc-10.2.0/libstdc++-v3/src/c++11/condition_variable.cc:53 #3 0x0000000000950728 in magma::TaskQueue::dequeue(std::atomic&) () at /home/couchbase/jenkins/workspace/couchbase-server-unix/magma/util/worker.cc:101 #4 0x0000000000950b11 in magma::TaskWorker::loop(void*) () at /home/couchbase/jenkins/workspace/couchbase-server-unix/magma/util/worker.cc:160 #5 0x0000000000af2889 in run (this=0x7f66e2b75940) at /home/couchbase/jenkins/workspace/couchbase-server-unix/platform/src/cb_pthreads.cc:51 #6 platform_thread_wrap(void*) () at /home/couchbase/jenkins/workspace/couchbase-server-unix/platform/src/cb_pthreads.cc:64 #7 0x00007f66e69a6ea5 in start_thread () from /lib64/libpthread.so.0 #8 0x00007f66e42d38dd in clone () from /lib64/libc.so.6 Thread 42 (LWP 40793): #0 0x00007f66e69aaa35 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f66e4b858bc in __gthread_cond_wait (__mutex=, __cond=) at /tmp/deploy/objdir/x86_64-pc-linux-gnu/libstdc++-v3/include/x86_64-pc-linux-gnu/bits/gthr-default.h:865 #2 std::condition_variable::wait (this=, __lock=...) at /tmp/deploy/objdir/../gcc-10.2.0/libstdc++-v3/src/c++11/condition_variable.cc:53 #3 0x0000000000950728 in magma::TaskQueue::dequeue(std::atomic&) () at /home/couchbase/jenkins/workspace/couchbase-server-unix/magma/util/worker.cc:101 #4 0x0000000000950b11 in magma::TaskWorker::loop(void*) () at /home/couchbase/jenkins/workspace/couchbase-server-unix/magma/util/worker.cc:160 #5 0x0000000000af2889 in run (this=0x7f66e2b75910) at /home/couchbase/jenkins/workspace/couchbase-server-unix/platform/src/cb_pthreads.cc:51 #6 platform_thread_wrap(void*) () at /home/couchbase/jenkins/workspace/couchbase-server-unix/platform/src/cb_pthreads.cc:64 #7 0x00007f66e69a6ea5 in start_thread () from /lib64/libpthread.so.0 #8 0x00007f66e42d38dd in clone () from /lib64/libc.so.6 Thread 41 (LWP 40792): #0 0x00007f66e69aaa35 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f66e4b858bc in __gthread_cond_wait (__mutex=, __cond=) at /tmp/deploy/objdir/x86_64-pc-linux-gnu/libstdc++-v3/include/x86_64-pc-linux-gnu/bits/gthr-default.h:865 #2 std::condition_variable::wait (this=, __lock=...) at /tmp/deploy/objdir/../gcc-10.2.0/libstdc++-v3/src/c++11/condition_variable.cc:53 #3 0x0000000000950728 in magma::TaskQueue::dequeue(std::atomic&) () at /home/couchbase/jenkins/workspace/couchbase-server-unix/magma/util/worker.cc:101 #4 0x0000000000950b11 in magma::TaskWorker::loop(void*) () at /home/couchbase/jenkins/workspace/couchbase-server-unix/magma/util/worker.cc:160 #5 0x0000000000af2889 in run (this=0x7f66e2b758e0) at /home/couchbase/jenkins/workspace/couchbase-server-unix/platform/src/cb_pthreads.cc:51 #6 platform_thread_wrap(void*) () at /home/couchbase/jenkins/workspace/couchbase-server-unix/platform/src/cb_pthreads.cc:64 #7 0x00007f66e69a6ea5 in start_thread () from /lib64/libpthread.so.0 #8 0x00007f66e42d38dd in clone () from /lib64/libc.so.6 Thread 40 (LWP 40791): #0 0x00007f66e69aaa35 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f66e4b858bc in __gthread_cond_wait (__mutex=, __cond=) at /tmp/deploy/objdir/x86_64-pc-linux-gnu/libstdc++-v3/include/x86_64-pc-linux-gnu/bits/gthr-default.h:865 #2 std::condition_variable::wait (this=, __lock=...) at /tmp/deploy/objdir/../gcc-10.2.0/libstdc++-v3/src/c++11/condition_variable.cc:53 #3 0x0000000000950728 in magma::TaskQueue::dequeue(std::atomic&) () at /home/couchbase/jenkins/workspace/couchbase-server-unix/magma/util/worker.cc:101 #4 0x0000000000950b11 in magma::TaskWorker::loop(void*) () at /home/couchbase/jenkins/workspace/couchbase-server-unix/magma/util/worker.cc:160 #5 0x0000000000af2889 in run (this=0x7f66e2b758b0) at /home/couchbase/jenkins/workspace/couchbase-server-unix/platform/src/cb_pthreads.cc:51 #6 platform_thread_wrap(void*) () at /home/couchbase/jenkins/workspace/couchbase-server-unix/platform/src/cb_pthreads.cc:64 #7 0x00007f66e69a6ea5 in start_thread () from /lib64/libpthread.so.0 #8 0x00007f66e42d38dd in clone () from /lib64/libc.so.6 Thread 39 (LWP 40790): #0 0x00007f66e69aaa35 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f66e4b858bc in __gthread_cond_wait (__mutex=, __cond=) at /tmp/deploy/objdir/x86_64-pc-linux-gnu/libstdc++-v3/include/x86_64-pc-linux-gnu/bits/gthr-default.h:865 #2 std::condition_variable::wait (this=, __lock=...) at /tmp/deploy/objdir/../gcc-10.2.0/libstdc++-v3/src/c++11/condition_variable.cc:53 #3 0x0000000000950728 in magma::TaskQueue::dequeue(std::atomic&) () at /home/couchbase/jenkins/workspace/couchbase-server-unix/magma/util/worker.cc:101 #4 0x0000000000950b11 in magma::TaskWorker::loop(void*) () at /home/couchbase/jenkins/workspace/couchbase-server-unix/magma/util/worker.cc:160 #5 0x0000000000af2889 in run (this=0x7f66e2b75880) at /home/couchbase/jenkins/workspace/couchbase-server-unix/platform/src/cb_pthreads.cc:51 #6 platform_thread_wrap(void*) () at /home/couchbase/jenkins/workspace/couchbase-server-unix/platform/src/cb_pthreads.cc:64 #7 0x00007f66e69a6ea5 in start_thread () from /lib64/libpthread.so.0 #8 0x00007f66e42d38dd in clone () from /lib64/libc.so.6 Thread 38 (LWP 40789): #0 0x00007f66e69aade2 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00000000009509bf in __gthread_cond_timedwait (__abs_timeout=0x7f6663fed660, __mutex=, __cond=0x7f66e2bf7e58) at /opt/gcc-10.2.0/include/c++/10.2.0/x86_64-pc-linux-gnu/bits/gthr-default.h:872 #2 __wait_until_impl > > (__lock=..., __lock=..., __atime=, this=0x7f66e2bf7e58) at /opt/gcc-10.2.0/include/c++/10.2.0/condition_variable:232 #3 wait_until > > (__atime=..., __lock=..., this=0x7f66e2bf7e58) at /opt/gcc-10.2.0/include/c++/10.2.0/condition_variable:141 #4 wait_for > (__rtime=..., __rtime=..., __lock=..., this=0x7f66e2bf7e58) at /opt/gcc-10.2.0/include/c++/10.2.0/condition_variable:172 #5 magma::TaskQueue::dequeue(std::atomic&) () at /home/couchbase/jenkins/workspace/couchbase-server-unix/magma/util/worker.cc:99 #6 0x0000000000950b11 in magma::TaskWorker::loop(void*) () at /home/couchbase/jenkins/workspace/couchbase-server-unix/magma/util/worker.cc:160 #7 0x0000000000af2889 in run (this=0x7f66e2b75850) at /home/couchbase/jenkins/workspace/couchbase-server-unix/platform/src/cb_pthreads.cc:51 #8 platform_thread_wrap(void*) () at /home/couchbase/jenkins/workspace/couchbase-server-unix/platform/src/cb_pthreads.cc:64 #9 0x00007f66e69a6ea5 in start_thread () from /lib64/libpthread.so.0 #10 0x00007f66e42d38dd in clone () from /lib64/libc.so.6 Thread 37 (LWP 40788): #0 0x00007f66e69aade2 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00000000009509bf in __gthread_cond_timedwait (__abs_timeout=0x7f6694fe7660, __mutex=, __cond=0x7f66e2bf7e58) at /opt/gcc-10.2.0/include/c++/10.2.0/x86_64-pc-linux-gnu/bits/gthr-default.h:872 #2 __wait_until_impl > > (__lock=..., __lock=..., __atime=, this=0x7f66e2bf7e58) at /opt/gcc-10.2.0/include/c++/10.2.0/condition_variable:232 #3 wait_until > > (__atime=..., __lock=..., this=0x7f66e2bf7e58) at /opt/gcc-10.2.0/include/c++/10.2.0/condition_variable:141 #4 wait_for > (__rtime=..., __rtime=..., __lock=..., this=0x7f66e2bf7e58) at /opt/gcc-10.2.0/include/c++/10.2.0/condition_variable:172 #5 magma::TaskQueue::dequeue(std::atomic&) () at /home/couchbase/jenkins/workspace/couchbase-server-unix/magma/util/worker.cc:99 #6 0x0000000000950b11 in magma::TaskWorker::loop(void*) () at /home/couchbase/jenkins/workspace/couchbase-server-unix/magma/util/worker.cc:160 #7 0x0000000000af2889 in run (this=0x7f66e2b75820) at /home/couchbase/jenkins/workspace/couchbase-server-unix/platform/src/cb_pthreads.cc:51 #8 platform_thread_wrap(void*) () at /home/couchbase/jenkins/workspace/couchbase-server-unix/platform/src/cb_pthreads.cc:64 #9 0x00007f66e69a6ea5 in start_thread () from /lib64/libpthread.so.0 #10 0x00007f66e42d38dd in clone () from /lib64/libc.so.6 Thread 36 (LWP 40787): #0 0x00007f66e69aade2 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00000000009509bf in __gthread_cond_timedwait (__abs_timeout=0x7f66957e8660, __mutex=, __cond=0x7f66e2bf7e58) at /opt/gcc-10.2.0/include/c++/10.2.0/x86_64-pc-linux-gnu/bits/gthr-default.h:872 #2 __wait_until_impl > > (__lock=..., __lock=..., __atime=, this=0x7f66e2bf7e58) at /opt/gcc-10.2.0/include/c++/10.2.0/condition_variable:232 #3 wait_until > > (__atime=..., __lock=..., this=0x7f66e2bf7e58) at /opt/gcc-10.2.0/include/c++/10.2.0/condition_variable:141 #4 wait_for > (__rtime=..., __rtime=..., __lock=..., this=0x7f66e2bf7e58) at /opt/gcc-10.2.0/include/c++/10.2.0/condition_variable:172 #5 magma::TaskQueue::dequeue(std::atomic&) () at /home/couchbase/jenkins/workspace/couchbase-server-unix/magma/util/worker.cc:99 #6 0x0000000000950b11 in magma::TaskWorker::loop(void*) () at /home/couchbase/jenkins/workspace/couchbase-server-unix/magma/util/worker.cc:160 #7 0x0000000000af2889 in run (this=0x7f66e2b757f0) at /home/couchbase/jenkins/workspace/couchbase-server-unix/platform/src/cb_pthreads.cc:51 #8 platform_thread_wrap(void*) () at /home/couchbase/jenkins/workspace/couchbase-server-unix/platform/src/cb_pthreads.cc:64 #9 0x00007f66e69a6ea5 in start_thread () from /lib64/libpthread.so.0 #10 0x00007f66e42d38dd in clone () from /lib64/libc.so.6 Thread 35 (LWP 40786): #0 0x00007f66e69aade2 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00000000009509bf in __gthread_cond_timedwait (__abs_timeout=0x7f6695fe9660, __mutex=, __cond=0x7f66e2bf7e58) at /opt/gcc-10.2.0/include/c++/10.2.0/x86_64-pc-linux-gnu/bits/gthr-default.h:872 #2 __wait_until_impl > > (__lock=..., __lock=..., __atime=, this=0x7f66e2bf7e58) at /opt/gcc-10.2.0/include/c++/10.2.0/condition_variable:232 #3 wait_until > > (__atime=..., __lock=..., this=0x7f66e2bf7e58) at /opt/gcc-10.2.0/include/c++/10.2.0/condition_variable:141 #4 wait_for > (__rtime=..., __rtime=..., __lock=..., this=0x7f66e2bf7e58) at /opt/gcc-10.2.0/include/c++/10.2.0/condition_variable:172 #5 magma::TaskQueue::dequeue(std::atomic&) () at /home/couchbase/jenkins/workspace/couchbase-server-unix/magma/util/worker.cc:99 #6 0x0000000000950b11 in magma::TaskWorker::loop(void*) () at /home/couchbase/jenkins/workspace/couchbase-server-unix/magma/util/worker.cc:160 #7 0x0000000000af2889 in run (this=0x7f66e2b757c0) at /home/couchbase/jenkins/workspace/couchbase-server-unix/platform/src/cb_pthreads.cc:51 #8 platform_thread_wrap(void*) () at /home/couchbase/jenkins/workspace/couchbase-server-unix/platform/src/cb_pthreads.cc:64 #9 0x00007f66e69a6ea5 in start_thread () from /lib64/libpthread.so.0 #10 0x00007f66e42d38dd in clone () from /lib64/libc.so.6 Thread 34 (LWP 40784): #0 0x00007f66e69aade2 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f66e5eaee51 in background_thread_sleep (tsdn=, interval=1000000000, info=) at src/background_thread.c:255 #2 background_work_sleep_once (ind=, info=, tsdn=) at src/background_thread.c:307 #3 background_work (ind=, tsd=0x7f66967eaf30) at src/background_thread.c:497 #4 background_thread_entry (ind_arg=) at src/background_thread.c:522 #5 0x00007f66e69a6ea5 in start_thread () from /lib64/libpthread.so.0 #6 0x00007f66e42d38dd in clone () from /lib64/libc.so.6 Thread 33 (LWP 37983): #0 0x00007f66e42d3eb3 in epoll_wait () from /lib64/libc.so.6 #1 0x00007f66e5c763c5 in epoll_dispatch (base=0x7f66e2ac3e00, tv=) at /home/couchbase/jenkins/workspace/cbdeps-platform-build-old/deps/packages/build/libevent/libevent-prefix/src/libevent/epoll.c:465 #2 0x00007f66e5c6d376 in event_base_loop (base=0x7f66e2ac3e00, flags=1) at /home/couchbase/jenkins/workspace/cbdeps-platform-build-old/deps/packages/build/libevent/libevent-prefix/src/libevent/event.c:1998 #3 0x0000000000b5f427 in folly::EventBase::loopBody (this=this@entry=0x7f66e3c36f00, flags=flags@entry=0, ignoreKeepAlive=ignoreKeepAlive@entry=false) at /home/couchbase/jenkins/workspace/cbdeps-platform-build-old/deps/packages/build/folly/folly-prefix/src/folly/folly/io/async/EventBase.cpp:397 #4 0x0000000000b5f8f6 in folly::EventBase::loop (this=this@entry=0x7f66e3c36f00) at /home/couchbase/jenkins/workspace/cbdeps-platform-build-old/deps/packages/build/folly/folly-prefix/src/folly/folly/io/async/EventBase.cpp:315 #5 0x0000000000b612b6 in folly::EventBase::loopForever (this=0x7f66e3c36f00) at /home/couchbase/jenkins/workspace/cbdeps-platform-build-old/deps/packages/build/folly/folly-prefix/src/folly/folly/io/async/EventBase.cpp:538 #6 0x0000000000b4d569 in folly::IOThreadPoolExecutor::threadRun (this=0x7f66e3c49780, thread=...) at /home/couchbase/jenkins/workspace/cbdeps-platform-build-old/deps/packages/build/folly/folly-prefix/src/folly/folly/executors/IOThreadPoolExecutor.cpp:195 #7 0x0000000000b58cf9 in __invoke_impl), folly::ThreadPoolExecutor*&, std::shared_ptr&> ( __t=, __f=) at /usr/local/include/c++/7.3.0/bits/invoke.h:73 #8 __invoke), folly::ThreadPoolExecutor*&, std::shared_ptr&> (__fn=) at /usr/local/include/c++/7.3.0/bits/invoke.h:95 #9 __call (__args=, this=) at /usr/local/include/c++/7.3.0/functional:467 #10 operator()<> (this=) at /usr/local/include/c++/7.3.0/functional:551 #11 folly::detail::function::FunctionTraits::callBig))(std::shared_ptr)> >(folly::detail::function::Data&) (p=...) at /home/couchbase/jenkins/workspace/cbdeps-platform-build-old/deps/packages/build/folly/folly-prefix/src/folly/folly/Function.h:401 #12 0x00007f66e4b8ad40 in execute_native_thread_routine () at /tmp/deploy/objdir/../gcc-10.2.0/libstdc++-v3/src/c++11/thread.cc:80 #13 0x00007f66e69a6ea5 in start_thread () from /lib64/libpthread.so.0 #14 0x00007f66e42d38dd in clone () from /lib64/libc.so.6 Thread 32 (LWP 37982): #0 0x00007f66e42cdbf9 in syscall () from /lib64/libc.so.6 #1 0x0000000000b35d4d in nativeFutexWaitImpl (waitMask=waitMask@entry=2541667096, absSteadyTime=absSteadyTime@entry=0x7f66977ec2f8, absSystemTime=absSystemTime@entry=0x0, expected=, addr=) at /home/couchbase/jenkins/workspace/cbdeps-platform-build-old/deps/packages/build/folly/folly-prefix/src/folly/folly/detail/Futex.cpp:123 #2 folly::detail::futexWaitImpl (futex=, expected=, absSystemTime=absSystemTime@entry=0x0, absSteadyTime=absSteadyTime@entry=0x7f66977ec318, waitMask=) at /home/couchbase/jenkins/workspace/cbdeps-platform-build-old/deps/packages/build/folly/folly-prefix/src/folly/folly/detail/Futex.cpp:254 #3 0x0000000000b3f8ea in futexWaitImpl const, std::chrono::time_point > > > (waitMask=, deadline=..., expected=, futex=) at /home/couchbase/jenkins/workspace/cbdeps-platform-build-old/deps/packages/build/folly/folly-prefix/src/folly/folly/detail/Futex-inl.h:85 #4 folly::detail::futexWaitUntil, std::chrono::_V2::steady_clock, std::chrono::duration > > (futex=, expected=, deadline=..., waitMask=) at /home/couchbase/jenkins/workspace/cbdeps-platform-build-old/deps/packages/build/folly/folly-prefix/src/folly/folly/detail/Futex-inl.h:123 #5 0x0000000000b45887 in folly::SaturatingSemaphore::tryWaitSlow > > (this=0x7f66cdfdaf60, deadline=..., opt=...) at /home/couchbase/jenkins/workspace/cbdeps-platform-build-old/deps/packages/build/folly/folly-prefix/src/folly/folly/detail/MemoryIdler.h:194 #6 0x0000000000b45db6 in folly::detail::LifoSemBase, std::atomic>::try_wait_until > > ( this=0x12d3f40, deadline=...) at /home/couchbase/jenkins/workspace/cbdeps-platform-build-old/deps/packages/build/folly/folly-prefix/src/folly/folly/synchronization/SaturatingSemaphore.h:179 #7 0x0000000000b49510 in folly::UnboundedBlockingQueue::try_take_for (this=0x12d3f00, time=...) at /home/couchbase/jenkins/workspace/cbdeps-platform-build-old/deps/packages/build/folly/folly-prefix/src/folly/folly/synchronization/LifoSem.h:498 #8 0x0000000000b3da87 in folly::CPUThreadPoolExecutor::threadRun (this=0x7f66e2b36c00, thread=...) at /home/couchbase/jenkins/workspace/cbdeps-platform-build-old/deps/packages/build/folly/folly-prefix/src/folly/folly/executors/CPUThreadPoolExecutor.cpp:243 #9 0x0000000000b58cf9 in __invoke_impl), folly::ThreadPoolExecutor*&, std::shared_ptr&> ( __t=, __f=) at /usr/local/include/c++/7.3.0/bits/invoke.h:73 #10 __invoke), folly::ThreadPoolExecutor*&, std::shared_ptr&> (__fn=) at /usr/local/include/c++/7.3.0/bits/invoke.h:95 #11 __call (__args=, this=) at /usr/local/include/c++/7.3.0/functional:467 #12 operator()<> (this=) at /usr/local/include/c++/7.3.0/functional:551 #13 folly::detail::function::FunctionTraits::callBig))(std::shared_ptr)> >(folly::detail::function::Data&) (p=...) at /home/couchbase/jenkins/workspace/cbdeps-platform-build-old/deps/packages/build/folly/folly-prefix/src/folly/folly/Function.h:401 #14 0x00000000009fa8b4 in operator() (this=0x7f66e2ab3940) at /home/couchbase/jenkins/workspace/couchbase-server-unix/kv_engine/executor/folly_executorpool.cc:47 #15 operator() (__closure=0x7f66e2ab3940) at /home/couchbase/jenkins/workspace/couchbase-server-unix/kv_engine/executor/folly_executorpool.cc:47 #16 folly::detail::function::FunctionTraits::callBig&&)::{lambda()#1}>(folly::detail::function::Data&) (p=...) at /home/couchbase/jenkins/workspace/couchbase-server-unix/server_build/tlm/deps/folly.exploded/include/folly/Function.h:401 #17 0x00007f66e4b8ad40 in execute_native_thread_routine () at /tmp/deploy/objdir/../gcc-10.2.0/libstdc++-v3/src/c++11/thread.cc:80 #18 0x00007f66e69a6ea5 in start_thread () from /lib64/libpthread.so.0 #19 0x00007f66e42d38dd in clone () from /lib64/libc.so.6 Thread 31 (LWP 37981): #0 0x00007f66e42cdbf9 in syscall () from /lib64/libc.so.6 #1 0x0000000000b35d4d in nativeFutexWaitImpl (waitMask=waitMask@entry=2550059800, absSteadyTime=absSteadyTime@entry=0x7f6697fed2f8, absSystemTime=absSystemTime@entry=0x0, expected=, addr=) at /home/couchbase/jenkins/workspace/cbdeps-platform-build-old/deps/packages/build/folly/folly-prefix/src/folly/folly/detail/Futex.cpp:123 #2 folly::detail::futexWaitImpl (futex=, expected=, absSystemTime=absSystemTime@entry=0x0, absSteadyTime=absSteadyTime@entry=0x7f6697fed318, waitMask=) at /home/couchbase/jenkins/workspace/cbdeps-platform-build-old/deps/packages/build/folly/folly-prefix/src/folly/folly/detail/Futex.cpp:254 #3 0x0000000000b3f8ea in futexWaitImpl const, std::chrono::time_point > > > (waitMask=, deadline=..., expected=, futex=) at /home/couchbase/jenkins/workspace/cbdeps-platform-build-old/deps/packages/build/folly/folly-prefix/src/folly/folly/detail/Futex-inl.h:85 #4 folly::detail::futexWaitUntil, std::chrono::_V2::steady_clock, std::chrono::duration > > (futex=, expected=, deadline=..., waitMask=) at /home/couchbase/jenkins/workspace/cbdeps-platform-build-old/deps/packages/build/folly/folly-prefix/src/folly/folly/detail/Futex-inl.h:123 #5 0x0000000000b45887 in folly::SaturatingSemaphore::tryWaitSlow > > (this=0x7f66cdfdec38, deadline=..., opt=...) at /home/couchbase/jenkins/workspace/cbdeps-platform-build-old/deps/packages/build/folly/folly-prefix/src/folly/folly/detail/MemoryIdler.h:194 #6 0x0000000000b45db6 in folly::detail::LifoSemBase, std::atomic>::try_wait_until > > ( this=0x12d3f40, deadline=...) at /home/couchbase/jenkins/workspace/cbdeps-platform-build-old/deps/packages/build/folly/folly-prefix/src/folly/folly/synchronization/SaturatingSemaphore.h:179 #7 0x0000000000b49510 in folly::UnboundedBlockingQueue::try_take_for (this=0x12d3f00, time=...) at /home/couchbase/jenkins/workspace/cbdeps-platform-build-old/deps/packages/build/folly/folly-prefix/src/folly/folly/synchronization/LifoSem.h:498 #8 0x0000000000b3da87 in folly::CPUThreadPoolExecutor::threadRun (this=0x7f66e2b36c00, thread=...) at /home/couchbase/jenkins/workspace/cbdeps-platform-build-old/deps/packages/build/folly/folly-prefix/src/folly/folly/executors/CPUThreadPoolExecutor.cpp:243 #9 0x0000000000b58cf9 in __invoke_impl), folly::ThreadPoolExecutor*&, std::shared_ptr&> ( __t=, __f=) at /usr/local/include/c++/7.3.0/bits/invoke.h:73 #10 __invoke), folly::ThreadPoolExecutor*&, std::shared_ptr&> (__fn=) at /usr/local/include/c++/7.3.0/bits/invoke.h:95 #11 __call (__args=, this=) at /usr/local/include/c++/7.3.0/functional:467 #12 operator()<> (this=) at /usr/local/include/c++/7.3.0/functional:551 #13 folly::detail::function::FunctionTraits::callBig))(std::shared_ptr)> >(folly::detail::function::Data&) (p=...) at /home/couchbase/jenkins/workspace/cbdeps-platform-build-old/deps/packages/build/folly/folly-prefix/src/folly/folly/Function.h:401 #14 0x00000000009fa8b4 in operator() (this=0x7f66e2ab3980) at /home/couchbase/jenkins/workspace/couchbase-server-unix/kv_engine/executor/folly_executorpool.cc:47 #15 operator() (__closure=0x7f66e2ab3980) at /home/couchbase/jenkins/workspace/couchbase-server-unix/kv_engine/executor/folly_executorpool.cc:47 #16 folly::detail::function::FunctionTraits::callBig&&)::{lambda()#1}>(folly::detail::function::Data&) (p=...) at /home/couchbase/jenkins/workspace/couchbase-server-unix/server_build/tlm/deps/folly.exploded/include/folly/Function.h:401 #17 0x00007f66e4b8ad40 in execute_native_thread_routine () at /tmp/deploy/objdir/../gcc-10.2.0/libstdc++-v3/src/c++11/thread.cc:80 #18 0x00007f66e69a6ea5 in start_thread () from /lib64/libpthread.so.0 #19 0x00007f66e42d38dd in clone () from /lib64/libc.so.6 Thread 30 (LWP 37979): #0 0x0000000000a980f2 in counts_inc_normalised (value=1, index=, h=0x7f664f3d6680) at /home/couchbase/jenkins/workspace/couchbase-server-unix/third_party/HdrHistogram_c/src/hdr_histogram.c:77 #1 hdr_record_values (h=0x7f664f3d6680, value=value@entry=11724, count=count@entry=1) at /home/couchbase/jenkins/workspace/couchbase-server-unix/third_party/HdrHistogram_c/src/hdr_histogram.c:499 #2 0x0000000000a8bc13 in HdrHistogram::addValueAndCount(unsigned long, unsigned long) () at /home/couchbase/jenkins/workspace/couchbase-server-unix/kv_engine/utilities/hdrhistogram.cc:97 #3 0x000000000084605a in MagmaKVStore::commitCallback(int, kvstats_ctx&) () at /home/couchbase/jenkins/workspace/couchbase-server-unix/kv_engine/utilities/hdrhistogram.h:580 #4 0x000000000084aae0 in MagmaKVStore::commit (this=0x7f664f297f00, commitData=...) at /home/couchbase/jenkins/workspace/couchbase-server-unix/kv_engine/engines/ep/src/magma-kvstore/magma-kvstore.cc:621 #5 0x00000000007e6a3e in EPBucket::commit(Vbid, KVStore&, VB::Commit&) () at /home/couchbase/jenkins/workspace/couchbase-server-unix/kv_engine/engines/ep/src/ep_bucket.cc:897 #6 0x00000000007ed7cd in EPBucket::flushVBucket_UNLOCKED(LockedVBucketPtr) () at /home/couchbase/jenkins/workspace/couchbase-server-unix/kv_engine/engines/ep/src/ep_bucket.cc:772 #7 0x00000000007edb6f in EPBucket::flushVBucket(Vbid) () at /home/couchbase/jenkins/workspace/couchbase-server-unix/kv_engine/engines/ep/src/ep_bucket.cc:369 #8 0x00000000006bd9d0 in Flusher::flushVB() () at /home/couchbase/jenkins/workspace/couchbase-server-unix/kv_engine/engines/ep/src/flusher.cc:296 #9 0x00000000006bddf0 in Flusher::step(GlobalTask*) () at /home/couchbase/jenkins/workspace/couchbase-server-unix/kv_engine/engines/ep/src/flusher.cc:199 #10 0x00000000009fdc12 in GlobalTask::execute() () at /home/couchbase/jenkins/workspace/couchbase-server-unix/kv_engine/executor/globaltask.cc:68 #11 0x00000000009fad35 in FollyExecutorPool::TaskProxy::scheduleViaCPUPool()::{lambda()#2}::operator()() const (__closure=0x7f66c57e8540) at /home/couchbase/jenkins/workspace/couchbase-server-unix/kv_engine/executor/folly_executorpool.cc:189 #12 0x0000000000b55d40 in operator() (this=0x7f66c57e8540) at /home/couchbase/jenkins/workspace/cbdeps-platform-build-old/deps/packages/build/folly/folly-prefix/src/folly/folly/Function.h:416 #13 folly::ThreadPoolExecutor::runTask(std::shared_ptr const&, folly::ThreadPoolExecutor::Task&&) (this=0x7f66e2aa2400, thread=..., task=) at /home/couchbase/jenkins/workspace/cbdeps-platform-build-old/deps/packages/build/folly/folly-prefix/src/folly/folly/executors/ThreadPoolExecutor.cpp:97 #14 0x0000000000b3dafa in folly::CPUThreadPoolExecutor::threadRun (this=0x7f66e2aa2400, thread=...) at /home/couchbase/jenkins/workspace/cbdeps-platform-build-old/deps/packages/build/folly/folly-prefix/src/folly/folly/executors/CPUThreadPoolExecutor.cpp:265 #15 0x0000000000b58cf9 in __invoke_impl), folly::ThreadPoolExecutor*&, std::shared_ptr&> ( __t=, __f=) at /usr/local/include/c++/7.3.0/bits/invoke.h:73 #16 __invoke), folly::ThreadPoolExecutor*&, std::shared_ptr&> (__fn=) at /usr/local/include/c++/7.3.0/bits/invoke.h:95 #17 __call (__args=, this=) at /usr/local/include/c++/7.3.0/functional:467 #18 operator()<> (this=) at /usr/local/include/c++/7.3.0/functional:551 #19 folly::detail::function::FunctionTraits::callBig))(std::shared_ptr)> >(folly::detail::function::Data&) (p=...) at /home/couchbase/jenkins/workspace/cbdeps-platform-build-old/deps/packages/build/folly/folly-prefix/src/folly/folly/Function.h:401 #20 0x00000000009fa8b4 in operator() (this=0x7f66e2ab3b40) at /home/couchbase/jenkins/workspace/couchbase-server-unix/kv_engine/executor/folly_executorpool.cc:47 #21 operator() (__closure=0x7f66e2ab3b40) at /home/couchbase/jenkins/workspace/couchbase-server-unix/kv_engine/executor/folly_executorpool.cc:47 #22 folly::detail::function::FunctionTraits::callBig&&)::{lambda()#1}>(folly::detail::function::Data&) (p=...) at /home/couchbase/jenkins/workspace/couchbase-server-unix/server_build/tlm/deps/folly.exploded/include/folly/Function.h:401 #23 0x00007f66e4b8ad40 in execute_native_thread_routine () at /tmp/deploy/objdir/../gcc-10.2.0/libstdc++-v3/src/c++11/thread.cc:80 #24 0x00007f66e69a6ea5 in start_thread () from /lib64/libpthread.so.0 #25 0x00007f66e42d38dd in clone () from /lib64/libc.so.6 Thread 29 (LWP 37978): #0 0x00007ffea95ae6c2 in ?? () #1 0x000000000000d2d0 in ?? () #2 0x00007f6644390c00 in ?? () #3 0x00007f66c5fe8380 in ?? () #4 0x00007f66c5fe83d0 in ?? () #5 0x00007f66c5fe8230 in ?? () #6 0x00007f66e42e97fd in clock_gettime () from /lib64/libc.so.6 #7 0x00007f66e4b82ca1 in std::chrono::_V2::steady_clock::now () at /tmp/deploy/objdir/../gcc-10.2.0/libstdc++-v3/src/c++11/chrono.cc:88 #8 0x00000000008553cc in std::_Function_handler >&), MagmaKVStore::saveDocs(VB::Commit&, kvstats_ctx&)::{lambda(std::vector >&)#2}>::_M_invoke(std::_Any_data const&, std::vector >&) () at /home/couchbase/jenkins/workspace/couchbase-server-unix/kv_engine/engines/ep/src/magma-kvstore/magma-kvstore.cc:1242 #9 0x00000000008fe69d in operator() (__args#0=..., this=0x7f66c5fe8590) at /opt/gcc-10.2.0/include/c++/10.2.0/bits/std_function.h:248 #10 magma::KVStore::WriteDocs(magma::WAL*, std::vector > const&, std::function, std::function >&)>) () at /home/couchbase/jenkins/workspace/couchbase-server-unix/magma/magma/kvstore/write.cc:104 #11 0x00000000008e0b05 in magma::Magma::Impl::WriteDocs(unsigned short, std::vector > const&, unsigned int, std::function, std::function >&)>) () at /opt/gcc-10.2.0/include/c++/10.2.0/bits/std_function.h:248 #12 0x00000000008e0cb2 in magma::Magma::WriteDocs(unsigned short, std::vector > const&, unsigned int, std::function, std::function >&)>) () at /opt/gcc-10.2.0/include/c++/10.2.0/bits/std_function.h:248 #13 0x000000000084dc22 in MagmaKVStore::saveDocs(VB::Commit&, kvstats_ctx&) () at /home/couchbase/jenkins/workspace/couchbase-server-unix/kv_engine/include/memcached/vbucket.h:62 #14 0x000000000084aab1 in MagmaKVStore::commit (this=0x7f664b097a00, commitData=...) at /home/couchbase/jenkins/workspace/couchbase-server-unix/kv_engine/engines/ep/src/magma-kvstore/magma-kvstore.cc:611 #15 0x00000000007e6a3e in EPBucket::commit(Vbid, KVStore&, VB::Commit&) () at /home/couchbase/jenkins/workspace/couchbase-server-unix/kv_engine/engines/ep/src/ep_bucket.cc:897 #16 0x00000000007ed7cd in EPBucket::flushVBucket_UNLOCKED(LockedVBucketPtr) () at /home/couchbase/jenkins/workspace/couchbase-server-unix/kv_engine/engines/ep/src/ep_bucket.cc:772 #17 0x00000000007edb6f in EPBucket::flushVBucket(Vbid) () at /home/couchbase/jenkins/workspace/couchbase-server-unix/kv_engine/engines/ep/src/ep_bucket.cc:369 #18 0x00000000006bd9d0 in Flusher::flushVB() () at /home/couchbase/jenkins/workspace/couchbase-server-unix/kv_engine/engines/ep/src/flusher.cc:296 #19 0x00000000006bddf0 in Flusher::step(GlobalTask*) () at /home/couchbase/jenkins/workspace/couchbase-server-unix/kv_engine/engines/ep/src/flusher.cc:199 #20 0x00000000009fdc12 in GlobalTask::execute() () at /home/couchbase/jenkins/workspace/couchbase-server-unix/kv_engine/executor/globaltask.cc:68 #21 0x00000000009fad35 in FollyExecutorPool::TaskProxy::scheduleViaCPUPool()::{lambda()#2}::operator()() const (__closure=0x7f66c5fe9540) at /home/couchbase/jenkins/workspace/couchbase-server-unix/kv_engine/executor/folly_executorpool.cc:189 #22 0x0000000000b55d40 in operator() (this=0x7f66c5fe9540) at /home/couchbase/jenkins/workspace/cbdeps-platform-build-old/deps/packages/build/folly/folly-prefix/src/folly/folly/Function.h:416 #23 folly::ThreadPoolExecutor::runTask(std::shared_ptr const&, folly::ThreadPoolExecutor::Task&&) (this=0x7f66e2aa2400, thread=..., task=) at /home/couchbase/jenkins/workspace/cbdeps-platform-build-old/deps/packages/build/folly/folly-prefix/src/folly/folly/executors/ThreadPoolExecutor.cpp:97 #24 0x0000000000b3dafa in folly::CPUThreadPoolExecutor::threadRun (this=0x7f66e2aa2400, thread=...) at /home/couchbase/jenkins/workspace/cbdeps-platform-build-old/deps/packages/build/folly/folly-prefix/src/folly/folly/executors/CPUThreadPoolExecutor.cpp:265 #25 0x0000000000b58cf9 in __invoke_impl), folly::ThreadPoolExecutor*&, std::shared_ptr&> ( __t=, __f=) at /usr/local/include/c++/7.3.0/bits/invoke.h:73 #26 __invoke), folly::ThreadPoolExecutor*&, std::shared_ptr&> (__fn=) at /usr/local/include/c++/7.3.0/bits/invoke.h:95 #27 __call (__args=, this=) at /usr/local/include/c++/7.3.0/functional:467 #28 operator()<> (this=) at /usr/local/include/c++/7.3.0/functional:551 #29 folly::detail::function::FunctionTraits::callBig))(std::shared_ptr)> >(folly::detail::function::Data&) (p=...) at /home/couchbase/jenkins/workspace/cbdeps-platform-build-old/deps/packages/build/folly/folly-prefix/src/folly/folly/Function.h:401 #30 0x00000000009fa8b4 in operator() (this=0x7f66e2ab3bc0) at /home/couchbase/jenkins/workspace/couchbase-server-unix/kv_engine/executor/folly_executorpool.cc:47 #31 operator() (__closure=0x7f66e2ab3bc0) at /home/couchbase/jenkins/workspace/couchbase-server-unix/kv_engine/executor/folly_executorpool.cc:47 #32 folly::detail::function::FunctionTraits::callBig&&)::{lambda()#1}>(folly::detail::function::Data&) (p=...) at /home/couchbase/jenkins/workspace/couchbase-server-unix/server_build/tlm/deps/folly.exploded/include/folly/Function.h:401 #33 0x00007f66e4b8ad40 in execute_native_thread_routine () at /tmp/deploy/objdir/../gcc-10.2.0/libstdc++-v3/src/c++11/thread.cc:80 #34 0x00007f66e69a6ea5 in start_thread () from /lib64/libpthread.so.0 #35 0x00007f66e42d38dd in clone () from /lib64/libc.so.6 Thread 28 (LWP 37977): #0 __atomic_add (__val=1, __mem=0x7f66500ed608) at /opt/gcc-10.2.0/include/c++/10.2.0/ext/atomicity.h:96 #1 __atomic_add_dispatch (__val=1, __mem=0x7f66500ed608) at /opt/gcc-10.2.0/include/c++/10.2.0/ext/atomicity.h:96 #2 _M_add_ref_copy (this=0x7f66500ed600) at /opt/gcc-10.2.0/include/c++/10.2.0/bits/shared_ptr_base.h:142 #3 __shared_count (__r=..., this=) at /opt/gcc-10.2.0/include/c++/10.2.0/bits/shared_ptr_base.h:740 #4 __shared_ptr (this=) at /opt/gcc-10.2.0/include/c++/10.2.0/bits/shared_ptr_base.h:1181 #5 shared_ptr (this=) at /opt/gcc-10.2.0/include/c++/10.2.0/bits/shared_ptr.h:149 #6 getSnapshot (this=0x7f664ff19a08) at /home/couchbase/jenkins/workspace/couchbase-server-unix/magma/lsm/checkpoint.cc:500 #7 PendingOverhead (currTreeSnapshot=..., this=0x7f6650b30080) at /home/couchbase/jenkins/workspace/couchbase-server-unix/magma/lsm/checkpoint.cc:456 #8 magma::LSMTree::GetPendingCheckpointOverhead() () at /home/couchbase/jenkins/workspace/couchbase-server-unix/magma/lsm/checkpoint.cc:72 #9 0x00000000008f021a in magma::KVStore::getFlushMode(magma::FlushMode) () at /home/couchbase/jenkins/workspace/couchbase-server-unix/magma/magma/kvstore/kvstore.cc:256 #10 0x00000000008f048b in magma::KVStore::NeedsFlushing (this=) at /opt/gcc-10.2.0/include/c++/10.2.0/atomic:87 #11 0x00000000008bfaeb in magma::Magma::Impl::tryWriteCacheFlush(magma::BlockingMode, magma::FlushMode) () at /home/couchbase/jenkins/workspace/couchbase-server-unix/magma/magma/db.cc:991 #12 0x00000000008e0b5e in magma::Magma::Impl::WriteDocs(unsigned short, std::vector > const&, unsigned int, std::function, std::function >&)>) () at /home/couchbase/jenkins/workspace/couchbase-server-unix/magma/magma/db_write.cc:50 #13 0x00000000008e0cb2 in magma::Magma::WriteDocs(unsigned short, std::vector > const&, unsigned int, std::function, std::function >&)>) () at /opt/gcc-10.2.0/include/c++/10.2.0/bits/std_function.h:248 #14 0x000000000084dc22 in MagmaKVStore::saveDocs(VB::Commit&, kvstats_ctx&) () at /home/couchbase/jenkins/workspace/couchbase-server-unix/kv_engine/include/memcached/vbucket.h:62 #15 0x000000000084aab1 in MagmaKVStore::commit (this=0x7f6650e98900, commitData=...) at /home/couchbase/jenkins/workspace/couchbase-server-unix/kv_engine/engines/ep/src/magma-kvstore/magma-kvstore.cc:611 #16 0x00000000007e6a3e in EPBucket::commit(Vbid, KVStore&, VB::Commit&) () at /home/couchbase/jenkins/workspace/couchbase-server-unix/kv_engine/engines/ep/src/ep_bucket.cc:897 #17 0x00000000007ed7cd in EPBucket::flushVBucket_UNLOCKED(LockedVBucketPtr) () at /home/couchbase/jenkins/workspace/couchbase-server-unix/kv_engine/engines/ep/src/ep_bucket.cc:772 #18 0x00000000007edb6f in EPBucket::flushVBucket(Vbid) () at /home/couchbase/jenkins/workspace/couchbase-server-unix/kv_engine/engines/ep/src/ep_bucket.cc:369 #19 0x00000000006bd9d0 in Flusher::flushVB() () at /home/couchbase/jenkins/workspace/couchbase-server-unix/kv_engine/engines/ep/src/flusher.cc:296 #20 0x00000000006bddf0 in Flusher::step(GlobalTask*) () at /home/couchbase/jenkins/workspace/couchbase-server-unix/kv_engine/engines/ep/src/flusher.cc:199 #21 0x00000000009fdc12 in GlobalTask::execute() () at /home/couchbase/jenkins/workspace/couchbase-server-unix/kv_engine/executor/globaltask.cc:68 #22 0x00000000009fad35 in FollyExecutorPool::TaskProxy::scheduleViaCPUPool()::{lambda()#2}::operator()() const (__closure=0x7f66c67ea540) at /home/couchbase/jenkins/workspace/couchbase-server-unix/kv_engine/executor/folly_executorpool.cc:189 #23 0x0000000000b55d40 in operator() (this=0x7f66c67ea540) at /home/couchbase/jenkins/workspace/cbdeps-platform-build-old/deps/packages/build/folly/folly-prefix/src/folly/folly/Function.h:416 #24 folly::ThreadPoolExecutor::runTask(std::shared_ptr const&, folly::ThreadPoolExecutor::Task&&) (this=0x7f66e2aa2400, thread=..., task=) at /home/couchbase/jenkins/workspace/cbdeps-platform-build-old/deps/packages/build/folly/folly-prefix/src/folly/folly/executors/ThreadPoolExecutor.cpp:97 #25 0x0000000000b3dafa in folly::CPUThreadPoolExecutor::threadRun (this=0x7f66e2aa2400, thread=...) at /home/couchbase/jenkins/workspace/cbdeps-platform-build-old/deps/packages/build/folly/folly-prefix/src/folly/folly/executors/CPUThreadPoolExecutor.cpp:265 #26 0x0000000000b58cf9 in __invoke_impl), folly::ThreadPoolExecutor*&, std::shared_ptr&> ( __t=, __f=) at /usr/local/include/c++/7.3.0/bits/invoke.h:73 #27 __invoke), folly::ThreadPoolExecutor*&, std::shared_ptr&> (__fn=) at /usr/local/include/c++/7.3.0/bits/invoke.h:95 #28 __call (__args=, this=) at /usr/local/include/c++/7.3.0/functional:467 #29 operator()<> (this=) at /usr/local/include/c++/7.3.0/functional:551 #30 folly::detail::function::FunctionTraits::callBig))(std::shared_ptr)> >(folly::detail::function::Data&) (p=...) at /home/couchbase/jenkins/workspace/cbdeps-platform-build-old/deps/packages/build/folly/folly-prefix/src/folly/folly/Function.h:401 #31 0x00000000009fa8b4 in operator() (this=0x7f66e2ab3c00) at /home/couchbase/jenkins/workspace/couchbase-server-unix/kv_engine/executor/folly_executorpool.cc:47 #32 operator() (__closure=0x7f66e2ab3c00) at /home/couchbase/jenkins/workspace/couchbase-server-unix/kv_engine/executor/folly_executorpool.cc:47 #33 folly::detail::function::FunctionTraits::callBig&&)::{lambda()#1}>(folly::detail::function::Data&) (p=...) at /home/couchbase/jenkins/workspace/couchbase-server-unix/server_build/tlm/deps/folly.exploded/include/folly/Function.h:401 #34 0x00007f66e4b8ad40 in execute_native_thread_routine () at /tmp/deploy/objdir/../gcc-10.2.0/libstdc++-v3/src/c++11/thread.cc:80 #35 0x00007f66e69a6ea5 in start_thread () from /lib64/libpthread.so.0 #36 0x00007f66e42d38dd in clone () from /lib64/libc.so.6 Thread 27 (LWP 37976): #0 0x0000000000809efb in basic_json (other=, this=0x7f664d465040) at /opt/gcc-10.2.0/include/c++/10.2.0/bits/stl_uninitialized.h:90 #1 moved_or_copied (this=0x7f66c6fe9b90, this=0x7f66c6fe9b90) at /home/couchbase/jenkins/workspace/couchbase-server-unix/server_build/tlm/deps/json.exploded/include/nlohmann/detail/json_ref.hpp:48 #2 basic_json (ref=..., this=0x7f664d465040) at /home/couchbase/jenkins/workspace/couchbase-server-unix/server_build/tlm/deps/json.exploded/include/nlohmann/json.hpp:1723 #3 _Construct, const nlohmann::detail::json_ref, std::allocator >, bool, long int, long unsigned int, double, std::allocator, nlohmann::adl_serializer> >&> (__p=0x7f664d465040) at /opt/gcc-10.2.0/include/c++/10.2.0/bits/stl_construct.h:109 #4 __uninit_copy >*, nlohmann::basic_json<>*> (__result=0x7f664d465040, __last=0x7f66c6fe9bd0, __first=0x7f66c6fe9b90) at /opt/gcc-10.2.0/include/c++/10.2.0/bits/stl_uninitialized.h:91 #5 uninitialized_copy >*, nlohmann::basic_json<>*> (__result=0x7f664d465040, __last=0x7f66c6fe9bd0, __first=) at /opt/gcc-10.2.0/include/c++/10.2.0/bits/stl_uninitialized.h:150 #6 __uninitialized_copy_a >*, nlohmann::basic_json<>*, nlohmann::basic_json<> > (__result=0x7f664d465040, __last=0x7f66c6fe9bd0, __first=) at /opt/gcc-10.2.0/include/c++/10.2.0/bits/stl_uninitialized.h:325 #7 _M_range_initialize >*> (__last=0x7f66c6fe9bd0, __first=, this=0x7f66408c99e0) at /opt/gcc-10.2.0/include/c++/10.2.0/bits/stl_vector.h:1585 #8 vector >*> (__a=..., __last=0x7f66c6fe9bd0, __first=, this=0x7f66408c99e0) at /opt/gcc-10.2.0/include/c++/10.2.0/bits/stl_vector.h:657 #9 construct, std::allocator > >, const nlohmann::detail::json_ref, std::allocator >, bool, long int, long unsigned int, double, std::allocator, nlohmann::adl_serializer> >*, const nlohmann::detail::json_ref, std::allocator >, bool, long int, long unsigned int, double, std::allocator, nlohmann::adl_serializer> >*> (__p=0x7f66408c99e0, this=) at /opt/gcc-10.2.0/include/c++/10.2.0/ext/new_allocator.h:150 #10 construct, std::allocator > >, const nlohmann::detail::json_ref, std::allocator >, bool, long int, long unsigned int, double, std::allocator, nlohmann::adl_serializer> >*, const nlohmann::detail::json_ref, std::allocator >, bool, long int, long unsigned int, double, std::allocator, nlohmann::adl_serializer> >*> (__p=0x7f66408c99e0, __a=...) at /opt/gcc-10.2.0/include/c++/10.2.0/bits/alloc_traits.h:512 #11 create, std::allocator > >, const nlohmann::detail::json_ref, std::allocator >, bool, long int, long unsigned int, double, std::allocator, nlohmann::adl_serializer> >*, const nlohmann::detail::json_ref, std::allocator >, bool, long int, long unsigned int, double, std::allocator, nlohmann::adl_serializer> >*> () at /home/couchbase/jenkins/workspace/couchbase-server-unix/server_build/tlm/deps/json.exploded/include/nlohmann/json.hpp:834 #12 nlohmann::basic_json, std::allocator >, bool, long, unsigned long, double, std::allocator, nlohmann::adl_serializer>::basic_json () at /home/couchbase/jenkins/workspace/couchbase-server-unix/server_build/tlm/deps/json.exploded/include/nlohmann/json.hpp:1449 #13 0x000000000080ee23 in json_ref (init=..., this=0x7f66c6fe9e30) at /opt/gcc-10.2.0/include/c++/10.2.0/bits/char_traits.h:322 #14 to_json(nlohmann::basic_json, std::allocator >, bool, long, unsigned long, double, std::allocator, nlohmann::adl_serializer>&, vbucket_state const&) () at /home/couchbase/jenkins/workspace/couchbase-server-unix/kv_engine/engines/ep/src/vbucket_state.cc:144 #15 0x000000000085075e in MagmaKVStore::addVBStateUpdateToLocalDbReqs(std::vector >&, vbucket_state const&, unsigned long) () at /home/couchbase/jenkins/workspace/couchbase-server-unix/kv_engine/engines/ep/src/magma-kvstore/magma-kvstore.cc:1881 #16 0x00000000008552c5 in std::_Function_handler >&), MagmaKVStore::saveDocs(VB::Commit&, kvstats_ctx&)::{lambda(std::vector >&)#2}>::_M_invoke(std::_Any_data const&, std::vector >&) () at /home/couchbase/jenkins/workspace/couchbase-server-unix/kv_engine/include/memcached/vbucket.h:62 #17 0x00000000008fe69d in operator() (__args#0=..., this=0x7f66c6fea590) at /opt/gcc-10.2.0/include/c++/10.2.0/bits/std_function.h:248 #18 magma::KVStore::WriteDocs(magma::WAL*, std::vector > const&, std::function, std::function >&)>) () at /home/couchbase/jenkins/workspace/couchbase-server-unix/magma/magma/kvstore/write.cc:104 #19 0x00000000008e0b05 in magma::Magma::Impl::WriteDocs(unsigned short, std::vector > const&, unsigned int, std::function, std::function >&)>) () at /opt/gcc-10.2.0/include/c++/10.2.0/bits/std_function.h:248 #20 0x00000000008e0cb2 in magma::Magma::WriteDocs(unsigned short, std::vector > const&, unsigned int, std::function, std::function >&)>) () at /opt/gcc-10.2.0/include/c++/10.2.0/bits/std_function.h:248 #21 0x000000000084dc22 in MagmaKVStore::saveDocs(VB::Commit&, kvstats_ctx&) () at /home/couchbase/jenkins/workspace/couchbase-server-unix/kv_engine/include/memcached/vbucket.h:62 #22 0x000000000084aab1 in MagmaKVStore::commit (this=0x7f664dc97a00, commitData=...) at /home/couchbase/jenkins/workspace/couchbase-server-unix/kv_engine/engines/ep/src/magma-kvstore/magma-kvstore.cc:611 #23 0x00000000007e6a3e in EPBucket::commit(Vbid, KVStore&, VB::Commit&) () at /home/couchbase/jenkins/workspace/couchbase-server-unix/kv_engine/engines/ep/src/ep_bucket.cc:897 #24 0x00000000007ed7cd in EPBucket::flushVBucket_UNLOCKED(LockedVBucketPtr) () at /home/couchbase/jenkins/workspace/couchbase-server-unix/kv_engine/engines/ep/src/ep_bucket.cc:772 #25 0x00000000007edb6f in EPBucket::flushVBucket(Vbid) () at /home/couchbase/jenkins/workspace/couchbase-server-unix/kv_engine/engines/ep/src/ep_bucket.cc:369 #26 0x00000000006bd9d0 in Flusher::flushVB() () at /home/couchbase/jenkins/workspace/couchbase-server-unix/kv_engine/engines/ep/src/flusher.cc:296 #27 0x00000000006bddf0 in Flusher::step(GlobalTask*) () at /home/couchbase/jenkins/workspace/couchbase-server-unix/kv_engine/engines/ep/src/flusher.cc:199 #28 0x00000000009fdc12 in GlobalTask::execute() () at /home/couchbase/jenkins/workspace/couchbase-server-unix/kv_engine/executor/globaltask.cc:68 #29 0x00000000009fad35 in FollyExecutorPool::TaskProxy::scheduleViaCPUPool()::{lambda()#2}::operator()() const (__closure=0x7f66c6feb540) at /home/couchbase/jenkins/workspace/couchbase-server-unix/kv_engine/executor/folly_executorpool.cc:189 #30 0x0000000000b55d40 in operator() (this=0x7f66c6feb540) at /home/couchbase/jenkins/workspace/cbdeps-platform-build-old/deps/packages/build/folly/folly-prefix/src/folly/folly/Function.h:416 #31 folly::ThreadPoolExecutor::runTask(std::shared_ptr const&, folly::ThreadPoolExecutor::Task&&) (this=0x7f66e2aa2400, thread=..., task=) at /home/couchbase/jenkins/workspace/cbdeps-platform-build-old/deps/packages/build/folly/folly-prefix/src/folly/folly/executors/ThreadPoolExecutor.cpp:97 #32 0x0000000000b3dafa in folly::CPUThreadPoolExecutor::threadRun (this=0x7f66e2aa2400, thread=...) at /home/couchbase/jenkins/workspace/cbdeps-platform-build-old/deps/packages/build/folly/folly-prefix/src/folly/folly/executors/CPUThreadPoolExecutor.cpp:265 #33 0x0000000000b58cf9 in __invoke_impl), folly::ThreadPoolExecutor*&, std::shared_ptr&> ( __t=, __f=) at /usr/local/include/c++/7.3.0/bits/invoke.h:73 #34 __invoke), folly::ThreadPoolExecutor*&, std::shared_ptr&> (__fn=) at /usr/local/include/c++/7.3.0/bits/invoke.h:95 #35 __call (__args=, this=) at /usr/local/include/c++/7.3.0/functional:467 #36 operator()<> (this=) at /usr/local/include/c++/7.3.0/functional:551 #37 folly::detail::function::FunctionTraits::callBig))(std::shared_ptr)> >(folly::detail::function::Data&) (p=...) at /home/couchbase/jenkins/workspace/cbdeps-platform-build-old/deps/packages/build/folly/folly-prefix/src/folly/folly/Function.h:401 #38 0x00000000009fa8b4 in operator() (this=0x7f66e2ab3c40) at /home/couchbase/jenkins/workspace/couchbase-server-unix/kv_engine/executor/folly_executorpool.cc:47 #39 operator() (__closure=0x7f66e2ab3c40) at /home/couchbase/jenkins/workspace/couchbase-server-unix/kv_engine/executor/folly_executorpool.cc:47 #40 folly::detail::function::FunctionTraits::callBig&&)::{lambda()#1}>(folly::detail::function::Data&) (p=...) at /home/couchbase/jenkins/workspace/couchbase-server-unix/server_build/tlm/deps/folly.exploded/include/folly/Function.h:401 #41 0x00007f66e4b8ad40 in execute_native_thread_routine () at /tmp/deploy/objdir/../gcc-10.2.0/libstdc++-v3/src/c++11/thread.cc:80 #42 0x00007f66e69a6ea5 in start_thread () from /lib64/libpthread.so.0 #43 0x00007f66e42d38dd in clone () from /lib64/libc.so.6 Thread 26 (LWP 37975): #0 0x00007f66e42cdbf9 in syscall () from /lib64/libc.so.6 #1 0x0000000000b35d4d in nativeFutexWaitImpl (waitMask=waitMask@entry=3346973464, absSteadyTime=absSteadyTime@entry=0x7f66c77ec2f8, absSystemTime=absSystemTime@entry=0x0, expected=, addr=) at /home/couchbase/jenkins/workspace/cbdeps-platform-build-old/deps/packages/build/folly/folly-prefix/src/folly/folly/detail/Futex.cpp:123 #2 folly::detail::futexWaitImpl (futex=, expected=, absSystemTime=absSystemTime@entry=0x0, absSteadyTime=absSteadyTime@entry=0x7f66c77ec318, waitMask=) at /home/couchbase/jenkins/workspace/cbdeps-platform-build-old/deps/packages/build/folly/folly-prefix/src/folly/folly/detail/Futex.cpp:254 #3 0x0000000000b3f8ea in futexWaitImpl const, std::chrono::time_point > > > (waitMask=, deadline=..., expected=, futex=) at /home/couchbase/jenkins/workspace/cbdeps-platform-build-old/deps/packages/build/folly/folly-prefix/src/folly/folly/detail/Futex-inl.h:85 #4 folly::detail::futexWaitUntil, std::chrono::_V2::steady_clock, std::chrono::duration > > (futex=, expected=, deadline=..., waitMask=) at /home/couchbase/jenkins/workspace/cbdeps-platform-build-old/deps/packages/build/folly/folly-prefix/src/folly/folly/detail/Futex-inl.h:123 #5 0x0000000000b4574d in folly::SaturatingSemaphore::tryWaitSlow > > (this=0x7f66cdfdd798, deadline=..., opt=...) at /home/couchbase/jenkins/workspace/cbdeps-platform-build-old/deps/packages/build/folly/folly-prefix/src/folly/folly/detail/MemoryIdler.h:164 #6 0x0000000000b45db6 in folly::detail::LifoSemBase, std::atomic>::try_wait_until > > ( this=0x12d1540, deadline=...) at /home/couchbase/jenkins/workspace/cbdeps-platform-build-old/deps/packages/build/folly/folly-prefix/src/folly/folly/synchronization/SaturatingSemaphore.h:179 #7 0x0000000000b49510 in folly::UnboundedBlockingQueue::try_take_for (this=0x12d1500, time=...) at /home/couchbase/jenkins/workspace/cbdeps-platform-build-old/deps/packages/build/folly/folly-prefix/src/folly/folly/synchronization/LifoSem.h:498 #8 0x0000000000b3da87 in folly::CPUThreadPoolExecutor::threadRun (this=0x7f66e2aa1800, thread=...) at /home/couchbase/jenkins/workspace/cbdeps-platform-build-old/deps/packages/build/folly/folly-prefix/src/folly/folly/executors/CPUThreadPoolExecutor.cpp:243 #9 0x0000000000b58cf9 in __invoke_impl), folly::ThreadPoolExecutor*&, std::shared_ptr&> ( __t=, __f=) at /usr/local/include/c++/7.3.0/bits/invoke.h:73 #10 __invoke), folly::ThreadPoolExecutor*&, std::shared_ptr&> (__fn=) at /usr/local/include/c++/7.3.0/bits/invoke.h:95 #11 __call (__args=, this=) at /usr/local/include/c++/7.3.0/functional:467 #12 operator()<> (this=) at /usr/local/include/c++/7.3.0/functional:551 #13 folly::detail::function::FunctionTraits::callBig))(std::shared_ptr)> >(folly::detail::function::Data&) (p=...) at /home/couchbase/jenkins/workspace/cbdeps-platform-build-old/deps/packages/build/folly/folly-prefix/src/folly/folly/Function.h:401 #14 0x00000000009fa8b4 in operator() (this=0x7f66e2b02d40) at /home/couchbase/jenkins/workspace/couchbase-server-unix/kv_engine/executor/folly_executorpool.cc:47 #15 operator() (__closure=0x7f66e2b02d40) at /home/couchbase/jenkins/workspace/couchbase-server-unix/kv_engine/executor/folly_executorpool.cc:47 #16 folly::detail::function::FunctionTraits::callBig&&)::{lambda()#1}>(folly::detail::function::Data&) (p=...) at /home/couchbase/jenkins/workspace/couchbase-server-unix/server_build/tlm/deps/folly.exploded/include/folly/Function.h:401 #17 0x00007f66e4b8ad40 in execute_native_thread_routine () at /tmp/deploy/objdir/../gcc-10.2.0/libstdc++-v3/src/c++11/thread.cc:80 #18 0x00007f66e69a6ea5 in start_thread () from /lib64/libpthread.so.0 #19 0x00007f66e42d38dd in clone () from /lib64/libc.so.6 Thread 25 (LWP 37974): #0 0x00007f66e42cdbf9 in syscall () from /lib64/libc.so.6 #1 0x0000000000b35d4d in nativeFutexWaitImpl (waitMask=waitMask@entry=3355366168, absSteadyTime=absSteadyTime@entry=0x7f66c7fed2f8, absSystemTime=absSystemTime@entry=0x0, expected=, addr=) at /home/couchbase/jenkins/workspace/cbdeps-platform-build-old/deps/packages/build/folly/folly-prefix/src/folly/folly/detail/Futex.cpp:123 #2 folly::detail::futexWaitImpl (futex=, expected=, absSystemTime=absSystemTime@entry=0x0, absSteadyTime=absSteadyTime@entry=0x7f66c7fed318, waitMask=) at /home/couchbase/jenkins/workspace/cbdeps-platform-build-old/deps/packages/build/folly/folly-prefix/src/folly/folly/detail/Futex.cpp:254 #3 0x0000000000b3f8ea in futexWaitImpl const, std::chrono::time_point > > > (waitMask=, deadline=..., expected=, futex=) at /home/couchbase/jenkins/workspace/cbdeps-platform-build-old/deps/packages/build/folly/folly-prefix/src/folly/folly/detail/Futex-inl.h:85 #4 folly::detail::futexWaitUntil, std::chrono::_V2::steady_clock, std::chrono::duration > > (futex=, expected=, deadline=..., waitMask=) at /home/couchbase/jenkins/workspace/cbdeps-platform-build-old/deps/packages/build/folly/folly-prefix/src/folly/folly/detail/Futex-inl.h:123 #5 0x0000000000b4574d in folly::SaturatingSemaphore::tryWaitSlow > > (this=0x7f66cdfdc9e8, deadline=..., opt=...) at /home/couchbase/jenkins/workspace/cbdeps-platform-build-old/deps/packages/build/folly/folly-prefix/src/folly/folly/detail/MemoryIdler.h:164 #6 0x0000000000b45db6 in folly::detail::LifoSemBase, std::atomic>::try_wait_until > > ( this=0x12d1540, deadline=...) at /home/couchbase/jenkins/workspace/cbdeps-platform-build-old/deps/packages/build/folly/folly-prefix/src/folly/folly/synchronization/SaturatingSemaphore.h:179 #7 0x0000000000b49510 in folly::UnboundedBlockingQueue::try_take_for (this=0x12d1500, time=...) at /home/couchbase/jenkins/workspace/cbdeps-platform-build-old/deps/packages/build/folly/folly-prefix/src/folly/folly/synchronization/LifoSem.h:498 #8 0x0000000000b3da87 in folly::CPUThreadPoolExecutor::threadRun (this=0x7f66e2aa1800, thread=...) at /home/couchbase/jenkins/workspace/cbdeps-platform-build-old/deps/packages/build/folly/folly-prefix/src/folly/folly/executors/CPUThreadPoolExecutor.cpp:243 #9 0x0000000000b58cf9 in __invoke_impl), folly::ThreadPoolExecutor*&, std::shared_ptr&> ( __t=