GNU gdb (Ubuntu 7.7.1-0ubuntu5~14.04.2) 7.7.1 Copyright (C) 2014 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-linux-gnu". Type "show configuration" for configuration details. For bug reporting instructions, please see: . Find the GDB manual and other documentation resources online at: . For help, type "help". Type "apropos word" to search for commands related to "word"... Reading symbols from /opt/couchbase/bin/memcached...done. [New LWP 13072] [New LWP 12764] [New LWP 12720] [New LWP 12768] [New LWP 12765] [New LWP 12769] [New LWP 12761] [New LWP 13064] [New LWP 13063] [New LWP 12767] [New LWP 12763] [New LWP 13073] [New LWP 12766] [New LWP 12762] [New LWP 13067] [New LWP 13066] [New LWP 12770] [New LWP 13065] [New LWP 12771] [New LWP 12772] [New LWP 13071] [New LWP 13069] [New LWP 13068] [New LWP 13070] [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". Core was generated by `/opt/couchbase/bin/memcached -C /opt/couchbase/var/lib/couchbase/config/memcach'. Program terminated with signal SIGABRT, Aborted. #0 0x00007fd33d408cc9 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56 (gdb) (gdb) t a a bt Thread 24 (Thread 0x7fd330a9d700 (LWP 13070)): #0 0x00007fd33d4c3fdd in fdatasync () at ../sysdeps/unix/syscall-template.S:81 #1 0x00007fd335e594f7 in couch_sync (errinfo=0x59d75d20, handle=0x53) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/couchstore/src/os.c:133 #2 0x00007fd3361489d6 in cfs_sync(struct {...} *, couch_file_handle) (errinfo=0x59d75d20, h=0xb1ea4f60) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/couch-kvstore/couch-fs-stats.cc:128 #3 0x00007fd335e4224c in couchstore_commit (db=0x59d75d00) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/couchstore/src/couch_db.cc:194 #4 0x00007fd33613be39 in CouchKVStore::saveDocs (this=, vbid=, rev=, docs=, docinfos=, docCount=, kvctx=..., snapStartSeqno=6163, snapEndSeqno=6163) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/couch-kvstore/couch-kvstore.cc:1779 #5 0x00007fd33613c362 in CouchKVStore::commit2couchstore (this=0x71fe800, cb=0x7fd330a9ca30, snapStartSeqno=, snapEndSeqno=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/couch-kvstore/couch-kvstore.cc:1671 #6 0x00007fd33613c5e5 in CouchKVStore::commit (this=0x53, cb=0x53, snapStartSeqno=7687230, snapEndSeqno=18446744073709551615) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/couch-kvstore/couch-kvstore.cc:1080 #7 0x00007fd33609726b in EventuallyPersistentStore::flushVBucket (this=0x60e5840, vbid=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/ep.cc:2804 #8 0x00007fd3360d2746 in Flusher::flushVB (this=0x17b6940) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/flusher.cc:281 #9 0x00007fd3360d3c4d in Flusher::step (this=0x17b6940, task=0x6083bd0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/flusher.cc:173 #10 0x00007fd3360dc5a5 in ExecutorThread::run (this=0x17acea0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:109 #11 0x00007fd3360dca8d in launch_executor_thread (arg=0x53) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:34 #12 0x00007fd33f65931a in platform_thread_wrap (arg=0x1758d10) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #13 0x00007fd33e3cb182 in start_thread (arg=0x7fd330a9d700) at pthread_create.c:312 #14 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 23 (Thread 0x7fd331a9f700 (LWP 13068)): #0 0x00007fd33d4c3fdd in fdatasync () at ../sysdeps/unix/syscall-template.S:81 #1 0x00007fd335e594f7 in couch_sync (errinfo=0xffa3cd20, handle=0x3) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/couchstore/src/os.c:133 #2 0x00007fd3361489d6 in cfs_sync(struct {...} *, couch_file_handle) (errinfo=0xffa3cd20, h=0xc319d3e0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/couch-kvstore/couch-fs-stats.cc:128 #3 0x00007fd335e4224c in couchstore_commit (db=0xffa3cd00) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/couchstore/src/couch_db.cc:194 #4 0x00007fd33613be39 in CouchKVStore::saveDocs (this=, vbid=, rev=, docs=, docinfos=, docCount=, kvctx=..., snapStartSeqno=15632, snapEndSeqno=15632) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/couch-kvstore/couch-kvstore.cc:1779 #5 0x00007fd33613c362 in CouchKVStore::commit2couchstore (this=0x60fa000, cb=0x7fd331a9ea30, snapStartSeqno=, snapEndSeqno=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/couch-kvstore/couch-kvstore.cc:1671 #6 0x00007fd33613c5e5 in CouchKVStore::commit (this=0x3, cb=0x3, snapStartSeqno=7687230, snapEndSeqno=18446744073709551615) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/couch-kvstore/couch-kvstore.cc:1080 #7 0x00007fd33609726b in EventuallyPersistentStore::flushVBucket (this=0x60e2000, vbid=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/ep.cc:2804 #8 0x00007fd3360d2746 in Flusher::flushVB (this=0x17b5540) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/flusher.cc:281 #9 0x00007fd3360d3c4d in Flusher::step (this=0x17b5540, task=0x6082730) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/flusher.cc:173 #10 0x00007fd3360dc5a5 in ExecutorThread::run (this=0x17acc60) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:109 #11 0x00007fd3360dca8d in launch_executor_thread (arg=0x3) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:34 #12 0x00007fd33f65931a in platform_thread_wrap (arg=0x1758cf0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #13 0x00007fd33e3cb182 in start_thread (arg=0x7fd331a9f700) at pthread_create.c:312 #14 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 22 (Thread 0x7fd33129e700 (LWP 13069)): #0 0x00007fd33d4c3fdd in fdatasync () at ../sysdeps/unix/syscall-template.S:81 #1 0x00007fd335e594f7 in couch_sync (errinfo=0x890428a0, handle=0x54) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/couchstore/src/os.c:133 #2 0x00007fd3361489d6 in cfs_sync(struct {...} *, couch_file_handle) (errinfo=0x890428a0, h=0x747bae20) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/couch-kvstore/couch-fs-stats.cc:128 #3 0x00007fd335e4224c in couchstore_commit (db=0x89042880) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/couchstore/src/couch_db.cc:194 #4 0x00007fd33613be39 in CouchKVStore::saveDocs (this=, vbid=, rev=, docs=, docinfos=, docCount=, kvctx=..., snapStartSeqno=15664, snapEndSeqno=15664) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/couch-kvstore/couch-kvstore.cc:1779 #5 0x00007fd33613c362 in CouchKVStore::commit2couchstore (this=0x60fc800, cb=0x7fd33129da30, snapStartSeqno=, snapEndSeqno=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/couch-kvstore/couch-kvstore.cc:1671 #6 0x00007fd33613c5e5 in CouchKVStore::commit (this=0x54, cb=0x54, snapStartSeqno=7687230, snapEndSeqno=18446744073709551615) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/couch-kvstore/couch-kvstore.cc:1080 #7 0x00007fd33609726b in EventuallyPersistentStore::flushVBucket (this=0x60e2000, vbid=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/ep.cc:2804 #8 0x00007fd3360d2746 in Flusher::flushVB (this=0x17b5680) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/flusher.cc:281 #9 0x00007fd3360d3c4d in Flusher::step (this=0x17b5680, task=0x60827d0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/flusher.cc:173 #10 0x00007fd3360dc5a5 in ExecutorThread::run (this=0x17acd80) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:109 #11 0x00007fd3360dca8d in launch_executor_thread (arg=0x54) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:34 #12 0x00007fd33f65931a in platform_thread_wrap (arg=0x1758d00) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #13 0x00007fd33e3cb182 in start_thread (arg=0x7fd33129e700) at pthread_create.c:312 #14 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 21 (Thread 0x7fd33029c700 (LWP 13071)): #0 0x00007fd33f04a007 in HashBytes (shift=, bytes=) at snappy.cc:49 #1 Hash (shift=, p=) at snappy.cc:52 #2 snappy::internal::CompressFragment ( input=0x1de94600b "{\"profile\":{\"likes\":39159,\"friends\":[\"8470C7DF7E\",\"986B4473EF\",\"76BB6A097B\",\"B8E1D4B050\",\"83C7D99911\"],\"name\":\"187EB88430074D92DE48\",\"online\":true},\"rating\":267.77,\"claim\":\"65DDDA98DFA2C795A426AEA84B4"..., input_size=, op=0xbe675afa "446696764DACC107C1\016\223\r\024\062\067\063E94\016G\032\034\070\061C51AFEe\263\f33A0\016\335\b\020\067\062C20\016\061\027\201\322\070\067\061\066\071\066\061\062\062\062\067B9547\016\204)", table=0x15cb6000, table_size=) at snappy.cc:358 #3 0x00007fd33f04aa48 in snappy::Compress (reader=0x7fd33029b4a0, writer=0x7fd33029b4c0) at snappy.cc:864 #4 0x00007fd33f04adfc in snappy::RawCompress (input=, input_length=, compressed=0x1de949b6b "764DA392773F631657AB765ACC7A05B7DDAC9743F7FDC998FB165AC944AB293D8876530E72BA0CE58A7D4430BF5D7CDB36A2C9A7464619DC50AE4C69AFC2C2E1A98280D3ADD8B29CAA74F0D75144673567D4847645A5C92D4C7AAE4E3183C606C0D2B109"..., compressed_length=0x12) at snappy.cc:1011 #5 0x00007fd335e42d8e in db_write_buf_compressed (file=0x74a53180, buf=0xe74a0eb0, pos=0x7fd33029b608, disk_size=0x7fd33029b618) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/couchstore/src/couch_file_write.cc:137 #6 0x00007fd335e437da in write_doc (writeopts=, disk_size=, bp=, doc=, db=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/couchstore/src/couch_save.cc:60 #7 add_doc_to_update_list (options=, seq=, idval=, seqval=, idterm=, seqterm=, fb=, info=, doc=, db=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/couchstore/src/couch_save.cc:299 #8 couchstore_save_documents (db=, docs=0x1f4d7d80, infos=0x1f4d8aa0, numdocs=, options=1363) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/couchstore/src/couch_save.cc:381 #9 0x00007fd33613bc0e in CouchKVStore::saveDocs (this=, vbid=, rev=, docs=, docinfos=, docCount=, kvctx=..., snapStartSeqno=15492, snapEndSeqno=15492) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/couch-kvstore/couch-kvstore.cc:1756 #10 0x00007fd33613c362 in CouchKVStore::commit2couchstore (this=0x61f6000, cb=0x7fd33029ba30, snapStartSeqno=, snapEndSeqno=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/couch-kvstore/couch-kvstore.cc:1671 #11 0x00007fd33613c5e5 in CouchKVStore::commit (this=0x23, cb=0x1de949b67, snapStartSeqno=8029248363, snapEndSeqno=18) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/couch-kvstore/couch-kvstore.cc:1080 #12 0x00007fd33609726b in EventuallyPersistentStore::flushVBucket (this=0x60e2000, vbid=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/ep.cc:2804 #13 0x00007fd3360d2746 in Flusher::flushVB (this=0x17b5900) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/flusher.cc:281 #14 0x00007fd3360d3c4d in Flusher::step (this=0x17b5900, task=0x6082780) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/flusher.cc:173 #15 0x00007fd3360dc5a5 in ExecutorThread::run (this=0x17acfc0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:109 #16 0x00007fd3360dca8d in launch_executor_thread (arg=0x23) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:34 #17 0x00007fd33f65931a in platform_thread_wrap (arg=0x1758d20) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #18 0x00007fd33e3cb182 in start_thread (arg=0x7fd33029c700) at pthread_create.c:312 #19 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 20 (Thread 0x7fd337399700 (LWP 12772)): #0 0x00007fd33d4ccb13 in epoll_wait () at ../sysdeps/unix/syscall-template.S:81 #1 0x00007fd33ebee5a6 in epoll_dispatch (base=0x6063900, tv=) at epoll.c:404 #2 0x00007fd33ebd9a04 in event_base_loop (base=0x6063900, flags=) at event.c:1558 #3 0x00007fd33f65931a in platform_thread_wrap (arg=0x1758230) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #4 0x00007fd33e3cb182 in start_thread (arg=0x7fd337399700) at pthread_create.c:312 #5 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 19 (Thread 0x7fd337b9a700 (LWP 12771)): #0 0x00007fd33d4ccb13 in epoll_wait () at ../sysdeps/unix/syscall-template.S:81 #1 0x00007fd33ebee5a6 in epoll_dispatch (base=0x6063680, tv=) at epoll.c:404 #2 0x00007fd33ebd9a04 in event_base_loop (base=0x6063680, flags=) at event.c:1558 #3 0x00007fd33f65931a in platform_thread_wrap (arg=0x1758220) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #4 0x00007fd33e3cb182 in start_thread (arg=0x7fd337b9a700) at pthread_create.c:312 #5 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 18 (Thread 0x7fd3332a2700 (LWP 13065)): #0 pthread_cond_timedwait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:238 #1 0x00007fd33f65977b in cb_cond_timedwait (cond=0x60b0460, mutex=0x60b0428, ms=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:156 #2 0x00007fd3360fefc0 in wait (nanoSecs=, this=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/syncobject.h:49 #3 TaskQueue::_doSleep (this=0x60b0420, t=...) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/taskqueue.cc:94 #4 0x00007fd33610166b in TaskQueue::_fetchNextTask (this=0x60b0420, t=..., toSleep=true) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/taskqueue.cc:117 #5 0x00007fd33610186f in TaskQueue::fetchNextTask (this=0x60b0420, thread=..., toSleep=142) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/taskqueue.cc:161 #6 0x00007fd3360c9c89 in ExecutorPool::_nextTask (this=0x6251a20, t=..., tick=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorpool.cc:217 #7 0x00007fd3360c9d0f in ExecutorPool::nextTask (this=0x6251a20, t=..., tick=142 '\216') at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorpool.cc:232 #8 0x00007fd3360dc221 in ExecutorThread::run (this=0x17ac900) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:81 #9 0x00007fd3360dca8d in launch_executor_thread (arg=0x60b0464) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:34 #10 0x00007fd33f65931a in platform_thread_wrap (arg=0x1758cd0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #11 0x00007fd33e3cb182 in start_thread (arg=0x7fd3332a2700) at pthread_create.c:312 #12 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 17 (Thread 0x7fd33839b700 (LWP 12770)): #0 0x00007fd33d4ccb13 in epoll_wait () at ../sysdeps/unix/syscall-template.S:81 #1 0x00007fd33ebee5a6 in epoll_dispatch (base=0x6063400, tv=) at epoll.c:404 #2 0x00007fd33ebd9a04 in event_base_loop (base=0x6063400, flags=) at event.c:1558 #3 0x00007fd33f65931a in platform_thread_wrap (arg=0x1758210) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #4 0x00007fd33e3cb182 in start_thread (arg=0x7fd33839b700) at pthread_create.c:312 #5 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 16 (Thread 0x7fd332aa1700 (LWP 13066)): #0 pthread_cond_timedwait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:238 #1 0x00007fd33f65977b in cb_cond_timedwait (cond=0x60b0460, mutex=0x60b0428, ms=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:156 #2 0x00007fd3360fefc0 in wait (nanoSecs=, this=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/syncobject.h:49 #3 TaskQueue::_doSleep (this=0x60b0420, t=...) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/taskqueue.cc:94 #4 0x00007fd33610166b in TaskQueue::_fetchNextTask (this=0x60b0420, t=..., toSleep=true) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/taskqueue.cc:117 #5 0x00007fd33610186f in TaskQueue::fetchNextTask (this=0x60b0420, thread=..., toSleep=136) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/taskqueue.cc:161 #6 0x00007fd3360c9c89 in ExecutorPool::_nextTask (this=0x6251a20, t=..., tick=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorpool.cc:217 #7 0x00007fd3360c9d0f in ExecutorPool::nextTask (this=0x6251a20, t=..., tick=136 '\210') at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorpool.cc:232 #8 0x00007fd3360dc221 in ExecutorThread::run (this=0x17aca20) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:81 #9 0x00007fd3360dca8d in launch_executor_thread (arg=0x60b0464) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:34 #10 0x00007fd33f65931a in platform_thread_wrap (arg=0x1758cc0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #11 0x00007fd33e3cb182 in start_thread (arg=0x7fd332aa1700) at pthread_create.c:312 #12 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 15 (Thread 0x7fd3322a0700 (LWP 13067)): #0 pthread_cond_timedwait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:238 #1 0x00007fd33f65977b in cb_cond_timedwait (cond=0x60b0460, mutex=0x60b0428, ms=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:156 #2 0x00007fd3360fefc0 in wait (nanoSecs=, this=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/syncobject.h:49 #3 TaskQueue::_doSleep (this=0x60b0420, t=...) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/taskqueue.cc:94 #4 0x00007fd33610166b in TaskQueue::_fetchNextTask (this=0x60b0420, t=..., toSleep=true) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/taskqueue.cc:117 #5 0x00007fd33610186f in TaskQueue::fetchNextTask (this=0x60b0420, thread=..., toSleep=138) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/taskqueue.cc:161 #6 0x00007fd3360c9c89 in ExecutorPool::_nextTask (this=0x6251a20, t=..., tick=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorpool.cc:217 #7 0x00007fd3360c9d0f in ExecutorPool::nextTask (this=0x6251a20, t=..., tick=138 '\212') at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorpool.cc:232 #8 0x00007fd3360dc221 in ExecutorThread::run (this=0x17acb40) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:81 #9 0x00007fd3360dca8d in launch_executor_thread (arg=0x60b0464) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:34 #10 0x00007fd33f65931a in platform_thread_wrap (arg=0x1758ce0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #11 0x00007fd33e3cb182 in start_thread (arg=0x7fd3322a0700) at pthread_create.c:312 #12 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 14 (Thread 0x7fd33c5b0700 (LWP 12762)): #0 pthread_cond_timedwait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:238 #1 0x00007fd33f65977b in cb_cond_timedwait (cond=0x7fd33c7b4200 , mutex=0x7fd33c7b41c0 , ms=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:156 #2 0x00007fd33c5b33fa in logger_thead_main (arg=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/memcached/extensions/loggers/file_logger.c:405 #3 0x00007fd33f65931a in platform_thread_wrap (arg=0x1758070) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #4 0x00007fd33e3cb182 in start_thread (arg=0x7fd33c5b0700) at pthread_create.c:312 #5 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 13 (Thread 0x7fd33a39f700 (LWP 12766)): #0 0x00007fd33d4ccb13 in epoll_wait () at ../sysdeps/unix/syscall-template.S:81 #1 0x00007fd33ebee5a6 in epoll_dispatch (base=0x6062a00, tv=) at epoll.c:404 #2 0x00007fd33ebd9a04 in event_base_loop (base=0x6062a00, flags=) at event.c:1558 #3 0x00007fd33f65931a in platform_thread_wrap (arg=0x17581d0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #4 0x00007fd33e3cb182 in start_thread (arg=0x7fd33a39f700) at pthread_create.c:312 #5 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 12 (Thread 0x7fd32f29a700 (LWP 13073)): #0 pthread_cond_timedwait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:238 #1 0x00007fd33f65977b in cb_cond_timedwait (cond=0x60b0720, mutex=0x60b06e8, ms=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:156 #2 0x00007fd3360fefc0 in wait (nanoSecs=, this=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/syncobject.h:49 #3 TaskQueue::_doSleep (this=0x60b06e0, t=...) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/taskqueue.cc:94 #4 0x00007fd33610166b in TaskQueue::_fetchNextTask (this=0x60b06e0, t=..., toSleep=true) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/taskqueue.cc:117 #5 0x00007fd33610186f in TaskQueue::fetchNextTask (this=0x60b06e0, thread=..., toSleep=81) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/taskqueue.cc:161 #6 0x00007fd3360c9c89 in ExecutorPool::_nextTask (this=0x6251a20, t=..., tick=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorpool.cc:217 #7 0x00007fd3360c9d0f in ExecutorPool::nextTask (this=0x6251a20, t=..., tick=81 'Q') at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorpool.cc:232 #8 0x00007fd3360dc221 in ExecutorThread::run (this=0x17ad200) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:81 #9 0x00007fd3360dca8d in launch_executor_thread (arg=0x60b0724) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:34 #10 0x00007fd33f65931a in platform_thread_wrap (arg=0x1758d40) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #11 0x00007fd33e3cb182 in start_thread (arg=0x7fd32f29a700) at pthread_create.c:312 #12 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 11 (Thread 0x7fd33bba2700 (LWP 12763)): #0 0x00007fd33d4ccb13 in epoll_wait () at ../sysdeps/unix/syscall-template.S:81 #1 0x00007fd33ebee5a6 in epoll_dispatch (base=0x6062280, tv=) at epoll.c:404 #2 0x00007fd33ebd9a04 in event_base_loop (base=0x6062280, flags=) at event.c:1558 #3 0x00007fd33f65931a in platform_thread_wrap (arg=0x17581a0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #4 0x00007fd33e3cb182 in start_thread (arg=0x7fd33bba2700) at pthread_create.c:312 #5 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 10 (Thread 0x7fd339b9e700 (LWP 12767)): #0 0x00007fd33d4ccb13 in epoll_wait () at ../sysdeps/unix/syscall-template.S:81 #1 0x00007fd33ebee5a6 in epoll_dispatch (base=0x6062c80, tv=) at epoll.c:404 #2 0x00007fd33ebd9a04 in event_base_loop (base=0x6062c80, flags=) at event.c:1558 #3 0x00007fd33f65931a in platform_thread_wrap (arg=0x17581e0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #4 0x00007fd33e3cb182 in start_thread (arg=0x7fd339b9e700) at pthread_create.c:312 #5 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 9 (Thread 0x7fd336b98700 (LWP 13063)): #0 0x00007fd33d492f3d in nanosleep () at ../sysdeps/unix/syscall-template.S:81 #1 0x00007fd33d4c44a4 in usleep (useconds=) at ../sysdeps/unix/sysv/linux/usleep.c:32 #2 0x00007fd3360da965 in updateStatsThread (arg=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/memory_tracker.cc:36 #3 0x00007fd33f65931a in platform_thread_wrap (arg=0x17582d0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #4 0x00007fd33e3cb182 in start_thread (arg=0x7fd336b98700) at pthread_create.c:312 #5 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 8 (Thread 0x7fd333aa3700 (LWP 13064)): #0 pthread_cond_timedwait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:238 #1 0x00007fd33f65977b in cb_cond_timedwait (cond=0x60b0460, mutex=0x60b0428, ms=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:156 #2 0x00007fd3360fefc0 in wait (nanoSecs=, this=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/syncobject.h:49 #3 TaskQueue::_doSleep (this=0x60b0420, t=...) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/taskqueue.cc:94 #4 0x00007fd33610166b in TaskQueue::_fetchNextTask (this=0x60b0420, t=..., toSleep=true) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/taskqueue.cc:117 #5 0x00007fd33610186f in TaskQueue::fetchNextTask (this=0x60b0420, thread=..., toSleep=140) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/taskqueue.cc:161 #6 0x00007fd3360c9c89 in ExecutorPool::_nextTask (this=0x6251a20, t=..., tick=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorpool.cc:217 #7 0x00007fd3360c9d0f in ExecutorPool::nextTask (this=0x6251a20, t=..., tick=140 '\214') at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorpool.cc:232 #8 0x00007fd3360dc221 in ExecutorThread::run (this=0x17ac7e0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:81 #9 0x00007fd3360dca8d in launch_executor_thread (arg=0x60b0464) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:34 #10 0x00007fd33f65931a in platform_thread_wrap (arg=0x1758cb0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #11 0x00007fd33e3cb182 in start_thread (arg=0x7fd333aa3700) at pthread_create.c:312 #12 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 7 (Thread 0x7fd33cfb5700 (LWP 12761)): #0 0x00007fd33d4bf12d in poll () at ../sysdeps/unix/syscall-template.S:81 #1 0x00007fd33cfb6c45 in get_command (buffer=, buffsize=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/memcached/extensions/daemon/stdin_check.c:51 #2 check_stdin_thread (arg=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/memcached/extensions/daemon/stdin_check.c:93 #3 0x00007fd33f65931a in platform_thread_wrap (arg=0x1758060) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #4 0x00007fd33e3cb182 in start_thread (arg=0x7fd33cfb5700) at pthread_create.c:312 #5 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 6 (Thread 0x7fd338b9c700 (LWP 12769)): #0 0x00007fd33d4ccb13 in epoll_wait () at ../sysdeps/unix/syscall-template.S:81 #1 0x00007fd33ebee5a6 in epoll_dispatch (base=0x6063180, tv=) at epoll.c:404 #2 0x00007fd33ebd9a04 in event_base_loop (base=0x6063180, flags=) at event.c:1558 #3 0x00007fd33f65931a in platform_thread_wrap (arg=0x1758200) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #4 0x00007fd33e3cb182 in start_thread (arg=0x7fd338b9c700) at pthread_create.c:312 #5 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 5 (Thread 0x7fd33aba0700 (LWP 12765)): #0 0x00007fd33d4ccb13 in epoll_wait () at ../sysdeps/unix/syscall-template.S:81 #1 0x00007fd33ebee5a6 in epoll_dispatch (base=0x6062780, tv=) at epoll.c:404 #2 0x00007fd33ebd9a04 in event_base_loop (base=0x6062780, flags=) at event.c:1558 #3 0x00007fd33f65931a in platform_thread_wrap (arg=0x17581c0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #4 0x00007fd33e3cb182 in start_thread (arg=0x7fd33aba0700) at pthread_create.c:312 #5 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 4 (Thread 0x7fd33939d700 (LWP 12768)): #0 0x00007fd33d4ccb13 in epoll_wait () at ../sysdeps/unix/syscall-template.S:81 #1 0x00007fd33ebee5a6 in epoll_dispatch (base=0x6062f00, tv=) at epoll.c:404 #2 0x00007fd33ebd9a04 in event_base_loop (base=0x6062f00, flags=) at event.c:1558 #3 0x00007fd33f65931a in platform_thread_wrap (arg=0x17581f0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #4 0x00007fd33e3cb182 in start_thread (arg=0x7fd33939d700) at pthread_create.c:312 #5 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 3 (Thread 0x7fd33fe79780 (LWP 12720)): #0 0x00007fd33d4ccb13 in epoll_wait () at ../sysdeps/unix/syscall-template.S:81 #1 0x00007fd33ebee5a6 in epoll_dispatch (base=0x6062000, tv=) at epoll.c:404 #2 0x00007fd33ebd9a04 in event_base_loop (base=0x6062000, flags=) at event.c:1558 #3 0x000000000041177d in main (argc=, argv=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/memcached/daemon/memcached.c:8931 Thread 2 (Thread 0x7fd33b3a1700 (LWP 12764)): #0 0x00007fd33d4ccb13 in epoll_wait () at ../sysdeps/unix/syscall-template.S:81 #1 0x00007fd33ebee5a6 in epoll_dispatch (base=0x6062500, tv=) at epoll.c:404 #2 0x00007fd33ebd9a04 in event_base_loop (base=0x6062500, flags=) at event.c:1558 #3 0x00007fd33f65931a in platform_thread_wrap (arg=0x17581b0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #4 0x00007fd33e3cb182 in start_thread (arg=0x7fd33b3a1700) at pthread_create.c:312 #5 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 1 (Thread 0x7fd32fa9b700 (LWP 13072)): #0 0x00007fd33d408cc9 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56 #1 0x00007fd33d40c0d8 in __GI_abort () at abort.c:89 #2 0x00007fd336085c0e in CheckpointManager::registerTAPCursorBySeqno (this=0x669ad98, name=..., startBySeqno=15533) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/checkpoint.cc:433 #3 0x00007fd336116990 in ActiveStream::markDiskSnapshot (this=0x7ccb3b80, startSeqno=0, endSeqno=15533) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/dcp-stream.cc:396 #4 0x00007fd33613cdee in CouchKVStore::loadDB (this=0x6184000, cb=, cl=, sr=..., keysOnly=, vbid=709, startSeqno=1, options=0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/couch-kvstore/couch-kvstore.cc:1212 #5 0x00007fd33613d226 in CouchKVStore::dump (this=0x31b0, vb=, stSeqno=, cb=..., cl=..., sr=...) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/couch-kvstore/couch-kvstore.cc:1047 #6 0x00007fd336113e06 in DCPBackfill::run (this=0x3934cf60) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/dcp-stream.cc:167 #7 0x00007fd3360dc5a5 in ExecutorThread::run (this=0x17ad0e0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:109 #8 0x00007fd3360dca8d in launch_executor_thread (arg=0x31b0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:34 #9 0x00007fd33f65931a in platform_thread_wrap (arg=0x1758d30) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #10 0x00007fd33e3cb182 in start_thread (arg=0x7fd32fa9b700) at pthread_create.c:312 #11 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 (gdb) Thread 24 (Thread 0x7fd330a9d700 (LWP 13070)): #0 0x00007fd33d4c3fdd in fdatasync () at ../sysdeps/unix/syscall-template.S:81 #1 0x00007fd335e594f7 in couch_sync (errinfo=0x59d75d20, handle=0x53) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/couchstore/src/os.c:133 #2 0x00007fd3361489d6 in cfs_sync(struct {...} *, couch_file_handle) (errinfo=0x59d75d20, h=0xb1ea4f60) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/couch-kvstore/couch-fs-stats.cc:128 #3 0x00007fd335e4224c in couchstore_commit (db=0x59d75d00) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/couchstore/src/couch_db.cc:194 #4 0x00007fd33613be39 in CouchKVStore::saveDocs (this=, vbid=, rev=, docs=, docinfos=, docCount=, kvctx=..., snapStartSeqno=6163, snapEndSeqno=6163) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/couch-kvstore/couch-kvstore.cc:1779 #5 0x00007fd33613c362 in CouchKVStore::commit2couchstore (this=0x71fe800, cb=0x7fd330a9ca30, snapStartSeqno=, snapEndSeqno=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/couch-kvstore/couch-kvstore.cc:1671 #6 0x00007fd33613c5e5 in CouchKVStore::commit (this=0x53, cb=0x53, snapStartSeqno=7687230, snapEndSeqno=18446744073709551615) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/couch-kvstore/couch-kvstore.cc:1080 #7 0x00007fd33609726b in EventuallyPersistentStore::flushVBucket (this=0x60e5840, vbid=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/ep.cc:2804 #8 0x00007fd3360d2746 in Flusher::flushVB (this=0x17b6940) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/flusher.cc:281 #9 0x00007fd3360d3c4d in Flusher::step (this=0x17b6940, task=0x6083bd0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/flusher.cc:173 #10 0x00007fd3360dc5a5 in ExecutorThread::run (this=0x17acea0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:109 #11 0x00007fd3360dca8d in launch_executor_thread (arg=0x53) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:34 #12 0x00007fd33f65931a in platform_thread_wrap (arg=0x1758d10) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #13 0x00007fd33e3cb182 in start_thread (arg=0x7fd330a9d700) at pthread_create.c:312 #14 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 23 (Thread 0x7fd331a9f700 (LWP 13068)): #0 0x00007fd33d4c3fdd in fdatasync () at ../sysdeps/unix/syscall-template.S:81 #1 0x00007fd335e594f7 in couch_sync (errinfo=0xffa3cd20, handle=0x3) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/couchstore/src/os.c:133 #2 0x00007fd3361489d6 in cfs_sync(struct {...} *, couch_file_handle) (errinfo=0xffa3cd20, h=0xc319d3e0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/couch-kvstore/couch-fs-stats.cc:128 #3 0x00007fd335e4224c in couchstore_commit (db=0xffa3cd00) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/couchstore/src/couch_db.cc:194 #4 0x00007fd33613be39 in CouchKVStore::saveDocs (this=, vbid=, rev=, docs=, docinfos=, docCount=, kvctx=..., snapStartSeqno=15632, snapEndSeqno=15632) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/couch-kvstore/couch-kvstore.cc:1779 #5 0x00007fd33613c362 in CouchKVStore::commit2couchstore (this=0x60fa000, cb=0x7fd331a9ea30, snapStartSeqno=, snapEndSeqno=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/couch-kvstore/couch-kvstore.cc:1671 #6 0x00007fd33613c5e5 in CouchKVStore::commit (this=0x3, cb=0x3, snapStartSeqno=7687230, snapEndSeqno=18446744073709551615) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/couch-kvstore/couch-kvstore.cc:1080 #7 0x00007fd33609726b in EventuallyPersistentStore::flushVBucket (this=0x60e2000, vbid=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/ep.cc:2804 #8 0x00007fd3360d2746 in Flusher::flushVB (this=0x17b5540) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/flusher.cc:281 #9 0x00007fd3360d3c4d in Flusher::step (this=0x17b5540, task=0x6082730) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/flusher.cc:173 #10 0x00007fd3360dc5a5 in ExecutorThread::run (this=0x17acc60) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:109 #11 0x00007fd3360dca8d in launch_executor_thread (arg=0x3) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:34 #12 0x00007fd33f65931a in platform_thread_wrap (arg=0x1758cf0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #13 0x00007fd33e3cb182 in start_thread (arg=0x7fd331a9f700) at pthread_create.c:312 #14 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 22 (Thread 0x7fd33129e700 (LWP 13069)): #0 0x00007fd33d4c3fdd in fdatasync () at ../sysdeps/unix/syscall-template.S:81 #1 0x00007fd335e594f7 in couch_sync (errinfo=0x890428a0, handle=0x54) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/couchstore/src/os.c:133 #2 0x00007fd3361489d6 in cfs_sync(struct {...} *, couch_file_handle) (errinfo=0x890428a0, h=0x747bae20) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/couch-kvstore/couch-fs-stats.cc:128 #3 0x00007fd335e4224c in couchstore_commit (db=0x89042880) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/couchstore/src/couch_db.cc:194 #4 0x00007fd33613be39 in CouchKVStore::saveDocs (this=, vbid=, rev=, docs=, docinfos=, docCount=, kvctx=..., snapStartSeqno=15664, snapEndSeqno=15664) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/couch-kvstore/couch-kvstore.cc:1779 #5 0x00007fd33613c362 in CouchKVStore::commit2couchstore (this=0x60fc800, cb=0x7fd33129da30, snapStartSeqno=, snapEndSeqno=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/couch-kvstore/couch-kvstore.cc:1671 #6 0x00007fd33613c5e5 in CouchKVStore::commit (this=0x54, cb=0x54, snapStartSeqno=7687230, snapEndSeqno=18446744073709551615) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/couch-kvstore/couch-kvstore.cc:1080 #7 0x00007fd33609726b in EventuallyPersistentStore::flushVBucket (this=0x60e2000, vbid=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/ep.cc:2804 #8 0x00007fd3360d2746 in Flusher::flushVB (this=0x17b5680) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/flusher.cc:281 #9 0x00007fd3360d3c4d in Flusher::step (this=0x17b5680, task=0x60827d0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/flusher.cc:173 #10 0x00007fd3360dc5a5 in ExecutorThread::run (this=0x17acd80) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:109 #11 0x00007fd3360dca8d in launch_executor_thread (arg=0x54) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:34 #12 0x00007fd33f65931a in platform_thread_wrap (arg=0x1758d00) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #13 0x00007fd33e3cb182 in start_thread (arg=0x7fd33129e700) at pthread_create.c:312 #14 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 21 (Thread 0x7fd33029c700 (LWP 13071)): #0 0x00007fd33f04a007 in HashBytes (shift=, bytes=) at snappy.cc:49 #1 Hash (shift=, p=) at snappy.cc:52 #2 snappy::internal::CompressFragment ( input=0x1de94600b "{\"profile\":{\"likes\":39159,\"friends\":[\"8470C7DF7E\",\"986B4473EF\",\"76BB6A097B\",\"B8E1D4B050\",\"83C7D99911\"],\"name\":\"187EB88430074D92DE48\",\"online\":true},\"rating\":267.77,\"claim\":\"65DDDA98DFA2C795A426AEA84B4"..., input_size=, op=0xbe675afa "446696764DACC107C1\016\223\r\024\062\067\063E94\016G\032\034\070\061C51AFEe\263\f33A0\016\335\b\020\067\062C20\016\061\027\201\322\070\067\061\066\071\066\061\062\062\062\067B9547\016\204)", table=0x15cb6000, table_size=) at snappy.cc:358 #3 0x00007fd33f04aa48 in snappy::Compress (reader=0x7fd33029b4a0, writer=0x7fd33029b4c0) at snappy.cc:864 #4 0x00007fd33f04adfc in snappy::RawCompress (input=, input_length=, compressed=0x1de949b6b "764DA392773F631657AB765ACC7A05B7DDAC9743F7FDC998FB165AC944AB293D8876530E72BA0CE58A7D4430BF5D7CDB36A2C9A7464619DC50AE4C69AFC2C2E1A98280D3ADD8B29CAA74F0D75144673567D4847645A5C92D4C7AAE4E3183C606C0D2B109"..., compressed_length=0x12) at snappy.cc:1011 #5 0x00007fd335e42d8e in db_write_buf_compressed (file=0x74a53180, buf=0xe74a0eb0, pos=0x7fd33029b608, disk_size=0x7fd33029b618) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/couchstore/src/couch_file_write.cc:137 #6 0x00007fd335e437da in write_doc (writeopts=, disk_size=, bp=, doc=, db=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/couchstore/src/couch_save.cc:60 #7 add_doc_to_update_list (options=, seq=, idval=, seqval=, idterm=, seqterm=, fb=, info=, doc=, db=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/couchstore/src/couch_save.cc:299 #8 couchstore_save_documents (db=, docs=0x1f4d7d80, infos=0x1f4d8aa0, numdocs=, options=1363) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/couchstore/src/couch_save.cc:381 #9 0x00007fd33613bc0e in CouchKVStore::saveDocs (this=, vbid=, rev=, docs=, docinfos=, docCount=, kvctx=..., snapStartSeqno=15492, snapEndSeqno=15492) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/couch-kvstore/couch-kvstore.cc:1756 #10 0x00007fd33613c362 in CouchKVStore::commit2couchstore (this=0x61f6000, cb=0x7fd33029ba30, snapStartSeqno=, snapEndSeqno=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/couch-kvstore/couch-kvstore.cc:1671 #11 0x00007fd33613c5e5 in CouchKVStore::commit (this=0x23, cb=0x1de949b67, snapStartSeqno=8029248363, snapEndSeqno=18) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/couch-kvstore/couch-kvstore.cc:1080 #12 0x00007fd33609726b in EventuallyPersistentStore::flushVBucket (this=0x60e2000, vbid=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/ep.cc:2804 #13 0x00007fd3360d2746 in Flusher::flushVB (this=0x17b5900) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/flusher.cc:281 #14 0x00007fd3360d3c4d in Flusher::step (this=0x17b5900, task=0x6082780) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/flusher.cc:173 #15 0x00007fd3360dc5a5 in ExecutorThread::run (this=0x17acfc0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:109 #16 0x00007fd3360dca8d in launch_executor_thread (arg=0x23) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:34 #17 0x00007fd33f65931a in platform_thread_wrap (arg=0x1758d20) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #18 0x00007fd33e3cb182 in start_thread (arg=0x7fd33029c700) at pthread_create.c:312 #19 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 20 (Thread 0x7fd337399700 (LWP 12772)): #0 0x00007fd33d4ccb13 in epoll_wait () at ../sysdeps/unix/syscall-template.S:81 #1 0x00007fd33ebee5a6 in epoll_dispatch (base=0x6063900, tv=) at epoll.c:404 #2 0x00007fd33ebd9a04 in event_base_loop (base=0x6063900, flags=) at event.c:1558 #3 0x00007fd33f65931a in platform_thread_wrap (arg=0x1758230) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #4 0x00007fd33e3cb182 in start_thread (arg=0x7fd337399700) at pthread_create.c:312 #5 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 19 (Thread 0x7fd337b9a700 (LWP 12771)): #0 0x00007fd33d4ccb13 in epoll_wait () at ../sysdeps/unix/syscall-template.S:81 #1 0x00007fd33ebee5a6 in epoll_dispatch (base=0x6063680, tv=) at epoll.c:404 #2 0x00007fd33ebd9a04 in event_base_loop (base=0x6063680, flags=) at event.c:1558 #3 0x00007fd33f65931a in platform_thread_wrap (arg=0x1758220) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #4 0x00007fd33e3cb182 in start_thread (arg=0x7fd337b9a700) at pthread_create.c:312 #5 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 18 (Thread 0x7fd3332a2700 (LWP 13065)): #0 pthread_cond_timedwait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:238 #1 0x00007fd33f65977b in cb_cond_timedwait (cond=0x60b0460, mutex=0x60b0428, ms=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:156 #2 0x00007fd3360fefc0 in wait (nanoSecs=, this=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/syncobject.h:49 #3 TaskQueue::_doSleep (this=0x60b0420, t=...) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/taskqueue.cc:94 #4 0x00007fd33610166b in TaskQueue::_fetchNextTask (this=0x60b0420, t=..., toSleep=true) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/taskqueue.cc:117 #5 0x00007fd33610186f in TaskQueue::fetchNextTask (this=0x60b0420, thread=..., toSleep=142) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/taskqueue.cc:161 #6 0x00007fd3360c9c89 in ExecutorPool::_nextTask (this=0x6251a20, t=..., tick=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorpool.cc:217 #7 0x00007fd3360c9d0f in ExecutorPool::nextTask (this=0x6251a20, t=..., tick=142 '\216') at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorpool.cc:232 #8 0x00007fd3360dc221 in ExecutorThread::run (this=0x17ac900) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:81 #9 0x00007fd3360dca8d in launch_executor_thread (arg=0x60b0464) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:34 #10 0x00007fd33f65931a in platform_thread_wrap (arg=0x1758cd0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #11 0x00007fd33e3cb182 in start_thread (arg=0x7fd3332a2700) at pthread_create.c:312 #12 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 17 (Thread 0x7fd33839b700 (LWP 12770)): #0 0x00007fd33d4ccb13 in epoll_wait () at ../sysdeps/unix/syscall-template.S:81 #1 0x00007fd33ebee5a6 in epoll_dispatch (base=0x6063400, tv=) at epoll.c:404 #2 0x00007fd33ebd9a04 in event_base_loop (base=0x6063400, flags=) at event.c:1558 #3 0x00007fd33f65931a in platform_thread_wrap (arg=0x1758210) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #4 0x00007fd33e3cb182 in start_thread (arg=0x7fd33839b700) at pthread_create.c:312 #5 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 16 (Thread 0x7fd332aa1700 (LWP 13066)): #0 pthread_cond_timedwait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:238 #1 0x00007fd33f65977b in cb_cond_timedwait (cond=0x60b0460, mutex=0x60b0428, ms=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:156 #2 0x00007fd3360fefc0 in wait (nanoSecs=, this=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/syncobject.h:49 #3 TaskQueue::_doSleep (this=0x60b0420, t=...) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/taskqueue.cc:94 #4 0x00007fd33610166b in TaskQueue::_fetchNextTask (this=0x60b0420, t=..., toSleep=true) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/taskqueue.cc:117 #5 0x00007fd33610186f in TaskQueue::fetchNextTask (this=0x60b0420, thread=..., toSleep=136) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/taskqueue.cc:161 #6 0x00007fd3360c9c89 in ExecutorPool::_nextTask (this=0x6251a20, t=..., tick=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorpool.cc:217 #7 0x00007fd3360c9d0f in ExecutorPool::nextTask (this=0x6251a20, t=..., tick=136 '\210') at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorpool.cc:232 #8 0x00007fd3360dc221 in ExecutorThread::run (this=0x17aca20) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:81 #9 0x00007fd3360dca8d in launch_executor_thread (arg=0x60b0464) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:34 #10 0x00007fd33f65931a in platform_thread_wrap (arg=0x1758cc0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #11 0x00007fd33e3cb182 in start_thread (arg=0x7fd332aa1700) at pthread_create.c:312 #12 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 15 (Thread 0x7fd3322a0700 (LWP 13067)): #0 pthread_cond_timedwait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:238 #1 0x00007fd33f65977b in cb_cond_timedwait (cond=0x60b0460, mutex=0x60b0428, ms=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:156 #2 0x00007fd3360fefc0 in wait (nanoSecs=, this=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/syncobject.h:49 #3 TaskQueue::_doSleep (this=0x60b0420, t=...) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/taskqueue.cc:94 #4 0x00007fd33610166b in TaskQueue::_fetchNextTask (this=0x60b0420, t=..., toSleep=true) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/taskqueue.cc:117 #5 0x00007fd33610186f in TaskQueue::fetchNextTask (this=0x60b0420, thread=..., toSleep=138) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/taskqueue.cc:161 #6 0x00007fd3360c9c89 in ExecutorPool::_nextTask (this=0x6251a20, t=..., tick=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorpool.cc:217 #7 0x00007fd3360c9d0f in ExecutorPool::nextTask (this=0x6251a20, t=..., tick=138 '\212') at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorpool.cc:232 #8 0x00007fd3360dc221 in ExecutorThread::run (this=0x17acb40) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:81 #9 0x00007fd3360dca8d in launch_executor_thread (arg=0x60b0464) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:34 #10 0x00007fd33f65931a in platform_thread_wrap (arg=0x1758ce0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #11 0x00007fd33e3cb182 in start_thread (arg=0x7fd3322a0700) at pthread_create.c:312 #12 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 14 (Thread 0x7fd33c5b0700 (LWP 12762)): #0 pthread_cond_timedwait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:238 #1 0x00007fd33f65977b in cb_cond_timedwait (cond=0x7fd33c7b4200 , mutex=0x7fd33c7b41c0 , ms=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:156 #2 0x00007fd33c5b33fa in logger_thead_main (arg=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/memcached/extensions/loggers/file_logger.c:405 #3 0x00007fd33f65931a in platform_thread_wrap (arg=0x1758070) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #4 0x00007fd33e3cb182 in start_thread (arg=0x7fd33c5b0700) at pthread_create.c:312 #5 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 13 (Thread 0x7fd33a39f700 (LWP 12766)): #0 0x00007fd33d4ccb13 in epoll_wait () at ../sysdeps/unix/syscall-template.S:81 #1 0x00007fd33ebee5a6 in epoll_dispatch (base=0x6062a00, tv=) at epoll.c:404 #2 0x00007fd33ebd9a04 in event_base_loop (base=0x6062a00, flags=) at event.c:1558 #3 0x00007fd33f65931a in platform_thread_wrap (arg=0x17581d0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #4 0x00007fd33e3cb182 in start_thread (arg=0x7fd33a39f700) at pthread_create.c:312 #5 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 12 (Thread 0x7fd32f29a700 (LWP 13073)): #0 pthread_cond_timedwait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:238 #1 0x00007fd33f65977b in cb_cond_timedwait (cond=0x60b0720, mutex=0x60b06e8, ms=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:156 #2 0x00007fd3360fefc0 in wait (nanoSecs=, this=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/syncobject.h:49 #3 TaskQueue::_doSleep (this=0x60b06e0, t=...) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/taskqueue.cc:94 #4 0x00007fd33610166b in TaskQueue::_fetchNextTask (this=0x60b06e0, t=..., toSleep=true) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/taskqueue.cc:117 #5 0x00007fd33610186f in TaskQueue::fetchNextTask (this=0x60b06e0, thread=..., toSleep=81) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/taskqueue.cc:161 #6 0x00007fd3360c9c89 in ExecutorPool::_nextTask (this=0x6251a20, t=..., tick=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorpool.cc:217 #7 0x00007fd3360c9d0f in ExecutorPool::nextTask (this=0x6251a20, t=..., tick=81 'Q') at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorpool.cc:232 #8 0x00007fd3360dc221 in ExecutorThread::run (this=0x17ad200) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:81 #9 0x00007fd3360dca8d in launch_executor_thread (arg=0x60b0724) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:34 #10 0x00007fd33f65931a in platform_thread_wrap (arg=0x1758d40) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #11 0x00007fd33e3cb182 in start_thread (arg=0x7fd32f29a700) at pthread_create.c:312 #12 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 11 (Thread 0x7fd33bba2700 (LWP 12763)): #0 0x00007fd33d4ccb13 in epoll_wait () at ../sysdeps/unix/syscall-template.S:81 #1 0x00007fd33ebee5a6 in epoll_dispatch (base=0x6062280, tv=) at epoll.c:404 #2 0x00007fd33ebd9a04 in event_base_loop (base=0x6062280, flags=) at event.c:1558 #3 0x00007fd33f65931a in platform_thread_wrap (arg=0x17581a0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #4 0x00007fd33e3cb182 in start_thread (arg=0x7fd33bba2700) at pthread_create.c:312 #5 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 10 (Thread 0x7fd339b9e700 (LWP 12767)): #0 0x00007fd33d4ccb13 in epoll_wait () at ../sysdeps/unix/syscall-template.S:81 #1 0x00007fd33ebee5a6 in epoll_dispatch (base=0x6062c80, tv=) at epoll.c:404 #2 0x00007fd33ebd9a04 in event_base_loop (base=0x6062c80, flags=) at event.c:1558 #3 0x00007fd33f65931a in platform_thread_wrap (arg=0x17581e0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #4 0x00007fd33e3cb182 in start_thread (arg=0x7fd339b9e700) at pthread_create.c:312 #5 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 9 (Thread 0x7fd336b98700 (LWP 13063)): #0 0x00007fd33d492f3d in nanosleep () at ../sysdeps/unix/syscall-template.S:81 #1 0x00007fd33d4c44a4 in usleep (useconds=) at ../sysdeps/unix/sysv/linux/usleep.c:32 #2 0x00007fd3360da965 in updateStatsThread (arg=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/memory_tracker.cc:36 #3 0x00007fd33f65931a in platform_thread_wrap (arg=0x17582d0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #4 0x00007fd33e3cb182 in start_thread (arg=0x7fd336b98700) at pthread_create.c:312 #5 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 8 (Thread 0x7fd333aa3700 (LWP 13064)): #0 pthread_cond_timedwait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:238 #1 0x00007fd33f65977b in cb_cond_timedwait (cond=0x60b0460, mutex=0x60b0428, ms=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:156 #2 0x00007fd3360fefc0 in wait (nanoSecs=, this=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/syncobject.h:49 #3 TaskQueue::_doSleep (this=0x60b0420, t=...) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/taskqueue.cc:94 #4 0x00007fd33610166b in TaskQueue::_fetchNextTask (this=0x60b0420, t=..., toSleep=true) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/taskqueue.cc:117 #5 0x00007fd33610186f in TaskQueue::fetchNextTask (this=0x60b0420, thread=..., toSleep=140) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/taskqueue.cc:161 #6 0x00007fd3360c9c89 in ExecutorPool::_nextTask (this=0x6251a20, t=..., tick=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorpool.cc:217 #7 0x00007fd3360c9d0f in ExecutorPool::nextTask (this=0x6251a20, t=..., tick=140 '\214') at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorpool.cc:232 #8 0x00007fd3360dc221 in ExecutorThread::run (this=0x17ac7e0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:81 #9 0x00007fd3360dca8d in launch_executor_thread (arg=0x60b0464) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:34 #10 0x00007fd33f65931a in platform_thread_wrap (arg=0x1758cb0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #11 0x00007fd33e3cb182 in start_thread (arg=0x7fd333aa3700) at pthread_create.c:312 #12 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 7 (Thread 0x7fd33cfb5700 (LWP 12761)): #0 0x00007fd33d4bf12d in poll () at ../sysdeps/unix/syscall-template.S:81 #1 0x00007fd33cfb6c45 in get_command (buffer=, buffsize=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/memcached/extensions/daemon/stdin_check.c:51 #2 check_stdin_thread (arg=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/memcached/extensions/daemon/stdin_check.c:93 #3 0x00007fd33f65931a in platform_thread_wrap (arg=0x1758060) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #4 0x00007fd33e3cb182 in start_thread (arg=0x7fd33cfb5700) at pthread_create.c:312 #5 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 6 (Thread 0x7fd338b9c700 (LWP 12769)): #0 0x00007fd33d4ccb13 in epoll_wait () at ../sysdeps/unix/syscall-template.S:81 #1 0x00007fd33ebee5a6 in epoll_dispatch (base=0x6063180, tv=) at epoll.c:404 #2 0x00007fd33ebd9a04 in event_base_loop (base=0x6063180, flags=) at event.c:1558 #3 0x00007fd33f65931a in platform_thread_wrap (arg=0x1758200) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #4 0x00007fd33e3cb182 in start_thread (arg=0x7fd338b9c700) at pthread_create.c:312 #5 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 5 (Thread 0x7fd33aba0700 (LWP 12765)): #0 0x00007fd33d4ccb13 in epoll_wait () at ../sysdeps/unix/syscall-template.S:81 #1 0x00007fd33ebee5a6 in epoll_dispatch (base=0x6062780, tv=) at epoll.c:404 #2 0x00007fd33ebd9a04 in event_base_loop (base=0x6062780, flags=) at event.c:1558 #3 0x00007fd33f65931a in platform_thread_wrap (arg=0x17581c0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #4 0x00007fd33e3cb182 in start_thread (arg=0x7fd33aba0700) at pthread_create.c:312 #5 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 4 (Thread 0x7fd33939d700 (LWP 12768)): #0 0x00007fd33d4ccb13 in epoll_wait () at ../sysdeps/unix/syscall-template.S:81 #1 0x00007fd33ebee5a6 in epoll_dispatch (base=0x6062f00, tv=) at epoll.c:404 #2 0x00007fd33ebd9a04 in event_base_loop (base=0x6062f00, flags=) at event.c:1558 #3 0x00007fd33f65931a in platform_thread_wrap (arg=0x17581f0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #4 0x00007fd33e3cb182 in start_thread (arg=0x7fd33939d700) at pthread_create.c:312 #5 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 3 (Thread 0x7fd33fe79780 (LWP 12720)): #0 0x00007fd33d4ccb13 in epoll_wait () at ../sysdeps/unix/syscall-template.S:81 #1 0x00007fd33ebee5a6 in epoll_dispatch (base=0x6062000, tv=) at epoll.c:404 #2 0x00007fd33ebd9a04 in event_base_loop (base=0x6062000, flags=) at event.c:1558 #3 0x000000000041177d in main (argc=, argv=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/memcached/daemon/memcached.c:8931 Thread 2 (Thread 0x7fd33b3a1700 (LWP 12764)): #0 0x00007fd33d4ccb13 in epoll_wait () at ../sysdeps/unix/syscall-template.S:81 #1 0x00007fd33ebee5a6 in epoll_dispatch (base=0x6062500, tv=) at epoll.c:404 #2 0x00007fd33ebd9a04 in event_base_loop (base=0x6062500, flags=) at event.c:1558 #3 0x00007fd33f65931a in platform_thread_wrap (arg=0x17581b0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #4 0x00007fd33e3cb182 in start_thread (arg=0x7fd33b3a1700) at pthread_create.c:312 #5 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 1 (Thread 0x7fd32fa9b700 (LWP 13072)): #0 0x00007fd33d408cc9 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56 #1 0x00007fd33d40c0d8 in __GI_abort () at abort.c:89 #2 0x00007fd336085c0e in CheckpointManager::registerTAPCursorBySeqno (this=0x669ad98, name=..., startBySeqno=15533) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/checkpoint.cc:433 #3 0x00007fd336116990 in ActiveStream::markDiskSnapshot (this=0x7ccb3b80, startSeqno=0, endSeqno=15533) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/dcp-stream.cc:396 #4 0x00007fd33613cdee in CouchKVStore::loadDB (this=0x6184000, cb=, cl=, sr=..., keysOnly=, vbid=709, startSeqno=1, options=0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/couch-kvstore/couch-kvstore.cc:1212 #5 0x00007fd33613d226 in CouchKVStore::dump (this=0x31b0, vb=, stSeqno=, cb=..., cl=..., sr=...) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/couch-kvstore/couch-kvstore.cc:1047 #6 0x00007fd336113e06 in DCPBackfill::run (this=0x3934cf60) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/dcp-stream.cc:167 #7 0x00007fd3360dc5a5 in ExecutorThread::run (this=0x17ad0e0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:109 #8 0x00007fd3360dca8d in launch_executor_thread (arg=0x31b0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:34 #9 0x00007fd33f65931a in platform_thread_wrap (arg=0x1758d30) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #10 0x00007fd33e3cb182 in start_thread (arg=0x7fd32fa9b700) at pthread_create.c:312 #11 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 (gdb) (gdb) Thread 24 (Thread 0x7fd330a9d700 (LWP 13070)): #0 0x00007fd33d4c3fdd in fdatasync () at ../sysdeps/unix/syscall-template.S:81 #1 0x00007fd335e594f7 in couch_sync (errinfo=0x59d75d20, handle=0x53) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/couchstore/src/os.c:133 #2 0x00007fd3361489d6 in cfs_sync(struct {...} *, couch_file_handle) (errinfo=0x59d75d20, h=0xb1ea4f60) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/couch-kvstore/couch-fs-stats.cc:128 #3 0x00007fd335e4224c in couchstore_commit (db=0x59d75d00) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/couchstore/src/couch_db.cc:194 #4 0x00007fd33613be39 in CouchKVStore::saveDocs (this=, vbid=, rev=, docs=, docinfos=, docCount=, kvctx=..., snapStartSeqno=6163, snapEndSeqno=6163) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/couch-kvstore/couch-kvstore.cc:1779 #5 0x00007fd33613c362 in CouchKVStore::commit2couchstore (this=0x71fe800, cb=0x7fd330a9ca30, snapStartSeqno=, snapEndSeqno=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/couch-kvstore/couch-kvstore.cc:1671 #6 0x00007fd33613c5e5 in CouchKVStore::commit (this=0x53, cb=0x53, snapStartSeqno=7687230, snapEndSeqno=18446744073709551615) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/couch-kvstore/couch-kvstore.cc:1080 #7 0x00007fd33609726b in EventuallyPersistentStore::flushVBucket (this=0x60e5840, vbid=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/ep.cc:2804 #8 0x00007fd3360d2746 in Flusher::flushVB (this=0x17b6940) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/flusher.cc:281 #9 0x00007fd3360d3c4d in Flusher::step (this=0x17b6940, task=0x6083bd0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/flusher.cc:173 #10 0x00007fd3360dc5a5 in ExecutorThread::run (this=0x17acea0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:109 #11 0x00007fd3360dca8d in launch_executor_thread (arg=0x53) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:34 #12 0x00007fd33f65931a in platform_thread_wrap (arg=0x1758d10) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #13 0x00007fd33e3cb182 in start_thread (arg=0x7fd330a9d700) at pthread_create.c:312 #14 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 23 (Thread 0x7fd331a9f700 (LWP 13068)): #0 0x00007fd33d4c3fdd in fdatasync () at ../sysdeps/unix/syscall-template.S:81 #1 0x00007fd335e594f7 in couch_sync (errinfo=0xffa3cd20, handle=0x3) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/couchstore/src/os.c:133 #2 0x00007fd3361489d6 in cfs_sync(struct {...} *, couch_file_handle) (errinfo=0xffa3cd20, h=0xc319d3e0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/couch-kvstore/couch-fs-stats.cc:128 #3 0x00007fd335e4224c in couchstore_commit (db=0xffa3cd00) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/couchstore/src/couch_db.cc:194 #4 0x00007fd33613be39 in CouchKVStore::saveDocs (this=, vbid=, rev=, docs=, docinfos=, docCount=, kvctx=..., snapStartSeqno=15632, snapEndSeqno=15632) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/couch-kvstore/couch-kvstore.cc:1779 #5 0x00007fd33613c362 in CouchKVStore::commit2couchstore (this=0x60fa000, cb=0x7fd331a9ea30, snapStartSeqno=, snapEndSeqno=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/couch-kvstore/couch-kvstore.cc:1671 #6 0x00007fd33613c5e5 in CouchKVStore::commit (this=0x3, cb=0x3, snapStartSeqno=7687230, snapEndSeqno=18446744073709551615) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/couch-kvstore/couch-kvstore.cc:1080 #7 0x00007fd33609726b in EventuallyPersistentStore::flushVBucket (this=0x60e2000, vbid=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/ep.cc:2804 #8 0x00007fd3360d2746 in Flusher::flushVB (this=0x17b5540) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/flusher.cc:281 #9 0x00007fd3360d3c4d in Flusher::step (this=0x17b5540, task=0x6082730) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/flusher.cc:173 #10 0x00007fd3360dc5a5 in ExecutorThread::run (this=0x17acc60) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:109 #11 0x00007fd3360dca8d in launch_executor_thread (arg=0x3) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:34 #12 0x00007fd33f65931a in platform_thread_wrap (arg=0x1758cf0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #13 0x00007fd33e3cb182 in start_thread (arg=0x7fd331a9f700) at pthread_create.c:312 #14 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 22 (Thread 0x7fd33129e700 (LWP 13069)): #0 0x00007fd33d4c3fdd in fdatasync () at ../sysdeps/unix/syscall-template.S:81 #1 0x00007fd335e594f7 in couch_sync (errinfo=0x890428a0, handle=0x54) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/couchstore/src/os.c:133 #2 0x00007fd3361489d6 in cfs_sync(struct {...} *, couch_file_handle) (errinfo=0x890428a0, h=0x747bae20) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/couch-kvstore/couch-fs-stats.cc:128 #3 0x00007fd335e4224c in couchstore_commit (db=0x89042880) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/couchstore/src/couch_db.cc:194 #4 0x00007fd33613be39 in CouchKVStore::saveDocs (this=, vbid=, rev=, docs=, docinfos=, docCount=, kvctx=..., snapStartSeqno=15664, snapEndSeqno=15664) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/couch-kvstore/couch-kvstore.cc:1779 #5 0x00007fd33613c362 in CouchKVStore::commit2couchstore (this=0x60fc800, cb=0x7fd33129da30, snapStartSeqno=, snapEndSeqno=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/couch-kvstore/couch-kvstore.cc:1671 #6 0x00007fd33613c5e5 in CouchKVStore::commit (this=0x54, cb=0x54, snapStartSeqno=7687230, snapEndSeqno=18446744073709551615) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/couch-kvstore/couch-kvstore.cc:1080 #7 0x00007fd33609726b in EventuallyPersistentStore::flushVBucket (this=0x60e2000, vbid=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/ep.cc:2804 #8 0x00007fd3360d2746 in Flusher::flushVB (this=0x17b5680) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/flusher.cc:281 #9 0x00007fd3360d3c4d in Flusher::step (this=0x17b5680, task=0x60827d0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/flusher.cc:173 #10 0x00007fd3360dc5a5 in ExecutorThread::run (this=0x17acd80) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:109 #11 0x00007fd3360dca8d in launch_executor_thread (arg=0x54) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:34 #12 0x00007fd33f65931a in platform_thread_wrap (arg=0x1758d00) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #13 0x00007fd33e3cb182 in start_thread (arg=0x7fd33129e700) at pthread_create.c:312 #14 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 21 (Thread 0x7fd33029c700 (LWP 13071)): #0 0x00007fd33f04a007 in HashBytes (shift=, bytes=) at snappy.cc:49 #1 Hash (shift=, p=) at snappy.cc:52 #2 snappy::internal::CompressFragment ( input=0x1de94600b "{\"profile\":{\"likes\":39159,\"friends\":[\"8470C7DF7E\",\"986B4473EF\",\"76BB6A097B\",\"B8E1D4B050\",\"83C7D99911\"],\"name\":\"187EB88430074D92DE48\",\"online\":true},\"rating\":267.77,\"claim\":\"65DDDA98DFA2C795A426AEA84B4"..., input_size=, op=0xbe675afa "446696764DACC107C1\016\223\r\024\062\067\063E94\016G\032\034\070\061C51AFEe\263\f33A0\016\335\b\020\067\062C20\016\061\027\201\322\070\067\061\066\071\066\061\062\062\062\067B9547\016\204)", table=0x15cb6000, table_size=) at snappy.cc:358 #3 0x00007fd33f04aa48 in snappy::Compress (reader=0x7fd33029b4a0, writer=0x7fd33029b4c0) at snappy.cc:864 #4 0x00007fd33f04adfc in snappy::RawCompress (input=, input_length=, compressed=0x1de949b6b "764DA392773F631657AB765ACC7A05B7DDAC9743F7FDC998FB165AC944AB293D8876530E72BA0CE58A7D4430BF5D7CDB36A2C9A7464619DC50AE4C69AFC2C2E1A98280D3ADD8B29CAA74F0D75144673567D4847645A5C92D4C7AAE4E3183C606C0D2B109"..., compressed_length=0x12) at snappy.cc:1011 #5 0x00007fd335e42d8e in db_write_buf_compressed (file=0x74a53180, buf=0xe74a0eb0, pos=0x7fd33029b608, disk_size=0x7fd33029b618) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/couchstore/src/couch_file_write.cc:137 #6 0x00007fd335e437da in write_doc (writeopts=, disk_size=, bp=, doc=, db=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/couchstore/src/couch_save.cc:60 #7 add_doc_to_update_list (options=, seq=, idval=, seqval=, idterm=, seqterm=, fb=, info=, doc=, db=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/couchstore/src/couch_save.cc:299 #8 couchstore_save_documents (db=, docs=0x1f4d7d80, infos=0x1f4d8aa0, numdocs=, options=1363) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/couchstore/src/couch_save.cc:381 #9 0x00007fd33613bc0e in CouchKVStore::saveDocs (this=, vbid=, rev=, docs=, docinfos=, docCount=, kvctx=..., snapStartSeqno=15492, snapEndSeqno=15492) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/couch-kvstore/couch-kvstore.cc:1756 #10 0x00007fd33613c362 in CouchKVStore::commit2couchstore (this=0x61f6000, cb=0x7fd33029ba30, snapStartSeqno=, snapEndSeqno=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/couch-kvstore/couch-kvstore.cc:1671 #11 0x00007fd33613c5e5 in CouchKVStore::commit (this=0x23, cb=0x1de949b67, snapStartSeqno=8029248363, snapEndSeqno=18) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/couch-kvstore/couch-kvstore.cc:1080 #12 0x00007fd33609726b in EventuallyPersistentStore::flushVBucket (this=0x60e2000, vbid=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/ep.cc:2804 #13 0x00007fd3360d2746 in Flusher::flushVB (this=0x17b5900) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/flusher.cc:281 #14 0x00007fd3360d3c4d in Flusher::step (this=0x17b5900, task=0x6082780) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/flusher.cc:173 #15 0x00007fd3360dc5a5 in ExecutorThread::run (this=0x17acfc0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:109 #16 0x00007fd3360dca8d in launch_executor_thread (arg=0x23) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:34 #17 0x00007fd33f65931a in platform_thread_wrap (arg=0x1758d20) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #18 0x00007fd33e3cb182 in start_thread (arg=0x7fd33029c700) at pthread_create.c:312 #19 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 20 (Thread 0x7fd337399700 (LWP 12772)): #0 0x00007fd33d4ccb13 in epoll_wait () at ../sysdeps/unix/syscall-template.S:81 #1 0x00007fd33ebee5a6 in epoll_dispatch (base=0x6063900, tv=) at epoll.c:404 #2 0x00007fd33ebd9a04 in event_base_loop (base=0x6063900, flags=) at event.c:1558 #3 0x00007fd33f65931a in platform_thread_wrap (arg=0x1758230) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #4 0x00007fd33e3cb182 in start_thread (arg=0x7fd337399700) at pthread_create.c:312 #5 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 19 (Thread 0x7fd337b9a700 (LWP 12771)): #0 0x00007fd33d4ccb13 in epoll_wait () at ../sysdeps/unix/syscall-template.S:81 #1 0x00007fd33ebee5a6 in epoll_dispatch (base=0x6063680, tv=) at epoll.c:404 #2 0x00007fd33ebd9a04 in event_base_loop (base=0x6063680, flags=) at event.c:1558 #3 0x00007fd33f65931a in platform_thread_wrap (arg=0x1758220) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #4 0x00007fd33e3cb182 in start_thread (arg=0x7fd337b9a700) at pthread_create.c:312 #5 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 18 (Thread 0x7fd3332a2700 (LWP 13065)): #0 pthread_cond_timedwait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:238 #1 0x00007fd33f65977b in cb_cond_timedwait (cond=0x60b0460, mutex=0x60b0428, ms=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:156 #2 0x00007fd3360fefc0 in wait (nanoSecs=, this=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/syncobject.h:49 #3 TaskQueue::_doSleep (this=0x60b0420, t=...) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/taskqueue.cc:94 #4 0x00007fd33610166b in TaskQueue::_fetchNextTask (this=0x60b0420, t=..., toSleep=true) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/taskqueue.cc:117 #5 0x00007fd33610186f in TaskQueue::fetchNextTask (this=0x60b0420, thread=..., toSleep=142) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/taskqueue.cc:161 #6 0x00007fd3360c9c89 in ExecutorPool::_nextTask (this=0x6251a20, t=..., tick=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorpool.cc:217 #7 0x00007fd3360c9d0f in ExecutorPool::nextTask (this=0x6251a20, t=..., tick=142 '\216') at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorpool.cc:232 #8 0x00007fd3360dc221 in ExecutorThread::run (this=0x17ac900) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:81 #9 0x00007fd3360dca8d in launch_executor_thread (arg=0x60b0464) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:34 #10 0x00007fd33f65931a in platform_thread_wrap (arg=0x1758cd0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #11 0x00007fd33e3cb182 in start_thread (arg=0x7fd3332a2700) at pthread_create.c:312 #12 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 17 (Thread 0x7fd33839b700 (LWP 12770)): #0 0x00007fd33d4ccb13 in epoll_wait () at ../sysdeps/unix/syscall-template.S:81 #1 0x00007fd33ebee5a6 in epoll_dispatch (base=0x6063400, tv=) at epoll.c:404 #2 0x00007fd33ebd9a04 in event_base_loop (base=0x6063400, flags=) at event.c:1558 #3 0x00007fd33f65931a in platform_thread_wrap (arg=0x1758210) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #4 0x00007fd33e3cb182 in start_thread (arg=0x7fd33839b700) at pthread_create.c:312 #5 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 16 (Thread 0x7fd332aa1700 (LWP 13066)): #0 pthread_cond_timedwait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:238 #1 0x00007fd33f65977b in cb_cond_timedwait (cond=0x60b0460, mutex=0x60b0428, ms=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:156 #2 0x00007fd3360fefc0 in wait (nanoSecs=, this=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/syncobject.h:49 #3 TaskQueue::_doSleep (this=0x60b0420, t=...) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/taskqueue.cc:94 #4 0x00007fd33610166b in TaskQueue::_fetchNextTask (this=0x60b0420, t=..., toSleep=true) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/taskqueue.cc:117 #5 0x00007fd33610186f in TaskQueue::fetchNextTask (this=0x60b0420, thread=..., toSleep=136) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/taskqueue.cc:161 #6 0x00007fd3360c9c89 in ExecutorPool::_nextTask (this=0x6251a20, t=..., tick=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorpool.cc:217 #7 0x00007fd3360c9d0f in ExecutorPool::nextTask (this=0x6251a20, t=..., tick=136 '\210') at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorpool.cc:232 #8 0x00007fd3360dc221 in ExecutorThread::run (this=0x17aca20) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:81 #9 0x00007fd3360dca8d in launch_executor_thread (arg=0x60b0464) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:34 #10 0x00007fd33f65931a in platform_thread_wrap (arg=0x1758cc0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #11 0x00007fd33e3cb182 in start_thread (arg=0x7fd332aa1700) at pthread_create.c:312 #12 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 15 (Thread 0x7fd3322a0700 (LWP 13067)): #0 pthread_cond_timedwait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:238 #1 0x00007fd33f65977b in cb_cond_timedwait (cond=0x60b0460, mutex=0x60b0428, ms=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:156 #2 0x00007fd3360fefc0 in wait (nanoSecs=, this=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/syncobject.h:49 #3 TaskQueue::_doSleep (this=0x60b0420, t=...) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/taskqueue.cc:94 #4 0x00007fd33610166b in TaskQueue::_fetchNextTask (this=0x60b0420, t=..., toSleep=true) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/taskqueue.cc:117 #5 0x00007fd33610186f in TaskQueue::fetchNextTask (this=0x60b0420, thread=..., toSleep=138) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/taskqueue.cc:161 #6 0x00007fd3360c9c89 in ExecutorPool::_nextTask (this=0x6251a20, t=..., tick=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorpool.cc:217 #7 0x00007fd3360c9d0f in ExecutorPool::nextTask (this=0x6251a20, t=..., tick=138 '\212') at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorpool.cc:232 #8 0x00007fd3360dc221 in ExecutorThread::run (this=0x17acb40) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:81 #9 0x00007fd3360dca8d in launch_executor_thread (arg=0x60b0464) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:34 #10 0x00007fd33f65931a in platform_thread_wrap (arg=0x1758ce0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #11 0x00007fd33e3cb182 in start_thread (arg=0x7fd3322a0700) at pthread_create.c:312 #12 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 14 (Thread 0x7fd33c5b0700 (LWP 12762)): #0 pthread_cond_timedwait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:238 #1 0x00007fd33f65977b in cb_cond_timedwait (cond=0x7fd33c7b4200 , mutex=0x7fd33c7b41c0 , ms=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:156 #2 0x00007fd33c5b33fa in logger_thead_main (arg=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/memcached/extensions/loggers/file_logger.c:405 #3 0x00007fd33f65931a in platform_thread_wrap (arg=0x1758070) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #4 0x00007fd33e3cb182 in start_thread (arg=0x7fd33c5b0700) at pthread_create.c:312 #5 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 13 (Thread 0x7fd33a39f700 (LWP 12766)): #0 0x00007fd33d4ccb13 in epoll_wait () at ../sysdeps/unix/syscall-template.S:81 #1 0x00007fd33ebee5a6 in epoll_dispatch (base=0x6062a00, tv=) at epoll.c:404 #2 0x00007fd33ebd9a04 in event_base_loop (base=0x6062a00, flags=) at event.c:1558 #3 0x00007fd33f65931a in platform_thread_wrap (arg=0x17581d0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #4 0x00007fd33e3cb182 in start_thread (arg=0x7fd33a39f700) at pthread_create.c:312 #5 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 12 (Thread 0x7fd32f29a700 (LWP 13073)): #0 pthread_cond_timedwait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:238 #1 0x00007fd33f65977b in cb_cond_timedwait (cond=0x60b0720, mutex=0x60b06e8, ms=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:156 #2 0x00007fd3360fefc0 in wait (nanoSecs=, this=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/syncobject.h:49 #3 TaskQueue::_doSleep (this=0x60b06e0, t=...) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/taskqueue.cc:94 #4 0x00007fd33610166b in TaskQueue::_fetchNextTask (this=0x60b06e0, t=..., toSleep=true) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/taskqueue.cc:117 #5 0x00007fd33610186f in TaskQueue::fetchNextTask (this=0x60b06e0, thread=..., toSleep=81) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/taskqueue.cc:161 #6 0x00007fd3360c9c89 in ExecutorPool::_nextTask (this=0x6251a20, t=..., tick=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorpool.cc:217 #7 0x00007fd3360c9d0f in ExecutorPool::nextTask (this=0x6251a20, t=..., tick=81 'Q') at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorpool.cc:232 #8 0x00007fd3360dc221 in ExecutorThread::run (this=0x17ad200) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:81 #9 0x00007fd3360dca8d in launch_executor_thread (arg=0x60b0724) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:34 #10 0x00007fd33f65931a in platform_thread_wrap (arg=0x1758d40) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #11 0x00007fd33e3cb182 in start_thread (arg=0x7fd32f29a700) at pthread_create.c:312 #12 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 11 (Thread 0x7fd33bba2700 (LWP 12763)): #0 0x00007fd33d4ccb13 in epoll_wait () at ../sysdeps/unix/syscall-template.S:81 #1 0x00007fd33ebee5a6 in epoll_dispatch (base=0x6062280, tv=) at epoll.c:404 #2 0x00007fd33ebd9a04 in event_base_loop (base=0x6062280, flags=) at event.c:1558 #3 0x00007fd33f65931a in platform_thread_wrap (arg=0x17581a0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #4 0x00007fd33e3cb182 in start_thread (arg=0x7fd33bba2700) at pthread_create.c:312 #5 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 10 (Thread 0x7fd339b9e700 (LWP 12767)): #0 0x00007fd33d4ccb13 in epoll_wait () at ../sysdeps/unix/syscall-template.S:81 #1 0x00007fd33ebee5a6 in epoll_dispatch (base=0x6062c80, tv=) at epoll.c:404 #2 0x00007fd33ebd9a04 in event_base_loop (base=0x6062c80, flags=) at event.c:1558 #3 0x00007fd33f65931a in platform_thread_wrap (arg=0x17581e0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #4 0x00007fd33e3cb182 in start_thread (arg=0x7fd339b9e700) at pthread_create.c:312 #5 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 9 (Thread 0x7fd336b98700 (LWP 13063)): #0 0x00007fd33d492f3d in nanosleep () at ../sysdeps/unix/syscall-template.S:81 #1 0x00007fd33d4c44a4 in usleep (useconds=) at ../sysdeps/unix/sysv/linux/usleep.c:32 #2 0x00007fd3360da965 in updateStatsThread (arg=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/memory_tracker.cc:36 #3 0x00007fd33f65931a in platform_thread_wrap (arg=0x17582d0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #4 0x00007fd33e3cb182 in start_thread (arg=0x7fd336b98700) at pthread_create.c:312 #5 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 8 (Thread 0x7fd333aa3700 (LWP 13064)): #0 pthread_cond_timedwait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:238 #1 0x00007fd33f65977b in cb_cond_timedwait (cond=0x60b0460, mutex=0x60b0428, ms=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:156 #2 0x00007fd3360fefc0 in wait (nanoSecs=, this=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/syncobject.h:49 #3 TaskQueue::_doSleep (this=0x60b0420, t=...) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/taskqueue.cc:94 #4 0x00007fd33610166b in TaskQueue::_fetchNextTask (this=0x60b0420, t=..., toSleep=true) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/taskqueue.cc:117 #5 0x00007fd33610186f in TaskQueue::fetchNextTask (this=0x60b0420, thread=..., toSleep=140) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/taskqueue.cc:161 #6 0x00007fd3360c9c89 in ExecutorPool::_nextTask (this=0x6251a20, t=..., tick=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorpool.cc:217 #7 0x00007fd3360c9d0f in ExecutorPool::nextTask (this=0x6251a20, t=..., tick=140 '\214') at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorpool.cc:232 #8 0x00007fd3360dc221 in ExecutorThread::run (this=0x17ac7e0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:81 #9 0x00007fd3360dca8d in launch_executor_thread (arg=0x60b0464) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:34 #10 0x00007fd33f65931a in platform_thread_wrap (arg=0x1758cb0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #11 0x00007fd33e3cb182 in start_thread (arg=0x7fd333aa3700) at pthread_create.c:312 #12 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 7 (Thread 0x7fd33cfb5700 (LWP 12761)): #0 0x00007fd33d4bf12d in poll () at ../sysdeps/unix/syscall-template.S:81 #1 0x00007fd33cfb6c45 in get_command (buffer=, buffsize=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/memcached/extensions/daemon/stdin_check.c:51 #2 check_stdin_thread (arg=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/memcached/extensions/daemon/stdin_check.c:93 #3 0x00007fd33f65931a in platform_thread_wrap (arg=0x1758060) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #4 0x00007fd33e3cb182 in start_thread (arg=0x7fd33cfb5700) at pthread_create.c:312 #5 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 6 (Thread 0x7fd338b9c700 (LWP 12769)): #0 0x00007fd33d4ccb13 in epoll_wait () at ../sysdeps/unix/syscall-template.S:81 #1 0x00007fd33ebee5a6 in epoll_dispatch (base=0x6063180, tv=) at epoll.c:404 #2 0x00007fd33ebd9a04 in event_base_loop (base=0x6063180, flags=) at event.c:1558 #3 0x00007fd33f65931a in platform_thread_wrap (arg=0x1758200) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #4 0x00007fd33e3cb182 in start_thread (arg=0x7fd338b9c700) at pthread_create.c:312 #5 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 5 (Thread 0x7fd33aba0700 (LWP 12765)): #0 0x00007fd33d4ccb13 in epoll_wait () at ../sysdeps/unix/syscall-template.S:81 #1 0x00007fd33ebee5a6 in epoll_dispatch (base=0x6062780, tv=) at epoll.c:404 #2 0x00007fd33ebd9a04 in event_base_loop (base=0x6062780, flags=) at event.c:1558 #3 0x00007fd33f65931a in platform_thread_wrap (arg=0x17581c0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #4 0x00007fd33e3cb182 in start_thread (arg=0x7fd33aba0700) at pthread_create.c:312 #5 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 4 (Thread 0x7fd33939d700 (LWP 12768)): #0 0x00007fd33d4ccb13 in epoll_wait () at ../sysdeps/unix/syscall-template.S:81 #1 0x00007fd33ebee5a6 in epoll_dispatch (base=0x6062f00, tv=) at epoll.c:404 #2 0x00007fd33ebd9a04 in event_base_loop (base=0x6062f00, flags=) at event.c:1558 #3 0x00007fd33f65931a in platform_thread_wrap (arg=0x17581f0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #4 0x00007fd33e3cb182 in start_thread (arg=0x7fd33939d700) at pthread_create.c:312 #5 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 3 (Thread 0x7fd33fe79780 (LWP 12720)): #0 0x00007fd33d4ccb13 in epoll_wait () at ../sysdeps/unix/syscall-template.S:81 #1 0x00007fd33ebee5a6 in epoll_dispatch (base=0x6062000, tv=) at epoll.c:404 #2 0x00007fd33ebd9a04 in event_base_loop (base=0x6062000, flags=) at event.c:1558 #3 0x000000000041177d in main (argc=, argv=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/memcached/daemon/memcached.c:8931 Thread 2 (Thread 0x7fd33b3a1700 (LWP 12764)): #0 0x00007fd33d4ccb13 in epoll_wait () at ../sysdeps/unix/syscall-template.S:81 #1 0x00007fd33ebee5a6 in epoll_dispatch (base=0x6062500, tv=) at epoll.c:404 #2 0x00007fd33ebd9a04 in event_base_loop (base=0x6062500, flags=) at event.c:1558 #3 0x00007fd33f65931a in platform_thread_wrap (arg=0x17581b0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #4 0x00007fd33e3cb182 in start_thread (arg=0x7fd33b3a1700) at pthread_create.c:312 #5 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 1 (Thread 0x7fd32fa9b700 (LWP 13072)): #0 0x00007fd33d408cc9 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56 #1 0x00007fd33d40c0d8 in __GI_abort () at abort.c:89 #2 0x00007fd336085c0e in CheckpointManager::registerTAPCursorBySeqno (this=0x669ad98, name=..., startBySeqno=15533) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/checkpoint.cc:433 #3 0x00007fd336116990 in ActiveStream::markDiskSnapshot (this=0x7ccb3b80, startSeqno=0, endSeqno=15533) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/dcp-stream.cc:396 #4 0x00007fd33613cdee in CouchKVStore::loadDB (this=0x6184000, cb=, cl=, sr=..., keysOnly=, vbid=709, startSeqno=1, options=0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/couch-kvstore/couch-kvstore.cc:1212 #5 0x00007fd33613d226 in CouchKVStore::dump (this=0x31b0, vb=, stSeqno=, cb=..., cl=..., sr=...) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/couch-kvstore/couch-kvstore.cc:1047 #6 0x00007fd336113e06 in DCPBackfill::run (this=0x3934cf60) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/dcp-stream.cc:167 #7 0x00007fd3360dc5a5 in ExecutorThread::run (this=0x17ad0e0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:109 #8 0x00007fd3360dca8d in launch_executor_thread (arg=0x31b0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:34 #9 0x00007fd33f65931a in platform_thread_wrap (arg=0x1758d30) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #10 0x00007fd33e3cb182 in start_thread (arg=0x7fd32fa9b700) at pthread_create.c:312 #11 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 (gdb) Thread 24 (Thread 0x7fd330a9d700 (LWP 13070)): #0 0x00007fd33d4c3fdd in fdatasync () at ../sysdeps/unix/syscall-template.S:81 #1 0x00007fd335e594f7 in couch_sync (errinfo=0x59d75d20, handle=0x53) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/couchstore/src/os.c:133 #2 0x00007fd3361489d6 in cfs_sync(struct {...} *, couch_file_handle) (errinfo=0x59d75d20, h=0xb1ea4f60) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/couch-kvstore/couch-fs-stats.cc:128 #3 0x00007fd335e4224c in couchstore_commit (db=0x59d75d00) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/couchstore/src/couch_db.cc:194 #4 0x00007fd33613be39 in CouchKVStore::saveDocs (this=, vbid=, rev=, docs=, docinfos=, docCount=, kvctx=..., snapStartSeqno=6163, snapEndSeqno=6163) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/couch-kvstore/couch-kvstore.cc:1779 #5 0x00007fd33613c362 in CouchKVStore::commit2couchstore (this=0x71fe800, cb=0x7fd330a9ca30, snapStartSeqno=, snapEndSeqno=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/couch-kvstore/couch-kvstore.cc:1671 #6 0x00007fd33613c5e5 in CouchKVStore::commit (this=0x53, cb=0x53, snapStartSeqno=7687230, snapEndSeqno=18446744073709551615) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/couch-kvstore/couch-kvstore.cc:1080 #7 0x00007fd33609726b in EventuallyPersistentStore::flushVBucket (this=0x60e5840, vbid=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/ep.cc:2804 #8 0x00007fd3360d2746 in Flusher::flushVB (this=0x17b6940) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/flusher.cc:281 #9 0x00007fd3360d3c4d in Flusher::step (this=0x17b6940, task=0x6083bd0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/flusher.cc:173 #10 0x00007fd3360dc5a5 in ExecutorThread::run (this=0x17acea0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:109 #11 0x00007fd3360dca8d in launch_executor_thread (arg=0x53) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:34 #12 0x00007fd33f65931a in platform_thread_wrap (arg=0x1758d10) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #13 0x00007fd33e3cb182 in start_thread (arg=0x7fd330a9d700) at pthread_create.c:312 #14 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 23 (Thread 0x7fd331a9f700 (LWP 13068)): #0 0x00007fd33d4c3fdd in fdatasync () at ../sysdeps/unix/syscall-template.S:81 #1 0x00007fd335e594f7 in couch_sync (errinfo=0xffa3cd20, handle=0x3) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/couchstore/src/os.c:133 #2 0x00007fd3361489d6 in cfs_sync(struct {...} *, couch_file_handle) (errinfo=0xffa3cd20, h=0xc319d3e0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/couch-kvstore/couch-fs-stats.cc:128 #3 0x00007fd335e4224c in couchstore_commit (db=0xffa3cd00) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/couchstore/src/couch_db.cc:194 #4 0x00007fd33613be39 in CouchKVStore::saveDocs (this=, vbid=, rev=, docs=, docinfos=, docCount=, kvctx=..., snapStartSeqno=15632, snapEndSeqno=15632) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/couch-kvstore/couch-kvstore.cc:1779 #5 0x00007fd33613c362 in CouchKVStore::commit2couchstore (this=0x60fa000, cb=0x7fd331a9ea30, snapStartSeqno=, snapEndSeqno=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/couch-kvstore/couch-kvstore.cc:1671 #6 0x00007fd33613c5e5 in CouchKVStore::commit (this=0x3, cb=0x3, snapStartSeqno=7687230, snapEndSeqno=18446744073709551615) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/couch-kvstore/couch-kvstore.cc:1080 #7 0x00007fd33609726b in EventuallyPersistentStore::flushVBucket (this=0x60e2000, vbid=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/ep.cc:2804 #8 0x00007fd3360d2746 in Flusher::flushVB (this=0x17b5540) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/flusher.cc:281 #9 0x00007fd3360d3c4d in Flusher::step (this=0x17b5540, task=0x6082730) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/flusher.cc:173 #10 0x00007fd3360dc5a5 in ExecutorThread::run (this=0x17acc60) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:109 #11 0x00007fd3360dca8d in launch_executor_thread (arg=0x3) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:34 #12 0x00007fd33f65931a in platform_thread_wrap (arg=0x1758cf0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #13 0x00007fd33e3cb182 in start_thread (arg=0x7fd331a9f700) at pthread_create.c:312 #14 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 22 (Thread 0x7fd33129e700 (LWP 13069)): #0 0x00007fd33d4c3fdd in fdatasync () at ../sysdeps/unix/syscall-template.S:81 #1 0x00007fd335e594f7 in couch_sync (errinfo=0x890428a0, handle=0x54) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/couchstore/src/os.c:133 #2 0x00007fd3361489d6 in cfs_sync(struct {...} *, couch_file_handle) (errinfo=0x890428a0, h=0x747bae20) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/couch-kvstore/couch-fs-stats.cc:128 #3 0x00007fd335e4224c in couchstore_commit (db=0x89042880) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/couchstore/src/couch_db.cc:194 #4 0x00007fd33613be39 in CouchKVStore::saveDocs (this=, vbid=, rev=, docs=, docinfos=, docCount=, kvctx=..., snapStartSeqno=15664, snapEndSeqno=15664) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/couch-kvstore/couch-kvstore.cc:1779 #5 0x00007fd33613c362 in CouchKVStore::commit2couchstore (this=0x60fc800, cb=0x7fd33129da30, snapStartSeqno=, snapEndSeqno=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/couch-kvstore/couch-kvstore.cc:1671 #6 0x00007fd33613c5e5 in CouchKVStore::commit (this=0x54, cb=0x54, snapStartSeqno=7687230, snapEndSeqno=18446744073709551615) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/couch-kvstore/couch-kvstore.cc:1080 #7 0x00007fd33609726b in EventuallyPersistentStore::flushVBucket (this=0x60e2000, vbid=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/ep.cc:2804 #8 0x00007fd3360d2746 in Flusher::flushVB (this=0x17b5680) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/flusher.cc:281 #9 0x00007fd3360d3c4d in Flusher::step (this=0x17b5680, task=0x60827d0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/flusher.cc:173 #10 0x00007fd3360dc5a5 in ExecutorThread::run (this=0x17acd80) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:109 #11 0x00007fd3360dca8d in launch_executor_thread (arg=0x54) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:34 #12 0x00007fd33f65931a in platform_thread_wrap (arg=0x1758d00) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #13 0x00007fd33e3cb182 in start_thread (arg=0x7fd33129e700) at pthread_create.c:312 #14 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 21 (Thread 0x7fd33029c700 (LWP 13071)): #0 0x00007fd33f04a007 in HashBytes (shift=, bytes=) at snappy.cc:49 #1 Hash (shift=, p=) at snappy.cc:52 #2 snappy::internal::CompressFragment ( input=0x1de94600b "{\"profile\":{\"likes\":39159,\"friends\":[\"8470C7DF7E\",\"986B4473EF\",\"76BB6A097B\",\"B8E1D4B050\",\"83C7D99911\"],\"name\":\"187EB88430074D92DE48\",\"online\":true},\"rating\":267.77,\"claim\":\"65DDDA98DFA2C795A426AEA84B4"..., input_size=, op=0xbe675afa "446696764DACC107C1\016\223\r\024\062\067\063E94\016G\032\034\070\061C51AFEe\263\f33A0\016\335\b\020\067\062C20\016\061\027\201\322\070\067\061\066\071\066\061\062\062\062\067B9547\016\204)", table=0x15cb6000, table_size=) at snappy.cc:358 #3 0x00007fd33f04aa48 in snappy::Compress (reader=0x7fd33029b4a0, writer=0x7fd33029b4c0) at snappy.cc:864 #4 0x00007fd33f04adfc in snappy::RawCompress (input=, input_length=, compressed=0x1de949b6b "764DA392773F631657AB765ACC7A05B7DDAC9743F7FDC998FB165AC944AB293D8876530E72BA0CE58A7D4430BF5D7CDB36A2C9A7464619DC50AE4C69AFC2C2E1A98280D3ADD8B29CAA74F0D75144673567D4847645A5C92D4C7AAE4E3183C606C0D2B109"..., compressed_length=0x12) at snappy.cc:1011 #5 0x00007fd335e42d8e in db_write_buf_compressed (file=0x74a53180, buf=0xe74a0eb0, pos=0x7fd33029b608, disk_size=0x7fd33029b618) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/couchstore/src/couch_file_write.cc:137 #6 0x00007fd335e437da in write_doc (writeopts=, disk_size=, bp=, doc=, db=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/couchstore/src/couch_save.cc:60 #7 add_doc_to_update_list (options=, seq=, idval=, seqval=, idterm=, seqterm=, fb=, info=, doc=, db=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/couchstore/src/couch_save.cc:299 #8 couchstore_save_documents (db=, docs=0x1f4d7d80, infos=0x1f4d8aa0, numdocs=, options=1363) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/couchstore/src/couch_save.cc:381 #9 0x00007fd33613bc0e in CouchKVStore::saveDocs (this=, vbid=, rev=, docs=, docinfos=, docCount=, kvctx=..., snapStartSeqno=15492, snapEndSeqno=15492) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/couch-kvstore/couch-kvstore.cc:1756 #10 0x00007fd33613c362 in CouchKVStore::commit2couchstore (this=0x61f6000, cb=0x7fd33029ba30, snapStartSeqno=, snapEndSeqno=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/couch-kvstore/couch-kvstore.cc:1671 #11 0x00007fd33613c5e5 in CouchKVStore::commit (this=0x23, cb=0x1de949b67, snapStartSeqno=8029248363, snapEndSeqno=18) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/couch-kvstore/couch-kvstore.cc:1080 #12 0x00007fd33609726b in EventuallyPersistentStore::flushVBucket (this=0x60e2000, vbid=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/ep.cc:2804 #13 0x00007fd3360d2746 in Flusher::flushVB (this=0x17b5900) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/flusher.cc:281 #14 0x00007fd3360d3c4d in Flusher::step (this=0x17b5900, task=0x6082780) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/flusher.cc:173 #15 0x00007fd3360dc5a5 in ExecutorThread::run (this=0x17acfc0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:109 #16 0x00007fd3360dca8d in launch_executor_thread (arg=0x23) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:34 #17 0x00007fd33f65931a in platform_thread_wrap (arg=0x1758d20) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #18 0x00007fd33e3cb182 in start_thread (arg=0x7fd33029c700) at pthread_create.c:312 #19 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 20 (Thread 0x7fd337399700 (LWP 12772)): #0 0x00007fd33d4ccb13 in epoll_wait () at ../sysdeps/unix/syscall-template.S:81 #1 0x00007fd33ebee5a6 in epoll_dispatch (base=0x6063900, tv=) at epoll.c:404 #2 0x00007fd33ebd9a04 in event_base_loop (base=0x6063900, flags=) at event.c:1558 #3 0x00007fd33f65931a in platform_thread_wrap (arg=0x1758230) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #4 0x00007fd33e3cb182 in start_thread (arg=0x7fd337399700) at pthread_create.c:312 #5 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 19 (Thread 0x7fd337b9a700 (LWP 12771)): #0 0x00007fd33d4ccb13 in epoll_wait () at ../sysdeps/unix/syscall-template.S:81 #1 0x00007fd33ebee5a6 in epoll_dispatch (base=0x6063680, tv=) at epoll.c:404 #2 0x00007fd33ebd9a04 in event_base_loop (base=0x6063680, flags=) at event.c:1558 #3 0x00007fd33f65931a in platform_thread_wrap (arg=0x1758220) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #4 0x00007fd33e3cb182 in start_thread (arg=0x7fd337b9a700) at pthread_create.c:312 #5 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 18 (Thread 0x7fd3332a2700 (LWP 13065)): #0 pthread_cond_timedwait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:238 #1 0x00007fd33f65977b in cb_cond_timedwait (cond=0x60b0460, mutex=0x60b0428, ms=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:156 #2 0x00007fd3360fefc0 in wait (nanoSecs=, this=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/syncobject.h:49 #3 TaskQueue::_doSleep (this=0x60b0420, t=...) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/taskqueue.cc:94 #4 0x00007fd33610166b in TaskQueue::_fetchNextTask (this=0x60b0420, t=..., toSleep=true) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/taskqueue.cc:117 #5 0x00007fd33610186f in TaskQueue::fetchNextTask (this=0x60b0420, thread=..., toSleep=142) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/taskqueue.cc:161 #6 0x00007fd3360c9c89 in ExecutorPool::_nextTask (this=0x6251a20, t=..., tick=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorpool.cc:217 #7 0x00007fd3360c9d0f in ExecutorPool::nextTask (this=0x6251a20, t=..., tick=142 '\216') at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorpool.cc:232 #8 0x00007fd3360dc221 in ExecutorThread::run (this=0x17ac900) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:81 #9 0x00007fd3360dca8d in launch_executor_thread (arg=0x60b0464) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:34 #10 0x00007fd33f65931a in platform_thread_wrap (arg=0x1758cd0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #11 0x00007fd33e3cb182 in start_thread (arg=0x7fd3332a2700) at pthread_create.c:312 #12 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 17 (Thread 0x7fd33839b700 (LWP 12770)): #0 0x00007fd33d4ccb13 in epoll_wait () at ../sysdeps/unix/syscall-template.S:81 #1 0x00007fd33ebee5a6 in epoll_dispatch (base=0x6063400, tv=) at epoll.c:404 #2 0x00007fd33ebd9a04 in event_base_loop (base=0x6063400, flags=) at event.c:1558 #3 0x00007fd33f65931a in platform_thread_wrap (arg=0x1758210) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #4 0x00007fd33e3cb182 in start_thread (arg=0x7fd33839b700) at pthread_create.c:312 #5 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 16 (Thread 0x7fd332aa1700 (LWP 13066)): #0 pthread_cond_timedwait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:238 #1 0x00007fd33f65977b in cb_cond_timedwait (cond=0x60b0460, mutex=0x60b0428, ms=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:156 #2 0x00007fd3360fefc0 in wait (nanoSecs=, this=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/syncobject.h:49 #3 TaskQueue::_doSleep (this=0x60b0420, t=...) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/taskqueue.cc:94 #4 0x00007fd33610166b in TaskQueue::_fetchNextTask (this=0x60b0420, t=..., toSleep=true) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/taskqueue.cc:117 #5 0x00007fd33610186f in TaskQueue::fetchNextTask (this=0x60b0420, thread=..., toSleep=136) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/taskqueue.cc:161 #6 0x00007fd3360c9c89 in ExecutorPool::_nextTask (this=0x6251a20, t=..., tick=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorpool.cc:217 #7 0x00007fd3360c9d0f in ExecutorPool::nextTask (this=0x6251a20, t=..., tick=136 '\210') at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorpool.cc:232 #8 0x00007fd3360dc221 in ExecutorThread::run (this=0x17aca20) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:81 #9 0x00007fd3360dca8d in launch_executor_thread (arg=0x60b0464) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:34 #10 0x00007fd33f65931a in platform_thread_wrap (arg=0x1758cc0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #11 0x00007fd33e3cb182 in start_thread (arg=0x7fd332aa1700) at pthread_create.c:312 #12 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 15 (Thread 0x7fd3322a0700 (LWP 13067)): #0 pthread_cond_timedwait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:238 #1 0x00007fd33f65977b in cb_cond_timedwait (cond=0x60b0460, mutex=0x60b0428, ms=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:156 #2 0x00007fd3360fefc0 in wait (nanoSecs=, this=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/syncobject.h:49 #3 TaskQueue::_doSleep (this=0x60b0420, t=...) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/taskqueue.cc:94 #4 0x00007fd33610166b in TaskQueue::_fetchNextTask (this=0x60b0420, t=..., toSleep=true) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/taskqueue.cc:117 #5 0x00007fd33610186f in TaskQueue::fetchNextTask (this=0x60b0420, thread=..., toSleep=138) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/taskqueue.cc:161 #6 0x00007fd3360c9c89 in ExecutorPool::_nextTask (this=0x6251a20, t=..., tick=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorpool.cc:217 #7 0x00007fd3360c9d0f in ExecutorPool::nextTask (this=0x6251a20, t=..., tick=138 '\212') at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorpool.cc:232 #8 0x00007fd3360dc221 in ExecutorThread::run (this=0x17acb40) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:81 #9 0x00007fd3360dca8d in launch_executor_thread (arg=0x60b0464) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:34 #10 0x00007fd33f65931a in platform_thread_wrap (arg=0x1758ce0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #11 0x00007fd33e3cb182 in start_thread (arg=0x7fd3322a0700) at pthread_create.c:312 #12 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 14 (Thread 0x7fd33c5b0700 (LWP 12762)): #0 pthread_cond_timedwait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:238 #1 0x00007fd33f65977b in cb_cond_timedwait (cond=0x7fd33c7b4200 , mutex=0x7fd33c7b41c0 , ms=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:156 #2 0x00007fd33c5b33fa in logger_thead_main (arg=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/memcached/extensions/loggers/file_logger.c:405 #3 0x00007fd33f65931a in platform_thread_wrap (arg=0x1758070) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #4 0x00007fd33e3cb182 in start_thread (arg=0x7fd33c5b0700) at pthread_create.c:312 #5 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 13 (Thread 0x7fd33a39f700 (LWP 12766)): #0 0x00007fd33d4ccb13 in epoll_wait () at ../sysdeps/unix/syscall-template.S:81 #1 0x00007fd33ebee5a6 in epoll_dispatch (base=0x6062a00, tv=) at epoll.c:404 #2 0x00007fd33ebd9a04 in event_base_loop (base=0x6062a00, flags=) at event.c:1558 #3 0x00007fd33f65931a in platform_thread_wrap (arg=0x17581d0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #4 0x00007fd33e3cb182 in start_thread (arg=0x7fd33a39f700) at pthread_create.c:312 #5 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 12 (Thread 0x7fd32f29a700 (LWP 13073)): #0 pthread_cond_timedwait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:238 #1 0x00007fd33f65977b in cb_cond_timedwait (cond=0x60b0720, mutex=0x60b06e8, ms=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:156 #2 0x00007fd3360fefc0 in wait (nanoSecs=, this=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/syncobject.h:49 #3 TaskQueue::_doSleep (this=0x60b06e0, t=...) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/taskqueue.cc:94 #4 0x00007fd33610166b in TaskQueue::_fetchNextTask (this=0x60b06e0, t=..., toSleep=true) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/taskqueue.cc:117 #5 0x00007fd33610186f in TaskQueue::fetchNextTask (this=0x60b06e0, thread=..., toSleep=81) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/taskqueue.cc:161 #6 0x00007fd3360c9c89 in ExecutorPool::_nextTask (this=0x6251a20, t=..., tick=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorpool.cc:217 #7 0x00007fd3360c9d0f in ExecutorPool::nextTask (this=0x6251a20, t=..., tick=81 'Q') at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorpool.cc:232 #8 0x00007fd3360dc221 in ExecutorThread::run (this=0x17ad200) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:81 #9 0x00007fd3360dca8d in launch_executor_thread (arg=0x60b0724) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:34 #10 0x00007fd33f65931a in platform_thread_wrap (arg=0x1758d40) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #11 0x00007fd33e3cb182 in start_thread (arg=0x7fd32f29a700) at pthread_create.c:312 #12 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 11 (Thread 0x7fd33bba2700 (LWP 12763)): #0 0x00007fd33d4ccb13 in epoll_wait () at ../sysdeps/unix/syscall-template.S:81 #1 0x00007fd33ebee5a6 in epoll_dispatch (base=0x6062280, tv=) at epoll.c:404 #2 0x00007fd33ebd9a04 in event_base_loop (base=0x6062280, flags=) at event.c:1558 #3 0x00007fd33f65931a in platform_thread_wrap (arg=0x17581a0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #4 0x00007fd33e3cb182 in start_thread (arg=0x7fd33bba2700) at pthread_create.c:312 #5 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 10 (Thread 0x7fd339b9e700 (LWP 12767)): #0 0x00007fd33d4ccb13 in epoll_wait () at ../sysdeps/unix/syscall-template.S:81 #1 0x00007fd33ebee5a6 in epoll_dispatch (base=0x6062c80, tv=) at epoll.c:404 #2 0x00007fd33ebd9a04 in event_base_loop (base=0x6062c80, flags=) at event.c:1558 #3 0x00007fd33f65931a in platform_thread_wrap (arg=0x17581e0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #4 0x00007fd33e3cb182 in start_thread (arg=0x7fd339b9e700) at pthread_create.c:312 #5 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 9 (Thread 0x7fd336b98700 (LWP 13063)): #0 0x00007fd33d492f3d in nanosleep () at ../sysdeps/unix/syscall-template.S:81 #1 0x00007fd33d4c44a4 in usleep (useconds=) at ../sysdeps/unix/sysv/linux/usleep.c:32 #2 0x00007fd3360da965 in updateStatsThread (arg=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/memory_tracker.cc:36 #3 0x00007fd33f65931a in platform_thread_wrap (arg=0x17582d0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #4 0x00007fd33e3cb182 in start_thread (arg=0x7fd336b98700) at pthread_create.c:312 #5 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 8 (Thread 0x7fd333aa3700 (LWP 13064)): #0 pthread_cond_timedwait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:238 #1 0x00007fd33f65977b in cb_cond_timedwait (cond=0x60b0460, mutex=0x60b0428, ms=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:156 #2 0x00007fd3360fefc0 in wait (nanoSecs=, this=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/syncobject.h:49 #3 TaskQueue::_doSleep (this=0x60b0420, t=...) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/taskqueue.cc:94 #4 0x00007fd33610166b in TaskQueue::_fetchNextTask (this=0x60b0420, t=..., toSleep=true) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/taskqueue.cc:117 #5 0x00007fd33610186f in TaskQueue::fetchNextTask (this=0x60b0420, thread=..., toSleep=140) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/taskqueue.cc:161 #6 0x00007fd3360c9c89 in ExecutorPool::_nextTask (this=0x6251a20, t=..., tick=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorpool.cc:217 #7 0x00007fd3360c9d0f in ExecutorPool::nextTask (this=0x6251a20, t=..., tick=140 '\214') at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorpool.cc:232 #8 0x00007fd3360dc221 in ExecutorThread::run (this=0x17ac7e0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:81 #9 0x00007fd3360dca8d in launch_executor_thread (arg=0x60b0464) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:34 #10 0x00007fd33f65931a in platform_thread_wrap (arg=0x1758cb0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #11 0x00007fd33e3cb182 in start_thread (arg=0x7fd333aa3700) at pthread_create.c:312 #12 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 7 (Thread 0x7fd33cfb5700 (LWP 12761)): #0 0x00007fd33d4bf12d in poll () at ../sysdeps/unix/syscall-template.S:81 #1 0x00007fd33cfb6c45 in get_command (buffer=, buffsize=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/memcached/extensions/daemon/stdin_check.c:51 #2 check_stdin_thread (arg=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/memcached/extensions/daemon/stdin_check.c:93 #3 0x00007fd33f65931a in platform_thread_wrap (arg=0x1758060) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #4 0x00007fd33e3cb182 in start_thread (arg=0x7fd33cfb5700) at pthread_create.c:312 #5 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 6 (Thread 0x7fd338b9c700 (LWP 12769)): #0 0x00007fd33d4ccb13 in epoll_wait () at ../sysdeps/unix/syscall-template.S:81 #1 0x00007fd33ebee5a6 in epoll_dispatch (base=0x6063180, tv=) at epoll.c:404 #2 0x00007fd33ebd9a04 in event_base_loop (base=0x6063180, flags=) at event.c:1558 #3 0x00007fd33f65931a in platform_thread_wrap (arg=0x1758200) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #4 0x00007fd33e3cb182 in start_thread (arg=0x7fd338b9c700) at pthread_create.c:312 #5 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 5 (Thread 0x7fd33aba0700 (LWP 12765)): #0 0x00007fd33d4ccb13 in epoll_wait () at ../sysdeps/unix/syscall-template.S:81 #1 0x00007fd33ebee5a6 in epoll_dispatch (base=0x6062780, tv=) at epoll.c:404 #2 0x00007fd33ebd9a04 in event_base_loop (base=0x6062780, flags=) at event.c:1558 #3 0x00007fd33f65931a in platform_thread_wrap (arg=0x17581c0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #4 0x00007fd33e3cb182 in start_thread (arg=0x7fd33aba0700) at pthread_create.c:312 #5 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 4 (Thread 0x7fd33939d700 (LWP 12768)): #0 0x00007fd33d4ccb13 in epoll_wait () at ../sysdeps/unix/syscall-template.S:81 #1 0x00007fd33ebee5a6 in epoll_dispatch (base=0x6062f00, tv=) at epoll.c:404 #2 0x00007fd33ebd9a04 in event_base_loop (base=0x6062f00, flags=) at event.c:1558 #3 0x00007fd33f65931a in platform_thread_wrap (arg=0x17581f0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #4 0x00007fd33e3cb182 in start_thread (arg=0x7fd33939d700) at pthread_create.c:312 #5 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 3 (Thread 0x7fd33fe79780 (LWP 12720)): #0 0x00007fd33d4ccb13 in epoll_wait () at ../sysdeps/unix/syscall-template.S:81 #1 0x00007fd33ebee5a6 in epoll_dispatch (base=0x6062000, tv=) at epoll.c:404 #2 0x00007fd33ebd9a04 in event_base_loop (base=0x6062000, flags=) at event.c:1558 #3 0x000000000041177d in main (argc=, argv=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/memcached/daemon/memcached.c:8931 Thread 2 (Thread 0x7fd33b3a1700 (LWP 12764)): #0 0x00007fd33d4ccb13 in epoll_wait () at ../sysdeps/unix/syscall-template.S:81 #1 0x00007fd33ebee5a6 in epoll_dispatch (base=0x6062500, tv=) at epoll.c:404 #2 0x00007fd33ebd9a04 in event_base_loop (base=0x6062500, flags=) at event.c:1558 #3 0x00007fd33f65931a in platform_thread_wrap (arg=0x17581b0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #4 0x00007fd33e3cb182 in start_thread (arg=0x7fd33b3a1700) at pthread_create.c:312 #5 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 1 (Thread 0x7fd32fa9b700 (LWP 13072)): #0 0x00007fd33d408cc9 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56 #1 0x00007fd33d40c0d8 in __GI_abort () at abort.c:89 #2 0x00007fd336085c0e in CheckpointManager::registerTAPCursorBySeqno (this=0x669ad98, name=..., startBySeqno=15533) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/checkpoint.cc:433 #3 0x00007fd336116990 in ActiveStream::markDiskSnapshot (this=0x7ccb3b80, startSeqno=0, endSeqno=15533) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/dcp-stream.cc:396 #4 0x00007fd33613cdee in CouchKVStore::loadDB (this=0x6184000, cb=, cl=, sr=..., keysOnly=, vbid=709, startSeqno=1, options=0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/couch-kvstore/couch-kvstore.cc:1212 #5 0x00007fd33613d226 in CouchKVStore::dump (this=0x31b0, vb=, stSeqno=, cb=..., cl=..., sr=...) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/couch-kvstore/couch-kvstore.cc:1047 #6 0x00007fd336113e06 in DCPBackfill::run (this=0x3934cf60) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/dcp-stream.cc:167 #7 0x00007fd3360dc5a5 in ExecutorThread::run (this=0x17ad0e0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:109 #8 0x00007fd3360dca8d in launch_executor_thread (arg=0x31b0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:34 #9 0x00007fd33f65931a in platform_thread_wrap (arg=0x1758d30) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #10 0x00007fd33e3cb182 in start_thread (arg=0x7fd32fa9b700) at pthread_create.c:312 #11 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 (gdb) Thread 24 (Thread 0x7fd330a9d700 (LWP 13070)): #0 0x00007fd33d4c3fdd in fdatasync () at ../sysdeps/unix/syscall-template.S:81 #1 0x00007fd335e594f7 in couch_sync (errinfo=0x59d75d20, handle=0x53) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/couchstore/src/os.c:133 #2 0x00007fd3361489d6 in cfs_sync(struct {...} *, couch_file_handle) (errinfo=0x59d75d20, h=0xb1ea4f60) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/couch-kvstore/couch-fs-stats.cc:128 #3 0x00007fd335e4224c in couchstore_commit (db=0x59d75d00) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/couchstore/src/couch_db.cc:194 #4 0x00007fd33613be39 in CouchKVStore::saveDocs (this=, vbid=, rev=, docs=, docinfos=, docCount=, kvctx=..., snapStartSeqno=6163, snapEndSeqno=6163) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/couch-kvstore/couch-kvstore.cc:1779 #5 0x00007fd33613c362 in CouchKVStore::commit2couchstore (this=0x71fe800, cb=0x7fd330a9ca30, snapStartSeqno=, snapEndSeqno=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/couch-kvstore/couch-kvstore.cc:1671 #6 0x00007fd33613c5e5 in CouchKVStore::commit (this=0x53, cb=0x53, snapStartSeqno=7687230, snapEndSeqno=18446744073709551615) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/couch-kvstore/couch-kvstore.cc:1080 #7 0x00007fd33609726b in EventuallyPersistentStore::flushVBucket (this=0x60e5840, vbid=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/ep.cc:2804 #8 0x00007fd3360d2746 in Flusher::flushVB (this=0x17b6940) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/flusher.cc:281 #9 0x00007fd3360d3c4d in Flusher::step (this=0x17b6940, task=0x6083bd0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/flusher.cc:173 #10 0x00007fd3360dc5a5 in ExecutorThread::run (this=0x17acea0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:109 #11 0x00007fd3360dca8d in launch_executor_thread (arg=0x53) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:34 #12 0x00007fd33f65931a in platform_thread_wrap (arg=0x1758d10) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #13 0x00007fd33e3cb182 in start_thread (arg=0x7fd330a9d700) at pthread_create.c:312 #14 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 23 (Thread 0x7fd331a9f700 (LWP 13068)): #0 0x00007fd33d4c3fdd in fdatasync () at ../sysdeps/unix/syscall-template.S:81 #1 0x00007fd335e594f7 in couch_sync (errinfo=0xffa3cd20, handle=0x3) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/couchstore/src/os.c:133 #2 0x00007fd3361489d6 in cfs_sync(struct {...} *, couch_file_handle) (errinfo=0xffa3cd20, h=0xc319d3e0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/couch-kvstore/couch-fs-stats.cc:128 #3 0x00007fd335e4224c in couchstore_commit (db=0xffa3cd00) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/couchstore/src/couch_db.cc:194 #4 0x00007fd33613be39 in CouchKVStore::saveDocs (this=, vbid=, rev=, docs=, docinfos=, docCount=, kvctx=..., snapStartSeqno=15632, snapEndSeqno=15632) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/couch-kvstore/couch-kvstore.cc:1779 #5 0x00007fd33613c362 in CouchKVStore::commit2couchstore (this=0x60fa000, cb=0x7fd331a9ea30, snapStartSeqno=, snapEndSeqno=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/couch-kvstore/couch-kvstore.cc:1671 #6 0x00007fd33613c5e5 in CouchKVStore::commit (this=0x3, cb=0x3, snapStartSeqno=7687230, snapEndSeqno=18446744073709551615) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/couch-kvstore/couch-kvstore.cc:1080 #7 0x00007fd33609726b in EventuallyPersistentStore::flushVBucket (this=0x60e2000, vbid=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/ep.cc:2804 #8 0x00007fd3360d2746 in Flusher::flushVB (this=0x17b5540) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/flusher.cc:281 #9 0x00007fd3360d3c4d in Flusher::step (this=0x17b5540, task=0x6082730) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/flusher.cc:173 #10 0x00007fd3360dc5a5 in ExecutorThread::run (this=0x17acc60) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:109 #11 0x00007fd3360dca8d in launch_executor_thread (arg=0x3) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:34 #12 0x00007fd33f65931a in platform_thread_wrap (arg=0x1758cf0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #13 0x00007fd33e3cb182 in start_thread (arg=0x7fd331a9f700) at pthread_create.c:312 #14 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 22 (Thread 0x7fd33129e700 (LWP 13069)): #0 0x00007fd33d4c3fdd in fdatasync () at ../sysdeps/unix/syscall-template.S:81 #1 0x00007fd335e594f7 in couch_sync (errinfo=0x890428a0, handle=0x54) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/couchstore/src/os.c:133 #2 0x00007fd3361489d6 in cfs_sync(struct {...} *, couch_file_handle) (errinfo=0x890428a0, h=0x747bae20) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/couch-kvstore/couch-fs-stats.cc:128 #3 0x00007fd335e4224c in couchstore_commit (db=0x89042880) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/couchstore/src/couch_db.cc:194 #4 0x00007fd33613be39 in CouchKVStore::saveDocs (this=, vbid=, rev=, docs=, docinfos=, docCount=, kvctx=..., snapStartSeqno=15664, snapEndSeqno=15664) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/couch-kvstore/couch-kvstore.cc:1779 #5 0x00007fd33613c362 in CouchKVStore::commit2couchstore (this=0x60fc800, cb=0x7fd33129da30, snapStartSeqno=, snapEndSeqno=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/couch-kvstore/couch-kvstore.cc:1671 #6 0x00007fd33613c5e5 in CouchKVStore::commit (this=0x54, cb=0x54, snapStartSeqno=7687230, snapEndSeqno=18446744073709551615) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/couch-kvstore/couch-kvstore.cc:1080 #7 0x00007fd33609726b in EventuallyPersistentStore::flushVBucket (this=0x60e2000, vbid=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/ep.cc:2804 #8 0x00007fd3360d2746 in Flusher::flushVB (this=0x17b5680) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/flusher.cc:281 #9 0x00007fd3360d3c4d in Flusher::step (this=0x17b5680, task=0x60827d0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/flusher.cc:173 #10 0x00007fd3360dc5a5 in ExecutorThread::run (this=0x17acd80) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:109 #11 0x00007fd3360dca8d in launch_executor_thread (arg=0x54) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:34 #12 0x00007fd33f65931a in platform_thread_wrap (arg=0x1758d00) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #13 0x00007fd33e3cb182 in start_thread (arg=0x7fd33129e700) at pthread_create.c:312 #14 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 21 (Thread 0x7fd33029c700 (LWP 13071)): #0 0x00007fd33f04a007 in HashBytes (shift=, bytes=) at snappy.cc:49 #1 Hash (shift=, p=) at snappy.cc:52 #2 snappy::internal::CompressFragment ( input=0x1de94600b "{\"profile\":{\"likes\":39159,\"friends\":[\"8470C7DF7E\",\"986B4473EF\",\"76BB6A097B\",\"B8E1D4B050\",\"83C7D99911\"],\"name\":\"187EB88430074D92DE48\",\"online\":true},\"rating\":267.77,\"claim\":\"65DDDA98DFA2C795A426AEA84B4"..., input_size=, op=0xbe675afa "446696764DACC107C1\016\223\r\024\062\067\063E94\016G\032\034\070\061C51AFEe\263\f33A0\016\335\b\020\067\062C20\016\061\027\201\322\070\067\061\066\071\066\061\062\062\062\067B9547\016\204)", table=0x15cb6000, table_size=) at snappy.cc:358 #3 0x00007fd33f04aa48 in snappy::Compress (reader=0x7fd33029b4a0, writer=0x7fd33029b4c0) at snappy.cc:864 #4 0x00007fd33f04adfc in snappy::RawCompress (input=, input_length=, compressed=0x1de949b6b "764DA392773F631657AB765ACC7A05B7DDAC9743F7FDC998FB165AC944AB293D8876530E72BA0CE58A7D4430BF5D7CDB36A2C9A7464619DC50AE4C69AFC2C2E1A98280D3ADD8B29CAA74F0D75144673567D4847645A5C92D4C7AAE4E3183C606C0D2B109"..., compressed_length=0x12) at snappy.cc:1011 #5 0x00007fd335e42d8e in db_write_buf_compressed (file=0x74a53180, buf=0xe74a0eb0, pos=0x7fd33029b608, disk_size=0x7fd33029b618) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/couchstore/src/couch_file_write.cc:137 #6 0x00007fd335e437da in write_doc (writeopts=, disk_size=, bp=, doc=, db=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/couchstore/src/couch_save.cc:60 #7 add_doc_to_update_list (options=, seq=, idval=, seqval=, idterm=, seqterm=, fb=, info=, doc=, db=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/couchstore/src/couch_save.cc:299 #8 couchstore_save_documents (db=, docs=0x1f4d7d80, infos=0x1f4d8aa0, numdocs=, options=1363) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/couchstore/src/couch_save.cc:381 #9 0x00007fd33613bc0e in CouchKVStore::saveDocs (this=, vbid=, rev=, docs=, docinfos=, docCount=, kvctx=..., snapStartSeqno=15492, snapEndSeqno=15492) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/couch-kvstore/couch-kvstore.cc:1756 #10 0x00007fd33613c362 in CouchKVStore::commit2couchstore (this=0x61f6000, cb=0x7fd33029ba30, snapStartSeqno=, snapEndSeqno=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/couch-kvstore/couch-kvstore.cc:1671 #11 0x00007fd33613c5e5 in CouchKVStore::commit (this=0x23, cb=0x1de949b67, snapStartSeqno=8029248363, snapEndSeqno=18) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/couch-kvstore/couch-kvstore.cc:1080 #12 0x00007fd33609726b in EventuallyPersistentStore::flushVBucket (this=0x60e2000, vbid=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/ep.cc:2804 #13 0x00007fd3360d2746 in Flusher::flushVB (this=0x17b5900) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/flusher.cc:281 #14 0x00007fd3360d3c4d in Flusher::step (this=0x17b5900, task=0x6082780) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/flusher.cc:173 #15 0x00007fd3360dc5a5 in ExecutorThread::run (this=0x17acfc0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:109 #16 0x00007fd3360dca8d in launch_executor_thread (arg=0x23) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:34 #17 0x00007fd33f65931a in platform_thread_wrap (arg=0x1758d20) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #18 0x00007fd33e3cb182 in start_thread (arg=0x7fd33029c700) at pthread_create.c:312 #19 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 20 (Thread 0x7fd337399700 (LWP 12772)): #0 0x00007fd33d4ccb13 in epoll_wait () at ../sysdeps/unix/syscall-template.S:81 #1 0x00007fd33ebee5a6 in epoll_dispatch (base=0x6063900, tv=) at epoll.c:404 #2 0x00007fd33ebd9a04 in event_base_loop (base=0x6063900, flags=) at event.c:1558 #3 0x00007fd33f65931a in platform_thread_wrap (arg=0x1758230) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #4 0x00007fd33e3cb182 in start_thread (arg=0x7fd337399700) at pthread_create.c:312 #5 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 19 (Thread 0x7fd337b9a700 (LWP 12771)): #0 0x00007fd33d4ccb13 in epoll_wait () at ../sysdeps/unix/syscall-template.S:81 #1 0x00007fd33ebee5a6 in epoll_dispatch (base=0x6063680, tv=) at epoll.c:404 #2 0x00007fd33ebd9a04 in event_base_loop (base=0x6063680, flags=) at event.c:1558 #3 0x00007fd33f65931a in platform_thread_wrap (arg=0x1758220) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #4 0x00007fd33e3cb182 in start_thread (arg=0x7fd337b9a700) at pthread_create.c:312 #5 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 18 (Thread 0x7fd3332a2700 (LWP 13065)): #0 pthread_cond_timedwait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:238 #1 0x00007fd33f65977b in cb_cond_timedwait (cond=0x60b0460, mutex=0x60b0428, ms=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:156 #2 0x00007fd3360fefc0 in wait (nanoSecs=, this=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/syncobject.h:49 #3 TaskQueue::_doSleep (this=0x60b0420, t=...) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/taskqueue.cc:94 #4 0x00007fd33610166b in TaskQueue::_fetchNextTask (this=0x60b0420, t=..., toSleep=true) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/taskqueue.cc:117 #5 0x00007fd33610186f in TaskQueue::fetchNextTask (this=0x60b0420, thread=..., toSleep=142) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/taskqueue.cc:161 #6 0x00007fd3360c9c89 in ExecutorPool::_nextTask (this=0x6251a20, t=..., tick=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorpool.cc:217 #7 0x00007fd3360c9d0f in ExecutorPool::nextTask (this=0x6251a20, t=..., tick=142 '\216') at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorpool.cc:232 #8 0x00007fd3360dc221 in ExecutorThread::run (this=0x17ac900) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:81 #9 0x00007fd3360dca8d in launch_executor_thread (arg=0x60b0464) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:34 #10 0x00007fd33f65931a in platform_thread_wrap (arg=0x1758cd0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #11 0x00007fd33e3cb182 in start_thread (arg=0x7fd3332a2700) at pthread_create.c:312 #12 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 17 (Thread 0x7fd33839b700 (LWP 12770)): #0 0x00007fd33d4ccb13 in epoll_wait () at ../sysdeps/unix/syscall-template.S:81 #1 0x00007fd33ebee5a6 in epoll_dispatch (base=0x6063400, tv=) at epoll.c:404 #2 0x00007fd33ebd9a04 in event_base_loop (base=0x6063400, flags=) at event.c:1558 #3 0x00007fd33f65931a in platform_thread_wrap (arg=0x1758210) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #4 0x00007fd33e3cb182 in start_thread (arg=0x7fd33839b700) at pthread_create.c:312 #5 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 16 (Thread 0x7fd332aa1700 (LWP 13066)): #0 pthread_cond_timedwait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:238 #1 0x00007fd33f65977b in cb_cond_timedwait (cond=0x60b0460, mutex=0x60b0428, ms=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:156 #2 0x00007fd3360fefc0 in wait (nanoSecs=, this=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/syncobject.h:49 #3 TaskQueue::_doSleep (this=0x60b0420, t=...) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/taskqueue.cc:94 #4 0x00007fd33610166b in TaskQueue::_fetchNextTask (this=0x60b0420, t=..., toSleep=true) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/taskqueue.cc:117 #5 0x00007fd33610186f in TaskQueue::fetchNextTask (this=0x60b0420, thread=..., toSleep=136) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/taskqueue.cc:161 #6 0x00007fd3360c9c89 in ExecutorPool::_nextTask (this=0x6251a20, t=..., tick=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorpool.cc:217 #7 0x00007fd3360c9d0f in ExecutorPool::nextTask (this=0x6251a20, t=..., tick=136 '\210') at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorpool.cc:232 #8 0x00007fd3360dc221 in ExecutorThread::run (this=0x17aca20) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:81 #9 0x00007fd3360dca8d in launch_executor_thread (arg=0x60b0464) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:34 #10 0x00007fd33f65931a in platform_thread_wrap (arg=0x1758cc0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #11 0x00007fd33e3cb182 in start_thread (arg=0x7fd332aa1700) at pthread_create.c:312 #12 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 15 (Thread 0x7fd3322a0700 (LWP 13067)): #0 pthread_cond_timedwait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:238 #1 0x00007fd33f65977b in cb_cond_timedwait (cond=0x60b0460, mutex=0x60b0428, ms=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:156 #2 0x00007fd3360fefc0 in wait (nanoSecs=, this=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/syncobject.h:49 #3 TaskQueue::_doSleep (this=0x60b0420, t=...) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/taskqueue.cc:94 #4 0x00007fd33610166b in TaskQueue::_fetchNextTask (this=0x60b0420, t=..., toSleep=true) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/taskqueue.cc:117 #5 0x00007fd33610186f in TaskQueue::fetchNextTask (this=0x60b0420, thread=..., toSleep=138) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/taskqueue.cc:161 #6 0x00007fd3360c9c89 in ExecutorPool::_nextTask (this=0x6251a20, t=..., tick=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorpool.cc:217 #7 0x00007fd3360c9d0f in ExecutorPool::nextTask (this=0x6251a20, t=..., tick=138 '\212') at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorpool.cc:232 #8 0x00007fd3360dc221 in ExecutorThread::run (this=0x17acb40) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:81 #9 0x00007fd3360dca8d in launch_executor_thread (arg=0x60b0464) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:34 #10 0x00007fd33f65931a in platform_thread_wrap (arg=0x1758ce0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #11 0x00007fd33e3cb182 in start_thread (arg=0x7fd3322a0700) at pthread_create.c:312 #12 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 14 (Thread 0x7fd33c5b0700 (LWP 12762)): #0 pthread_cond_timedwait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:238 #1 0x00007fd33f65977b in cb_cond_timedwait (cond=0x7fd33c7b4200 , mutex=0x7fd33c7b41c0 , ms=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:156 #2 0x00007fd33c5b33fa in logger_thead_main (arg=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/memcached/extensions/loggers/file_logger.c:405 #3 0x00007fd33f65931a in platform_thread_wrap (arg=0x1758070) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #4 0x00007fd33e3cb182 in start_thread (arg=0x7fd33c5b0700) at pthread_create.c:312 #5 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 13 (Thread 0x7fd33a39f700 (LWP 12766)): #0 0x00007fd33d4ccb13 in epoll_wait () at ../sysdeps/unix/syscall-template.S:81 #1 0x00007fd33ebee5a6 in epoll_dispatch (base=0x6062a00, tv=) at epoll.c:404 #2 0x00007fd33ebd9a04 in event_base_loop (base=0x6062a00, flags=) at event.c:1558 #3 0x00007fd33f65931a in platform_thread_wrap (arg=0x17581d0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #4 0x00007fd33e3cb182 in start_thread (arg=0x7fd33a39f700) at pthread_create.c:312 #5 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 12 (Thread 0x7fd32f29a700 (LWP 13073)): #0 pthread_cond_timedwait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:238 #1 0x00007fd33f65977b in cb_cond_timedwait (cond=0x60b0720, mutex=0x60b06e8, ms=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:156 #2 0x00007fd3360fefc0 in wait (nanoSecs=, this=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/syncobject.h:49 #3 TaskQueue::_doSleep (this=0x60b06e0, t=...) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/taskqueue.cc:94 #4 0x00007fd33610166b in TaskQueue::_fetchNextTask (this=0x60b06e0, t=..., toSleep=true) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/taskqueue.cc:117 #5 0x00007fd33610186f in TaskQueue::fetchNextTask (this=0x60b06e0, thread=..., toSleep=81) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/taskqueue.cc:161 #6 0x00007fd3360c9c89 in ExecutorPool::_nextTask (this=0x6251a20, t=..., tick=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorpool.cc:217 #7 0x00007fd3360c9d0f in ExecutorPool::nextTask (this=0x6251a20, t=..., tick=81 'Q') at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorpool.cc:232 #8 0x00007fd3360dc221 in ExecutorThread::run (this=0x17ad200) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:81 #9 0x00007fd3360dca8d in launch_executor_thread (arg=0x60b0724) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:34 #10 0x00007fd33f65931a in platform_thread_wrap (arg=0x1758d40) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #11 0x00007fd33e3cb182 in start_thread (arg=0x7fd32f29a700) at pthread_create.c:312 #12 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 11 (Thread 0x7fd33bba2700 (LWP 12763)): #0 0x00007fd33d4ccb13 in epoll_wait () at ../sysdeps/unix/syscall-template.S:81 #1 0x00007fd33ebee5a6 in epoll_dispatch (base=0x6062280, tv=) at epoll.c:404 #2 0x00007fd33ebd9a04 in event_base_loop (base=0x6062280, flags=) at event.c:1558 #3 0x00007fd33f65931a in platform_thread_wrap (arg=0x17581a0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #4 0x00007fd33e3cb182 in start_thread (arg=0x7fd33bba2700) at pthread_create.c:312 #5 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 10 (Thread 0x7fd339b9e700 (LWP 12767)): #0 0x00007fd33d4ccb13 in epoll_wait () at ../sysdeps/unix/syscall-template.S:81 #1 0x00007fd33ebee5a6 in epoll_dispatch (base=0x6062c80, tv=) at epoll.c:404 #2 0x00007fd33ebd9a04 in event_base_loop (base=0x6062c80, flags=) at event.c:1558 #3 0x00007fd33f65931a in platform_thread_wrap (arg=0x17581e0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #4 0x00007fd33e3cb182 in start_thread (arg=0x7fd339b9e700) at pthread_create.c:312 #5 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 9 (Thread 0x7fd336b98700 (LWP 13063)): #0 0x00007fd33d492f3d in nanosleep () at ../sysdeps/unix/syscall-template.S:81 #1 0x00007fd33d4c44a4 in usleep (useconds=) at ../sysdeps/unix/sysv/linux/usleep.c:32 #2 0x00007fd3360da965 in updateStatsThread (arg=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/memory_tracker.cc:36 #3 0x00007fd33f65931a in platform_thread_wrap (arg=0x17582d0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #4 0x00007fd33e3cb182 in start_thread (arg=0x7fd336b98700) at pthread_create.c:312 #5 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 8 (Thread 0x7fd333aa3700 (LWP 13064)): #0 pthread_cond_timedwait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:238 #1 0x00007fd33f65977b in cb_cond_timedwait (cond=0x60b0460, mutex=0x60b0428, ms=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:156 #2 0x00007fd3360fefc0 in wait (nanoSecs=, this=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/syncobject.h:49 #3 TaskQueue::_doSleep (this=0x60b0420, t=...) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/taskqueue.cc:94 #4 0x00007fd33610166b in TaskQueue::_fetchNextTask (this=0x60b0420, t=..., toSleep=true) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/taskqueue.cc:117 #5 0x00007fd33610186f in TaskQueue::fetchNextTask (this=0x60b0420, thread=..., toSleep=140) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/taskqueue.cc:161 #6 0x00007fd3360c9c89 in ExecutorPool::_nextTask (this=0x6251a20, t=..., tick=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorpool.cc:217 #7 0x00007fd3360c9d0f in ExecutorPool::nextTask (this=0x6251a20, t=..., tick=140 '\214') at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorpool.cc:232 #8 0x00007fd3360dc221 in ExecutorThread::run (this=0x17ac7e0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:81 #9 0x00007fd3360dca8d in launch_executor_thread (arg=0x60b0464) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:34 #10 0x00007fd33f65931a in platform_thread_wrap (arg=0x1758cb0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #11 0x00007fd33e3cb182 in start_thread (arg=0x7fd333aa3700) at pthread_create.c:312 #12 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 7 (Thread 0x7fd33cfb5700 (LWP 12761)): #0 0x00007fd33d4bf12d in poll () at ../sysdeps/unix/syscall-template.S:81 #1 0x00007fd33cfb6c45 in get_command (buffer=, buffsize=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/memcached/extensions/daemon/stdin_check.c:51 #2 check_stdin_thread (arg=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/memcached/extensions/daemon/stdin_check.c:93 #3 0x00007fd33f65931a in platform_thread_wrap (arg=0x1758060) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #4 0x00007fd33e3cb182 in start_thread (arg=0x7fd33cfb5700) at pthread_create.c:312 #5 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 6 (Thread 0x7fd338b9c700 (LWP 12769)): #0 0x00007fd33d4ccb13 in epoll_wait () at ../sysdeps/unix/syscall-template.S:81 #1 0x00007fd33ebee5a6 in epoll_dispatch (base=0x6063180, tv=) at epoll.c:404 #2 0x00007fd33ebd9a04 in event_base_loop (base=0x6063180, flags=) at event.c:1558 #3 0x00007fd33f65931a in platform_thread_wrap (arg=0x1758200) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #4 0x00007fd33e3cb182 in start_thread (arg=0x7fd338b9c700) at pthread_create.c:312 #5 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 5 (Thread 0x7fd33aba0700 (LWP 12765)): #0 0x00007fd33d4ccb13 in epoll_wait () at ../sysdeps/unix/syscall-template.S:81 #1 0x00007fd33ebee5a6 in epoll_dispatch (base=0x6062780, tv=) at epoll.c:404 #2 0x00007fd33ebd9a04 in event_base_loop (base=0x6062780, flags=) at event.c:1558 #3 0x00007fd33f65931a in platform_thread_wrap (arg=0x17581c0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #4 0x00007fd33e3cb182 in start_thread (arg=0x7fd33aba0700) at pthread_create.c:312 #5 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 4 (Thread 0x7fd33939d700 (LWP 12768)): #0 0x00007fd33d4ccb13 in epoll_wait () at ../sysdeps/unix/syscall-template.S:81 #1 0x00007fd33ebee5a6 in epoll_dispatch (base=0x6062f00, tv=) at epoll.c:404 #2 0x00007fd33ebd9a04 in event_base_loop (base=0x6062f00, flags=) at event.c:1558 #3 0x00007fd33f65931a in platform_thread_wrap (arg=0x17581f0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #4 0x00007fd33e3cb182 in start_thread (arg=0x7fd33939d700) at pthread_create.c:312 #5 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 3 (Thread 0x7fd33fe79780 (LWP 12720)): #0 0x00007fd33d4ccb13 in epoll_wait () at ../sysdeps/unix/syscall-template.S:81 #1 0x00007fd33ebee5a6 in epoll_dispatch (base=0x6062000, tv=) at epoll.c:404 #2 0x00007fd33ebd9a04 in event_base_loop (base=0x6062000, flags=) at event.c:1558 #3 0x000000000041177d in main (argc=, argv=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/memcached/daemon/memcached.c:8931 Thread 2 (Thread 0x7fd33b3a1700 (LWP 12764)): #0 0x00007fd33d4ccb13 in epoll_wait () at ../sysdeps/unix/syscall-template.S:81 #1 0x00007fd33ebee5a6 in epoll_dispatch (base=0x6062500, tv=) at epoll.c:404 #2 0x00007fd33ebd9a04 in event_base_loop (base=0x6062500, flags=) at event.c:1558 #3 0x00007fd33f65931a in platform_thread_wrap (arg=0x17581b0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #4 0x00007fd33e3cb182 in start_thread (arg=0x7fd33b3a1700) at pthread_create.c:312 #5 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 1 (Thread 0x7fd32fa9b700 (LWP 13072)): #0 0x00007fd33d408cc9 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56 #1 0x00007fd33d40c0d8 in __GI_abort () at abort.c:89 #2 0x00007fd336085c0e in CheckpointManager::registerTAPCursorBySeqno (this=0x669ad98, name=..., startBySeqno=15533) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/checkpoint.cc:433 #3 0x00007fd336116990 in ActiveStream::markDiskSnapshot (this=0x7ccb3b80, startSeqno=0, endSeqno=15533) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/dcp-stream.cc:396 #4 0x00007fd33613cdee in CouchKVStore::loadDB (this=0x6184000, cb=, cl=, sr=..., keysOnly=, vbid=709, startSeqno=1, options=0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/couch-kvstore/couch-kvstore.cc:1212 #5 0x00007fd33613d226 in CouchKVStore::dump (this=0x31b0, vb=, stSeqno=, cb=..., cl=..., sr=...) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/couch-kvstore/couch-kvstore.cc:1047 #6 0x00007fd336113e06 in DCPBackfill::run (this=0x3934cf60) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/dcp-stream.cc:167 #7 0x00007fd3360dc5a5 in ExecutorThread::run (this=0x17ad0e0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:109 #8 0x00007fd3360dca8d in launch_executor_thread (arg=0x31b0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:34 #9 0x00007fd33f65931a in platform_thread_wrap (arg=0x1758d30) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #10 0x00007fd33e3cb182 in start_thread (arg=0x7fd32fa9b700) at pthread_create.c:312 #11 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 (gdb) (gdb) Thread 24 (Thread 0x7fd330a9d700 (LWP 13070)): #0 0x00007fd33d4c3fdd in fdatasync () at ../sysdeps/unix/syscall-template.S:81 #1 0x00007fd335e594f7 in couch_sync (errinfo=0x59d75d20, handle=0x53) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/couchstore/src/os.c:133 #2 0x00007fd3361489d6 in cfs_sync(struct {...} *, couch_file_handle) (errinfo=0x59d75d20, h=0xb1ea4f60) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/couch-kvstore/couch-fs-stats.cc:128 #3 0x00007fd335e4224c in couchstore_commit (db=0x59d75d00) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/couchstore/src/couch_db.cc:194 #4 0x00007fd33613be39 in CouchKVStore::saveDocs (this=, vbid=, rev=, docs=, docinfos=, docCount=, kvctx=..., snapStartSeqno=6163, snapEndSeqno=6163) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/couch-kvstore/couch-kvstore.cc:1779 #5 0x00007fd33613c362 in CouchKVStore::commit2couchstore (this=0x71fe800, cb=0x7fd330a9ca30, snapStartSeqno=, snapEndSeqno=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/couch-kvstore/couch-kvstore.cc:1671 #6 0x00007fd33613c5e5 in CouchKVStore::commit (this=0x53, cb=0x53, snapStartSeqno=7687230, snapEndSeqno=18446744073709551615) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/couch-kvstore/couch-kvstore.cc:1080 #7 0x00007fd33609726b in EventuallyPersistentStore::flushVBucket (this=0x60e5840, vbid=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/ep.cc:2804 #8 0x00007fd3360d2746 in Flusher::flushVB (this=0x17b6940) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/flusher.cc:281 #9 0x00007fd3360d3c4d in Flusher::step (this=0x17b6940, task=0x6083bd0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/flusher.cc:173 #10 0x00007fd3360dc5a5 in ExecutorThread::run (this=0x17acea0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:109 #11 0x00007fd3360dca8d in launch_executor_thread (arg=0x53) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:34 #12 0x00007fd33f65931a in platform_thread_wrap (arg=0x1758d10) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #13 0x00007fd33e3cb182 in start_thread (arg=0x7fd330a9d700) at pthread_create.c:312 #14 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 23 (Thread 0x7fd331a9f700 (LWP 13068)): #0 0x00007fd33d4c3fdd in fdatasync () at ../sysdeps/unix/syscall-template.S:81 #1 0x00007fd335e594f7 in couch_sync (errinfo=0xffa3cd20, handle=0x3) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/couchstore/src/os.c:133 #2 0x00007fd3361489d6 in cfs_sync(struct {...} *, couch_file_handle) (errinfo=0xffa3cd20, h=0xc319d3e0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/couch-kvstore/couch-fs-stats.cc:128 #3 0x00007fd335e4224c in couchstore_commit (db=0xffa3cd00) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/couchstore/src/couch_db.cc:194 #4 0x00007fd33613be39 in CouchKVStore::saveDocs (this=, vbid=, rev=, docs=, docinfos=, docCount=, kvctx=..., snapStartSeqno=15632, snapEndSeqno=15632) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/couch-kvstore/couch-kvstore.cc:1779 #5 0x00007fd33613c362 in CouchKVStore::commit2couchstore (this=0x60fa000, cb=0x7fd331a9ea30, snapStartSeqno=, snapEndSeqno=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/couch-kvstore/couch-kvstore.cc:1671 #6 0x00007fd33613c5e5 in CouchKVStore::commit (this=0x3, cb=0x3, snapStartSeqno=7687230, snapEndSeqno=18446744073709551615) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/couch-kvstore/couch-kvstore.cc:1080 #7 0x00007fd33609726b in EventuallyPersistentStore::flushVBucket (this=0x60e2000, vbid=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/ep.cc:2804 #8 0x00007fd3360d2746 in Flusher::flushVB (this=0x17b5540) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/flusher.cc:281 #9 0x00007fd3360d3c4d in Flusher::step (this=0x17b5540, task=0x6082730) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/flusher.cc:173 #10 0x00007fd3360dc5a5 in ExecutorThread::run (this=0x17acc60) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:109 #11 0x00007fd3360dca8d in launch_executor_thread (arg=0x3) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:34 #12 0x00007fd33f65931a in platform_thread_wrap (arg=0x1758cf0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #13 0x00007fd33e3cb182 in start_thread (arg=0x7fd331a9f700) at pthread_create.c:312 #14 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 22 (Thread 0x7fd33129e700 (LWP 13069)): #0 0x00007fd33d4c3fdd in fdatasync () at ../sysdeps/unix/syscall-template.S:81 #1 0x00007fd335e594f7 in couch_sync (errinfo=0x890428a0, handle=0x54) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/couchstore/src/os.c:133 #2 0x00007fd3361489d6 in cfs_sync(struct {...} *, couch_file_handle) (errinfo=0x890428a0, h=0x747bae20) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/couch-kvstore/couch-fs-stats.cc:128 #3 0x00007fd335e4224c in couchstore_commit (db=0x89042880) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/couchstore/src/couch_db.cc:194 #4 0x00007fd33613be39 in CouchKVStore::saveDocs (this=, vbid=, rev=, docs=, docinfos=, docCount=, kvctx=..., snapStartSeqno=15664, snapEndSeqno=15664) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/couch-kvstore/couch-kvstore.cc:1779 #5 0x00007fd33613c362 in CouchKVStore::commit2couchstore (this=0x60fc800, cb=0x7fd33129da30, snapStartSeqno=, snapEndSeqno=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/couch-kvstore/couch-kvstore.cc:1671 #6 0x00007fd33613c5e5 in CouchKVStore::commit (this=0x54, cb=0x54, snapStartSeqno=7687230, snapEndSeqno=18446744073709551615) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/couch-kvstore/couch-kvstore.cc:1080 #7 0x00007fd33609726b in EventuallyPersistentStore::flushVBucket (this=0x60e2000, vbid=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/ep.cc:2804 #8 0x00007fd3360d2746 in Flusher::flushVB (this=0x17b5680) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/flusher.cc:281 #9 0x00007fd3360d3c4d in Flusher::step (this=0x17b5680, task=0x60827d0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/flusher.cc:173 #10 0x00007fd3360dc5a5 in ExecutorThread::run (this=0x17acd80) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:109 #11 0x00007fd3360dca8d in launch_executor_thread (arg=0x54) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:34 #12 0x00007fd33f65931a in platform_thread_wrap (arg=0x1758d00) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #13 0x00007fd33e3cb182 in start_thread (arg=0x7fd33129e700) at pthread_create.c:312 #14 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 21 (Thread 0x7fd33029c700 (LWP 13071)): #0 0x00007fd33f04a007 in HashBytes (shift=, bytes=) at snappy.cc:49 #1 Hash (shift=, p=) at snappy.cc:52 #2 snappy::internal::CompressFragment ( input=0x1de94600b "{\"profile\":{\"likes\":39159,\"friends\":[\"8470C7DF7E\",\"986B4473EF\",\"76BB6A097B\",\"B8E1D4B050\",\"83C7D99911\"],\"name\":\"187EB88430074D92DE48\",\"online\":true},\"rating\":267.77,\"claim\":\"65DDDA98DFA2C795A426AEA84B4"..., input_size=, op=0xbe675afa "446696764DACC107C1\016\223\r\024\062\067\063E94\016G\032\034\070\061C51AFEe\263\f33A0\016\335\b\020\067\062C20\016\061\027\201\322\070\067\061\066\071\066\061\062\062\062\067B9547\016\204)", table=0x15cb6000, table_size=) at snappy.cc:358 #3 0x00007fd33f04aa48 in snappy::Compress (reader=0x7fd33029b4a0, writer=0x7fd33029b4c0) at snappy.cc:864 #4 0x00007fd33f04adfc in snappy::RawCompress (input=, input_length=, compressed=0x1de949b6b "764DA392773F631657AB765ACC7A05B7DDAC9743F7FDC998FB165AC944AB293D8876530E72BA0CE58A7D4430BF5D7CDB36A2C9A7464619DC50AE4C69AFC2C2E1A98280D3ADD8B29CAA74F0D75144673567D4847645A5C92D4C7AAE4E3183C606C0D2B109"..., compressed_length=0x12) at snappy.cc:1011 #5 0x00007fd335e42d8e in db_write_buf_compressed (file=0x74a53180, buf=0xe74a0eb0, pos=0x7fd33029b608, disk_size=0x7fd33029b618) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/couchstore/src/couch_file_write.cc:137 #6 0x00007fd335e437da in write_doc (writeopts=, disk_size=, bp=, doc=, db=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/couchstore/src/couch_save.cc:60 #7 add_doc_to_update_list (options=, seq=, idval=, seqval=, idterm=, seqterm=, fb=, info=, doc=, db=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/couchstore/src/couch_save.cc:299 #8 couchstore_save_documents (db=, docs=0x1f4d7d80, infos=0x1f4d8aa0, numdocs=, options=1363) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/couchstore/src/couch_save.cc:381 #9 0x00007fd33613bc0e in CouchKVStore::saveDocs (this=, vbid=, rev=, docs=, docinfos=, docCount=, kvctx=..., snapStartSeqno=15492, snapEndSeqno=15492) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/couch-kvstore/couch-kvstore.cc:1756 #10 0x00007fd33613c362 in CouchKVStore::commit2couchstore (this=0x61f6000, cb=0x7fd33029ba30, snapStartSeqno=, snapEndSeqno=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/couch-kvstore/couch-kvstore.cc:1671 #11 0x00007fd33613c5e5 in CouchKVStore::commit (this=0x23, cb=0x1de949b67, snapStartSeqno=8029248363, snapEndSeqno=18) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/couch-kvstore/couch-kvstore.cc:1080 #12 0x00007fd33609726b in EventuallyPersistentStore::flushVBucket (this=0x60e2000, vbid=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/ep.cc:2804 #13 0x00007fd3360d2746 in Flusher::flushVB (this=0x17b5900) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/flusher.cc:281 #14 0x00007fd3360d3c4d in Flusher::step (this=0x17b5900, task=0x6082780) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/flusher.cc:173 #15 0x00007fd3360dc5a5 in ExecutorThread::run (this=0x17acfc0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:109 #16 0x00007fd3360dca8d in launch_executor_thread (arg=0x23) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:34 #17 0x00007fd33f65931a in platform_thread_wrap (arg=0x1758d20) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #18 0x00007fd33e3cb182 in start_thread (arg=0x7fd33029c700) at pthread_create.c:312 #19 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 20 (Thread 0x7fd337399700 (LWP 12772)): #0 0x00007fd33d4ccb13 in epoll_wait () at ../sysdeps/unix/syscall-template.S:81 #1 0x00007fd33ebee5a6 in epoll_dispatch (base=0x6063900, tv=) at epoll.c:404 #2 0x00007fd33ebd9a04 in event_base_loop (base=0x6063900, flags=) at event.c:1558 #3 0x00007fd33f65931a in platform_thread_wrap (arg=0x1758230) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #4 0x00007fd33e3cb182 in start_thread (arg=0x7fd337399700) at pthread_create.c:312 #5 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 19 (Thread 0x7fd337b9a700 (LWP 12771)): #0 0x00007fd33d4ccb13 in epoll_wait () at ../sysdeps/unix/syscall-template.S:81 #1 0x00007fd33ebee5a6 in epoll_dispatch (base=0x6063680, tv=) at epoll.c:404 #2 0x00007fd33ebd9a04 in event_base_loop (base=0x6063680, flags=) at event.c:1558 #3 0x00007fd33f65931a in platform_thread_wrap (arg=0x1758220) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #4 0x00007fd33e3cb182 in start_thread (arg=0x7fd337b9a700) at pthread_create.c:312 #5 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 18 (Thread 0x7fd3332a2700 (LWP 13065)): #0 pthread_cond_timedwait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:238 #1 0x00007fd33f65977b in cb_cond_timedwait (cond=0x60b0460, mutex=0x60b0428, ms=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:156 #2 0x00007fd3360fefc0 in wait (nanoSecs=, this=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/syncobject.h:49 #3 TaskQueue::_doSleep (this=0x60b0420, t=...) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/taskqueue.cc:94 #4 0x00007fd33610166b in TaskQueue::_fetchNextTask (this=0x60b0420, t=..., toSleep=true) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/taskqueue.cc:117 #5 0x00007fd33610186f in TaskQueue::fetchNextTask (this=0x60b0420, thread=..., toSleep=142) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/taskqueue.cc:161 #6 0x00007fd3360c9c89 in ExecutorPool::_nextTask (this=0x6251a20, t=..., tick=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorpool.cc:217 #7 0x00007fd3360c9d0f in ExecutorPool::nextTask (this=0x6251a20, t=..., tick=142 '\216') at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorpool.cc:232 #8 0x00007fd3360dc221 in ExecutorThread::run (this=0x17ac900) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:81 #9 0x00007fd3360dca8d in launch_executor_thread (arg=0x60b0464) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:34 #10 0x00007fd33f65931a in platform_thread_wrap (arg=0x1758cd0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #11 0x00007fd33e3cb182 in start_thread (arg=0x7fd3332a2700) at pthread_create.c:312 #12 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 17 (Thread 0x7fd33839b700 (LWP 12770)): #0 0x00007fd33d4ccb13 in epoll_wait () at ../sysdeps/unix/syscall-template.S:81 #1 0x00007fd33ebee5a6 in epoll_dispatch (base=0x6063400, tv=) at epoll.c:404 #2 0x00007fd33ebd9a04 in event_base_loop (base=0x6063400, flags=) at event.c:1558 #3 0x00007fd33f65931a in platform_thread_wrap (arg=0x1758210) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #4 0x00007fd33e3cb182 in start_thread (arg=0x7fd33839b700) at pthread_create.c:312 #5 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 16 (Thread 0x7fd332aa1700 (LWP 13066)): #0 pthread_cond_timedwait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:238 #1 0x00007fd33f65977b in cb_cond_timedwait (cond=0x60b0460, mutex=0x60b0428, ms=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:156 #2 0x00007fd3360fefc0 in wait (nanoSecs=, this=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/syncobject.h:49 #3 TaskQueue::_doSleep (this=0x60b0420, t=...) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/taskqueue.cc:94 #4 0x00007fd33610166b in TaskQueue::_fetchNextTask (this=0x60b0420, t=..., toSleep=true) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/taskqueue.cc:117 #5 0x00007fd33610186f in TaskQueue::fetchNextTask (this=0x60b0420, thread=..., toSleep=136) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/taskqueue.cc:161 #6 0x00007fd3360c9c89 in ExecutorPool::_nextTask (this=0x6251a20, t=..., tick=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorpool.cc:217 #7 0x00007fd3360c9d0f in ExecutorPool::nextTask (this=0x6251a20, t=..., tick=136 '\210') at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorpool.cc:232 #8 0x00007fd3360dc221 in ExecutorThread::run (this=0x17aca20) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:81 #9 0x00007fd3360dca8d in launch_executor_thread (arg=0x60b0464) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:34 #10 0x00007fd33f65931a in platform_thread_wrap (arg=0x1758cc0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #11 0x00007fd33e3cb182 in start_thread (arg=0x7fd332aa1700) at pthread_create.c:312 #12 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 15 (Thread 0x7fd3322a0700 (LWP 13067)): #0 pthread_cond_timedwait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:238 #1 0x00007fd33f65977b in cb_cond_timedwait (cond=0x60b0460, mutex=0x60b0428, ms=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:156 #2 0x00007fd3360fefc0 in wait (nanoSecs=, this=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/syncobject.h:49 #3 TaskQueue::_doSleep (this=0x60b0420, t=...) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/taskqueue.cc:94 #4 0x00007fd33610166b in TaskQueue::_fetchNextTask (this=0x60b0420, t=..., toSleep=true) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/taskqueue.cc:117 #5 0x00007fd33610186f in TaskQueue::fetchNextTask (this=0x60b0420, thread=..., toSleep=138) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/taskqueue.cc:161 #6 0x00007fd3360c9c89 in ExecutorPool::_nextTask (this=0x6251a20, t=..., tick=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorpool.cc:217 #7 0x00007fd3360c9d0f in ExecutorPool::nextTask (this=0x6251a20, t=..., tick=138 '\212') at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorpool.cc:232 #8 0x00007fd3360dc221 in ExecutorThread::run (this=0x17acb40) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:81 #9 0x00007fd3360dca8d in launch_executor_thread (arg=0x60b0464) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:34 #10 0x00007fd33f65931a in platform_thread_wrap (arg=0x1758ce0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #11 0x00007fd33e3cb182 in start_thread (arg=0x7fd3322a0700) at pthread_create.c:312 #12 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 14 (Thread 0x7fd33c5b0700 (LWP 12762)): #0 pthread_cond_timedwait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:238 #1 0x00007fd33f65977b in cb_cond_timedwait (cond=0x7fd33c7b4200 , mutex=0x7fd33c7b41c0 , ms=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:156 #2 0x00007fd33c5b33fa in logger_thead_main (arg=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/memcached/extensions/loggers/file_logger.c:405 #3 0x00007fd33f65931a in platform_thread_wrap (arg=0x1758070) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #4 0x00007fd33e3cb182 in start_thread (arg=0x7fd33c5b0700) at pthread_create.c:312 #5 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 13 (Thread 0x7fd33a39f700 (LWP 12766)): #0 0x00007fd33d4ccb13 in epoll_wait () at ../sysdeps/unix/syscall-template.S:81 #1 0x00007fd33ebee5a6 in epoll_dispatch (base=0x6062a00, tv=) at epoll.c:404 #2 0x00007fd33ebd9a04 in event_base_loop (base=0x6062a00, flags=) at event.c:1558 #3 0x00007fd33f65931a in platform_thread_wrap (arg=0x17581d0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #4 0x00007fd33e3cb182 in start_thread (arg=0x7fd33a39f700) at pthread_create.c:312 #5 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 12 (Thread 0x7fd32f29a700 (LWP 13073)): #0 pthread_cond_timedwait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:238 #1 0x00007fd33f65977b in cb_cond_timedwait (cond=0x60b0720, mutex=0x60b06e8, ms=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:156 #2 0x00007fd3360fefc0 in wait (nanoSecs=, this=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/syncobject.h:49 #3 TaskQueue::_doSleep (this=0x60b06e0, t=...) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/taskqueue.cc:94 #4 0x00007fd33610166b in TaskQueue::_fetchNextTask (this=0x60b06e0, t=..., toSleep=true) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/taskqueue.cc:117 #5 0x00007fd33610186f in TaskQueue::fetchNextTask (this=0x60b06e0, thread=..., toSleep=81) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/taskqueue.cc:161 #6 0x00007fd3360c9c89 in ExecutorPool::_nextTask (this=0x6251a20, t=..., tick=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorpool.cc:217 #7 0x00007fd3360c9d0f in ExecutorPool::nextTask (this=0x6251a20, t=..., tick=81 'Q') at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorpool.cc:232 #8 0x00007fd3360dc221 in ExecutorThread::run (this=0x17ad200) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:81 #9 0x00007fd3360dca8d in launch_executor_thread (arg=0x60b0724) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:34 #10 0x00007fd33f65931a in platform_thread_wrap (arg=0x1758d40) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #11 0x00007fd33e3cb182 in start_thread (arg=0x7fd32f29a700) at pthread_create.c:312 #12 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 11 (Thread 0x7fd33bba2700 (LWP 12763)): #0 0x00007fd33d4ccb13 in epoll_wait () at ../sysdeps/unix/syscall-template.S:81 #1 0x00007fd33ebee5a6 in epoll_dispatch (base=0x6062280, tv=) at epoll.c:404 #2 0x00007fd33ebd9a04 in event_base_loop (base=0x6062280, flags=) at event.c:1558 #3 0x00007fd33f65931a in platform_thread_wrap (arg=0x17581a0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #4 0x00007fd33e3cb182 in start_thread (arg=0x7fd33bba2700) at pthread_create.c:312 #5 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 10 (Thread 0x7fd339b9e700 (LWP 12767)): #0 0x00007fd33d4ccb13 in epoll_wait () at ../sysdeps/unix/syscall-template.S:81 #1 0x00007fd33ebee5a6 in epoll_dispatch (base=0x6062c80, tv=) at epoll.c:404 #2 0x00007fd33ebd9a04 in event_base_loop (base=0x6062c80, flags=) at event.c:1558 #3 0x00007fd33f65931a in platform_thread_wrap (arg=0x17581e0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #4 0x00007fd33e3cb182 in start_thread (arg=0x7fd339b9e700) at pthread_create.c:312 #5 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 9 (Thread 0x7fd336b98700 (LWP 13063)): #0 0x00007fd33d492f3d in nanosleep () at ../sysdeps/unix/syscall-template.S:81 #1 0x00007fd33d4c44a4 in usleep (useconds=) at ../sysdeps/unix/sysv/linux/usleep.c:32 #2 0x00007fd3360da965 in updateStatsThread (arg=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/memory_tracker.cc:36 #3 0x00007fd33f65931a in platform_thread_wrap (arg=0x17582d0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #4 0x00007fd33e3cb182 in start_thread (arg=0x7fd336b98700) at pthread_create.c:312 #5 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 8 (Thread 0x7fd333aa3700 (LWP 13064)): #0 pthread_cond_timedwait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:238 #1 0x00007fd33f65977b in cb_cond_timedwait (cond=0x60b0460, mutex=0x60b0428, ms=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:156 #2 0x00007fd3360fefc0 in wait (nanoSecs=, this=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/syncobject.h:49 #3 TaskQueue::_doSleep (this=0x60b0420, t=...) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/taskqueue.cc:94 #4 0x00007fd33610166b in TaskQueue::_fetchNextTask (this=0x60b0420, t=..., toSleep=true) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/taskqueue.cc:117 #5 0x00007fd33610186f in TaskQueue::fetchNextTask (this=0x60b0420, thread=..., toSleep=140) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/taskqueue.cc:161 #6 0x00007fd3360c9c89 in ExecutorPool::_nextTask (this=0x6251a20, t=..., tick=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorpool.cc:217 #7 0x00007fd3360c9d0f in ExecutorPool::nextTask (this=0x6251a20, t=..., tick=140 '\214') at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorpool.cc:232 #8 0x00007fd3360dc221 in ExecutorThread::run (this=0x17ac7e0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:81 #9 0x00007fd3360dca8d in launch_executor_thread (arg=0x60b0464) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:34 #10 0x00007fd33f65931a in platform_thread_wrap (arg=0x1758cb0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #11 0x00007fd33e3cb182 in start_thread (arg=0x7fd333aa3700) at pthread_create.c:312 #12 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 7 (Thread 0x7fd33cfb5700 (LWP 12761)): #0 0x00007fd33d4bf12d in poll () at ../sysdeps/unix/syscall-template.S:81 #1 0x00007fd33cfb6c45 in get_command (buffer=, buffsize=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/memcached/extensions/daemon/stdin_check.c:51 #2 check_stdin_thread (arg=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/memcached/extensions/daemon/stdin_check.c:93 #3 0x00007fd33f65931a in platform_thread_wrap (arg=0x1758060) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #4 0x00007fd33e3cb182 in start_thread (arg=0x7fd33cfb5700) at pthread_create.c:312 #5 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 6 (Thread 0x7fd338b9c700 (LWP 12769)): #0 0x00007fd33d4ccb13 in epoll_wait () at ../sysdeps/unix/syscall-template.S:81 #1 0x00007fd33ebee5a6 in epoll_dispatch (base=0x6063180, tv=) at epoll.c:404 #2 0x00007fd33ebd9a04 in event_base_loop (base=0x6063180, flags=) at event.c:1558 #3 0x00007fd33f65931a in platform_thread_wrap (arg=0x1758200) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #4 0x00007fd33e3cb182 in start_thread (arg=0x7fd338b9c700) at pthread_create.c:312 #5 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 5 (Thread 0x7fd33aba0700 (LWP 12765)): #0 0x00007fd33d4ccb13 in epoll_wait () at ../sysdeps/unix/syscall-template.S:81 #1 0x00007fd33ebee5a6 in epoll_dispatch (base=0x6062780, tv=) at epoll.c:404 #2 0x00007fd33ebd9a04 in event_base_loop (base=0x6062780, flags=) at event.c:1558 #3 0x00007fd33f65931a in platform_thread_wrap (arg=0x17581c0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #4 0x00007fd33e3cb182 in start_thread (arg=0x7fd33aba0700) at pthread_create.c:312 #5 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 4 (Thread 0x7fd33939d700 (LWP 12768)): #0 0x00007fd33d4ccb13 in epoll_wait () at ../sysdeps/unix/syscall-template.S:81 #1 0x00007fd33ebee5a6 in epoll_dispatch (base=0x6062f00, tv=) at epoll.c:404 #2 0x00007fd33ebd9a04 in event_base_loop (base=0x6062f00, flags=) at event.c:1558 #3 0x00007fd33f65931a in platform_thread_wrap (arg=0x17581f0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #4 0x00007fd33e3cb182 in start_thread (arg=0x7fd33939d700) at pthread_create.c:312 #5 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 3 (Thread 0x7fd33fe79780 (LWP 12720)): #0 0x00007fd33d4ccb13 in epoll_wait () at ../sysdeps/unix/syscall-template.S:81 #1 0x00007fd33ebee5a6 in epoll_dispatch (base=0x6062000, tv=) at epoll.c:404 #2 0x00007fd33ebd9a04 in event_base_loop (base=0x6062000, flags=) at event.c:1558 #3 0x000000000041177d in main (argc=, argv=) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/memcached/daemon/memcached.c:8931 Thread 2 (Thread 0x7fd33b3a1700 (LWP 12764)): #0 0x00007fd33d4ccb13 in epoll_wait () at ../sysdeps/unix/syscall-template.S:81 #1 0x00007fd33ebee5a6 in epoll_dispatch (base=0x6062500, tv=) at epoll.c:404 #2 0x00007fd33ebd9a04 in event_base_loop (base=0x6062500, flags=) at event.c:1558 #3 0x00007fd33f65931a in platform_thread_wrap (arg=0x17581b0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #4 0x00007fd33e3cb182 in start_thread (arg=0x7fd33b3a1700) at pthread_create.c:312 #5 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 1 (Thread 0x7fd32fa9b700 (LWP 13072)): #0 0x00007fd33d408cc9 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56 #1 0x00007fd33d40c0d8 in __GI_abort () at abort.c:89 #2 0x00007fd336085c0e in CheckpointManager::registerTAPCursorBySeqno (this=0x669ad98, name=..., startBySeqno=15533) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/checkpoint.cc:433 #3 0x00007fd336116990 in ActiveStream::markDiskSnapshot (this=0x7ccb3b80, startSeqno=0, endSeqno=15533) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/dcp-stream.cc:396 #4 0x00007fd33613cdee in CouchKVStore::loadDB (this=0x6184000, cb=, cl=, sr=..., keysOnly=, vbid=709, startSeqno=1, options=0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/couch-kvstore/couch-kvstore.cc:1212 #5 0x00007fd33613d226 in CouchKVStore::dump (this=0x31b0, vb=, stSeqno=, cb=..., cl=..., sr=...) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/couch-kvstore/couch-kvstore.cc:1047 #6 0x00007fd336113e06 in DCPBackfill::run (this=0x3934cf60) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/dcp-stream.cc:167 #7 0x00007fd3360dc5a5 in ExecutorThread::run (this=0x17ad0e0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:109 #8 0x00007fd3360dca8d in launch_executor_thread (arg=0x31b0) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/ep-engine/src/executorthread.cc:34 #9 0x00007fd33f65931a in platform_thread_wrap (arg=0x1758d30) at /home/buildbot/buildbot_slave/ubuntu-1004-x64-30x-builder/build/build/platform/src/cb_pthreads.c:19 #10 0x00007fd33e3cb182 in start_thread (arg=0x7fd32fa9b700) at pthread_create.c:312 #11 0x00007fd33d4cc47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 (gdb) quit