memcached binary incr abort()

While running memcapable against moxi to memcached, I see memcached crash/assert(0).

Looks like there might be some new ENGINE_XXXX return codes from engine->arithmetic() which are not handled by memcached.c complete_incr_bin()...

------------

Process: memcached [51048]
Path: /Users/steveyen/work/northscale/dev/tmp/nss1/northscale-server/bin/memcached/memcached
Identifier: memcached
Version: ??? (???)
Code Type: X86-64 (Native)
Parent Process: port_adaptor [51047]

Date/Time: 2010-07-19 09:55:37.101 -0700
OS Version: Mac OS X 10.6.3 (10D578)
Report Version: 6

Exception Type: EXC_CRASH (SIGABRT)
Exception Codes: 0x0000000000000000, 0x0000000000000000
Crashed Thread: 4

Application Specific Information:
abort() called

Thread 0: Dispatch queue: com.apple.main-thread
0 libSystem.B.dylib 0x00007fff841bc4ea kevent + 10
1 libevent-1.4.2.dylib 0x0000000100035213 kq_dispatch + 83
2 libevent-1.4.2.dylib 0x0000000100028d28 event_base_loop + 440
3 memcached 0x0000000100004c29 main + 4281 (memcached.c:6511)
4 memcached 0x0000000100000ba4 start + 52

Thread 1:
0 libSystem.B.dylib 0x00007fff841de316 __semwait_signal + 10
1 libSystem.B.dylib 0x00007fff841e2131 _pthread_cond_wait + 1286
2 ep.so 0x000000010020160b Dispatcher::run() + 1179 (syncobject.hh:42)
3 ep.so 0x0000000100201bd5 launch_dispatcher_thread + 21 (dispatcher.cc:34)
4 libSystem.B.dylib 0x00007fff841dc8b6 _pthread_start + 331
5 libSystem.B.dylib 0x00007fff841dc769 thread_start + 13

Thread 2:
0 libSystem.B.dylib 0x00007fff841de316 __semwait_signal + 10
1 libSystem.B.dylib 0x00007fff841e2131 _pthread_cond_wait + 1286
2 ep.so 0x0000000100221b99 EventuallyPersistentEngine::notifyTapIoThreadMain() + 1513 (syncobject.hh:31)
3 ep.so 0x0000000100213fa4 EvpNotifyTapIo + 292 (ep_engine.h:1462)
4 libSystem.B.dylib 0x00007fff841dc8b6 _pthread_start + 331
5 libSystem.B.dylib 0x00007fff841dc769 thread_start + 13

Thread 3:
0 libSystem.B.dylib 0x00007fff841bc4ea kevent + 10
1 libevent-1.4.2.dylib 0x0000000100035213 kq_dispatch + 83
2 libevent-1.4.2.dylib 0x0000000100028d28 event_base_loop + 440
3 memcached 0x0000000100010411 worker_libevent + 65 (thread.c:254)
4 libSystem.B.dylib 0x00007fff841dc8b6 _pthread_start + 331
5 libSystem.B.dylib 0x00007fff841dc769 thread_start + 13

Thread 4 Crashed:
0 libSystem.B.dylib 0x00007fff8421601e __semwait_signal_nocancel + 10
1 libSystem.B.dylib 0x00007fff84215f20 nanosleep$NOCANCEL + 129
2 libSystem.B.dylib 0x00007fff84272902 usleep$NOCANCEL + 57
3 libSystem.B.dylib 0x00007fff84291eb8 abort + 93
4 memcached 0x0000000100009127 complete_incr_bin + 455 (memcached.c:1414)
5 memcached 0x000000010000d70f complete_nread_binary + 767 (memcached.c:3051)
6 memcached 0x000000010000e90a conn_nread + 970 (memcached.c:3134)
7 memcached 0x0000000100002626 event_handler + 166 (memcached.c:4916)
8 libevent-1.4.2.dylib 0x0000000100029031 event_base_loop + 1217
9 memcached 0x0000000100010411 worker_libevent + 65 (thread.c:254)
10 libSystem.B.dylib 0x00007fff841dc8b6 _pthread_start + 331
11 libSystem.B.dylib 0x00007fff841dc769 thread_start + 13

Thread 5:
0 libSystem.B.dylib 0x00007fff841bc4ea kevent + 10
1 libevent-1.4.2.dylib 0x0000000100035213 kq_dispatch + 83
2 libevent-1.4.2.dylib 0x0000000100028d28 event_base_loop + 440
3 memcached 0x0000000100010411 worker_libevent + 65 (thread.c:254)
4 libSystem.B.dylib 0x00007fff841dc8b6 _pthread_start + 331
5 libSystem.B.dylib 0x00007fff841dc769 thread_start + 13

Thread 6:
0 libSystem.B.dylib 0x00007fff841bc4ea kevent + 10
1 libevent-1.4.2.dylib 0x0000000100035213 kq_dispatch + 83
2 libevent-1.4.2.dylib 0x0000000100028d28 event_base_loop + 440
3 memcached 0x0000000100010411 worker_libevent + 65 (thread.c:254)
4 libSystem.B.dylib 0x00007fff841dc8b6 _pthread_start + 331
5 libSystem.B.dylib 0x00007fff841dc769 thread_start + 13

Thread 7:
0 libSystem.B.dylib 0x00007fff841bc4ea kevent + 10
1 libevent-1.4.2.dylib 0x0000000100035213 kq_dispatch + 83
2 libevent-1.4.2.dylib 0x0000000100028d28 event_base_loop + 440
3 memcached 0x0000000100010411 worker_libevent + 65 (thread.c:254)
4 libSystem.B.dylib 0x00007fff841dc8b6 _pthread_start + 331
5 libSystem.B.dylib 0x00007fff841dc769 thread_start + 13

Thread 4 crashed with X86 Thread State (64-bit):
rax: 0x000000000000003c rbx: 0x00000001006033c0 rcx: 0x0000000100603378 rdx: 0x0000000000000001
rdi: 0x0000000000000c03 rsi: 0x0000000000000000 rbp: 0x00000001006033b0 rsp: 0x0000000100603378
r8: 0x0000000000000000 r9: 0x0000000000989680 r10: 0x0000000000000001 r11: 0x0000000000000246
r12: 0x0000000000000000 r13: 0x000000000040a501 r14: 0x0000000102246e18 r15: 0x0000000102246e00
rip: 0x00007fff8421601e rfl: 0x0000000000000247 cr2: 0x0000000101f49868

Binary Images:
0x100000000 - 0x10001afef +memcached ??? (???) <8806F3FA-5C9C-9C67-1770-18398C56E934> /Users/steveyen/work/northscale/dev/ns_server/bin/memcached/memcached
0x100027000 - 0x10003bfef +libevent-1.4.2.dylib 4.3.0 (compatibility 4.0.0) <D80CD763-4718-5709-2720-784D73D3D508> /Users/steveyen/work/northscale/dev/grommix/libevent-1.4.13-stable/lib/libevent-1.4.2.dylib
0x100200000 - 0x100314fef +ep.so ??? (???) <E75CE4A8-A95C-C61D-F1E0-A77E5ACDA356> /Users/steveyen/work/northscale/dev/tmp/nss1/northscale-server/bin/ep_engine/ep.so
0x7fff5fc00000 - 0x7fff5fc3bdef dyld 132.1 (???) <B633F790-4DDB-53CD-7ACF-2A3682BCEA9F> /usr/lib/dyld
0x7fff80842000 - 0x7fff80863fff libresolv.9.dylib 40.0.0 (compatibility 1.0.0) <1AE68BBB-6536-125C-DE2A-13CA916D0EC4> /usr/lib/libresolv.9.dylib
0x7fff83247000 - 0x7fff8324bff7 libmathCommon.A.dylib 315.0.0 (compatibility 1.0.0) <95718673-FEEE-B6ED-B127-BCDBDB60D4E5> /usr/lib/system/libmathCommon.A.dylib
0x7fff841a2000 - 0x7fff84361ff7 libSystem.B.dylib 125.0.1 (compatibility 1.0.0) <CB9A4929-61AF-DE71-5635-133E9EC95783> /usr/lib/libSystem.B.dylib
0x7fff87818000 - 0x7fff87895fef libstdc+.6.dylib 7.9.0 (compatibility 7.0.0) <35ECA411-2C08-FD7D-11B1-1B7A04921A5C> /usr/lib/libstdc+.6.dylib
0x7fffffe00000 - 0x7fffffe01fff libSystem.B.dylib ??? (???) <CB9A4929-61AF-DE71-5635-133E9EC95783> /usr/lib/libSystem.B.dylib