Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Cannot Reproduce
    • Affects Version/s: 2.0-beta
    • Fix Version/s: 3.0
    • Component/s: couchbase-bucket
    • Security Level: Public
    • Labels:
      None

      Description

      Was running cluster_run cluster and stopped it by Ctrl-C (which is not delivered to memcached). All memcached's died normally. One seems to hang. Here's backtrace:

      (gdb) thread app all bt

      Thread 2 (Thread 0xf298ab70 (LWP 4846)):
      #0 0xf772f430 in __kernel_vsyscall ()
      #1 0xf767af02 in __lll_lock_wait () at ../nptl/sysdeps/unix/sysv/linux/i386/i686/../i486/lowlevellock.S:142
      #2 0xf767639b in _L_lock_728 () from /lib/i386-linux-gnu/i686/cmov/libpthread.so.0
      #3 0xf76761c1 in __pthread_mutex_lock (mutex=mutex@entry=0xc90bb64) at pthread_mutex_lock.c:61
      #4 0x0804d9a4 in release_cookie (cookie=0xc8da8d8) at daemon/memcached.c:6691
      #5 0xf771fc12 in bucket_engine_release_cookie (cookie=0xc8da8d8) at bucket_engine.c:2519
      #6 0xf4343fa4 in EventuallyPersistentEngine::releaseCookie (this=0xc93a4c8, cookie=0xc8da8d8) at ep_engine.cc:1192
      #7 0xf437d04b in TapConnection::releaseReference (this=0xf1800a10, force=false) at tapconnection.cc:65
      #8 0xf438f72b in TapConnectionReaperCallback::callback (this=0xc90bd38) at tapconnmap.cc:29
      #9 0xf4326855 in Task::run (this=0xca3dc40, d=..., t=...) at dispatcher.hh:139
      #10 0xf4324713 in Dispatcher::run (this=0xc95ea08) at dispatcher.cc:123
      #11 0xf43261ea in launch_dispatcher_thread (arg=0xc95ea08) at dispatcher.cc:28
      #12 0xf7673c39 in start_thread (arg=0xf298ab70) at pthread_create.c:304
      #13 0xf75e127e in clone () at ../sysdeps/unix/sysv/linux/i386/clone.S:130

      Thread 1 (Thread 0xf750f6c0 (LWP 4793)):
      #0 0xf772f430 in __kernel_vsyscall ()
      #1 0xf7674e65 in pthread_join (threadid=4070091632, thread_return=0x0) at pthread_join.c:89
      #2 0xf43203d0 in Dispatcher::stop (this=0xc95ea08, force=false) at dispatcher.cc:162
      #3 0xf432c8f7 in EventuallyPersistentStore::~EventuallyPersistentStore (this=0xc94b9b0, __in_chrg=<optimized out>) at ep.cc:676
      #4 0xf43595b4 in EventuallyPersistentEngine::~EventuallyPersistentEngine (this=0xc93a4c8, __in_chrg=<optimized out>) at ep_engine.h:511
      #5 0xf4347fc5 in EvpDestroy (handle=0xc93a4c8, force=false) at ep_engine.cc:125
      #6 0xf771c5fa in bucket_shutdown_engine (key=0xc93a4b8, nkey=7, val=0xc90c150, nval=0, args=0x0) at bucket_engine.c:1289
      #7 0xf7721c59 in genhash_iter (h=0xc8f5330, iterfunc=iterfunc@entry=0xf771c5a0 <bucket_shutdown_engine>, arg=arg@entry=0x0) at genhash.c:275
      #8 0xf7720ae2 in bucket_destroy (handle=0xf77253e0, force=<optimized out>) at bucket_engine.c:1326
      #9 bucket_destroy (handle=0xf77253e0, force=false) at bucket_engine.c:1306
      #10 0x0804be30 in main (argc=19, argv=0xffeb7ab4) at daemon/memcached.c:7921

      Both threads seem to be stuck and unable to proceed

      No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

        Hide
        karan Karan Kumar (Inactive) added a comment -

        Not sure. How can I get per thread cpu consumption?

        Show
        karan Karan Kumar (Inactive) added a comment - Not sure. How can I get per thread cpu consumption?
        Hide
        trond Trond Norbye added a comment -

        I believe you may see that from pressing H in top on linux.

        Show
        trond Trond Norbye added a comment - I believe you may see that from pressing H in top on linux.
        Hide
        peter peter added a comment -

        Trond, do you have a patch in gerrit for this?

        Show
        peter peter added a comment - Trond, do you have a patch in gerrit for this?
        Hide
        karan Karan Kumar (Inactive) added a comment -

        Hitting the issue again on build 1862
        https://friendpaste.com/2RoA2uswrK1WgTtjDNrQei

        Show
        karan Karan Kumar (Inactive) added a comment - Hitting the issue again on build 1862 https://friendpaste.com/2RoA2uswrK1WgTtjDNrQei
        Hide
        mikew Mike Wiederhold added a comment -

        Closing since this is over 1 year old. If this is still a valid issue please reopen.

        Show
        mikew Mike Wiederhold added a comment - Closing since this is over 1 year old. If this is still a valid issue please reopen.

          People

          • Assignee:
            trond Trond Norbye
            Reporter:
            alkondratenko Aleksey Kondratenko (Inactive)
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Gerrit Reviews

              There are no open Gerrit changes