Details

    • Type: Bug
    • Status: Open
    • Priority: Critical
    • Resolution: Unresolved
    • Affects Version/s: 2.7.3
    • Fix Version/s: .next
    • Component/s: library
    • Security Level: Public
    • Environment:
      Membased 1.7.1
      Spylib 2.7.3
      Java 6

      Description

      We have a small round robin pool of MemcachedClient objects.
      On application shutdown we do call shutdown on each MemcachedClient,but it hangs and application shutdown hangs too.
      Do you have any ideas why?

      I have a thread dump

      java.lang.Thread.State: TIMED_WAITING (parking)
      at sun.misc.Unsafe.park(Native Method)

      • parking to wait for <0x0000000700b24368> (a java.util.concurrent.CountDownLatch$Sync)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1011)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1303)
        at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:253)
        at net.spy.memcached.MemcachedClient.waitForQueues(MemcachedClient.java:2120)
        at net.spy.memcached.MemcachedClient.shutdown(MemcachedClient.java:2073)

        Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

          Hide
          akhimich Alexander Khimich added a comment -

          You could probably think that it's waiting for Queues, for async operations.
          But application doesn't use async operations, all calls are sync and app waits for result, to make sure data was persisted.

          When i do call simple client.shutdown(30, TimeUnit.SECONDS) it hangs(see bug description).

          When i added timeout for shutdown call, like:
          future = executor.submit(new Callable<Boolean>() {

          public Boolean call() throws Exception

          { return client.shutdown(30, TimeUnit.SECONDS); }

          });
          future.get(1, TimeUnit.MINUTES);

          Application able to exit with messages :
          WARN 22 Feb 2012 12:35:21,337 [Memcached IO over

          {MemcachedConnection to xxxx}

          - SHUTTING DOWN (waiting)] (?:?) - Closing, and reopening

          {QA sa=xxx, #Rops=1, #Wops=0, #iq=0, topRop=Cmd: 10 Opaque: 224540867, topWop=null, toWrite=0, interested=1}

          , attempt 0.
          WARN 22 Feb 2012 12:35:21,339 [Memcached IO over

          {MemcachedConnection to xxxxx }

          - SHUTTING DOWN (waiting)] (?:?) - Discarding partially completed op: Cmd: 10 Opaque: 224540867

          Cmd:10 it's a NoopOperationImpl
          Does it mean that Noop hangs?

          Show
          akhimich Alexander Khimich added a comment - You could probably think that it's waiting for Queues, for async operations. But application doesn't use async operations, all calls are sync and app waits for result, to make sure data was persisted. When i do call simple client.shutdown(30, TimeUnit.SECONDS) it hangs(see bug description). When i added timeout for shutdown call, like: future = executor.submit(new Callable<Boolean>() { public Boolean call() throws Exception { return client.shutdown(30, TimeUnit.SECONDS); } }); future.get(1, TimeUnit.MINUTES); Application able to exit with messages : WARN 22 Feb 2012 12:35:21,337 [Memcached IO over {MemcachedConnection to xxxx} - SHUTTING DOWN (waiting)] (?:?) - Closing, and reopening {QA sa=xxx, #Rops=1, #Wops=0, #iq=0, topRop=Cmd: 10 Opaque: 224540867, topWop=null, toWrite=0, interested=1} , attempt 0. WARN 22 Feb 2012 12:35:21,339 [Memcached IO over {MemcachedConnection to xxxxx } - SHUTTING DOWN (waiting)] (?:?) - Discarding partially completed op: Cmd: 10 Opaque: 224540867 Cmd:10 it's a NoopOperationImpl Does it mean that Noop hangs?
          Hide
          akhimich Alexander Khimich added a comment -

          Another observation it happens only with spymemcached Vbucket feature http://127.0.0.1:8091/pools
          But doesn't happens when you use node list like 127.0.0.1:11211 127.0.0.1:11212

          Show
          akhimich Alexander Khimich added a comment - Another observation it happens only with spymemcached Vbucket feature http://127.0.0.1:8091/pools But doesn't happens when you use node list like 127.0.0.1:11211 127.0.0.1:11212
          Hide
          daschl Michael Nitschinger added a comment -

          Assigning to .next for now until we have decided how to progress with it.

          Show
          daschl Michael Nitschinger added a comment - Assigning to .next for now until we have decided how to progress with it.

            People

            • Assignee:
              daschl Michael Nitschinger
              Reporter:
              akhimich Alexander Khimich
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:

                Gerrit Reviews

                There are no open Gerrit changes