Uploaded image for project: 'Couchbase Java Client'
  1. Couchbase Java Client
  2. JCBC-33

Reconfiguration strategy used in TapConnectionProvider can lead to temporary deadlock of the ConfigurationProvider thread


    • Type: Bug
    • Status: Open
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: 1.0.2
    • Fix Version/s: .backlog1.x
    • Component/s: Core
    • Security Level: Public
    • Labels:
    • Environment:
      tested on linux


      Let's assume for now that a new node has been added to the cluster.

      1. When a configuration change is detected, the reconfigure() method of TapConnectionProvider is called.
      2. TapConnectionProvider will call ((CouchbaseConnection)conn).reconfigure(bucket);
      3. Inside CouchbaseConnection reconfigure() the new server will be found and added to the list newServers
      4. Then createConnections(newServers) will be called in the parent class MemcachedConnection
      5. Depending on the log level you'll see the message logged from getLogger().info("Added %s to connect queue", qa);
      6. Then the code will hang at the line: https://github.com/couchbase/spymemcached/blob/master/src/main/java/net/spy/memcached/MemcachedConnection.java#L155

      qa.setSk(ch.register(selector, ops, qa));

      The code will hang until another packet is received on the channel. This is the expected behavior in Java NIO. The recommended practice is perform registrations from the same thread as selects. In this case we're registering from the thread that was monitoring for configuration changes, and selecting from the main run loop of the MemcachedConnection.

      See http://stackoverflow.com/questions/1057224/thread-is-stuck-while-registering-channel-with-selector-in-java-nio-server

      It's not a huge problem for us, because even in an idle situation we eventually receive a NOOP. However, this combined with another bug I was hitting and made it really hard to troubleshoot. I'd suggest we look at ways to avoid this problem.

      It's also possible this is a bug in Spy and not the Java client, but I haven't studied how createConnections() is used in other contexts within spy, so it might just be how we use it when reconfiguring from the java client.

      No reviews matched the request. Check your Options in the drop-down menu of this sections header.


        There are no comments yet on this issue.


          • Assignee:
            daschl Michael Nitschinger
            mschoch Marty Schoch
          • Votes:
            0 Vote for this issue
            0 Start watching this issue


            • Created:

              Gerrit Reviews

              There are no open Gerrit changes