Uploaded image for project: 'Couchbase C client library libcouchbase'
  1. Couchbase C client library libcouchbase
  2. CCBC-192

Failure to handle host that has been removed from a cluster

    XMLWordPrintable

Details

    • Improvement
    • Resolution: Fixed
    • Blocker
    • 2.1.1
    • 2.0.4
    • library
    • Security Level: Public
    • None
    • Linux, Ubuntu 12.04 Precise

    Description

      We're encountering an issue where both the Ruby and Perl clients get() method will consistently, repeatedly just return an error. On the Perl client it is "Temporary error. try again later", and on Ruby it is "Bucket not found".

      This occurs when we have several couchbase servers, and have connected using the node list option to supply all of them to the library.

      We remove the first server in the node list via the Couchbase GUI, trigger a rebalance, and wait for it to complete.
      Once the rebalance has completed, we start a script that simply attempts to connect and retrieve some keys we previously inserted. Instead of success, we find that it repeatedly gives the errors mentioned above.

      note that if we remain connected to the cluster, then everything seems fine. The errors only start occurring when a client connects.

      note also that the issue only occurs if the first server (or servers) in the list are down; if the first server listed is currently actively in the cluster, then we're OK.

      It seems like the problem is that the now-removed server is still accepting connections on port 8091, and thus the client library thinks that this means it is a valid server. However because it's removed from the pool, it causes confusion and errors to the client.

      Obviously, the correct/desired behaviour would be for the client library to behave as if the removed-server was not accepting connections, and to move onto the next server in the node list.

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              avsej Sergey Avseyev
              wintrmute Toby Corkindale
              Votes:
              2 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty