Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-5602

auto-failover fails over a node if some of the buckets are already rebalanced out but rebalance has been stopped or interrupted ( auto-failover should failover if all buckets are down)


    • Type: Bug
    • Status: Closed
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: 1.8.1-release-candidate
    • Fix Version/s: 1.8.1
    • Component/s: ns_server
    • Security Level: Public
    • Labels:
    • Environment:
      18 node cluster, Centos
      Build 181-918
      2 buckets, 1024vbuckets


      1.Setup a 18 node cluster with 2 buckets- bucket1, bucket2
      2. Enable auto-failover
      3. Add a new node 126
      4. Rebalance

      1. Rebalance works fine. But seeing these log messages -

      Could not automatically failover node 'ns_1@<ns_1@><ns_1@<ns_1@>>' because I think rebalance is running auto_failover000 ns_1@<ns_1@><ns_1@<ns_1@>> 19:32:12 - Sun Jun 17, 2012
      Bucket "bucket1" loaded on node 'ns_1@<ns_1@><ns_1@<ns_1@>>' in 0 seconds. ns_memcached001 ns_1@<ns_1@><ns_1@<ns_1@>> 19:32:04 - Sun Jun 17, 2012
      Started rebalancing bucket bucket2 ns_rebalancer000 ns_1@<ns_1@><ns_1@<ns_1@>> 19:31:36 - Sun Jun 17, 2012
      Starting rebalance, KeepNodes = ['ns_1@<ns_1@><ns_1@<ns_1@>>','ns_1@<ns_1@><ns_1@<ns_1@>>',
      'ns_1@<ns_1@><ns_1@<ns_1@>>'], EjectNodes = []

      Attached are the web-logs and logs from master node-104.


      Other related conversation
      I have enabled auto-failover on the large-cluster and every time I rebalance In a node, I get an error message showing " Could not automatically failover node 'ns_1@<ns_1@><ns_1@<ns_1@>>' because I think rebalance is running" .
      The node 126 is newly added and rebalance issued, is this message displayed because the node is not yet ready to join the cluster ?
      The rebalance works fine, but I do not understand why is auto-failover attempted in here. Any idea?

      No. according to logs at 19:32:04 bucket1 was loaded. Maybe there are some other buckets that are still not ready on this node. May I have logs?



          • Assignee:
            alkondratenko Aleksey Kondratenko (Inactive)
            ketaki Ketaki Gangal
          • Votes:
            0 Vote for this issue
            1 Start watching this issue


            • Created:

              Gerrit Reviews

              There are no open Gerrit changes