Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-6385

Rebalance 5->4 nodes is failed with reason not_all_nodes_are_ready_yet

    XMLWordPrintable

Details

    • Bug
    • Resolution: Fixed
    • Blocker
    • 2.0-beta
    • None
    • ns_server
    • Security Level: Public
    • centOS, 64 -bit, 4 cores VMs, build #1620

    Description

      1.Rebalance in 1->5 nodes
      2. Load data (1M), no views or ddocs are created,
      data doc is like: {{ "age": some_integer, "first_name": some_string }}
      3. Start rebalance out
      4. Created 2 ddocs, 3 view per ddoc, map function: 'function (doc)

      { emit(null, doc);}

      '
      ddoc_40->views0, views1,
      ddoc_41->views0, views1,
      ddoc_42->views0,views1
      5. Rebalance is failed

      2012-08-22 18:47:09.143 menelaus_sup:1:info:web start ok(ns_1@10.3.3.73) - Couchbase Server has started on web port 8091 on node 'ns_1@10.3.3.73'.
      2012-08-22 18:47:09.143 ns_node_disco:4:info:node up(ns_1@10.3.3.73) - Node 'ns_1@10.3.3.73' saw that node 'ns_1@10.3.3.71' came up.
      2012-08-22 18:47:09.214 ns_cluster:3:info:message(ns_1@10.3.3.73) - Node ns_1@10.3.3.73 joined cluster
      2012-08-22 18:47:09.249 ns_orchestrator:4:info:message(ns_1@10.3.3.58) - Starting rebalance, KeepNodes = ['ns_1@10.3.3.73','ns_1@10.3.3.68',
      'ns_1@10.3.3.58','ns_1@10.3.3.71'], EjectNodes = ['ns_1@10.3.3.64']

      2012-08-22 18:47:09.331 ns_rebalancer:0:info:message(ns_1@10.3.3.58) - Started rebalancing bucket default
      2012-08-22 18:47:09.591 ns_memcached:2:info:message(ns_1@10.3.3.73) - Shutting down bucket "default" on 'ns_1@10.3.3.73' for server shutdown
      2012-08-22 18:48:09.080 ns_memcached:2:info:message(ns_1@10.3.3.73) - Shutting down bucket "default" on 'ns_1@10.3.3.73' for server shutdown (repeated 1544 times)
      2012-08-22 18:48:09.105 ns_memcached:2:info:message(ns_1@10.3.3.73) - Shutting down bucket "default" on 'ns_1@10.3.3.73' for server shutdown
      2012-08-22 18:48:09.418 ns_orchestrator:2:info:message(ns_1@10.3.3.58) - Rebalance exited with reason

      {not_all_nodes_are_ready_yet,['ns_1@10.3.3.73']}

      Attachments

        1. 10.3.3.58-8091-diag.txt.gz
          15.72 MB
        2. 10.3.3.64-8091-diag.txt.gz
          6.19 MB
        3. 10.3.3.71-8091-diag.txt.gz
          5.83 MB
        4. 10.3.3.68-8091-diag.txt.gz
          14.42 MB
        5. 10.3.3.73-8091-diag.txt.gz
          3.59 MB
        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            iryna iryna
            iryna iryna
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty