Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-6507

adding new node to cluster while rebalance is running causes rebalance to fail with detected_nodes_change condition

    XMLWordPrintable

Details

    • Bug
    • Resolution: Duplicate
    • Major
    • feature-backlog
    • 2.0-beta
    • ns_server
    • Security Level: Public
    • None

    Description

      build-1666
      steps:
      1. node in cluster - 10.3.121.92
      2. rebalance in 10.5.2.11 and start rebalance
      3. when rebalance is in progress from api initialize new rebalance with params:

      password=password&ejectedNodes=&user=Administrator&knownNodes=ns_1%4010.3.121.93%2Cns_1%4010.3.121.94%2Cns_1%4010.5.2.11%2Cns_1%4010.3.121.92%2Cns_1%4010.3.121.95

      as a result the first rebalance is failed:

      Rebalance exited with reason {detected_nodes_change,
      {ns_node_disco_events,
      ['ns_1@10.3.121.92','ns_1@10.5.2.11'],
      ['ns_1@10.3.121.92','ns_1@10.3.121.93',
      'ns_1@10.5.2.11']}}

      [error_logger:error,2012-08-31T12:27:47.902,ns_1@10.3.121.92:error_logger:ale_error_logger_handler:log_report:72]
      =========================CRASH REPORT=========================
      crasher:
      initial call: ns_vbucket_mover:init/1
      pid: <0.31823.0>
      registered_name: []
      exception exit: {detected_nodes_change,
      {ns_node_disco_events,
      ['ns_1@10.3.121.92','ns_1@10.5.2.11'],
      ['ns_1@10.3.121.92','ns_1@10.3.121.93',
      'ns_1@10.5.2.11']}}
      in function gen_server:terminate/6
      ancestors: [<0.31774.0>]
      messages: [{'EXIT',<0.936.1>,
      {detected_nodes_change,

      {ns_node_disco_events, ['ns_1@10.3.121.92','ns_1@10.5.2.11'], ['ns_1@10.3.121.92','ns_1@10.3.121.93', 'ns_1@10.5.2.11']}

      }}]
      links: [<0.31774.0>,<0.31827.0>,<0.57.0>]
      dictionary: [

      {bucket_name,"standard_bucket0"}

      ,

      {i_am_master_mover,true}

      ,

      {child_processes,[<0.936.1>,<0.878.1>,<0.833.1>,<0.800.1>, <0.763.1>,<0.727.1>,<0.693.1>,<0.650.1>, <0.614.1>,<0.581.1>,<0.547.1>,<0.511.1>, <0.478.1>,<0.448.1>,<0.427.1>,<0.406.1>, <0.385.1>,<0.364.1>,<0.340.1>,<0.319.1>, <0.298.1>,<0.275.1>,<0.248.1>,<0.227.1>, <0.206.1>,<0.185.1>,<0.164.1>,<0.143.1>, <0.122.1>,<0.101.1>,<0.77.1>,<0.56.1>, <0.35.1>,<0.14.1>,<0.32761.0>,<0.32740.0>, <0.32719.0>,<0.32698.0>,<0.32677.0>, <0.32656.0>,<0.32635.0>,<0.32614.0>, <0.32593.0>,<0.32568.0>,<0.32547.0>, <0.32526.0>,<0.32505.0>,<0.32483.0>, <0.32461.0>,<0.32435.0>,<0.32414.0>, <0.32390.0>,<0.32369.0>,<0.32348.0>, <0.32326.0>,<0.32305.0>,<0.32284.0>, <0.32259.0>,<0.32235.0>,<0.32214.0>, <0.32193.0>,<0.32172.0>,<0.32148.0>, <0.32127.0>,<0.32106.0>,<0.32085.0>, <0.32064.0>,<0.32043.0>,<0.32022.0>, <0.32001.0>,<0.31980.0>,<0.31959.0>, <0.31937.0>,<0.31916.0>,<0.31895.0>, <0.31871.0>,<0.31850.0>,<0.31829.0>]}

      ]
      trap_exit: true
      status: running
      heap_size: 6765
      stack_size: 24
      reductions: 609293
      neighbours:
      neighbour: [

      {pid,<0.31774.0>}

      ,

      {registered_name,[]}

      ,
      {initial_call,{erlang,apply,2}},
      {current_function,{ns_rebalancer,wait_for_mover_tail,2}},

      {ancestors,[]}

      ,

      {messages,[]}

      ,

      {links,[<0.31291.0>,<0.31823.0>]}

      ,
      {dictionary,[{random_seed,{8712,26823,9965}}]},

      {trap_exit,false}

      ,

      {status,waiting}

      ,

      {heap_size,28657}

      ,

      {stack_size,28}

      ,

      {reductions,175181}

      ]

      Attachments

        1. 10.3.121.92-8091-diag.txt.gz
          1.10 MB
          Andrei Baranouski
        2. 10.3.121.93-8091-diag.txt.gz
          841 kB
          Andrei Baranouski
        3. 10.3.121.94-8091-diag.txt.gz
          843 kB
          Andrei Baranouski
        4. 10.3.121.95-8091-diag.txt.gz
          838 kB
          Andrei Baranouski
        5. 10.5.2.11-8091-diag.txt.gz
          1.39 MB
          Andrei Baranouski
        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            alkondratenko Aleksey Kondratenko (Inactive)
            andreibaranouski Andrei Baranouski
            Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty