Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-47363

Cannot rebalance 7.1.0 nodes into 6.x clusters

    XMLWordPrintable

Details

    • Untriaged
    • Centos 64-bit
    • 1
    • Yes

    Description

      What is happening?

      When trying to rebalance 7.1.0-1061 nodes into a 6.x cluster (tested against 6.0 and 6.6), a 500 error is given and the rebalance does not go through.

      What did I expect to happen?

      I expected the rebalance to succeed

      Steps to reproduce

      1. Create a cluster of 6.x nodes
      2. Add nodes running 7.1.0-1061 (any method works)
      3. Rebalance the nodes in

      I've attached a cbcollect from the test against 6.0.0-1693, with a relevant extract here:

      [ns_server:error,2021-07-13T11:44:33.771Z,ns_1@10.112.212.101:<0.28378.0>:menelaus_web:loop:143]Server error during processing: ["web request failed",
                                       {path,"/controller/rebalance"},
                                       {method,'POST'},
                                       {type,exit},
                                       {what,
                                        {noproc,
                                         {gen_fsm,sync_send_all_state_event,
                                          [{via,leader_registry,ns_orchestrator},
                                           {maybe_start_rebalance,
                                            ['ns_1@10.112.212.101',
                                             'ns_1@10.112.212.102',
                                             'ns_1@10.112.212.103',
                                             'ns_1@10.112.212.104'],
                                            ['ns_1@10.112.212.101',
                                             'ns_1@10.112.212.102'],
                                            all}]}}},
                                       {trace,
                                        [{gen_fsm,sync_send_all_state_event,2,
                                          [{file,"gen_fsm.erl"},{line,232}]},
                                         {menelaus_web_cluster,do_handle_rebalance,
                                          4,
                                          [{file,"src/menelaus_web_cluster.erl"},
                                           {line,737}]},
                                         {request_throttler,do_request,3,
                                          [{file,"src/request_throttler.erl"},
                                           {line,59}]},
                                         {menelaus_web,loop,2,
                                          [{file,"src/menelaus_web.erl"},
                                           {line,121}]},
                                         {mochiweb_http,headers,5,
                                          [{file,
                                            "/home/couchbase/jenkins/workspace/couchbase-server-unix/couchdb/src/mochiweb/mochiweb_http.erl"},
                                           {line,94}]},
                                         {proc_lib,init_p_do_apply,3,
                                          [{file,"proc_lib.erl"},{line,239}]}]}]

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          joe.mitchelljones Joe Mitchell Jones created issue -
          joe.mitchelljones Joe Mitchell Jones made changes -
          Field Original Value New Value
          Component/s ns_server [ 10019 ]
          joe.mitchelljones Joe Mitchell Jones made changes -
          Labels rebalance upgrade functional-test rebalance upgrade
          raju Raju Suravarjjala made changes -
          Fix Version/s Neo [ 17615 ]
          raju Raju Suravarjjala made changes -
          Assignee Meni Hillel [ JIRAUSER25407 ]

          For each release, we need to bake in appropriate backwards compatibility mode logic that migrate and upgrade state. It is very ealy in 7.1 for us to build this logic. We have not yet even started building all features. It would be expected that upgrade wouldn't work for non-production builds.

          meni.hillel Meni Hillel (Inactive) added a comment - For each release, we need to bake in appropriate backwards compatibility mode logic that migrate and upgrade state. It is very ealy in 7.1 for us to build this logic. We have not yet even started building all features. It would be expected that upgrade wouldn't work for non-production builds.
          meni.hillel Meni Hillel (Inactive) made changes -
          Assignee Meni Hillel [ JIRAUSER25407 ] Joe Mitchell Jones [ JIRAUSER25420 ]
          meni.hillel Meni Hillel (Inactive) made changes -
          Resolution Not a Bug [ 10200 ]
          Status Open [ 1 ] Resolved [ 5 ]
          ritam.sharma Ritam Sharma made changes -
          VERIFICATION STEPS Closing all duplicates, not a bug and incomplete bugs.
          Status Resolved [ 5 ] Closed [ 6 ]

          People

            joe.mitchelljones Joe Mitchell Jones
            joe.mitchelljones Joe Mitchell Jones
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty