Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-25674

Server-Analytics: Adding, then removing and re-adding the same cbas node in cluster and rebalancing, it failed. 500 error while connecting to cbas service.

    XMLWordPrintable

Details

    • Bug
    • Resolution: Fixed
    • Blocker
    • CBAS DP3
    • CBAS DP3
    • analytics
    • Build: 5.0.0-649

    Description

      1. Initialize a cluster with 1 node, Node1: data, full text, index, query
      2. Add a node, Node2: cbas. Rabalance it(successful)
      3. Remove Node2. Rabalance(Successful)
      4. Add Node2:cbas again. Rebalance and it failed.
      5. In step4, rebalance failed but node gets added into the cluster. Now remove Node2 again and rebalance. Rebalance failed again and Node2 cannot be removed from the cluster anymore.

      Logs attached.

      =========================CRASH REPORT=========================
        crasher:
          initial call: service_rebalancer:-run_rebalance/7-fun-1-/0
          pid: <0.20732.0>
          registered_name: 'service_rebalancer-cbas-worker'
          exception exit: {{lost_connection,shutdown},
                           {gen_server,call,
                            [{'service_agent-cbas','ns_1@172.23.108.233'},
                             {if_rebalance,<0.20711.0>,
                              {start_rebalance,
                               <<"550d9126e57a9ef62cd72d5b2fd27a5c">>,rebalance,
                               [{[{node_id,<<"d5620a4b9c35b2a891252ab761227a29">>},
                                  {priority,0},
                                  {opaque,
                                   {[{<<"ccHttpPort">>,<<"8200">>},
                                     {<<"host">>,<<"172.23.108.233">>},
                                     {<<"iodevices">>,
                                      <<"/opt/couchbase/var/lib/couchbase/data/@analytics/iodevice">>}]}}],
                                 full}],
                               [],<0.20732.0>}},
                             90000]}}
            in function  gen_server:call/3 (gen_server.erl, line 188)
            in call from service_rebalancer:rebalance/8 (src/service_rebalancer.erl, line 116)
          ancestors: ['service_rebalancer-cbas',<0.20578.0>,<0.18244.0>,
                        ns_orchestrator_child_sup,ns_orchestrator_sup,mb_master_sup,
                        mb_master,<0.17040.0>,ns_server_sup,ns_server_nodes_sup,
                        <0.171.0>,ns_server_cluster_sup,<0.89.0>]
          messages: []
          links: [<0.20711.0>]
          dictionary: []
          trap_exit: false
          status: running
          heap_size: 2586
          stack_size: 27
          reductions: 5268
        neighbours:
      

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            ritesh.agarwal Ritesh Agarwal
            ritesh.agarwal Ritesh Agarwal
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty