Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-37167

Rebalance multiple kv nodes failed with Some apps are deploying or resuming on some or all Eventing nodes

    XMLWordPrintable

Details

    • Untriaged
    • Yes

    Description

      build: 6.5.0-4917 , last passed on 6.5.0-4908

      • Create 5 node cluster 3 kv , 1 eventing , 1 index and n1ql
      • Deploy SBM handler with timer
      • Start data loading in source bucket
      • Add 2 more kv nodes – failed
      • Verify results in source bucket
      • Remove above added nodes from the cluster

      ./testrunner -i /tmp/testexec.17696.ini -p get-cbcollect-info=True,GROUP=source_bucket_mutation_timers -t eventing.eventing_rebalance.EventingRebalance.test_eventing_rebalance_with_multiple_kv_nodes,doc-per-day=5,dataset=default,nodes_init=5,services_init=kv-kv-kv-eventing-index:n1ql,groups=simple,reset_services=True,handler_code=source_bucket_mutation_with_timers,source_bucket_mutation=True,GROUP=source_bucket_mutation_timers 

      2019-12-04 21:39:56,138] - [rest_client:3331] ERROR - {u'node': u'ns_1@172.23.104.238', u'code': 0, u'text': u'Rebalance exited with reason {service_rebalance_failed,eventing,\n                              {worker_died,\n                               {\'EXIT\',<0.29602.2>,\n                                {{badmatch,\n                                  {error,\n                                   {unknown_error,\n                                    <<"Some apps are deploying or resuming on some or all Eventing nodes">>}}},\n                                 [{service_rebalancer,rebalance_worker,1,\n                                   [{file,"src/service_rebalancer.erl"},\n                                    {line,170}]},\n                                  {proc_lib,init_p,3,\n                                   [{file,"proc_lib.erl"},{line,232}]}]}}}}.\nRebalance Operation Id = 6868fa7d4f59e481b6b7553fb2b8c450', u'shortText': u'message', u'serverTime': u'2019-12-04T21:39:48.893Z', u'module': u'ns_orchestrator', u'tstamp': 1575524388893, u'type': u'critical'}
       
       

      Attachments

        Issue Links

          For Gerrit Dashboard: MB-37167
          # Subject Branch Project Status CR V

          Activity

            People

              satya.nand Satya Nand (Inactive)
              vikas.chaudhary Vikas Chaudhary
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty