Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-51334

[System Test]Rebalance exited with reason {service_rebalance_failed,fts - worker_died - inactivity_timeout

    XMLWordPrintable

Details

    • Untriaged
    • 1
    • Unknown

    Description

      Build: 7.1.0-2436
      Test: -test tests/fts/cheshire-cat/test_fts_clusterops_cheshire_cat_coll_crud.yml -scope tests/fts/cheshire-cat/scope_fts_cheshire_cat.yml
      Scale: 1
      Test Cycle: 7

      Swap Rebalance fts node failed with below:

      2022-03-06T06:17:11.118-08:00, ns_orchestrator:0:critical:message(ns_1@172.23.106.253) - Rebalance exited with reason {service_rebalance_failed,fts,
                                    {worker_died,
                                     {'EXIT',<0.25528.919>,
                                      {rebalance_failed,inactivity_timeout}}}}.
      Rebalance Operation Id = e045a6badcdb3ba73c13036f7b7c87b7
      

      Test Step:

       
      [2022-03-06T05:59:29-08:00, sequoiatools/couchbase-cli:7.1:7b3b44] server-add -c 172.23.106.253:8091 --server-add https://172.23.107.89 -u Administrator -p password --server-add-username Administrator --server-add-password password --services fts
      [pull] sequoiatools/couchbase-cli:7.1
      [2022-03-06T05:59:41-08:00, sequoiatools/couchbase-cli:7.1:55acf5] rebalance -c 172.23.106.253:8091 --server-remove 172.23.105.186 -u Administrator -p password
       
      Error occurred on container - sequoiatools/couchbase-cli:7.1:[rebalance -c 172.23.106.253:8091 --server-remove 172.23.105.186 -u Administrator -p password]
       
      docker logs 55acf5
      docker start 55acf5
       
      *Unable to display progress bar on this os
      JERROR: Rebalance failed. See logs for detailed reason. You can try again.
      [pull] sequoiatools/cmd
      [2022-03-06T06:17:19-08:00, sequoiatools/cmd:88272f] 60
      

      Cluster config:

      ########## Cluster config ##################
      ######  n1ql : 2 ===== > [172.23.105.185:8091 172.23.106.182:8091]  ###########
      ######  fts : 7 ===== > [172.23.105.186:8091 172.23.105.190:8091 172.23.106.154:8091 172.23.106.243:8091 172.23.107.89:8091 172.23.97.213:8091 172.23.97.214:8091]  ###########
      ######  kv : 3 ===== > [172.23.106.242:8091 172.23.106.253:8091 172.23.106.255:8091]  ###########
      

      Logs:
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1646580677/collectinfo-2022-03-06T153118-ns_1%40172.23.105.185.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1646580677/collectinfo-2022-03-06T153118-ns_1%40172.23.105.186.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1646580677/collectinfo-2022-03-06T153118-ns_1%40172.23.105.190.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1646580677/collectinfo-2022-03-06T153118-ns_1%40172.23.106.182.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1646580677/collectinfo-2022-03-06T153118-ns_1%40172.23.106.242.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1646580677/collectinfo-2022-03-06T153118-ns_1%40172.23.106.243.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1646580677/collectinfo-2022-03-06T153118-ns_1%40172.23.106.253.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1646580677/collectinfo-2022-03-06T153118-ns_1%40172.23.106.255.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1646580677/collectinfo-2022-03-06T153118-ns_1%40172.23.107.89.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1646580677/collectinfo-2022-03-06T153118-ns_1%40172.23.97.213.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1646580677/collectinfo-2022-03-06T153118-ns_1%40172.23.97.214.zip

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              Sreekanth Sivasankaran Sreekanth Sivasankaran (Inactive)
              girish.benakappa Girish Benakappa
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty