Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-44824

Investigate chronicle timeout (was: [System Test] : Rebalance exited with reason - config_sync_failed, bad_node)

    XMLWordPrintable

Details

    • 1

    Description

      Build : 7.0.0-4626
      Test : -test tests/integration/cheshirecat/test_cheshirecat_kv_gsi_coll_xdcr_backup_sgw_fts_itemct_txns_eventing_cbas.yml -scope tests/integration/cheshirecat/scope_cheshirecat_with_backup.yml
      Scale : 2
      Iteration : 1st

      In the longevity test, there is a step to add 2 indexer nodes to the cluster.

      [2021-03-08T22:05:25-08:00, sequoiatools/couchbase-cli:7.0:667967] server-add -c 172.23.108.103:8091 --server-add https://172.23.104.67 -u Administrator -p password --server-add-username Administrator --server-add-password password --services index
      [2021-03-08T22:06:04-08:00, sequoiatools/couchbase-cli:7.0:5f8455] server-add -c 172.23.108.103:8091 --server-add https://172.23.105.107 -u Administrator -p password --server-add-username Administrator --server-add-password password --services index
      [2021-03-08T22:06:16-08:00, sequoiatools/couchbase-cli:7.0:cfcf8c] rebalance -c 172.23.108.103:8091 -u Administrator -p password
       
      Error occurred on container - sequoiatools/couchbase-cli:7.0:[rebalance -c 172.23.108.103:8091 -u Administrator -p password]
       
      docker logs cfcf8c
      docker start cfcf8c
       
      *Unable to display progress bar on this os
      JERROR: Rebalance failed. See logs for detailed reason. You can try again.
      [2021-03-08T22:06:59-08:00, sequoiatools/cmd:93402d] 60
      

      This step failed due to the following error -

      [user:error,2021-03-08T22:06:51.978-08:00,ns_1@172.23.108.103:<0.22677.0>:ns_orchestrator:log_rebalance_completion:1407]Rebalance exited with reason {config_sync_failed,
                                       {error,
                                           [{'ns_1@172.23.97.119',bad_node},
                                            {'ns_1@172.23.97.239',bad_node}]}}.
      Rebalance Operation Id = 1051b05366c69968c11d5111a775d7d3
      

      From the logs on 172.23.97.119 & 172.23.97.239, it could not be determined why these nodes were deemed to be "bad nodes"

      Logs to be uploaded soon.

      Attachments

        Issue Links

          For Gerrit Dashboard: MB-44824
          # Subject Branch Project Status CR V

          Activity

            People

              Aliaksey Artamonau Aliaksey Artamonau (Inactive)
              mihir.kamdar Mihir Kamdar (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty