Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-45551

FTS rebalance fails with error message: bad node

    XMLWordPrintable

Details

    • Bug
    • Resolution: Duplicate
    • Major
    • 7.0.0
    • Cheshire-Cat
    • fts
    • 7.0.0-4854
    • Untriaged
    • 1
    • Unknown

    Description

      Test run: 
                      FTS Rebalance-swap (min), 3 -> 3, 20M X 1KB, maxConcurrentPartitionMovesPerNode = 10, partitioned_index = 10, replica = 1
      Number of docs : 20M
      Index has 10 partitions and 1 replica

      The Rebalance doesn't take place and it fails with the following error message: 
      Rebalance exited with reason {config_sync_failed, {error,[

      Unknown macro: {'ns_1@172.23.99.38',bad_node}

      ]}}.

      Link to the job: http://perf.jenkins.couchbase.com/view/FTS/job/atlas_rebalance/181/console
      Logs: https://s3-us-west-2.amazonaws.com/perf-artifacts/jenkins-atlas_rebalance-181/172.23.99.211.zip
      [https://s3-us-west-2.amazonaws.com/perf-artifacts/jenkins-atlas_rebalance-181/172.23.99.38.zip
      ]**[https://s3-us-west-2.amazonaws.com/perf-artifacts/jenkins-atlas_rebalance-181/172.23.99.39.zip
      ] [https://s3-us-west-2.amazonaws.com/perf-artifacts/jenkins-atlas_rebalance-181/172.23.99.40.zip
      ]

      Rebalance report in the cbcollect :

      {{{"stageInfo":{"search":

      Unknown macro: {"startTime"}

      ,"data":{"startTime":"2021-04-08T07:53:07.261-07:00","completedTime":false,"timeTaken":30373}},"rebalanceId":"70e887e0e3af559541f0a39fad1322a9","nodesInfo":{"active_nodes":["ns_1@172.23.99.211","ns_1@172.23.99.38","ns_1@172.23.99.40","ns_1@172.23.99.39"],"keep_nodes":["ns_1@172.23.99.211","ns_1@172.23.99.38","ns_1@172.23.99.40"],"eject_nodes":["ns_1@172.23.99.39"],"delta_nodes":[],"failed_nodes":[]},"masterNode":"ns_1@172.23.99.211","completionMessage":"Rebalance exited with reason {config_sync_failed,\n {error,[

      Unknown macro: {'ns_1@172.23.99.38',bad_node}

      ]}}."}}}

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              abhinav Abhi Dangeti
              jyotsna.nayak Jyotsna Nayak
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty