Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-49215

[ARM][System Test] rebalance failure observed in component level test - buckets_shutdown_wait_failed

    XMLWordPrintable

Details

    • Untriaged
    • 1
    • No

    Description

      7.1.0-1572

      Test:
      -test tests/rebalance/test_allRebalance_collections.yml -scope tests/rebalance/scope_rebalance_collection.yml
      Scale 2

      Following rebalance exit observed in diag.log of .18.170:

      2021-10-27T13:35:54.092Z, ns_rebalancer:0:critical:message(ns_1@172.31.18.170) - Failed to wait deletion of some buckets on some nodes: [{'ns_1@172.31.18.172',
                                                               {'EXIT',
                                                                {old_buckets_shutdown_wait_failed,
                                                                 ["bucket9",
                                                                  "bucket8",
                                                                  "bucket7",
                                                                  "bucket6",
                                                                  "bucket5",
                                                                  "bucket4",
                                                                  "bucket3",
                                                                  "bucket2",
                                                                  "default"]}}}]
       
      2021-10-27T13:35:54.093Z, ns_orchestrator:0:critical:message(ns_1@172.31.18.170) - Rebalance exited with reason {buckets_shutdown_wait_failed,
                                    [{'ns_1@172.31.18.172',
                                      {'EXIT',
                                       {old_buckets_shutdown_wait_failed,
                                        ["bucket9","bucket8","bucket7","bucket6",
                                         "bucket5","bucket4","bucket3","bucket2",
                                         "default"]}}}]}.
      Rebalance Operation Id = 17f1b5233746078450b971127d6d84be
      

      Logs:
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1635349113/collectinfo-2021-10-27T153833-ns_1%40172.31.17.156.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1635349113/collectinfo-2021-10-27T153833-ns_1%40172.31.17.66.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1635349113/collectinfo-2021-10-27T153833-ns_1%40172.31.18.170.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1635349113/collectinfo-2021-10-27T153833-ns_1%40172.31.18.172.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1635349113/collectinfo-2021-10-27T153833-ns_1%40172.31.21.95.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1635349113/collectinfo-2021-10-27T153833-ns_1%40172.31.22.132.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1635349113/collectinfo-2021-10-27T153833-ns_1%40172.31.22.32.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1635349113/collectinfo-2021-10-27T153833-ns_1%40172.31.25.241.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1635349113/collectinfo-2021-10-27T153833-ns_1%40172.31.26.148.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1635349113/collectinfo-2021-10-27T153833-ns_1%40172.31.29.243.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1635349113/collectinfo-2021-10-27T153833-ns_1%40172.31.30.96.zip

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              arunkumar Arunkumar Senthilnathan (Inactive)
              arunkumar Arunkumar Senthilnathan (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty