Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-45594

[System Test][couchbase-bucket] Rebalance failures with error buckets_cleanup_failed

    XMLWordPrintable

Details

    • Untriaged
    • 1
    • Unknown
    • KV-Engine 2021-March

    Description

      7.0.0-4910

      Test:
      -test tests/integration/cheshirecat/test_cheshirecat_kv_gsi_coll_xdcr_backup_sgw_fts_itemct_txns_eventing_cbas_scale3.yml -scope tests/integration/cheshirecat/scope_cheshirecat_with_backup.yml
      Scale 3
      Iteration 1

      Two rebalance failures observed with this error:

      2021-04-11T10:26:42.904-07:00, ns_orchestrator:0:critical:message(ns_1@172.23.108.103) - Rebalance exited with reason {buckets_cleanup_failed,
      		 
                                       ['ns_1@172.23.97.239','ns_1@172.23.97.242',
      		 
                                        'ns_1@172.23.97.119']}.
      		 
      Rebalance Operation Id = bdd600f5cddb7bf8fa180aaaac694f63


      2021-04-11T06:55:18.109-07:00, ns_orchestrator:0:critical:message(ns_1@172.23.108.103) - Rebalance exited with reason {buckets_cleanup_failed,['ns_1@172.23.97.119']}.
      		 
      Rebalance Operation Id = 95124008717744bd960d9b7f508dec3e

      Logs:
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1618168029/collectinfo-2021-04-11T190711-ns_1%40172.23.104.137.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1618168029/collectinfo-2021-04-11T190711-ns_1%40172.23.104.155.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1618168029/collectinfo-2021-04-11T190711-ns_1%40172.23.104.157.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1618168029/collectinfo-2021-04-11T190711-ns_1%40172.23.104.5.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1618168029/collectinfo-2021-04-11T190711-ns_1%40172.23.104.67.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1618168029/collectinfo-2021-04-11T190711-ns_1%40172.23.104.69.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1618168029/collectinfo-2021-04-11T190711-ns_1%40172.23.105.107.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1618168029/collectinfo-2021-04-11T190711-ns_1%40172.23.105.111.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1618168029/collectinfo-2021-04-11T190711-ns_1%40172.23.106.100.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1618168029/collectinfo-2021-04-11T190711-ns_1%40172.23.106.188.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1618168029/collectinfo-2021-04-11T190711-ns_1%40172.23.108.103.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1618168029/collectinfo-2021-04-11T190711-ns_1%40172.23.120.245.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1618168029/collectinfo-2021-04-11T190711-ns_1%40172.23.121.3.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1618168029/collectinfo-2021-04-11T190711-ns_1%40172.23.123.27.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1618168029/collectinfo-2021-04-11T190711-ns_1%40172.23.123.28.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1618168029/collectinfo-2021-04-11T190711-ns_1%40172.23.96.148.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1618168029/collectinfo-2021-04-11T190711-ns_1%40172.23.96.251.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1618168029/collectinfo-2021-04-11T190711-ns_1%40172.23.96.252.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1618168029/collectinfo-2021-04-11T190711-ns_1%40172.23.96.253.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1618168029/collectinfo-2021-04-11T190711-ns_1%40172.23.97.119.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1618168029/collectinfo-2021-04-11T190711-ns_1%40172.23.97.121.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1618168029/collectinfo-2021-04-11T190711-ns_1%40172.23.97.122.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1618168029/collectinfo-2021-04-11T190711-ns_1%40172.23.97.239.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1618168029/collectinfo-2021-04-11T190711-ns_1%40172.23.97.242.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1618168029/collectinfo-2021-04-11T190711-ns_1%40172.23.98.135.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1618168029/collectinfo-2021-04-11T190711-ns_1%40172.23.99.11.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1618168029/collectinfo-2021-04-11T190711-ns_1%40172.23.99.20.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1618168029/collectinfo-2021-04-11T190711-ns_1%40172.23.99.21.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1618168029/collectinfo-2021-04-11T190711-ns_1%40172.23.99.25.zip

      Attachments

        Issue Links

          duplicates

          Bug - A problem which impairs or prevents the functions of the product. MB-46099 130 node testing: Rebalance in 1 node in a 85 node cluster failed due to buckets_cleanup_failed

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed
          is duplicated by

          Bug - A problem which impairs or prevents the functions of the product. MB-45429 [System Test] Rebalance exited with timeout error

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MB-46067 [Collections] - Rebalance in fails with Rebalance exited with reason {buckets_cleanup_failed,

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MB-46099 130 node testing: Rebalance in 1 node in a 85 node cluster failed due to buckets_cleanup_failed

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed
          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              Aliaksey Artamonau Aliaksey Artamonau (Inactive)
              arunkumar Arunkumar Senthilnathan (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty