Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-62348

Cluster stuck in repetitive rebalance state

    XMLWordPrintable

Details

    • Bug
    • Resolution: Duplicate
    • Critical
    • 7.2.4
    • 7.2.2
    • analytics
    • Untriaged
    • 0
    • Unknown

    Description

      I was upgrading capella from 7.2.2 to 7.6.2.

      Upgrade failed and cluster state stuck as `upgrading` : 

      Stats while upgarde : 

      1. 14GB bucket disk usage
      2. Cloud : Azure 
      3. Upgrade Build : 7.6.2-3710

      Rebalance failure message :

      Rebalance exited with reason {service_rebalance_failed,cbas, {worker_died, {'EXIT',<0.16272.347>, {task_failed,rebalance,

      {service_error, <<"Rebalance dc3c34680b29b864dfb485b7f8bb2c09 failed: CBAS0001: Analytics collection GbXmBwazXx.ghWyakAcOZ in different partitions has different DCP states. Seqno gap = 11846. User action: Try again later">>}

      }}}}. Rebalance Operation Id = 3008e9f4c9a1ba16ee1671a6843950b5

       

      Cluster Logs : 
      https://cb-engineering.s3.amazonaws.com/Vipul/collectinfo-2024-06-14T065910-ns_1%40svc-dqisea-node-002.gwklayfuz3a8ydf.sandbox.nonprod-project-avengers.com.zip
      https://cb-engineering.s3.amazonaws.com/Vipul/collectinfo-2024-06-14T065910-ns_1%40svc-dqisea-node-003.gwklayfuz3a8ydf.sandbox.nonprod-project-avengers.com.zip
      https://cb-engineering.s3.amazonaws.com/Vipul/collectinfo-2024-06-14T065910-ns_1%40svc-dqisea-node-004.gwklayfuz3a8ydf.sandbox.nonprod-project-avengers.com.zip
      https://cb-engineering.s3.amazonaws.com/Vipul/collectinfo-2024-06-14T065910-ns_1%40svc-dqisea-node-005.gwklayfuz3a8ydf.sandbox.nonprod-project-avengers.com.zip
      https://cb-engineering.s3.amazonaws.com/Vipul/collectinfo-2024-06-14T065910-ns_1%40svc-dqisea-node-006.gwklayfuz3a8ydf.sandbox.nonprod-project-avengers.com.zip
      https://cb-engineering.s3.amazonaws.com/Vipul/collectinfo-2024-06-14T065910-ns_1%40svc-dqisea-node-007.gwklayfuz3a8ydf.sandbox.nonprod-project-avengers.com.zip

      Attachments

        Issue Links

          duplicates

          Bug - A problem which impairs or prevents the functions of the product. MB-59582 Out of sync DCP states may cause Analytics service to fail rebalances indefinitely

          • Major - Major loss of function.
          • Closed
          is duplicated by

          Bug - A problem which impairs or prevents the functions of the product. MB-62726 Analytics rebalance during upgrade to 7.6.2 is continuously failing

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved
          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              vipul.bhardwaj Vipul Bhardwaj
              vipul.bhardwaj Vipul Bhardwaj
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty