Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-63026

[BP 7.2.6] Rebalance hung while scaling from 4 to 8 nodes.

    XMLWordPrintable

Details

    • Untriaged
    • 0
    • Yes

    Description

      1. Create a 2 node 16C, 64G columnar cluster.
      2. Create 2 remote collections each ingesting 1B items and wait for ingestion completion
      3. Create 2 more new collections under the same link.
      4. Perform scaling 2 -> 4 -> 8
      5. Last scaling operation seems to be stuck forever. Its been 4+ hours

       

      https://cb-engineering.s3.amazonaws.com/QE/collectinfo-2024-07-09T233227-ns_1%40svc-da-node-002.uwnad3cifi3tr05w.sandbox.nonprod-project-avengers.com.zip

      https://cb-engineering.s3.amazonaws.com/QE/collectinfo-2024-07-09T233227-ns_1%40svc-da-node-005.uwnad3cifi3tr05w.sandbox.nonprod-project-avengers.com.zip

      https://cb-engineering.s3.amazonaws.com/QE/collectinfo-2024-07-09T233227-ns_1%40svc-da-node-006.uwnad3cifi3tr05w.sandbox.nonprod-project-avengers.com.zip

      https://cb-engineering.s3.amazonaws.com/QE/collectinfo-2024-07-09T233227-ns_1%40svc-da-node-007.uwnad3cifi3tr05w.sandbox.nonprod-project-avengers.com.zip

      https://cb-engineering.s3.amazonaws.com/QE/collectinfo-2024-07-09T233227-ns_1%40svc-da-node-008.uwnad3cifi3tr05w.sandbox.nonprod-project-avengers.com.zip

      https://cb-engineering.s3.amazonaws.com/QE/collectinfo-2024-07-09T233227-ns_1%40svc-da-node-009.uwnad3cifi3tr05w.sandbox.nonprod-project-avengers.com.zip

      https://cb-engineering.s3.amazonaws.com/QE/collectinfo-2024-07-09T233227-ns_1%40svc-da-node-010.uwnad3cifi3tr05w.sandbox.nonprod-project-avengers.com.zip

      https://cb-engineering.s3.amazonaws.com/QE/collectinfo-2024-07-09T233227-ns_1%40svc-da-node-011.uwnad3cifi3tr05w.sandbox.nonprod-project-avengers.com.zip

      Attachments

        Issue Links

          is a backport of

          Bug - A problem which impairs or prevents the functions of the product. MB-62668 Rebalance hung while scaling from 4 to 8 nodes.

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed
          is duplicated by

          Bug - A problem which impairs or prevents the functions of the product. MB-63004 [7.2.6 RC1] [System Test] Analytics Rebalance stuck for over 35+ hrs

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed
          For Gerrit Dashboard: MB-63026
          # Subject Branch Project Status CR V
          213781,2 MB-63026: don't clear fixer inbox on rollback response neo analytics-dcp-client Status: MERGED +2 +1
          213887,1 MB-63026: merge branch 'neo' into 'trinity' trinity analytics-dcp-client Status: MERGED +2 +1
          214487,5 MB-63026: [BP 7.6.2] slip analytics-dcp-client to pick up fix master manifest Status: MERGED +2 +1

          Activity

            People

              pulkit.matta Pulkit Matta
              michael.blow Michael Blow
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty