Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-62780

After scaling down cluster being unusable for ~50 Minutes

    XMLWordPrintable

Details

    Description

      Steps we did:

      1. Create a 2 node 8vCPU, 32GiB and a capella provisioned cluster with 2 buckets
      		 
      2. Create 5 remote links, 50 remote collection, 1 S3 link and 1 S3 collection.
      		 
      3. Start data load on provisioned with 20GB per bucket - Total 1TB in columnar and scale the columnar instance to 8 nodes
      		 
      4. Wait for ingestion complete and take a backup of the columnar instance
      		 
      5. Restore the backup
      		 
      6. Start data load on provisioned to reach 40GB per bucket - Total 2TB in columnar
      		 
      7. Wait for load to complete and take the backup
      		 
      8. Start data load on provisioned with 60GB per bucket - Total 3TB in columnar
      		 
      9. Wait for ingestion complete and restore with 2TB backup
      		 
      10. Wait for ingestion complete (columnar have 3TB after this step)
      		 
      11.  Start a backup job
      		 
      12. While backup is in progress scale down the instance to 2 nodes.
      		 
      The cluster becomes unusable for 50 minutes after the scaling is complete

      It can be because of the cc node get rebalanced out
      References: https://issues.couchbase.com/browse/MB-62680?focusedId=785211&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-785211

      CC Node Ticket: https://couchbasecloud.atlassian.net/browse/AV-82021

      Attachments

        Issue Links

          duplicates

          Bug - A problem which impairs or prevents the functions of the product. MB-62680 [System Test] [Intermittent] Cluster unusable for a brief period after scale down operations

          • Major - Major loss of function.
          • Open
          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              abhay.aggrawal Abhay Aggrawal
              abhay.aggrawal Abhay Aggrawal
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty