Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-57943

[AWS, Collocated]: Cluster upgrade from 7.1.4-3632 failed due to analytics rebalance failure.

    XMLWordPrintable

Details

    • Bug
    • Resolution: Won't Do
    • Blocker
    • None
    • 7.2.1
    • analytics
    • 7.1.4-3632-v1.0.14 -> 7.2.1-5855-v1.0.19 Upgrade

    Description

      1. Create a 3 node all services co-located 7.1.4 cluster. Create a bucket, 2 collections, and load 75M items in each collection
      2. Create GSI/FTS indexes, CBAS datasets/indexes. Let ingestion for all to complete.
      3. Start 5000/s kv mutations with 80% reads and 20% sets
      4. Start the query load for all the services.
      5. Upgrade cluster from 7.1.4->7.2.1
      6. Upgrade triggers a swap rebalance and it failed due to analytics.

      Test

       
      [global]
      username:root
      password:couchbase
      port:18091
      n1ql_port:18093
      index_port:19102
      eventing_port:18096
       
      [capella]
      pod:cloudapi.qe-7.sandbox.nonprod-project-avengers.com
      tenant_id:c2cbbba7-fbea-482f-a92e-88471e3425ad
      project:defa4f2c-2e72-470f-9529-e1500e48ebe9
      capella_user:ritesh.agarwal+004@couchbase.com
      capella_pwd:Couch@123
      clusters:d41b59de-836c-4d9f-9e11-998ed8ecc9ad
       
      override_token:cb-645AD509-7639-4EE8-B58E-2DE04EFCE1CB
       
      upgrade_image:couchbase-cloud-server-7.2.1-5855-v1.0.19
      upgrade_server_version:7.2.1
      upgrade_release_id:1.0.19
      override_key:the-secret-test-override-key
      image:couchbase-cloud-server-7.1.4-3632-v1.0.14
      server_version:7.1.4
      release_id:1.0.18
       
      secret_key:GqMeLJhUnjFtC!%JY#PV4!6jTrlFy#yLRLsXGVOu91tg!!BlXGAF9BHKcerLgtwc
      access_key:4gTFwgY2gtVW94Dms0ZNKw6DsQPXhIMl
       
      [membase]
      rest_username:Administrator
      rest_password:Couch@123
       
      sudo guides/gradlew --refresh-dependencies testrunner -P jython=/opt/jython/bin/jython -P 'args=-i /tmp/couchbase_capella_volume_2_new.ini -p bucket_storage=magma,bucket_eviction_policy=fullEviction,rerun=False -t aGoodDoctor.hostedHospital.Murphy.test_rebalance,num_items=75000000,num_buckets=1,bucket_names=GleamBook,bucket_type=membase,iterations=5,batch_size=1000,sdk_timeout=60,log_level=debug,infra_log_level=debug,rerun=False,skip_cleanup=True,key_size=18,randomize_doc_size=False,randomize_value=True,maxttl=10,pc=20,gsi_nodes=3,cbas_nodes=3,fts_nodes=3,kv_nodes=3,n1ql_nodes=3,kv_disk=500,n1ql_disk=50,gsi_disk=500,fts_disk=500,cbas_disk=500,kv_compute=m5.4xlarge,gsi_compute=m5.4xlarge,n1ql_compute=m5.4xlarge,fts_compute=m5.4xlarge,cbas_compute=m5.4xlarge,mutation_perc=20,key_type=CircularKey,capella_run=true,services=data:query:index:analytics:search,max_rebl_nodes=27,provider=AWS,region=us-east-1,type=GP3,size=500,skip_teardown_cleanup=true,wait_timeout=14400,index_timeout=28800,runtype=dedicated,skip_init=true -m rest'
      

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              shelby.ramsey Shelby Ramsey
              ritesh.agarwal Ritesh Agarwal
              Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty