Details
-
Bug
-
Resolution: Unresolved
-
Major
-
7.2.0
-
Enterprise Edition 7.2.0 build 5330
-
Untriaged
-
cbas_rebl_failed → http://supportal.couchbase.com/snapshot/dc2a307855ffacbbeb85ec20daeeb8b0::0
-
0
-
Unknown
Description
Test steps
- Deploy an GCP cluster having 3 KV, GSI-Query and FTS, CBAS nodes each separately.
- Create a Magma bucket having single replica and 1 scope + 2 collections in addition to _default._default keyspace.
- Load 5M docs in each of the 2 collections.
- Create GSI Indexes, wait for the Indexes to come online and run queries against them.
- Increase disk size by 5G for all service groups.
- Online scaling operation goes through fine without any issues.
- Decrease the disk size by 5G for all the service groups. This triggers a swap rebalance for all the nodes one at a time. When it comes to cbas, swap rebalance for cbas failing:
Rebalance exited with reason {service_rebalance_failed,cbas,
|
{worker_died,
|
{'EXIT',<0.15712.37>,
|
{rebalance_failed,
|
{service_error,
|
<<"Rebalance 04e84f10c3df260ab582180f9b550230 failed: CBAS0001: Analytics collections in different partitions have different DCP states. Mutations needed to catch up = 11404155. User action: Try again later">>}}}}}.
|
Rebalance Operation Id = 04d96440e4b72ca88bf6344484fb940a
|
|
QE Test |
sudo guides/gradlew --refresh-dependencies testrunner -P jython=/opt/jython/bin/jython -P 'args=-i /tmp/capella.ini -p bucket_storage=magma,bucket_eviction_policy=fullEviction,rerun=False -t aGoodDoctor.hostedHospital.Murphy.test_rebalance,graceful=True,skip_cleanup=True,num_buckets=1,bucket_names=GleamBook,bucket_type=membase,eviction_policy=fullEviction,iterations=10,batch_size=1000,sdk_timeout=60,log_level=debug,infra_log_level=debug,rerun=False,skip_cleanup=True,key_size=24,randomize_doc_size=False,randomize_value=True,maxttl=10,pc=20,gsi_nodes=3,cbas_nodes=3,fts_nodes=3,kv_nodes=3,n1ql_nodes=3,mutation_perc=100,key_type=RandomKey,capella_run=true,services=data-query:index-search-analytics,max_rebl_nodes=27,kv_compute=n2-standard-4,gsi_compute=n2-standard-4,n1ql_compute=n2-standard-4,fts_compute=n2-standard-4,cbas_compute=n2-standard-4,kv_disk=500,n1ql_disk=50,gsi_disk=500,cbas_disk=500,provider=GCP,region=us-central1,type=PD-SSD,skip_teardown_cleanup=true,wait_timeout=14400,index_timeout=28800,runtype=dedicated,track_failures=True,skip_init=False,key_type=CircularKey,rebalance_type=disk,clients_per_db=1 -m rest'
|