Details
-
Bug
-
Resolution: Cannot Reproduce
-
Critical
-
None
-
7.1.5
-
7.1.5
-
Untriaged
-
0
-
Unknown
Description
- Create a 6 nodes(c5.2xlarge) colocated services cluster.
- Create bucket, 2 collections, 50M items in each.
- Create cbas datasets. Start the CBAS query load
- Create GSI indexes. Start the n1ql query load
- Create FTS indexes. While FTS indexes are being built very few queries saw:
2023-07-31 16:08:39,234 | infra | CRITICAL | query_thread_default08 | [hostedN1QL:_run_query:465] com.couchbase.client.core.error.RequestCanceledException: QueryRequest, Reason: NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT) {"cancelled":true,"completed":true,"coreId":"0x57f4b0c000000001","idempotent":false,"lastDispatchedFrom":"172.23.120.26:49828","lastDispatchedTo":"svc-dqisa-node-004.iox3on3eipylahap.sandbox.nonprod-project-avengers.com:18093","reason":"NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT)","requestId":100129848,"requestType":"QueryRequest","retried":0,"service":{"bucket":"default0","operationId":"query_thread_default082540","scope":"_default","statement":"select meta().id from VolumeCollection0 where country is not null and `type` is not null and (any r in reviews satisfies r.ratings.`Check in / front desk` is not null end) limit 100","type":"query"},"timeoutMs":75000,"timings":{"totalMicros":3619212}}4 out of 106563 queries ran so far have seen this exception.
- Same is being observed during cluster scale up from 6 to 9 nodes.
2023-07-31 16:58:27,657 | test | INFO | Thread-1 | [table_view:display:72] N1QL Query Statistics+----------+---------------+----------------+-----------------+------------------+-------------------+-----------------+-----------------+| Bucket | Total Queries | Failed Queries | Success Queries | Rejected Queries | Cancelled Queries | Timeout Queries | Errored Queries |+----------+---------------+----------------+-----------------+------------------+-------------------+-----------------+-----------------+| default0 | count(313258) | count(0) | count(312707) | count(0) | count(19) | count(0) | count(0) |+----------+---------------+----------------+-----------------+------------------+-------------------+-----------------+-----------------+ - Test Logs: consoleText.txt
|
QE Test |
sudo guides/gradlew --refresh-dependencies testrunner -P jython=/opt/jython/bin/jython -P 'args=-i /tmp/gcp_capella_volume_1TB.ini -p bucket_storage=magma,bucket_eviction_policy=fullEviction,rerun=False -t aGoodDoctor.hostedHospital.Murphy.test_rebalance,nodes_init=6,graceful=True,skip_cleanup=True,num_items=50000000,num_buckets=1,bucket_names=GleamBook,doc_size=1024,bucket_type=membase,eviction_policy=fullEviction,iterations=1,batch_size=1000,sdk_timeout=60,log_level=debug,infra_log_level=debug,rerun=False,skip_cleanup=True,key_size=18,randomize_doc_size=False,randomize_value=True,maxttl=10,pc=20,gsi_nodes=6,cbas_nodes=6,fts_nodes=6,kv_nodes=6,n1ql_nodes=6,kv_disk=300,n1ql_disk=50,gsi_disk=300,fts_disk=300,cbas_disk=300,kv_compute=c5.4xlarge,gsi_compute=c5.4xlarge,n1ql_compute=c5.4xlarge,fts_compute=c5.4xlarge,cbas_compute=c5.4xlarge,doc_ops=create:update:delete:read,mutation_perc=20,key_type=RandomKey,capella_run=true,services=data:query:index:analytics:search,max_rebl_nodes=27,provider=AWS,region=ca-central-1,type=GP3,size=300,skip_teardown_cleanup=false,wait_timeout=14400,index_timeout=28800,runtype=dedicated -m rest'
|