Details
-
Bug
-
Resolution: Fixed
-
Critical
-
Elixir
-
7.5.0-3593
-
Untriaged
-
-
0
-
Unknown
Description
- Create 20 serverless databases. wait for kv to scale from 3 to 6 kv nodes.
- Create gsi index and fts indexes on all the databases. Number of indexer varies based on the dataplase load pattern here: https://github.com/couchbaselabs/TAF/blob/master/pytests/aGoodDoctor/serverlessHospital.py#L142
- Start the data load when all the indexes are ready
- Data ingestion and index building started in FTS as in data started loading in kv.
- FTS cpu started going up. It qualifies for Auto-Scaling and 2 more nodes are added to the cluster and rebalanced. Rebalance fails.
|
Node svc-s-node-007.nomghki5newnypa.sandbox.nonprod-project-avengers.com |
Service 'fts' exited with status 2. Restarting. Messages:
|
/home/couchbase/.cbdepscache/exploded/aarch64/go-1.18.7/go/src/bufio/bufio.go:236 +0x1d4
|
io.ReadAtLeast({0xffff680bece0, 0x40fa8f2e40}, {0x41e2210400, 0x3a8, 0x3a8}, 0x3a8)
|
/home/couchbase/.cbdepscache/exploded/aarch64/go-1.18.7/go/src/io/io.go:331 +0xac
|
io.ReadFull(...)
|
/home/couchbase/.cbdepscache/exploded/aarch64/go-1.18.7/go/src/io/io.go:350
|
github.com/couchbase/gocbcore/v10/memd.(*Conn).ReadPacket(0x41afb5d880)
|
/home/couchbase/.cbdepscache/gomodcache/pkg/mod/github.com/couchbase/gocbcore/v10@v10.1.6/memd/conn.go:353 +0xe8
|
github.com/couchbase/gocbcore/v10.(*memdConnWrap).ReadPacket(0x437ca80960?)
|
/home/couchbase/.cbdepscache/gomodcache/pkg/mod/github.com/couchbase/gocbcore/v10@v10.1.6/memdconn.go:103 +0x24
|
github.com/couchbase/gocbcore/v10.(*memdClient).run.func2()
|
/home/couchbase/.cbdepscache/gomodcache/pkg/mod/github.com/couchbase/gocbcore/v10@v10.1.6/memdclient.go:392 +0x54
|
created by github.com/couchbase/gocbcore/v10.(*memdClient).run
|
/home/couchbase/.cbdepscache/gomodcache/pkg/mod/github.com/couchbase/gocbcore/v10@v10.1.6/memdclient.go:390 +0x120
|
|
QE Test |
git fetch https://review.couchbase.org/TAF refs/changes/05/185605/1 && git checkout FETCH_HEAD
|
|
|
sudo guides/gradlew --refresh-dependencies testrunner -P jython=/opt/jython/bin/jython -P "args=-i /tmp/ElixirVolumeTest.ini -p bucket_storage=magma,bucket_eviction_policy=fullEviction,rerun=False -t aGoodDoctor.serverlessHospital.Murphy.ElixirVolume,skip_cleanup=True,num_buckets=${num_buckets},bucket_names=GleamBook,doc_size=1024,bucket_type=membase,eviction_policy=fullEviction,iterations=${iterations},batch_size=1000,sdk_timeout=60,log_level=debug,infra_log_level=debug,key_size=18,randomize_doc_size=False,randomize_value=True,assert_crashes_on_load=True,maxttl=10,pc=${pc},mutation_perc=20,key_type=RandomKey,capella_run=true,skip_teardown_cleanup=true,wait_timeout=14400,index_timeout=36000,dataplane_id=${dataplane_id},cb_image=${cb_image},dn_image=${dn_image},dapi_image=${dapi_image},num_dataplanes=${num_dataplanes},runtype=serverless,kv_disk_size=${kv_disk_size},index_disk_size=300 -m rest"
|
Attachments
Issue Links
- relates to
-
-
- Resolved
-
-
-
- Resolved
-