Details
-
Bug
-
Resolution: Not a Bug
-
Major
-
Cheshire-Cat
-
7.0.0-4291
-
Untriaged
-
Centos 64-bit
-
1
-
Unknown
Description
Script to Repro
./sequoia -client 172.23.96.162:2375 -provider file:centos_third_cluster.yml -test tests/integration/cheshirecat/test_cheshirecat_kv_gsi_coll_xdcr_backup_sgw_fts_itemct_txns_eventing_cbas_scale3.yml -scope tests/integration/cheshirecat/scope_cheshirecat_with_backup.yml -scale 3 -repeat 0 -log_level 0 -version 7.0.0-5194 -skip_setup=false -skip_test=false -skip_teardown=true -skip_cleanup=false -continue=false -collect_on_error=false -stop_on_error=false -duration=604800 -show_topology=true
|
We are also running the following script in background during this test.
for i in {0..500000}
|
do
|
`curl -X PUT -u Administrator:password http://localhost:8091/_metakv/key{$i} -d 'value=foo1'`
|
sleep 15
|
`curl -X DELETE -v -u Administrator:password http://localhost:8091/_metakv/key{$i}`
|
sleep 15
|
done
|
During that time frame we are trying to do a Auto failover
[2021-05-20T10:44:38-07:00, sequoiatools/couchbase-cli:7.0:db8cb1] setting-autofailover -c 172.23.104.232:8091 -u Administrator -p password --enable-auto-failover=1 --auto-failover-timeout=5 --max-failovers=1
|
[2021-05-20T10:44:44-07:00, sequoiatools/cmd:8b4eb2] 10
|
[pull] sequoiatools/cbutil
|
[2021-05-20T10:45:01-07:00, sequoiatools/cbutil:56985f] /cbinit.py 172.23.104.244 root couchbase stop
|
[2021-05-20T10:45:19-07:00, sequoiatools/cmd:77a48e] 10
|
[2021-05-20T10:45:35-07:00, sequoiatools/couchbase-cli:7.0:4d3865] rebalance -c 172.23.104.232:8091 -u Administrator -p password
|
[2021-05-20T11:14:13-07:00, sequoiatools/cmd:6f4dab] 60
|
[2021-05-20T11:15:19-07:00, sequoiatools/cmd:b162b5] 180
|
[2021-05-20T11:18:25-07:00, sequoiatools/cbutil:f2b973] /cbinit.py 172.23.104.244 root couchbase start
|
[2021-05-20T11:18:31-07:00, sequoiatools/cmd:03ef43] 300
|
172.23.106.209 : fts
/opt/couchbase/var/lib/couchbase/logs/fts.log:2021-05-20T10:45:04.424-07:00 [ERRO] (GOCBCORE) Pipeline client `172.23.104.244:11207/0xc20cc88cc0` failed to shut down errored client socket (tls: use of closed connection) -- cbgt.GocbcoreLogger.Log() at gocbcore_utils.go:615
|
/opt/couchbase/var/lib/couchbase/logs/fts.log:2021-05-20T10:45:04.483-07:00 [ERRO] (GOCBCORE) Pipeline client `172.23.104.244:11207/0xc0a0a36840` failed to shut down errored client socket (tls: use of closed connection) -- cbgt.GocbcoreLogger.Log() at gocbcore_utils.go:615
|
/opt/couchbase/var/lib/couchbase/logs/fts.log:2021-05-20T10:45:04.523-07:00 [ERRO] (GOCBCORE) Pipeline client `172.23.104.244:11207/0xc0c0008d20` failed to shut down errored client socket (tls: use of closed connection) -- cbgt.GocbcoreLogger.Log() at gocbcore_utils.go:615
|
/opt/couchbase/var/lib/couchbase/logs/fts.log:2021-05-20T10:45:04.753-07:00 [ERRO] (GOCBCORE) Pipeline client `172.23.104.244:11207/0xc0aeb46960` failed to shut down errored client socket (tls: use of closed connection) -- cbgt.GocbcoreLogger.Log() at gocbcore_utils.go:615
|
/opt/couchbase/var/lib/couchbase/logs/fts.log:2021-05-20T10:45:04.753-07:00 [ERRO] (GOCBCORE) Pipeline client `172.23.104.244:11207/0xc14a3f7560` failed to shut down errored client socket (tls: use of closed connection) -- cbgt.GocbcoreLogger.Log() at gocbcore_utils.go:615
|
/opt/couchbase/var/lib/couchbase/logs/fts.log:2021-05-20T10:45:04.930-07:00 [ERRO] (GOCBCORE) Pipeline client `172.23.104.244:11207/0xc0a6ca0060` failed to shut down errored client socket (tls: use of closed connection) -- cbgt.GocbcoreLogger.Log() at gocbcore_utils.go:615
|
/opt/couchbase/var/lib/couchbase/logs/fts.log:2021-05-20T10:45:04.940-07:00 [ERRO] (GOCBCORE) Pipeline client `172.23.104.244:11207/0xc16f7aab40` failed to shut down errored client socket (tls: use of closed connection) -- cbgt.GocbcoreLogger.Log() at gocbcore_utils.go:615
|
172.23.106.225 : fts
2021-05-20T10:45:04.368-07:00 [ERRO] (GOCBCORE) Pipeline client `172.23.104.244:11207/0xc09dce2720` failed to shut down errored client socket (tls: use of closed connection) -- cbgt.GocbcoreLogger.Log() at gocbcore_utils.go:615
|
2021-05-20T10:45:04.489-07:00 [ERRO] (GOCBCORE) Pipeline client `172.23.104.244:11207/0xc0ee220b40` failed to shut down errored client socket (tls: use of closed connection) -- cbgt.GocbcoreLogger.Log() at gocbcore_utils.go:615
|
2021-05-20T10:45:04.609-07:00 [ERRO] (GOCBCORE) Pipeline client `172.23.104.244:11207/0xc0c5e39f20` failed to shut down errored client socket (tls: use of closed connection) -- cbgt.GocbcoreLogger.Log() at gocbcore_utils.go:615
|
2021-05-20T10:45:04.747-07:00 [ERRO] (GOCBCORE) Pipeline client `172.23.104.244:11207/0xc0884e79e0` failed to shut down errored client socket (tls: use of closed connection) -- cbgt.GocbcoreLogger.Log() at gocbcore_utils.go:615
|
2021-05-20T10:45:04.760-07:00 [ERRO] (GOCBCORE) Pipeline client `172.23.104.244:11207/0xc0f4847200` failed to shut down errored client socket (tls: use of closed connection) -- cbgt.GocbcoreLogger.Log() at gocbcore_utils.go:615
|
2021-05-20T10:45:04.929-07:00 [ERRO] (GOCBCORE) Pipeline client `172.23.104.244:11207/0xc02831fda0` failed to shut down errored client socket (tls: use of closed connection) -- cbgt.GocbcoreLogger.Log() at gocbcore_utils.go:615
|
2021-05-20T10:45:04.937-07:00 [ERRO] (GOCBCORE) Pipeline client `172.23.104.244:11207/0xc0a1bfc540` failed to shut down errored client socket (tls: use of closed connection) -- cbgt.GocbcoreLogger.Log() at gocbcore_utils.go:615
|
cbcollect_info attached.