Details
-
Bug
-
Resolution: Fixed
-
Major
-
Cheshire-Cat
-
Untriaged
-
-
1
-
Unknown
Description
Build : 7.0.0-4626
Test : -test tests/integration/cheshirecat/test_cheshirecat_kv_gsi_coll_xdcr_backup_sgw_fts_itemct_txns_eventing_cbas.yml -scope tests/integration/cheshirecat/scope_cheshirecat_with_backup.yml
Scale : 2
Iteration : 1st
The query select count
from system:active_requests where state=="closed" shows 27099 queries. Some of them are there since 32hrs.
cbq> select statement,requestTime,elapsedTime,state from system:active_requests where state=="closed" limit 5;
|
{
|
"requestID": "9511f356-1fc6-4924-9821-3be705873186",
|
"signature": {
|
"elapsedTime": "json",
|
"requestTime": "json",
|
"state": "json",
|
"statement": "json"
|
},
|
"results": [
|
{
|
"elapsedTime": "32h3m45.231338861s",
|
"requestTime": "2021-03-09T01:31:41.305-08:00",
|
"state": "closed",
|
"statement": "select * from `default` where result=\"SUCCESS\" limit 20"
|
},
|
{
|
"elapsedTime": "32h3m10.075343787s",
|
"requestTime": "2021-03-09T01:32:16.462-08:00",
|
"state": "closed",
|
"statement": "select * from `default` where result=\"SUCCESS\" limit 20"
|
},
|
{
|
"elapsedTime": "31h33m2.36324449s",
|
"requestTime": "2021-03-09T02:02:24.174-08:00",
|
"state": "closed",
|
"statement": "select * from `default` where result=\"SUCCESS\" limit 20"
|
},
|
{
|
"elapsedTime": "22h58m37.422609497s",
|
"requestTime": "2021-03-09T10:36:49.116-08:00",
|
"state": "closed",
|
"statement": "select * from `default` where result=\"SUCCESS\" limit 20"
|
},
|
{
|
"elapsedTime": "22h55m18.086693803s",
|
"requestTime": "2021-03-09T10:40:08.453-08:00",
|
"state": "closed",
|
"statement": "select * from `default` where result=\"SUCCESS\" limit 20"
|
}
|
],
|
"status": "success",
|
"metrics": {
|
"elapsedTime": "55.896692ms",
|
"executionTime": "55.726685ms",
|
"resultCount": 5,
|
"resultSize": 1081,
|
"serviceLoad": 15
|
}
|
}
|
It needs to be investigated what is causing such a large number of queries to be stuck like this, are they holding up any resources etc.
Query service is responding fine - no issues seen in the functioning as such.
I have attached the cpu, heap and memory profiles along with dump of active and completed requests for both the query nodes - 172.23.120.245 and 172.23.96.251