Details
-
Bug
-
Resolution: Duplicate
-
Critical
-
None
-
7.1.0
-
Untriaged
-
1
-
Yes
Description
Build: 7.1.0 build 2304
Test: -test tests/fts/cheshire-cat/test_fts_clusterops_cheshire_cat_coll_crud.yml -scope tests/fts/cheshire-cat/scope_fts_cheshire_cat.yml
Scale: 1
Test Cycle: 6
Cluster - comp cluster 2
KV bucket fragmentation seen high in one of the node (172.23.106.154) and seen disk full on same node and below errors:
2022-02-16T22:41:32.721-08:00, menelaus_web_alerts_srv:0:info:message(ns_1@172.23.106.154) - Write Commit Failure. Disk write failed for item in Bucket "bucket4" on node 172.23.106.154.
|
2022-02-16T22:41:32.722-08:00, menelaus_web_alerts_srv:0:info:message(ns_1@172.23.106.154) - Write Commit Failure. Disk write failed for item in Bucket "bucket3" on node 172.23.106.154.
|
2022-02-16T22:42:32.722-08:00, menelaus_web_alerts_srv:0:info:message(ns_1@172.23.106.154) - Write Commit Failure. Disk write failed for item in Bucket "bucket4" on node 172.23.106.154.
|
2022-02-16T22:42:32.723-08:00, menelaus_web_alerts_srv:0:info:message(ns_1@172.23.106.154) - Write Commit Failure. Disk write failed for item in Bucket "bucket3" on node 172.23.106.154.
|
2022-02-16T22:43:32.723-08:00, menelaus_web_alerts_srv:0:info:message(ns_1@172.23.106.154) - Write Commit Failure. Disk write failed for item in Bucket "bucket4" on node 172.23.106.154.
|
2022-02-16T22:43:32.724-08:00, menelaus_web_alerts_srv:0:info:message(ns_1@172.23.106.154) - Write Commit Failure. Disk write failed for item in Bucket "bucket3" on node 172.23.106.154.
|
2022-02-16T22:44:32.724-08:00, menelaus_web_alerts_srv:0:info:message(ns_1@172.23.106.154) - Write Commit Failure. Disk write failed for item in Bucket "bucket4" on node 172.23.106.154.
|
2022-02-16T22:44:32.725-08:00, menelaus_web_alerts_srv:0:info:message(ns_1@172.23.106.154) - Write Commit Failure. Disk write failed for item in Bucket "bucket3" on node 172.23.106.154.
|
2022-02-16T22:45:32.727-08:00, menelaus_web_alerts_srv:0:info:message(ns_1@172.23.106.154) - Write Commit Failure. Disk write failed for item in Bucket "bucket4" on node 172.23.106.154.
|
Test step around the time when the errors started:
[2022-02-16T22:35:12-08:00, sequoiatools/couchbase-cli:7.1:bb883d] server-add -c 172.23.106.253:8091 --server-add https://172.23.105.186 -u Administrator -p password --server-add-username Administrator --server-add-password password --services fts
|
[pull] sequoiatools/couchbase-cli:7.1
|
[2022-02-16T22:35:24-08:00, sequoiatools/couchbase-cli:7.1:8a13cd] rebalance -c 172.23.106.253:8091 -u Administrator -p password
|
[pull] sequoiatools/cmd
|
[2022-02-16T22:38:47-08:00, sequoiatools/cmd:4e1c54] 60
|
[pull] sequoiatools/cmd
|
[2022-02-16T22:39:55-08:00, sequoiatools/cmd:99f181] 900
|
Logs around the time the above errors started:
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1645083830/collectinfo-2022-02-17T074352-ns_1%40172.23.105.185.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1645083830/collectinfo-2022-02-17T074352-ns_1%40172.23.105.186.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1645083830/collectinfo-2022-02-17T074352-ns_1%40172.23.105.190.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1645083830/collectinfo-2022-02-17T074352-ns_1%40172.23.106.154.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1645083830/collectinfo-2022-02-17T074352-ns_1%40172.23.106.182.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1645083830/collectinfo-2022-02-17T074352-ns_1%40172.23.106.242.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1645083830/collectinfo-2022-02-17T074352-ns_1%40172.23.106.243.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1645083830/collectinfo-2022-02-17T074352-ns_1%40172.23.106.253.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1645083830/collectinfo-2022-02-17T074352-ns_1%40172.23.106.255.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1645083830/collectinfo-2022-02-17T074352-ns_1%40172.23.107.89.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1645083830/collectinfo-2022-02-17T074352-ns_1%40172.23.97.213.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1645083830/collectinfo-2022-02-17T074352-ns_1%40172.23.97.214.zip
Attachments
Issue Links
- duplicates
-
MB-50941 Compaction reschedules but never runs (sleeping for ~0 seconds?)
- Closed