Details
-
Bug
-
Resolution: Fixed
-
Major
-
6.6.0
-
Untriaged
-
1
-
Unknown
Description
Build: 6.6.0-7909
- Cluster with 3 kv, 2 index+n1ql, 4 search nodes
- 6 bkts with 5000 docs each
- Built 200 GSI indexes with replica 1 (50 indexes on 4 buckets)
- Created 30 fts custom indexes (10 indexes on 3 buckets), just to add more entries to metakv
- Create and Drop 100 gsi indexes sequentially on 4 buckets ( so this would be adding more entries of create/drop of 400 indexes)
- Create and drop 50 fts indexes on 3 buckets.
Failover/addback/rebalance of search node failed in below error:
[ns_server:info,2020-12-14T19:44:24.220-08:00,ns_1@172.23.106.245:rebalance_agent<0.23812.0>:rebalance_agent:handle_down:296]Rebalancer process <0.8594.994> died (reason {service_rebalance_failed,fts,
|
{worker_died,
|
{'EXIT',<0.15724.994>,
|
{rebalance_failed,
|
{service_error,
|
<<"nodes: sample, res: (*http.Response)(nil), urlUUID: monitor.UrlUUID{Url:\"http://172.23.121.45:8094\", UUID:\"6935eebb7e2cf4303b771ac2796f6158\"}, kind: /api/stats, err: Get http://%40fts-cbauth:***@172.23.121.45:8094/api/stats: EOF">>}}}}}).
|
[user:error,2020-12-14T19:44:24.220-08:00,ns_1@172.23.106.245:<0.29120.245>:ns_orchestrator:log_rebalance_completion:1445]Rebalance exited with reason {service_rebalance_failed,fts,
|
{worker_died,
|
{'EXIT',<0.15724.994>,
|
{rebalance_failed,
|
{service_error,
|
<<"nodes: sample, res: (*http.Response)(nil), urlUUID: monitor.UrlUUID{Url:\"http://172.23.121.45:8094\", UUID:\"6935eebb7e2cf4303b771ac2796f6158\"}, kind: /api/stats, err: Get http://%40fts-cbauth:***@172.23.121.45:8094/api/stats: EOF">>}}}}}.
|
Rebalance Operation Id = f4b5b7f3fb0ebb704c43831d738e0060
|
[error_logger:error,2020-12-14T19:44:24.220-08:00,ns_1@172.23.106.245:error_logger<0.32.0>:ale_error_logger_handler:do_log:203]
|
Logs:
cbcollect logs:
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1608005762/collectinfo-2020-12-15T041603-ns_1%40172.23.106.245.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1608005762/collectinfo-2020-12-15T041603-ns_1%40172.23.107.104.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1608005762/collectinfo-2020-12-15T041603-ns_1%40172.23.107.197.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1608005762/collectinfo-2020-12-15T041603-ns_1%40172.23.107.87.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1608005762/collectinfo-2020-12-15T041603-ns_1%40172.23.121.41.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1608005762/collectinfo-2020-12-15T041603-ns_1%40172.23.121.45.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1608005762/collectinfo-2020-12-15T041603-ns_1%40172.23.121.46.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1608005762/collectinfo-2020-12-15T041603-ns_1%40172.23.121.47.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1608005762/collectinfo-2020-12-15T041603-ns_1%40172.23.121.49.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1608005762/collectinfo-2020-12-15T041603-ns_1%40172.23.121.52.zip
Attachments
Issue Links
- is a backport of
-
-
- Closed
-