Details
-
Bug
-
Resolution: Incomplete
-
Critical
-
6.5.0
-
component cluster
-
Untriaged
-
Unknown
Description
Build : 6.5.0-4471
Test : -test tests/integration/test_allFeatures_madhatter_durability.yml -scope tests/integration/scope_Xattrs_Madhatter.yml
Scale : 3
Iteration : 1st
Step:
[2019-10-04T10:18:51-07:00, sequoiatools/couchbase-cli:6.5:e082b3] failover -c 172.23.108.103:8091 --server-failover 172.23.104.67:8091 -u Administrator -p password --force |
[2019-10-04T10:19:05-07:00, sequoiatools/couchbase-cli:6.5:5bbe14] rebalance -c 172.23.108.103:8091 -u Administrator -p password |
→
|
|
Error occurred on container - sequoiatools/couchbase-cli:6.5:[rebalance -c 172.23.108.103:8091 -u Administrator -p password] |
|
docker logs 5bbe14
|
docker start 5bbe14
|
|
*Unable to display progress bar on this os |
JERROR: Rebalance failed. See logs for detailed reason. You can try again. |
Error: (diag.log in 172.23.108.103)
2019-10-04T10:19:06.159-07:00, ns_orchestrator:0:info:message(ns_1@172.23.108.103) - Starting rebalance, KeepNodes = ['ns_1@172.23.104.156','ns_1@172.23.104.157', |
'ns_1@172.23.104.164','ns_1@172.23.104.61', |
'ns_1@172.23.104.69','ns_1@172.23.104.70', |
'ns_1@172.23.104.87','ns_1@172.23.106.100', |
'ns_1@172.23.106.188','ns_1@172.23.108.103', |
'ns_1@172.23.96.148','ns_1@172.23.96.251', |
'ns_1@172.23.96.252','ns_1@172.23.96.253', |
'ns_1@172.23.96.56','ns_1@172.23.96.95', |
'ns_1@172.23.97.119','ns_1@172.23.97.121', |
'ns_1@172.23.97.122','ns_1@172.23.97.239', |
'ns_1@172.23.97.242','ns_1@172.23.98.135', |
'ns_1@172.23.99.20','ns_1@172.23.99.21', |
'ns_1@172.23.99.25'], EjectNodes = [], Failed over and being ejected nodes = ['ns_1@172.23.104.67']; no delta recovery nodes; Operation Id = def4081507c372eebd57c71153ed3478 |
2019-10-04T10:19:06.582-07:00, ns_orchestrator:0:critical:message(ns_1@172.23.108.103) - Rebalance exited with reason {{badmatch, |
{error,
|
{failed_nodes,['ns_1@172.23.97.119']}}}, |
[{ns_janitor,cleanup_apply_config_body,4, |
[{file,"src/ns_janitor.erl"},{line,297}]}, |
{ns_janitor,'-cleanup_apply_config/4-fun-0-', |
4, |
[{file,"src/ns_janitor.erl"},{line,217}]}, |
{async,'-async_init/4-fun-1-',3, |
[{file,"src/async.erl"},{line,197}]}]}. |
Rebalance Operation Id = def4081507c372eebd57c71153ed3478
|
|
logs:
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1570213139/collectinfo-2019-10-04T181901-ns_1%40172.23.104.156.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1570213139/collectinfo-2019-10-04T181901-ns_1%40172.23.104.157.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1570213139/collectinfo-2019-10-04T181901-ns_1%40172.23.104.164.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1570213139/collectinfo-2019-10-04T181901-ns_1%40172.23.104.61.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1570213139/collectinfo-2019-10-04T181901-ns_1%40172.23.104.67.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1570213139/collectinfo-2019-10-04T181901-ns_1%40172.23.104.69.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1570213139/collectinfo-2019-10-04T181901-ns_1%40172.23.104.70.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1570213139/collectinfo-2019-10-04T181901-ns_1%40172.23.104.87.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1570213139/collectinfo-2019-10-04T181901-ns_1%40172.23.106.100.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1570213139/collectinfo-2019-10-04T181901-ns_1%40172.23.106.188.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1570213139/collectinfo-2019-10-04T181901-ns_1%40172.23.108.103.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1570213139/collectinfo-2019-10-04T181901-ns_1%40172.23.96.148.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1570213139/collectinfo-2019-10-04T181901-ns_1%40172.23.96.251.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1570213139/collectinfo-2019-10-04T181901-ns_1%40172.23.96.252.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1570213139/collectinfo-2019-10-04T181901-ns_1%40172.23.96.253.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1570213139/collectinfo-2019-10-04T181901-ns_1%40172.23.96.56.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1570213139/collectinfo-2019-10-04T181901-ns_1%40172.23.96.95.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1570213139/collectinfo-2019-10-04T181901-ns_1%40172.23.97.119.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1570213139/collectinfo-2019-10-04T181901-ns_1%40172.23.97.121.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1570213139/collectinfo-2019-10-04T181901-ns_1%40172.23.97.122.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1570213139/collectinfo-2019-10-04T181901-ns_1%40172.23.97.239.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1570213139/collectinfo-2019-10-04T181901-ns_1%40172.23.97.242.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1570213139/collectinfo-2019-10-04T181901-ns_1%40172.23.98.135.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1570213139/collectinfo-2019-10-04T181901-ns_1%40172.23.99.11.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1570213139/collectinfo-2019-10-04T181901-ns_1%40172.23.99.20.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1570213139/collectinfo-2019-10-04T181901-ns_1%40172.23.99.21.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1570213139/collectinfo-2019-10-04T181901-ns_1%40172.23.99.25.zip