Details
-
Bug
-
Resolution: Fixed
-
Critical
-
Cheshire-Cat
-
Untriaged
-
-
1
-
Unknown
Description
Build : 7.0.0-5075
Test : -test tests/2i/cheshirecat/test_idx_clusterops_cheshire_cat_moi.yml -scope tests/2i/cheshirecat/scope_idx_cheshire_cat_moi.yml
Scale : 2
Iteration : 2nd
Rebalance for index node addition failed within a couple of mins.
[2021-05-02T00:32:56-07:00, sequoiatools/couchbase-cli:7.0:7dcdc7] server-add -c 172.23.97.215:8091 --server-add https://172.23.107.2 -u Administrator -p password --server-add-username Administrator --server-add-password password --services index
|
[pull] sequoiatools/couchbase-cli:7.0
|
[2021-05-02T00:33:10-07:00, sequoiatools/couchbase-cli:7.0:d3b0ee] rebalance -c 172.23.97.215:8091 -u Administrator -p password
|
→
|
|
|
Error occurred on container - sequoiatools/couchbase-cli:7.0:[rebalance -c 172.23.97.215:8091 -u Administrator -p password]
|
|
|
docker logs d3b0ee
|
docker start d3b0ee
|
|
|
*Unable to display progress bar on this os
|
JERROR: Rebalance failed. See logs for detailed reason. You can try again.
|
[pull] sequoiatools/cmd
|
[2021-05-02T00:34:57-07:00, sequoiatools/cmd:b58c46] 60
|
From the error.log of orchestrator node 172.23.97.215
[ns_server:error,2021-05-02T00:34:49.369-07:00,ns_1@172.23.97.215:service_rebalancer-index<0.12487.265>:service_rebalancer:run_rebalance_worker:130]Agent terminated during the rebalance: {'DOWN',
|
#Ref<0.918429405.365690887.175058>,
|
process,<30299.13344.83>,
|
{linked_process_died,
|
<30299.15071.83>,
|
{'ns_1@172.23.107.2',
|
{timeout,
|
{gen_server,call,
|
[<30299.14843.83>,
|
{call,"ServiceAPI.GetTaskList",
|
#Fun<json_rpc_connection.0.77329884>},
|
60000]}}}}}
|
[user:error,2021-05-02T00:34:49.372-07:00,ns_1@172.23.97.215:<0.8584.0>:ns_orchestrator:log_rebalance_completion:1405]Rebalance exited with reason {service_rebalance_failed,index,
|
{agent_died,<30299.13344.83>,
|
{linked_process_died,<30299.15071.83>,
|
{'ns_1@172.23.107.2',
|
{timeout,
|
{gen_server,call,
|
[<30299.14843.83>,
|
{call,"ServiceAPI.GetTaskList",
|
#Fun<json_rpc_connection.0.77329884>},
|
60000]}}}}}}.
|
Rebalance Operation Id = 8ccd5f6e15f5b47f561d0d8790d385c6
|
Index nodes : 172.23.107.3:8091 172.23.107.4:8091 172.23.107.5:8091 172.23.97.216:8091 172.23.97.217:8091
Attachments
| For Gerrit Dashboard: MB-46054 | ||||||
|---|---|---|---|---|---|---|
| # | Subject | Branch | Project | Status | CR | V |
| 153097,5 | MB-46054 Propagate UNKNOWN_COLLECTION status to indexer | unstable | indexing | Status: MERGED | +2 | +1 |
| 153295,8 | MB-46054 Handle UNKNOWN_COLLECTION status at timekeeper | unstable | indexing | Status: MERGED | +2 | +1 |
| 153359,2 | MB-46054 Process errors before processing seqnos | unstable | indexing | Status: MERGED | +2 | +1 |
| 162973,10 | MB-48754 Do not cleanup index data when flush is in progress | unstable | indexing | Status: MERGED | +2 | +1 |
| 182948,3 | MB-54690 Do not cleanup index data when flush is in progress | cheshire-cat | indexing | Status: MERGED | +2 | +1 |