Details
-
Bug
-
Resolution: Cannot Reproduce
-
Critical
-
Cheshire-Cat
-
Untriaged
-
-
1
-
Unknown
Description
Build : 7.0.0-5277
Test : -test tests/2i/cheshirecat/test_idx_clusterops_cheshire_cat_recovery.yml -scope tests/2i/cheshirecat/scope_idx_cheshire_cat_dgm.yml
Scale : 2
Iteration : 4th
Day : 2nd
The test has a step to swap rebalance a pair of index nodes - add 172.23.97.216 and remove 172.23.97.217. The rebalance started at 2021-06-09T04:02:02. Then indexer on 172.23.97.216 was killed at 2021-06-09T04:08:31. So, the rebalance failed. This rebalance was automatically retried after 5 mins. It appears that the rebalance is stuck since then with 72% progress.
curl -u Administrator:password 172.23.97.215:8091/pools/default/rebalanceProgress | jq
|
{
|
"status": "running",
|
"ns_1@172.23.97.232": {
|
"progress": 1
|
},
|
"ns_1@172.23.107.2": {
|
"progress": 0.7222222222222223
|
},
|
"ns_1@172.23.107.3": {
|
"progress": 0.7222222222222223
|
},
|
"ns_1@172.23.107.4": {
|
"progress": 0.7222222222222223
|
},
|
"ns_1@172.23.97.235": {
|
"progress": 1
|
},
|
"ns_1@172.23.97.215": {
|
"progress": 1
|
},
|
"ns_1@172.23.107.5": {
|
"progress": 0.7222222222222223
|
},
|
"ns_1@172.23.97.216": {
|
"progress": 0.7222222222222223
|
},
|
"ns_1@172.23.97.236": {
|
"progress": 0
|
},
|
"ns_1@172.23.97.237": {
|
"progress": 1
|
},
|
"ns_1@172.23.97.217": {
|
"progress": 0.7222222222222223
|
},
|
"ns_1@172.23.97.227": {
|
"progress": 0
|
}
|
}
|
The MetadataStore files from 172.23.97.216 are attached.
Attachments
Issue Links
- relates to
-
MB-46839 [System Test] Index swap rebalance stuck at 72%
- Closed