Details
-
Bug
-
Resolution: Not a Bug
-
Critical
-
Cheshire-Cat
-
Untriaged
-
Centos 64-bit
-
-
1
-
Unknown
Description
Build - 7.0.0 - 5249
Cluster consists of 10 nodes -
- 5 kv
- 5 eventing
Performed Quorum failover on 3 kv and 3 eventing nodes.
Vbshuffle does not seem to start/ progress after performing Quorum failover.
/getAppRebalanceProgress remained unchanged for half an hour.
curl -XGET http://Administrator:password@172.23.106.76:8096/getAggRebalanceProgress |
{
|
"CloseStreamVbsLen": 0, |
"StreamReqVbsLen": 0, |
"VbsRemainingToShuffle": 4100, |
"VbsOwnedPerPlan": 4096, |
"NodeLevelStats": { |
"172.23.106.76:8096": { |
"close_stream_vbs_len": 0, |
"stream_req_vbs_len": 1241, |
"vbs_owned_per_plan": 2048, |
"vbs_remaining_to_shuffle": 1241 |
},
|
"172.23.106.77:8096": { |
"close_stream_vbs_len": 811, |
"stream_req_vbs_len": 2048, |
"vbs_owned_per_plan": 2048, |
"vbs_remaining_to_shuffle": 2859 |
}
|
}
|
}
|
Hence /getAggRebalanceStatus returns true and we can't perform lifecycle operations.
Tried to perform rebalance on the cluster.
Following error is thrown when eventing rebalance is going on. KV rebalance went in fine.
Undeploy failed due to "Rebalance or Failover processing ongoing on some/all Eventing nodes, creating new functions, deployment or undeployment of existing functions is not allowed |