Details
-
Bug
-
Resolution: Done
-
Critical
-
6.5.0
-
Untriaged
-
Centos 64-bit
-
Yes
Description
Build: 6.5.0-4821 , last time passed on 6.5.0-4783
./testrunner -i /tmp/testexec.7233.ini -p get-cbcollect-info=True,GROUP=bucket_op_with_timers -t eventing.eventing_rebalance.EventingRebalance.test_eventing_rebalance_with_multiple_eventing_nodes,doc-per-day=5,dataset=default,nodes_init=5,services_init=kv-eventing-eventing-eventing-index:n1ql,groups=simple,reset_services=True,handler_code=bucket_op_with_timers,GROUP=bucket_op_with_timers
|
|
Observed following rebalance completed
[2019-11-15 08:04:13,709] - [rest_client:2105] INFO - Node 172.23.105.80 not part of cluster inactiveAdded
|
[2019-11-15 08:04:14,628] - [rest_client:1497] INFO - rebalance params : {'password': 'password', 'ejectedNodes': u'ns_1@172.23.105.94,ns_1@172.23.105.96,ns_1@172.23.105.59', 'user': 'Administrator', 'knownNodes': u'ns_1@172.23.105.32,ns_1@172.23.105.80,ns_1@172.23.105.82,ns_1@172.23.105.79,ns_1@172.23.105.96,ns_1@172.23.105.59,ns_1@172.23.105.94'}
|
[2019-11-15 08:04:14,642] - [rest_client:1502] INFO - rebalance operation started
|
[2019-11-15 08:04:14,649] - [rest_client:1659] INFO - rebalance percentage : 0.00 %
|
[2019-11-15 08:04:14,650] - [task:533] INFO - Rebalance - status: running, progress: 0.00%
|
[2019-11-15 08:04:16,761] - [rest_client:92] INFO - rebalance reached >100% in 3.00745296478 seconds
|
[2019-11-15 08:04:24,682] - [rest_client:1659] INFO - rebalance percentage : 21.43 %
|
[2019-11-15 08:04:24,683] - [task:533] INFO - Rebalance - status: running, progress: 21.43%
|
[2019-11-15 08:04:34,708] - [rest_client:1659] INFO - rebalance percentage : 36.29 %
|
[2019-11-15 08:04:34,708] - [task:533] INFO - Rebalance - status: running, progress: 36.29%
|
[2019-11-15 08:04:44,733] - [rest_client:1659] INFO - rebalance percentage : 65.81 %
|
[2019-11-15 08:04:44,733] - [task:533] INFO - Rebalance - status: running, progress: 65.81%
|
[2019-11-15 08:04:54,761] - [rest_client:1659] INFO - rebalance percentage : 91.23 %
|
[2019-11-15 08:04:54,761] - [task:533] INFO - Rebalance - status: running, progress: 91.23%
|
[2019-11-15 08:05:04,788] - [task:533] INFO - Rebalance - status: none, progress: 100.00%
|
[2019-11-15 08:05:04,844] - [task:592] INFO - rebalancing was completed with progress: 100% in 50.2019569874 sec
|
|
But undeployment fails with
[2019-11-15 08:05:12,632] - [rest_client:840] ERROR - POST http://172.23.105.80:8096/api/v1/functions/Function_771968608_test_eventing_rebalance_with_multiple_eventing_nodes/settings body: {"processing_status": false, "deployment_status": false} headers: {'Content-type': 'application/json', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==\n'} error: 406 reason: unknown {
|
"name": "ERR_REBALANCE_ONGOING",
|
"code": 36,
|
"description": "Rebalance ongoing on some/all Eventing nodes, creating new functions, deployment or undeployment of existing functions is not allowed",
|
"attributes": null,
|
"runtime_info": {
|
"code": 36,
|
"info": "Rebalance ongoing on some/all Eventing nodes, creating new functions, deployment or undeployment of existing functions is not allowed"
|
}
|
} auth: Administrator:password
|
ERROR
|
Attachments
Issue Links
- depends on
-
MB-36767 [System test]: service_rebalance_failed,eventing, worker died, timed out
- Closed