Description
http://qa.hq.northscale.net/view/2.0.1/job/centos-64-2.0-upgrade/51/consoleFull
./testrunner -i /tmp/upgrade.ini get-logs=False,upgrade_version=2.0.1-120-rel,initial_vbuckets=64 -t newupgradetests.MultiNodesUpgradeTests.offline_cluster_upgrade_and_rebalance,initial_version=2.0.0-1978-rel,nodes_init=4,num_stoped_nodes=1,nodes_in=1,nodes_out=1
steps:
1. cluster 2.0.0-1978-rel ns_1%4010.3.3.11%2Cns_1%4010.3.3.16%2Cns_1%4010.3.3.14%2Cns_1%4010.3.3.13
2. start rebalance with ejectedNodes=ns_1%4010.3.3.13
3.during rebalance upgrade 10.3.3.16 on 2.0.1-120-rel
result: [2013-01-09 04:28:20,650] - [newupgradetests:139] INFO - rebalance failed as expected
4.wait while upgrade 10.3.3.16 completed
[2013-01-09 04:29:47,536] - [cluster_helper:84] INFO - waiting for ns_server @ 10.3.3.16:8091
[2013-01-09 04:29:47,570] - [rest_client:36] WARNING - server 10.3.3.16:8091 status is warmup
[2013-01-09 04:29:48,591] - [rest_client:36] WARNING - server 10.3.3.16:8091 status is warmup
[2013-01-09 04:29:49,610] - [rest_client:36] WARNING - server 10.3.3.16:8091 status is warmup
[2013-01-09 04:29:50,623] - [rest_client:36] WARNING - server 10.3.3.16:8091 status is warmup
[2013-01-09 04:29:51,643] - [rest_client:36] WARNING - server 10.3.3.16:8091 status is warmup
[2013-01-09 04:29:52,661] - [rest_client:36] WARNING - server 10.3.3.16:8091 status is warmup
[2013-01-09 04:29:53,673] - [rest_client:36] WARNING - server 10.3.3.16:8091 status is warmup
[2013-01-09 04:29:54,685] - [rest_client:36] WARNING - server 10.3.3.16:8091 status is warmup
[2013-01-09 04:29:55,696] - [rest_client:36] WARNING - server 10.3.3.16:8091 status is warmup
[2013-01-09 04:29:56,709] - [rest_client:36] WARNING - server 10.3.3.16:8091 status is warmup
[2013-01-09 04:29:57,725] - [rest_client:36] WARNING - server 10.3.3.16:8091 status is warmup
[2013-01-09 04:29:58,736] - [rest_client:36] WARNING - server 10.3.3.16:8091 status is warmup
[2013-01-09 04:29:59,748] - [rest_client:36] WARNING - server 10.3.3.16:8091 status is warmup
[2013-01-09 04:30:00,761] - [rest_client:36] WARNING - server 10.3.3.16:8091 status is warmup
[2013-01-09 04:30:01,773] - [cluster_helper:86] INFO - ns_server @ 10.3.3.16:8091 is running
5. and retry rebalance
[2013-01-09 04:30:02,774] - [rest_client:788] INFO - rebalance params : password=password&ejectedNodes=ns_1%4010.3.3.13&user=Administrator&knownNodes=ns_1%4010.3.3.11%2Cns_1%4010.3.3.16%2Cns_1%4010.3.3.14%2Cns_1%4010.3.3.13
[2013-01-09 04:30:02,784] - [rest_client:792] INFO - rebalance operation started
[2013-01-09 04:30:02,797] - [rest_client:881] INFO - rebalance percentage : 0 %
[2013-01-09 04:30:12,817] - [rest_client:866] ERROR -
- rebalance failed
result:
2013-01-09 04:30:06.976 ns_orchestrator:2:info:message(ns_1@10.3.3.16) - Rebalance exited with reason {{badmatch,[]},
[
,
,
,
]}