Details
Description
Scenario:: Incremental Rebalane-out
Version:: 3.0.0-730, centos-64, vbuckets=128
7 Nodes participating :: 10.6.2.144-150
1. Add 7 Nodes to cluster
2. Create default bucket
3. Add 100 K items
4. Rebalance-out 2 Nodes with Create Ops
5. Rebalance-out 2 Nodes with Create Ops
Fails [hangs/stuck] in step 5
Test Case
./testrunner -i /tmp/centos-64-2.0-basic-rebalance-tests-P0.ini -t rebalance.rebalanceout.RebalanceOutTests.incremental_rebalance_out_with_ops,replicas=1,items=100000,max_verify=100000,doc_ops=create,GROUP=OUT
Rebalance-out that did not fail
ejectedNodes=ns_1%4010.6.2.149%2Cns_1%4010.6.2.150&user=Administrator&knownNodes=ns_1%4010.6.2.146%2Cns_1%4010.6.2.144%2Cns_1%4010.6.2.145%2Cns_1%4010.6.2.147%2Cns_1%4010.6.2.148%2Cns_1%4010.6.2.150%2Cns_1%4010.6.2.149
Rebalance that led to failure
2014-05-23 16:03:59 | INFO | MainProcess | Cluster_Thread | [rest_client.rebalance] rebalance params : password=password&ejectedNodes=ns_1%4010.6.2.147%2Cns_1%4010.6.2.148&user=Administrator&knownNodes=ns_1%4010.6.2.145%2Cns_1%4010.6.2.146%2Cns_1%4010.6.2.144%2Cns_1%4010.6.2.147%2Cns_1%4010.6.2.148
Log for run
http://qa.hq.northscale.net/view/3.0.0/job/centos_x64--02_04--Rebalance_tests_UPR-P0/143/console
The cbcollect info is attached