Details
Description
Setup:
1.Setup a 18 node cluster. Enable Auto-failover
2.Load data on all 3 buckets [around 50M, 22M, 500k] items.
3. Continue loading data..
4. Failover orchestrator node [104]
5. Issue rebalance on this cluster.
Output
1. Node 104 is failed over . As expected
2. Rebalance operation fails withe error" mover_failed,{wrong_number_takeovers,0,1}}
Output from web-log
error_logger:error] [2012-06-15 15:30:32] [ns_1@10.3.2.89:error_logger:ale_error_logger_handler:log_report:72]
=========================CRASH REPORT=========================
crasher:
initial call: ebucketmigrator_srv:init/1
pid: <0.23836.1>
registered_name: []
exception exit: downstream_closed
in function gen_server:terminate/6
ancestors: ['ns_vbm_sup-bucket1','single_bucket_sup-bucket1',<0.470.0>]
messages: []
links: [<0.519.0>,<0.23837.1>]
dictionary: []
trap_exit: true
status: running
heap_size: 2584
stack_size: 24
reductions: 13752727
neighbours:
[error_logger:error] [2012-06-15 15:30:32] [ns_1@10.3.2.89:error_logger:ale_error_logger_handler:log_report:72]
=========================SUPERVISOR REPORT=========================
Supervisor:
Context: child_terminated
Reason: downstream_closed
Offender: [
,
{name,
{child_id,
[144,145,146,147,148,149,851],
'ns_1@10.3.2.104'}},
{mfargs,
{ebucketmigrator_srv,start_link,
[
,
,
[
,
,
,
,
,
]]}},
,
,
]
Logs at https://s3.amazonaws.com/bugdb/jira/bug-rebalance-2/bug2.tar
Checked existing bugs -Bug 5343 and this one, well it looked different to me.