Details
-
Bug
-
Resolution: Cannot Reproduce
-
Blocker
-
2.5.0
-
Security Level: Public
-
centos 6.4 64-bit
-
Centos 64-bit
-
Description
Environment:
8 centos 6.4 64-bit servers with 15 GB RAM
1:tambran-s21701
2:tambran-s21702
3:tambran-s21703
4:tambran-s21704
5:tambran-s21705
6:tambran-s21706
7:tambran-s21707
8:tambran-s21708
Setup:
Install couchbase server 2.5.0-994 on 8 nodes
Create 6 nodes cluster with 3 groups: Group 1, Group 2 and Group 3. Each group has 2 nodes.
Create sasl-1 bucket with 2 replica
Load 49 M items into bucket with set, get, delete, expired, update operation
Do swap rebalance,
add node 51 to Group 1 and remove node 45 from Group 1 out of cluster
add node 52 to Group 2 and remove node 47 from Group 2 out of cluster
Click rebalace. Rebalance run a while and failed with error node down
Node 'ns_1@172.23.106.45' saw that node 'ns_1@172.23.106.51' went down. Details: [
{nodedown_reason, net_tick_timeout}] ns_node_disco005 ns_1@172.23.106.45 01:20:44 - Fri Dec 13, 2013Node 'ns_1@172.23.106.47' saw that node 'ns_1@172.23.106.51' went down. Details: [{nodedown_reason, net_tick_timeout}
]
Then I tried rebalance again, it also failed with error "{mover_failed,{badmatch,
{error,ehostunreach}}}I waited for few minute and retried rebalance, it also failed with same error
Rebalance exited with reason {mover_failed,{badmatch,{error,ehostunreach}
}}
ns_orchestrator002 ns_1@172.23.106.45 01:55:45 - Fri Dec 13, 2013
Cluster is in failed state now.