Details
-
Bug
-
Resolution: Fixed
-
Major
-
1.8.1
-
Security Level: Public
-
None
-
windows 2008 R2 64 bit
Description
Install couchbase server 1.8.1-916 on 10 nodes cluster in longevity test.
Load 40 million items size from 70 to 512 bytes to cluster with around 10K ops
Keep load running around 10K (update and delete items) during rebalance run below.
Do rebalance in and out one by one node.
Load 30 million items more to cluster.
Delete 20 million items from cluster.
Do swap rebalance one by one with couchbase server version 1.8.1-937 (4 nodes swapped to build 937)
During swap node (add 79 and remove 78), restart physical server a node. When this node was done warmup, replica items from other nodes dropdown to almost zero. Then gradually increase then item number.
After few days, the replica items from node 10 , 11, 12, 182 and 183 did not go back to the same of active items.
Cluster also gets a lot of Hard Out Of Memory Error from nodes 10 , 11, 12, 182 and 183
Here are their IPs:
1:10.3.2.11
2:10.3.2.12
3:10.3.2.16
4:10.3.2.10
5:10.3.2.75
6:10.3.2.76
7:10.3.2.77
8:10.3.2.78
9:10.3.2.79
10:10.3.121.185
11:10.3.121.182
12:10.3.121.183