Details
-
Bug
-
Resolution: Duplicate
-
Major
-
Cheshire-Cat
-
Centos 7 64 bit; Couchbase Enterprise Build 3239
-
Untriaged
-
Centos 64-bit
-
-
1
-
No
Description
Summary:
Resident ratio dropped low on nodes that were failovered and recovered.
Steps to Reproduce:
1. Create a 6 node cluster
2020-10-01 03:24:03,920 | test | INFO | pool-1-thread-7 | [table_view:display:72] Rebalance Overview
-----------------------++-------------
Nodes | Services | Status |
-----------------------++-------------
172.23.105.211 | kv | Cluster node |
172.23.105.212 | None | <--- IN — |
172.23.105.213 | None | <--- IN — |
172.23.105.215 | None | <--- IN — |
172.23.105.217 | None | <--- IN — |
172.23.105.219 | None | <--- IN — |
-----------------------++-------------
2. Create a single bucket with 1000 collections and a total of 10k items
-------------------------+----------------------------------------------------+----------
Bucket | Type | Replicas | Durability | TTL | Items | RAM Quota | RAM Used | Disk Used |
-------------------------+----------------------------------------------------+----------
3Y2OCngAZQmemyo | couchbase | 2 | none | 0 | 10000 | 12582912000 | 957175360 | 635367063 |
-------------------------+----------------------------------------------------+----------
3. Drop 400 collections and recreate them whilst doing the below steps (no more docs are created, and number of items get reduced as collections are dropped )
4. Failover .211 and .212
2020-10-01 03:32:50,365 | test | INFO | MainThread | [collections_drop_recreate_rebalance:wait_for_failover_or_assert:53] 2 nodes failed over as expected in 0.0750000476837 seconds
5. Delta recover and rebalance
2020-10-01 03:32:50,500 | test | INFO | pool-1-thread-28 | [table_view:display:72] Rebalance Overview
-----------------------++-------------
Nodes | Services | Status |
-----------------------++-------------
172.23.105.215 | kv | Cluster node |
172.23.105.212 | kv | Cluster node |
172.23.105.213 | kv | Cluster node |
172.23.105.217 | kv | Cluster node |
172.23.105.219 | kv | Cluster node |
172.23.105.211 | kv | Cluster node |
-----------------------++-------------
Now the resident ratio drops low as seen in the screenshot.
from supportal logs, the low resident ratio is on the .211 and .212 nodes