Details
-
Bug
-
Resolution: Fixed
-
Critical
-
Cheshire-Cat
-
Untriaged
-
1
-
Unknown
-
KV-Engine CC Final Sprint
Description
In the 4B rebalance runs, we saw auto failover happened during the load phase. We increased failover threshold from 5 seconds to 30 seconds in the second run (rebalance-in), but the run still hit auto failover. Both runs didn't hit the issue with 6.6.2 build.
Rebalance-out (min), 5 -> 4, 4B x 1KB, 15K ops/sec (90/10 R/W), 10%% cache miss rate
Build: 7.0.0-5071
Job: http://perf.jenkins.couchbase.com/job/titan-reb/2119/
[user:info,2021-05-05T12:00:57.796-07:00,ns_1@172.23.96.100:<0.5861.0>:auto_failover:log_failover_success:544]Node ('ns_1@172.23.96.100') was automatically failed over. Reason: The data service did not respond for the duration of the auto-failover threshold. Either none of the buckets have warmed up or there is an issue with the data service.
Rebalance-in (min), 4 -> 5, 4B x 1KB, 15K ops/sec (90/10 R/W), 10%% cache miss rate
Build: 7.0.0-5071
Job: http://perf.jenkins.couchbase.com/job/titan-reb/2124/
[user:info,2021-05-07T20:39:41.757-07:00,ns_1@172.23.96.100:<0.6131.0>:auto_failover:log_failover_success:544]Node ('ns_1@172.23.96.102') was automatically failed over. Reason: The data service did not respond for the duration of the auto-failover threshold. Either none of the buckets have warmed up or there is an issue with the data service.