Details
-
Bug
-
Resolution: Fixed
-
Critical
-
5.5.0, 5.5.1, 5.5.2, 5.5.3, 5.5.4, 5.5.5, 5.5.6, 6.0.0, 6.0.1, 6.0.2, 6.0.3, 6.0.4, 6.5.1, 6.5.0
-
Triaged
-
1
Description
Currently, to perform a hard failover using couchbase-cli failover, you must pass the --force flag.
If you do not pass this flag, then the failover will be a graceful failover, which will obviously not work in situations where the node is down.
As well as specifying hard failover, when you pass the --force flag it also passes the allowUnsafe=true as part of the REST request.
Since Couchbase Server 5.5.0 and the introduction of a quorum based orchestration mechanism (via leases), there needs to be a majority quorum to be able to perform failovers by default.
There are obviously cases where this is not possible (maybe the majority of nodes are irrecoverably down in the cluster), so ns_server has a mechanism to specify when performing a failover that you do not need it to wait for a quorum.
Due to its nature this is inherently unsafe and may cause 'split brain' effects if the exact nature of the issue is not well understood.
Most users will not want to use this allowUnsafe option and its use should generally be avoided unless absolutely necessary.
From the perspective of using couchbase-cli failover though, the two concepts of hard failover and unsafe failover are both merged together with the --force flag.
I think it would be useful to reconsider this API to be safer by default, while still allowing users to perform unsafe failovers if they are genuinely required.
Attachments
Issue Links
- relates to
-
DOC-6735 Failover docs should describe the concept of failover "unsafeness"
- Resolved
For Gerrit Dashboard: MB-39220 | ||||||
---|---|---|---|---|---|---|
# | Subject | Branch | Project | Status | CR | V |
129178,6 | MB-39220 Failover separates hard and force options | mad-hatter | couchbase-cli | Status: MERGED | +2 | +1 |
129315,2 | MB-39220: update tests | mad-hatter | cbas-core | Status: MERGED | +2 | +1 |
131490,3 | MB-39220: Adding --hard flag for cli failover tests | mad-hatter | testrunner | Status: MERGED | +2 | +1 |
131645,1 | Merge branch 'mad-hatter' | master | couchbase-cli | Status: MERGED | +2 | +1 |
131728,3 | MB-39220: update tests | master | cbas-core | Status: MERGED | +2 | +1 |
132088,4 | MB-39220: Adding --hard flag for cli failover tests | master | testrunner | Status: MERGED | +2 | +1 |