Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-39220

Couchbase-cli needs a way to perform hard failover without passing the unsafe flag

    XMLWordPrintable

Details

    • Bug
    • Resolution: Fixed
    • Critical
    • 6.6.0
    • 5.5.0, 5.5.1, 5.5.2, 5.5.3, 5.5.4, 5.5.5, 5.5.6, 6.0.0, 6.0.1, 6.0.2, 6.0.3, 6.0.4, 6.5.1, 6.5.0
    • tools
    • Triaged
    • 1

    Description

      Currently, to perform a hard failover using couchbase-cli failover, you must pass the --force flag.
      If you do not pass this flag, then the failover will be a graceful failover, which will obviously not work in situations where the node is down.

      As well as specifying hard failover, when you pass the --force flag it also passes the allowUnsafe=true as part of the REST request.
      Since Couchbase Server 5.5.0 and the introduction of a quorum based orchestration mechanism (via leases), there needs to be a majority quorum to be able to perform failovers by default.
      There are obviously cases where this is not possible (maybe the majority of nodes are irrecoverably down in the cluster), so ns_server has a mechanism to specify when performing a failover that you do not need it to wait for a quorum.
      Due to its nature this is inherently unsafe and may cause 'split brain' effects if the exact nature of the issue is not well understood.
      Most users will not want to use this allowUnsafe option and its use should generally be avoided unless absolutely necessary.

      From the perspective of using couchbase-cli failover though, the two concepts of hard failover and unsafe failover are both merged together with the --force flag.
      I think it would be useful to reconsider this API to be safer by default, while still allowing users to perform unsafe failovers if they are genuinely required.

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              pvarley Patrick Varley (Inactive)
              matt.carabine Matt Carabine (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              11 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty