Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-50422

MultiNodeFailover: Non-KV nodes got failed over when unable to failover KV node

    XMLWordPrintable

Details

    Description

      Steps to reproduce:

      • Multi node cluster as follows

        +----------------+-------------+-----------------+-----------+----------+-----------------------+-------------------+ | Node           | Services    | CPU_utilization | Mem_total | Mem_free | Swap_mem_used         | Active / Replica  | +----------------+-------------+-----------------+-----------+----------+-----------------------+-------------------+ | 172.23.100.21  | n1ql        | 1.78078755957   | 3.91 GiB  | 3.13 GiB | 7.50 MiB / 3.50 GiB   | 0 / 0             | | 172.23.105.212 | index, n1ql | 2.76019245379   | 3.91 GiB  | 3.13 GiB | 57.22 MiB / 3.50 GiB  | 0 / 0             | | 172.23.108.238 | n1ql        | 0.778307808185  | 3.69 GiB  | 3.01 GiB | 0.0 Byte / 3.50 GiB   | 0 / 0             | | 172.23.105.244 | index       | 3.06918238994   | 3.91 GiB  | 3.22 GiB | 116.25 MiB / 3.50 GiB | 0 / 0             | | 172.23.105.245 | index       | 2.69453538152   | 3.91 GiB  | 3.20 GiB | 157.75 MiB / 3.50 GiB | 0 / 0             | | 172.23.105.155 | kv          | 16.5342219944   | 3.91 GiB  | 2.94 GiB | 115.80 MiB / 3.50 GiB | 0 / 0             | | 172.23.105.213 | index       | 5.10101010101   | 3.91 GiB  | 3.22 GiB | 64.89 MiB / 3.50 GiB  | 0 / 0             | | 172.23.100.22  | n1ql        | 1.65912518854   | 3.91 GiB  | 3.25 GiB | 134.75 MiB / 3.50 GiB | 0 / 0             | | 172.23.105.211 | kv          | 13.910158244    | 3.91 GiB  | 3.19 GiB | 146.50 MiB / 3.50 GiB | 0 / 0             | +----------------+-------------+-----------------+-----------+----------+-----------------------+-------------------+

      • Couchbase bucket, replica=1
      • Auto-failover settings - maxCount=5
      • Bring down 4 nodes

        +----------------+----------+-------------+----------------+
        | Node           | Services | Node status | Failover type  |
        +----------------+----------+-------------+----------------+
        | 172.23.105.213 | index    | active      | stop_couchbase |
        | 172.23.100.21  | n1ql     | active      | stop_couchbase |
        | 172.23.105.155 | kv       | active      | stop_couchbase |
        | 172.23.100.22  | n1ql     | active      | stop_couchbase |
        +----------------+----------+-------------+----------------+

      Observation:

      Non-kv nodes .213, .21, .22 got failed over leaving out the KV node.

      Expected behavior:

      No failover should be allowed since KV failover is impossible here due to data-loss

       

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            ashwin.govindarajulu Ashwin Govindarajulu
            ashwin.govindarajulu Ashwin Govindarajulu
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty