Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-46725

Rebalance button not enabled post Quorum Loss failover even when indexing has partitioned indexes

    XMLWordPrintable

Details

    • Untriaged
    • Centos 64-bit
    • 1
    • Unknown

    Description

      Steps to Repro
      1. Create a 6 node cluster. (3 kv + 3 index,n1ql). See
      2. Create 2 partition indexes using the following commands. See

      CREATE INDEX idx_pe11 ON `travel-sample`(airline, sourceairport, destinationairport)
       PARTITION BY HASH(airline) WITH {"num_partition":9};
      CREATE INDEX idx_pe12 ON `travel-sample`(airline, sourceairport, destinationairport)
       PARTITION BY HASH(airline) WITH {"num_partition":9};
      

      3. Bring down 4 nodes. (2 kv + 2 index,n1ql) using systemctl stop couchbase-server.
      4. QL failover all the 4 nodes. See

      All the 4 nodes are out of the cluster. However rebalance button is not enabled.

      I think it's because there is only one KV node in the cluster and we think there is nothing to redistribute. However, as seen from post QL failover we can see there is only 3 partitions now and a rebalance would be required to rebuild other 6 partitions. For this to happen we need to have rebalance button enabled which is not the case here. See

      cc: Amit Kulkarni

      Attachments

        1. Cluster.png
          Cluster.png
          205 kB
        2. Indexes.png
          Indexes.png
          249 kB
        3. QL failover.png
          QL failover.png
          128 kB
        4. Indexes_post_QL.png
          Indexes_post_QL.png
          209 kB
        5. Servers_post_QL.png
          Servers_post_QL.png
          155 kB

        Issue Links

          For Gerrit Dashboard: MB-46725
          # Subject Branch Project Status CR V

          Activity

            kevin.cherkauer Kevin Cherkauer added a comment - - edited

            Fixes merged to unstable and master.

            Needs to be cherry-picked to cheshire-cat branch for 7.0.1 (only after 7.0.0 GA image on same branch has been finalized) before resolving.

            kevin.cherkauer Kevin Cherkauer added a comment - - edited Fixes merged to unstable and master. Needs to be cherry-picked to cheshire-cat branch for 7.0.1 (only after 7.0.0 GA image on same branch has been finalized) before resolving.

            Build couchbase-server-7.1.0-1024 contains indexing commit 393e40f with commit message:
            MB-46725 Set isBalanced = false after all failovers

            build-team Couchbase Build Team added a comment - Build couchbase-server-7.1.0-1024 contains indexing commit 393e40f with commit message: MB-46725 Set isBalanced = false after all failovers

            Backported to 7.0.1 (cheshire-cat branch).

            kevin.cherkauer Kevin Cherkauer added a comment - Backported to 7.0.1 (cheshire-cat branch).

            Build couchbase-server-7.0.1-5995 contains indexing commit 1a2854b with commit message:
            MB-46725 Set isBalanced = false after all failovers

            build-team Couchbase Build Team added a comment - Build couchbase-server-7.0.1-5995 contains indexing commit 1a2854b with commit message: MB-46725 Set isBalanced = false after all failovers

            Validated on 7.0.2 build 6522. Rebalance button was enabled and can be successfully use to rebalance out failover nodes

            hemant.rajput Hemant Rajput added a comment - Validated on 7.0.2 build 6522. Rebalance button was enabled and can be successfully use to rebalance out failover nodes

            People

              kevin.cherkauer Kevin Cherkauer
              Balakumaran.Gopal Balakumaran Gopal
              Votes:
              0 Vote for this issue
              Watchers:
              12 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty