Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-46808

Implement periodic background Planner runs to update isBalanced flag

    XMLWordPrintable

Details

    • Bug
    • Status: Open
    • Critical
    • Resolution: Unresolved
    • Cheshire-Cat
    • Morpheus
    • secondary-index
    • Enterprise Edition 7.0.0 build 5263
    • Untriaged
    • Centos 64-bit
    • 1
    • Unknown

    Description

      Cloned from MB-46725 (7.0.1) per discussion with Deepkaran Salooja to implement a feature of a periodic background Planner run to update the rebalance_service_manager.go ServiceMgr.state.isBalanced flag automatically. The issue is that currently there is nothing that makes an absolute calculation as to whether GSI is balanced or not. Thus if partitions or replicas become unavailable due to unsafe failovers or many indexes are added or dropped, currently GSI will not set isBalanced = false to enable the Rebalance button in the UI because it does not do any kind of check for these things.

      Jeelan Poola FYI.

       

      Repro from original MB for historical purposes:

      Steps to Repro
      1. Create a 6 node cluster. (3 kv + 3 index,n1ql). See
      2. Create 2 partition indexes using the following commands. See

      CREATE INDEX idx_pe11 ON `travel-sample`(airline, sourceairport, destinationairport)
       PARTITION BY HASH(airline) WITH {"num_partition":9};
      CREATE INDEX idx_pe12 ON `travel-sample`(airline, sourceairport, destinationairport)
       PARTITION BY HASH(airline) WITH {"num_partition":9};
      

      3. Bring down 4 nodes. (2 kv + 2 index,n1ql) using systemctl stop couchbase-server.
      4. QL failover all the 4 nodes. See

      All the 4 nodes are out of the cluster. However rebalance button is not enabled.

      I think it's because there is only one KV node in the cluster and we think there is nothing to redistribute. However, as seen from post QL failover we can see there is only 3 partitions now and a rebalance would be required to rebuild other 6 partitions. For this to happen we need to have rebalance button enabled which is not the case here. See

      cc: Amit Kulkarni

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            Amit Kulkarni Would this aspect be covered in the document that you are going to propose as per discussion today?

            jeelan.poola Jeelan Poola added a comment - Amit Kulkarni Would this aspect be covered in the document that you are going to propose as per discussion today?

            Hi Jeelan Poola, thanks for pointing to this.

            This MB focuses on using planner to check if the cluster is balanced.

            I am focussing on changes which keeps the cluster balanced with respect to server groups, whenever it gets a chance (like index creation / rebalance etc.)

            So, this MB represents the other side (the consumer side) of the solution I am going to propose.

            amit.kulkarni Amit Kulkarni added a comment - Hi Jeelan Poola , thanks for pointing to this. This MB focuses on using planner to check if the cluster is balanced. I am focussing on changes which keeps the cluster balanced with respect to server groups, whenever it gets a chance (like index creation / rebalance etc.) So, this MB represents the other side (the consumer side) of the solution I am going to propose.

            People

              kevin.cherkauer Kevin Cherkauer
              kevin.cherkauer Kevin Cherkauer
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty