Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-41897

Schedule indexes get stuck in schedule stage during rebalance out of assigned node

    XMLWordPrintable

Details

    • Untriaged
    • Centos 64-bit
    • 1
    • No

    Description

      Rebalancing out of assigned node of an index during schedule stage make index stuck on reassigned node.

      Steps to reproduce:-

      1. Have 3 nodes cluster (index-kv-query, index, index) with 1B-1S-1C and 1M docs.
      2. Create multiple indexes in parallel (say 10) with 1 replica num and preassigned deploy node. 

        CREATE INDEX idx_7 ON default:test_bucket.test_scope_1.test_collection_1(suffix) USING GSI  WITH {'nodes': ['10.112.205.102:8091', '10.112.205.103:8091'], 'defer_build': True}
        

      3. Rebalance out one of the assigned node before indexes moved out of schedule state. (Also observed similar behaviour if one of the assigned node is failed over)
      4. Observer that all the indexes get reassigned to other available index node in cluster and comes out of schedule state.

      Attachments

        1. node1-cb660-centos7.vagrants.zip
          54.36 MB
        2. node2-cb660-centos7.vagrants.zip
          12.96 MB
        3. node3-cb660-centos7.vagrants.zip
          15.62 MB
        4. node4-cb660-centos7.vagrants.zip
          17.19 MB
        5. Screenshot 2020-11-10 at 1.59.16 PM.png
          Screenshot 2020-11-10 at 1.59.16 PM.png
          1.46 MB
        6. test.log
          227 kB
        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            amit.kulkarni Amit Kulkarni
            hemant.rajput Hemant Rajput
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty