Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-60916

Optimise support for 10k index shard rebalance tests

    XMLWordPrintable

Details

    • Bug
    • Resolution: Unresolved
    • Critical
    • Morpheus, 7.6.3
    • 7.6.0
    • secondary-index
    • None
    • Untriaged
    • 0
    • Unknown

    Description

      Following areas can be improved for 10k index shard rebalance tests:

      a. waitForIndexState optimisations - Currently each transfer token will spawn a goroutine to gather metadata and stats. With 10K indexes, there can be 100's of tokens and this can lead to lot of garbage. This scheme can be optimised to only have one goroutine gather meta every second and all other go-routines can use the data.

      b. Drop Index optimisations - Currently, rebalancer issues one drop index request at a time. If 5000 indexes have to be dropped on a node, then 5000 rest calls have to be made. In each call, all the list of indexes have to be iterated. This is a time consuming operation and can slow down entire rebalance. An ideal way to approach this would be to batch all drop requests of a transfer token at a time and issue a single drop request. 

      c. Planner optimisations - Planner does multiple iterations of all the indexes. With 10K indexes, this is not practical. Identify and address bottlenecks in planner

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              amit.kulkarni Amit Kulkarni
              varun.velamuri Varun Velamuri
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty