Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-54528

Indexer created without replicas in Elixir DBAAS setup

    XMLWordPrintable

Details

    • Bug
    • Status: Resolved
    • Critical
    • Resolution: Fixed
    • Elixir
    • Elixir
    • secondary-index
    • None
    • Elixir DBAAS
    • Untriaged
    • 0
    • Unknown

    Description

      I see that in some cases, indexes get created without replicas. Not really sure when this happens, so requires some investigation. I've attached the cluster config and indexes page screenshots. As you can see from the screenshots the index 'magmaname' has been created without any replica.

      The current state of the dataplane is as follows:

      • 4 indexer nodes, 3 KV nodes, 2 FTS, and 2 Query nodes.

      Not really sure if it's relevant to this bug, but as per an earlier comment by Varun Velamuri, if there are 2 indexer nodes in the same server group, no replicas get created. In the real world scenario, there is a possibility that they end up on the same server group as there are only a limited no of server groups (auto-scaling groups in the AWS world). We will either need to revisit the planner implementation.

      Log bundles ->

      s3://cb-customers-secure/indexwithoutreplica/2022-11-11/collectinfo-2022-11-11t121258-ns_1@5iq8ka1ekrc3bxws.hsvngcdq1x5cqze2.sandbox.nonprod-project-avengers.com.zip
      s3://cb-customers-secure/indexwithoutreplica/2022-11-11/collectinfo-2022-11-11t121258-ns_1@8ayl-ap4iho1gqre.hsvngcdq1x5cqze2.sandbox.nonprod-project-avengers.com.zip
      s3://cb-customers-secure/indexwithoutreplica/2022-11-11/collectinfo-2022-11-11t121258-ns_1@9vidso5p5m92qwyz.hsvngcdq1x5cqze2.sandbox.nonprod-project-avengers.com.zip
      s3://cb-customers-secure/indexwithoutreplica/2022-11-11/collectinfo-2022-11-11t121258-ns_1@bp6iae4pc9o529fo.hsvngcdq1x5cqze2.sandbox.nonprod-project-avengers.com.zip
      s3://cb-customers-secure/indexwithoutreplica/2022-11-11/collectinfo-2022-11-11t121258-ns_1@e2cullprghoxw9xs.hsvngcdq1x5cqze2.sandbox.nonprod-project-avengers.com.zip
      s3://cb-customers-secure/indexwithoutreplica/2022-11-11/collectinfo-2022-11-11t121258-ns_1@fnkl6e7opwunhr0o.hsvngcdq1x5cqze2.sandbox.nonprod-project-avengers.com.zip
      s3://cb-customers-secure/indexwithoutreplica/2022-11-11/collectinfo-2022-11-11t121258-ns_1@fnnqxiyl9p8eqpa6.hsvngcdq1x5cqze2.sandbox.nonprod-project-avengers.com.zip
      s3://cb-customers-secure/indexwithoutreplica/2022-11-11/collectinfo-2022-11-11t121258-ns_1@jynioof-jbxixibl.hsvngcdq1x5cqze2.sandbox.nonprod-project-avengers.com.zip
      s3://cb-customers-secure/indexwithoutreplica/2022-11-11/collectinfo-2022-11-11t121258-ns_1@q9jw3pguekwzvei3.hsvngcdq1x5cqze2.sandbox.nonprod-project-avengers.com.zip
      s3://cb-customers-secure/indexwithoutreplica/2022-11-11/collectinfo-2022-11-11t121258-ns_1@qn2jesemsw1javo9.hsvngcdq1x5cqze2.sandbox.nonprod-project-avengers.com.zip
      s3://cb-customers-secure/indexwithoutreplica/2022-11-11/collectinfo-2022-11-11t121258-ns_1@xaqh0wrx6tyxx2v5.hsvngcdq1x5cqze2.sandbox.nonprod-project-avengers.com.zip

      Screenshots also attached.

      Attachments

        For Gerrit Dashboard: MB-54528
        # Subject Branch Project Status CR V

        Activity

          Build couchbase-server-7.5.0-3444 contains indexing commit 8b92e2c with commit message:
          MB-54528 Additional test for replica repair with defrag

          build-team Couchbase Build Team added a comment - Build couchbase-server-7.5.0-3444 contains indexing commit 8b92e2c with commit message: MB-54528 Additional test for replica repair with defrag

          Build couchbase-server-8.0.0-1197 contains indexing commit e2ab592 with commit message:
          MB-54528 Fix grouping of empty subclusters

          build-team Couchbase Build Team added a comment - Build couchbase-server-8.0.0-1197 contains indexing commit e2ab592 with commit message: MB-54528 Fix grouping of empty subclusters

          Build couchbase-server-8.0.0-1197 contains indexing commit f9fae14 with commit message:
          MB-54528 planner unit test for swap rebalance server group mismatch

          build-team Couchbase Build Team added a comment - Build couchbase-server-8.0.0-1197 contains indexing commit f9fae14 with commit message: MB-54528 planner unit test for swap rebalance server group mismatch

          Build couchbase-server-8.0.0-1197 contains indexing commit 45b7ade with commit message:
          MB-54528 planner - fix server group validation for swap rebalance

          build-team Couchbase Build Team added a comment - Build couchbase-server-8.0.0-1197 contains indexing commit 45b7ade with commit message: MB-54528 planner - fix server group validation for swap rebalance

          Build couchbase-server-8.0.0-1197 contains indexing commit 8b92e2c with commit message:
          MB-54528 Additional test for replica repair with defrag

          build-team Couchbase Build Team added a comment - Build couchbase-server-8.0.0-1197 contains indexing commit 8b92e2c with commit message: MB-54528 Additional test for replica repair with defrag

          People

            deepkaran.salooja Deepkaran Salooja
            pavan.pb Pavan PB
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty