Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-58967

Indexer panic seen - runtime error: index out of range [1] with length 1

    XMLWordPrintable

Details

    • Untriaged
    • 0
    • Unknown

    Description

      Seen on 172.23.106.30, 172.23.106.171, and 172.23.105.122.

      Excerpts from the log on 106.30 -

      2023-10-04T16:53:43.973-07:00 [Info] Planner::populateAlternateShardIds Using slot: 3390268807220357650 for placing replicas: Node: 172.23.105.122:8091, numShards: 122, minShardCapacity: 200, Index name: idx6_WlxXpdMk, defnId: 2919139178181059646, replicaId: 0, partnId: 2
      Node: 172.23.97.109:8091, numShards: 200, minShardCapacity: 200, Index name: idx6_WlxXpdMk, defnId: 2919139178181059646, replicaId: 1, partnId: 2
       as it is least loaded
      2023-10-04T16:53:43.978-07:00 [Info] Planner::populateAlternateShardIds Using slot: 3825379453870917546 for placing replicas: Node: 172.23.105.122:8091, numShards: 122, minShardCapacity: 200, Index name: idx1_F77s6Kif, defnId: 12696598913505312438, replicaId: 1, partnId: 2
      Node: 172.23.106.176:8091, numShards: 185, minShardCapacity: 200, Index name: idx1_F77s6Kif, defnId: 12696598913505312438, replicaId: 0, partnId: 2
       as it is least loaded
      2023-10-04T16:53:43.984-07:00 [Info] Planner::populateAlternateShardIds Using slot: 3825379453870917546 for placing replicas: Node: 172.23.97.109:8091, numShards: 200, minShardCapacity: 200, Index name: idx2_hRnSVM, defnId: 16934562483263348526, replicaId: 2, partnId: 3
      Node: 172.23.97.108:8091, numShards: 214, minShardCapacity: 200, Index name: idx2_hRnSVM, defnId: 16934562483263348526, replicaId: 0, partnId: 3
      Node: 172.23.106.176:8091, numShards: 187, minShardCapacity: 200, Index name: idx2_hRnSVM, defnId: 16934562483263348526, replicaId: 1, partnId: 3
      Node: 172.23.106.171:8091, numShards: 152, minShardCapacity: 200, Index name: idx2_hRnSVM, defnId: 16934562483263348526, replicaId: 3, partnId: 3
       as it is least loaded
      panic: runtime error: index out of range [1] with length 1
       
      goroutine 3003587 [running]:
      github.com/couchbase/indexing/secondary/planner.assignAlternateIds(0x1b2568f?, {0xc01accb350, 0xc01accd830}, 0x60?)
      	/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/indexing/secondary/planner/executor.go:6573 +0x76d
      github.com/couchbase/indexing/secondary/planner.PopulateAlternateShardIds(0xc027162fc0, {0xc019fffe00, 0x2d, 0x28?}, 0x0?)
      	/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/indexing/secondary/planner/executor.go:6000 +0x791
      github.com/couchbase/indexing/secondary/planner.rebalance({0x1a83f62, 0x9}, 0xc0169bd5e8, 0xc0169bd120, {0xc01af7b800, 0x536, 0x700}, {0x0, 0x0, 0x0}, ...)
      	/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/indexing/secondary/planner/executor.go:2612 +0x9ea
      github.com/couchbase/indexing/secondary/planner.rebalance({0x1a83f62, 0x9}, 0xc0169bd5e8, 0xc0439fa000, {0x0, 0x0, 0x0}, {0x2e3bb78, 0x0, 0x0}, ...)
      	/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/indexing/secondary/planner/executor.go:2608 +0xc65
      github.com/couchbase/indexing/secondary/planner.executeRebal(0x20?, {0x1a83f62?, 0x195?}, 0xc0169bd458?, {0x4741ee?, 0x4426c7?, 0xc0169bd440?}, {0x2e3bb78, 0x0, 0x0}, ...)
      	/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/indexing/secondary/planner/executor.go:1954 +0x65
      github.com/couchbase/indexing/secondary/planner.ExecuteRebalanceInternal({0x7fff5669f768, 0xe}, {{0xc098cddcc0, 0x20}, {0x0, 0x0, 0x0}, {0xc098cddce0, 0x19}, {0xc01b8aa2c0, ...}, ...}, ...)
      	/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/indexing/secondary/planner/executor.go:238 +0x846
      github.com/couchbase/indexing/secondary/planner.ExecuteRebalance({0x7fff5669f768, 0xe}, {{0xc098cddcc0, 0x20}, {0x0, 0x0, 0x0}, {0xc098cddce0, 0x19}, {0xc01b8aa2c0, ...}, ...}, ...)
      	/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/indexing/secondary/planner/executor.go:184 +0x190
      github.com/couchbase/indexing/secondary/indexer.(*ShardRebalancer).initRebalAsync(0xc0763fa900)
      	/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/indexing/secondary/indexer/shard_rebalancer.go:297 +0x9b0
      created by github.com/couchbase/indexing/secondary/indexer.NewShardRebalancer
      	/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/indexing/secondary/indexer/shard_rebalancer.go:199 +0x9b7
      

      Cbcollect logs:

      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1696479331/collectinfo-2023-10-05T042415-ns_1%40172.23.105.122.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1696479331/collectinfo-2023-10-05T042415-ns_1%40172.23.106.171.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1696479331/collectinfo-2023-10-05T042415-ns_1%40172.23.106.176.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1696479331/collectinfo-2023-10-05T042415-ns_1%40172.23.106.30.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1696479331/collectinfo-2023-10-05T042415-ns_1%40172.23.96.198.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1696479331/collectinfo-2023-10-05T042415-ns_1%40172.23.96.230.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1696479331/collectinfo-2023-10-05T042415-ns_1%40172.23.96.245.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1696479331/collectinfo-2023-10-05T042415-ns_1%40172.23.97.100.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1696479331/collectinfo-2023-10-05T042415-ns_1%40172.23.97.108.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1696479331/collectinfo-2023-10-05T042415-ns_1%40172.23.97.109.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1696479331/collectinfo-2023-10-05T042415-ns_1%40172.23.97.66.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1696479331/collectinfo-2023-10-05T042415-ns_1%40172.23.97.67.zip

      Attachments

        For Gerrit Dashboard: MB-58967
        # Subject Branch Project Status CR V

        Activity

          People

            pavan.pb Pavan PB
            pavan.pb Pavan PB
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty