Details
-
Bug
-
Resolution: Fixed
-
Critical
-
7.6.0
-
7.6.0-1591
-
Untriaged
-
0
-
Unknown
Description
Seen on 172.23.106.30, 172.23.106.171, and 172.23.105.122.
Excerpts from the log on 106.30 -
2023-10-04T16:53:43.973-07:00 [Info] Planner::populateAlternateShardIds Using slot: 3390268807220357650 for placing replicas: Node: 172.23.105.122:8091, numShards: 122, minShardCapacity: 200, Index name: idx6_WlxXpdMk, defnId: 2919139178181059646, replicaId: 0, partnId: 2 |
Node: 172.23.97.109:8091, numShards: 200, minShardCapacity: 200, Index name: idx6_WlxXpdMk, defnId: 2919139178181059646, replicaId: 1, partnId: 2 |
as it is least loaded
|
2023-10-04T16:53:43.978-07:00 [Info] Planner::populateAlternateShardIds Using slot: 3825379453870917546 for placing replicas: Node: 172.23.105.122:8091, numShards: 122, minShardCapacity: 200, Index name: idx1_F77s6Kif, defnId: 12696598913505312438, replicaId: 1, partnId: 2 |
Node: 172.23.106.176:8091, numShards: 185, minShardCapacity: 200, Index name: idx1_F77s6Kif, defnId: 12696598913505312438, replicaId: 0, partnId: 2 |
as it is least loaded
|
2023-10-04T16:53:43.984-07:00 [Info] Planner::populateAlternateShardIds Using slot: 3825379453870917546 for placing replicas: Node: 172.23.97.109:8091, numShards: 200, minShardCapacity: 200, Index name: idx2_hRnSVM, defnId: 16934562483263348526, replicaId: 2, partnId: 3 |
Node: 172.23.97.108:8091, numShards: 214, minShardCapacity: 200, Index name: idx2_hRnSVM, defnId: 16934562483263348526, replicaId: 0, partnId: 3 |
Node: 172.23.106.176:8091, numShards: 187, minShardCapacity: 200, Index name: idx2_hRnSVM, defnId: 16934562483263348526, replicaId: 1, partnId: 3 |
Node: 172.23.106.171:8091, numShards: 152, minShardCapacity: 200, Index name: idx2_hRnSVM, defnId: 16934562483263348526, replicaId: 3, partnId: 3 |
as it is least loaded
|
panic: runtime error: index out of range [1] with length 1 |
|
goroutine 3003587 [running]: |
github.com/couchbase/indexing/secondary/planner.assignAlternateIds(0x1b2568f?, {0xc01accb350, 0xc01accd830}, 0x60?) |
/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/indexing/secondary/planner/executor.go:6573 +0x76d |
github.com/couchbase/indexing/secondary/planner.PopulateAlternateShardIds(0xc027162fc0, {0xc019fffe00, 0x2d, 0x28?}, 0x0?) |
/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/indexing/secondary/planner/executor.go:6000 +0x791 |
github.com/couchbase/indexing/secondary/planner.rebalance({0x1a83f62, 0x9}, 0xc0169bd5e8, 0xc0169bd120, {0xc01af7b800, 0x536, 0x700}, {0x0, 0x0, 0x0}, ...) |
/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/indexing/secondary/planner/executor.go:2612 +0x9ea |
github.com/couchbase/indexing/secondary/planner.rebalance({0x1a83f62, 0x9}, 0xc0169bd5e8, 0xc0439fa000, {0x0, 0x0, 0x0}, {0x2e3bb78, 0x0, 0x0}, ...) |
/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/indexing/secondary/planner/executor.go:2608 +0xc65 |
github.com/couchbase/indexing/secondary/planner.executeRebal(0x20?, {0x1a83f62?, 0x195?}, 0xc0169bd458?, {0x4741ee?, 0x4426c7?, 0xc0169bd440?}, {0x2e3bb78, 0x0, 0x0}, ...) |
/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/indexing/secondary/planner/executor.go:1954 +0x65 |
github.com/couchbase/indexing/secondary/planner.ExecuteRebalanceInternal({0x7fff5669f768, 0xe}, {{0xc098cddcc0, 0x20}, {0x0, 0x0, 0x0}, {0xc098cddce0, 0x19}, {0xc01b8aa2c0, ...}, ...}, ...) |
/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/indexing/secondary/planner/executor.go:238 +0x846 |
github.com/couchbase/indexing/secondary/planner.ExecuteRebalance({0x7fff5669f768, 0xe}, {{0xc098cddcc0, 0x20}, {0x0, 0x0, 0x0}, {0xc098cddce0, 0x19}, {0xc01b8aa2c0, ...}, ...}, ...) |
/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/indexing/secondary/planner/executor.go:184 +0x190 |
github.com/couchbase/indexing/secondary/indexer.(*ShardRebalancer).initRebalAsync(0xc0763fa900) |
/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/indexing/secondary/indexer/shard_rebalancer.go:297 +0x9b0 |
created by github.com/couchbase/indexing/secondary/indexer.NewShardRebalancer
|
/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/indexing/secondary/indexer/shard_rebalancer.go:199 +0x9b7 |
Cbcollect logs:
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1696479331/collectinfo-2023-10-05T042415-ns_1%40172.23.105.122.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1696479331/collectinfo-2023-10-05T042415-ns_1%40172.23.106.171.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1696479331/collectinfo-2023-10-05T042415-ns_1%40172.23.106.176.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1696479331/collectinfo-2023-10-05T042415-ns_1%40172.23.106.30.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1696479331/collectinfo-2023-10-05T042415-ns_1%40172.23.96.198.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1696479331/collectinfo-2023-10-05T042415-ns_1%40172.23.96.230.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1696479331/collectinfo-2023-10-05T042415-ns_1%40172.23.96.245.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1696479331/collectinfo-2023-10-05T042415-ns_1%40172.23.97.100.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1696479331/collectinfo-2023-10-05T042415-ns_1%40172.23.97.108.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1696479331/collectinfo-2023-10-05T042415-ns_1%40172.23.97.109.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1696479331/collectinfo-2023-10-05T042415-ns_1%40172.23.97.66.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1696479331/collectinfo-2023-10-05T042415-ns_1%40172.23.97.67.zip
Attachments
For Gerrit Dashboard: MB-58967 | ||||||
---|---|---|---|---|---|---|
# | Subject | Branch | Project | Status | CR | V |
198412,7 | MB-58967 Compute targetNodes per definition | unstable | indexing | Status: MERGED | +2 | +1 |
200157,5 | MB-58967 handle shardsIds for primary index movements and minor fix | unstable | indexing | Status: MERGED | +2 | +1 |