Details
-
Bug
-
Resolution: Fixed
-
Critical
-
7.6.0
-
Untriaged
-
0
-
Unknown
Description
Stack trace ( on 108 )
2023-11-23T14:04:44.913-08:00 [Info] KVSender::sendMutationTopicRequest Projector 172.23.96.230:9999 Topic INIT_STREAM_TOPIC_ae1f817c72b2ab5903d521ce2af25b69 bucket2:scope_0:coll_0 |
Instances [indexInstance:<instId:815156907363401732 state:IndexPending definition:<defnID:3723100210287659059 bucket:bucket2 scope:scope_0 collection:coll_0 isPrimary:false name:idx6_zYa6 using:0 exprType:N1QL secExpressions:<ud>([`city`])</ud> partitionScheme:SINGLE whereExpression:<ud>()</ud> > singlePartn:endpoints:"172.23.97.108:9103" > ] |
2023-11-23T14:04:44.913-08:00 [Info] TK StreamBegin INIT_STREAM bucket3:scope_4:coll_1 979 81308018378475 131154 27179 172.23.96.230:8091. HWT [131154-131154,131154]. |
2023-11-23T14:04:44.913-08:00 [Fatal] PlasmaSlice::Statistics Error observed when processing Statistics on instId: 17681455313528138243, partnId: 3 |
panic: runtime error: invalid memory address or nil pointer dereference [recovered]
|
panic: runtime error: invalid memory address or nil pointer dereference
|
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x11d7754] |
|
goroutine 2577760 [running]: |
github.com/couchbase/indexing/secondary/indexer.(*plasmaSlice).Statistics.func1()
|
/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/indexing/secondary/indexer/plasma_slice.go:2849 +0xd4 |
panic({0x17b3680?, 0x2e11150?}) |
/home/couchbase/.cbdepscache/exploded/x86_64/go-1.21.3/go/src/runtime/panic.go:914 +0x21f |
github.com/couchbase/nitro/skiplist.(*Skiplist).GetStats(...)
|
/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/nitro/skiplist/stats.go:139 |
github.com/couchbase/plasma.(*Plasma).GetNumPages(...)
|
/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/plasma/plasma.go:1162 |
github.com/couchbase/plasma.(*Plasma).getStats(_, _)
|
/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/plasma/plasma.go:1175 +0xf4 |
github.com/couchbase/plasma.(*Plasma).GetPreparedStats(...)
|
/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/plasma/plasma.go:1157 |
github.com/couchbase/indexing/secondary/indexer.(*plasmaSlice).Statistics(0xc088661200, 0x0) |
/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/indexing/secondary/indexer/plasma_slice.go:2864 +0x11c |
github.com/couchbase/indexing/secondary/indexer.(*storageMgr).getIndexStorageStats(0xc00274b520, 0x0) |
/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/indexing/secondary/indexer/storage_manager.go:1835 +0xbde |
github.com/couchbase/indexing/secondary/indexer.(*storageMgr).handleStats.func1()
|
/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/indexing/secondary/indexer/storage_manager.go:1588 +0x105 |
created by github.com/couchbase/indexing/secondary/indexer.(*storageMgr).handleStats in goroutine 1492 |
/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/indexing/secondary/indexer/storage_manager.go:1582 +0xb6 |
2023-11-23T14:04:45.873-08:00 [Info] Indexer started with command line: [/opt/couchbase/bin/indexer -adminPort=9100 -scanPort=9101 -httpPort=9102 -streamInitPort=9103 -streamCatchupPort=9104 -streamMaintPort=9105 --httpsPort=19102 --certFile=/opt/couchbase/var/lib/couchbase/config/certs/chain.pem --keyFile=/opt/couchbase/var/lib/couchbase/config/certs/pkey.pem --caFile=/opt/couchbase/var/lib/couchbase/config/certs/ca.pem --clientCertFile=/opt/couchbase/var/lib/couchbase/config/certs/client_chain.pem --clientKeyFile=/opt/couchbase/var/lib/couchbase/config/certs/client_pkey.pem -ipv4=required -ipv6=optional -cluster=127.0.0.1:8091 -storageDir=/data/@2i -diagDir=/opt/couchbase/var/lib/couchbase/crash -logDir=/opt/couchbase/var/lib/couchbase/logs -nodeUUID=ae1f817c72b2ab5903d521ce2af25b69 -isEnterprise=true --deploymentModel=default] |
Second panic also on 108
2023-11-23T14:30:15.295-08:00 [Fatal] PlasmaSlice::Statistics Error observed when processing Statistics on instId: 1345553842805662843, partnId: 2 |
panic: runtime error: invalid memory address or nil pointer dereference [recovered]
|
panic: runtime error: invalid memory address or nil pointer dereference
|
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x11d7754] |
|
goroutine 4797252 [running]: |
github.com/couchbase/indexing/secondary/indexer.(*plasmaSlice).Statistics.func1()
|
/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/indexing/secondary/indexer/plasma_slice.go:2849 +0xd4 |
panic({0x17b3680?, 0x2e11150?}) |
/home/couchbase/.cbdepscache/exploded/x86_64/go-1.21.3/go/src/runtime/panic.go:914 +0x21f |
github.com/couchbase/nitro/skiplist.(*Skiplist).GetStats(...)
|
/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/nitro/skiplist/stats.go:139 |
github.com/couchbase/plasma.(*Plasma).GetNumPages(...)
|
/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/plasma/plasma.go:1162 |
github.com/couchbase/plasma.(*Plasma).getStats(_, _)
|
/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/plasma/plasma.go:1175 +0xf4 |
github.com/couchbase/plasma.(*Plasma).GetPreparedStats(...)
|
/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/plasma/plasma.go:1157 |
github.com/couchbase/indexing/secondary/indexer.(*plasmaSlice).Statistics(0xc084601800, 0x0) |
/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/indexing/secondary/indexer/plasma_slice.go:2864 +0x11c |
github.com/couchbase/indexing/secondary/indexer.(*storageMgr).getIndexStorageStats(0xc0064dc000, 0x0) |
/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/indexing/secondary/indexer/storage_manager.go:1835 +0xbde |
github.com/couchbase/indexing/secondary/indexer.(*storageMgr).handleStats.func1()
|
/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/indexing/secondary/indexer/storage_manager.go:1588 +0x105 |
created by github.com/couchbase/indexing/secondary/indexer.(*storageMgr).handleStats in goroutine 844 |
/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/indexing/secondary/indexer/storage_manager.go:1582 +0xb6 |
2023-11-23T14:30:16.263-08:00 [Info] Indexer started with command line: [/opt/couchbase/bin/indexer -adminPort=9100 -scanPort=9101 -httpPort=9102 -streamInitPort=9103 -streamCatchupPort=9104 -streamMaintPort=9105 --httpsPort=19102 --certFile=/opt/couchbase/var/lib/couchbase/config/certs/chain.pem --keyFile=/opt/couchbase/var/lib/couchbase/config/certs/pkey.pem --caFile=/opt/couchbase/var/lib/couchbase/config/certs/ca.pem --clientCertFile=/opt/couchbase/var/lib/couchbase/config/certs/client_chain.pem --clientKeyFile=/opt/couchbase/var/lib/couchbase/config/certs/client_pkey.pem -ipv4=required -ipv6=optional -cluster=127.0.0.1:8091 -storageDir=/data/@2i -diagDir=/opt/couchbase/var/lib/couchbase/crash -logDir=/opt/couchbase/var/lib/couchbase/logs -nodeUUID=ae1f817c72b2ab5903d521ce2af25b69 -isEnterprise=true --deploymentModel=default] |
2023-11-23T14:30:16.263-08:00 [Info] Indexer DeploymentModel is set to: default |
2023-11-23T14:30:16.263-08:00 [Info] Setting ipv6=false |
2023-11-23T14:30:16.269-08:00 [Info] Indexer::NewIndexer Status Warmup |
2023-11-23T14:30:16.285-08:00 [Info] IndexerSettingsManager: Setting numSliceWriters to 8 cgroup.max_cpu_percent: 800 runtime.NumCPU: 800 |
2023-11-23T14:30:16.290-08:00 [Info] Setting buffer block size to 16384 bytes |
2023-11-23T14:30:16.290-08:00 [Info] settingsManager::setGlobalSettings: Indexer # CPU cores: 8, memory quota: 10096738304 bytes (9860096.000 KB, 9629.000 MB, 9.403 GB) |
2023-11-23T14:30:16.290-08:00 [Info] Setting log level to Info |
2023-11-23T14:30:16.290-08:00 [Warn] SetCICLMgrTimeDiffToForceFetch: Singleton Manager in ClusterInfoCacheLite is not set |
2023-11-23T14:30:16.290-08:00 [Warn] SetCICLMgrSleepTimeOnNotifierRestart: Singleton Manager in ClusterInfoCacheLite is not set |
2023-11-23T14:30:16.290-08:00 [Info] CpuThrottle::SetCpuTarget: New cpuTarget: 0.95 |
2023-11-23T14:30:16.290-08:00 [Info] audit::InitAuditService using address http://127.0.0.1:8091 |
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1700779133/collectinfo-2023-11-23T224622-ns_1%40172.23.106.171.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1700779133/collectinfo-2023-11-23T224622-ns_1%40172.23.106.176.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1700779133/collectinfo-2023-11-23T224622-ns_1%40172.23.96.198.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1700779133/collectinfo-2023-11-23T224622-ns_1%40172.23.96.230.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1700779133/collectinfo-2023-11-23T224622-ns_1%40172.23.96.245.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1700779133/collectinfo-2023-11-23T224622-ns_1%40172.23.97.100.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1700779133/collectinfo-2023-11-23T224622-ns_1%40172.23.97.108.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1700779133/collectinfo-2023-11-23T224622-ns_1%40172.23.97.109.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1700779133/collectinfo-2023-11-23T224622-ns_1%40172.23.97.66.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1700779133/collectinfo-2023-11-23T224622-ns_1%40172.23.97.67.zip