Details
-
Bug
-
Resolution: Fixed
-
Major
-
7.0.0
-
Untriaged
-
Centos 64-bit
-
1
-
No
Description
Build : 7.0.0-5295
Test : -test tests/2i/cheshirecat/test_idx_clusterops_cheshire_cat_recovery.yml -scope tests/2i/cheshirecat/scope_idx_cheshire_cat_dgm.yml -scale 2
Scale : 3
Iteration : 9th
Day : 4th
Panic observed on Indexer at *.216
No Indexer crash observed.
2021-06-21T19:25:14.235-07:00 [Info] MutationStreamReader:Shutdown StreamReader INIT_STREAM
|
2021-06-21T19:25:14.237-07:00 [Error] DATP[->dataport ":9103"] Accept() Error: accept tcp4 0.0.0.0:9103: use of closed network connection
|
2021-06-21T19:25:14.237-07:00 [Info] DATP[->dataport ":9103"] ... stopped
|
2021-06-21T19:25:14.237-07:00 [Error] DATP[->dataport ":9103"] gen-server crashed: assignment to entry in nil map
|
2021-06-21T19:25:14.237-07:00 [Error] /home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/indexing/secondary/logging/logging.go:312 +0x22
|
github.com/couchbase/indexing/secondary/dataport.(*Server).genServer.func1(0xc015190600)
|
/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/indexing/secondary/dataport/server.go:245 +0xf7
|
panic(0x10e9b60, 0x14b8f90)
|
/home/couchbase/.cbdepscache/exploded/x86_64/go-1.13.7/go/src/runtime/panic.go:679 +0x1b2
|
github.com/couchbase/indexing/secondary/dataport.(*Server).genServer.func3(0xc01c08ed00, 0x1, 0x1)
|
/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/indexing/secondary/dataport/server.go:329 +0x36d
|
github.com/couchbase/indexing/secondary/dataport.(*Server).genServer(0xc015190600, 0xc022dca960, 0xc022dca9c0)
|
/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/indexing/secondary/dataport/server.go:372 +0x95b
|
created by github.com/couchbase/indexing/secondary/dataport.NewServer
|
/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/indexing/secondary/dataport/server.go:185 +0x628
|
|
|
2021-06-21T19:25:14.237-07:00 [Error] DATP[->dataport ":9103"] handleClose() crashed: close of closed channel
|
2021-06-21T19:25:14.238-07:00 [Error] /home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/indexing/secondary/logging/logging.go:312 +0x22
|
github.com/couchbase/indexing/secondary/dataport.(*Server).handleClose.func1(0xc015190600)
|
/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/indexing/secondary/dataport/server.go:409 +0xf7
|
panic(0x10e9b60, 0x14b90d0)
|
/home/couchbase/.cbdepscache/exploded/x86_64/go-1.13.7/go/src/runtime/panic.go:679 +0x1b2
|
github.com/couchbase/indexing/secondary/dataport.(*Server).handleClose(0xc015190600)
|
/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/indexing/secondary/dataport/server.go:424 +0x1bb
|
github.com/couchbase/indexing/secondary/dataport.(*Server).genServer.func1(0xc015190600)
|
/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/indexing/secondary/dataport/server.go:246 +0x173
|
panic(0x10e9b60, 0x14b8f90)
|
/home/couchbase/.cbdepscache/exploded/x86_64/go-1.13.7/go/src/runtime/panic.go:679 +0x1b2
|
github.com/couchbase/indexing/secondary/dataport.(*Server).genServer.func3(0xc01c08ed00, 0x1, 0x1)
|
/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/indexing/secondary/dataport/server.go:329 +0x36d
|
github.com/couchbase/indexing/secondary/dataport.(*Server).genServer(0xc015190600, 0xc022dca960, 0xc022dca9c0)
|
/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/indexing/secondary/dataport/server.go:372 +0x95b
|
created by github.com/couchbase/indexing/secondary/dataport.NewServer
|
/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/indexing/secondary/dataport/server.go:185 +0x628
|
|
|
2021-06-21T19:25:14.239-07:00 [Info] Timekeeper::handleRemoveKeyspaceFromStream
|
Message: MsgStreamUpdate
|
Type: REMOVE_KEYSPACE_FROM_STREAM
|
Stream: INIT_STREAM
|
KeyspaceId: bucket4:_default:U05XEHaY7h8cBao6RJcK03V
|
BuildTS: []
|
IndexList: []
|
Async: false
|
SessionId: 1
|
CollectionId:
|
CollectionAware: false
|
EnableOSO: false
|
RestartTs: <nil>
|
2021-06-21T19:25:14.239-07:00 [Info] Timekeeper::removeKeyspaceFromStream remove index 1945269381096944475 from stream INIT_STREAM keyspaceId bucket4:_default:U05XEHaY7h8cBao6RJcK03V
|
2021-06-21T19:25:14.239-07:00 [Info] Timekeeper::stopTimer INIT_STREAM bucket4:_default:U05XEHaY7h8cBao6RJcK03V
|
2021-06-21T19:25:14.239-07:00 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket4:_default:U05XEHaY7h8cBao6RJcK03V Deleted from Stream INIT_STREAM
|
2021-06-21T19:25:14.239-07:00 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap
|
2021-06-21T19:25:14.239-07:00 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed
|
Index nodes - 172.23.107.2, 172.23.107.3, 172.23.107.4, 172.23.107.5, 172.23.97.216
Logs are available here -
cbcollect logs:
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1624333623/collectinfo-2021-06-22T034704-ns_1%40172.23.107.2.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1624333623/collectinfo-2021-06-22T034704-ns_1%40172.23.107.3.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1624333623/collectinfo-2021-06-22T034704-ns_1%40172.23.107.4.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1624333623/collectinfo-2021-06-22T034704-ns_1%40172.23.107.5.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1624333623/collectinfo-2021-06-22T034704-ns_1%40172.23.97.215.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1624333623/collectinfo-2021-06-22T034704-ns_1%40172.23.97.216.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1624333623/collectinfo-2021-06-22T034704-ns_1%40172.23.97.227.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1624333623/collectinfo-2021-06-22T034704-ns_1%40172.23.97.232.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1624333623/collectinfo-2021-06-22T034704-ns_1%40172.23.97.235.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1624333623/collectinfo-2021-06-22T034704-ns_1%40172.23.97.236.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1624333623/collectinfo-2021-06-22T034704-ns_1%40172.23.97.237.zip
Attachments
Issue Links
- backports to
-
MB-47634 [BP 7.0.2 MB-47050] - [System Test] PANIC observed in System Test
- Closed