Details
-
Bug
-
Resolution: Fixed
-
Major
-
7.1.0
-
Untriaged
-
1
-
Unknown
Description
Eventing deployment stuck on eventing ci machine.
2022/01/12 03:53:43 Deploying app: TestCacheOverflowCustom. Item count src bucket: 0 dst bucket: 0
|
2022/01/12 03:53:44 Context: Post to main store, response code: 200 dump: {
|
"code": 0,
|
"info": "Function: TestCacheOverflowCustom stored in temp store"
|
}
|
2022/01/12 03:53:49 Waiting for app: TestCacheOverflowCustom to get deployed
|
2022/01/12 03:53:54 Waiting for app: TestCacheOverflowCustom to get deployed
|
2022/01/12 03:53:59 Waiting for app: TestCacheOverflowCustom to get deployed
|
2022/01/12 03:54:04 Waiting for app: TestCacheOverflowCustom to get deployed
|
2022/01/12 03:54:09 Waiting for app: TestCacheOverflowCustom to get deployed
|
2022/01/12 03:54:14 Waiting for app: TestCacheOverflowCustom to get deployed
|
2022/01/12 03:54:19 Waiting for app: TestCacheOverflowCustom to get deployed
|
Looking at the goroutine stacktrace its stuck in gocb Close.
1 @ 0x43f605 0x40b18f 0x40adcb 0x86e385 0x86c5fb 0x85f9db 0x830c65 0x837c13 0x83687d 0x8ab91f 0x8add6f 0xd893f5 0x9176d4 0xd85a0c 0xd84e12 0xd837d7 0xd81e57 0xd79805 0xd73277 0x7cbd3d 0x7c96b9 0x7c9b37 0x7c9ae0 0xd8eb51 0x4761e1
|
# 0x86e384 github.com/couchbase/gocbcore/v9.(*memdPipelineClient).Close+0x184 /opt/build/godeps/src/github.com/couchbase/gocbcore/v9/memdpipelineclient.go:283
|
# 0x86c5fa github.com/couchbase/gocbcore/v9.(*memdPipeline).Close+0xda /opt/build/godeps/src/github.com/couchbase/gocbcore/v9/memdpipeline.go:155
|
# 0x85f9da github.com/couchbase/gocbcore/v9.(*kvMux).Close+0xda /opt/build/godeps/src/github.com/couchbase/gocbcore/v9/kvmux.go:407
|
# 0x830c64 github.com/couchbase/gocbcore/v9.(*Agent).Close+0x44 /opt/build/godeps/src/github.com/couchbase/gocbcore/v9/agent.go:489
|
# 0x837c12 github.com/couchbase/gocbcore/v9.(*AgentGroup).maybeCloseGlobalAgent+0x152 /opt/build/godeps/src/github.com/couchbase/gocbcore/v9/agentgroup.go:247
|
# 0x83687c github.com/couchbase/gocbcore/v9.(*AgentGroup).OpenBucket+0x15c /opt/build/godeps/src/github.com/couchbase/gocbcore/v9/agentgroup.go:86
|
# 0x8ab91e github.com/couchbase/gocb/v2.(*stdConnectionMgr).openBucket+0xbe /opt/build/godeps/src/github.com/couchbase/gocb/v2/client.go:131
|
# 0x8add6e github.com/couchbase/gocb/v2.(*Cluster).Bucket+0x12e /opt/build/godeps/src/github.com/couchbase/gocb/v2/cluster.go:305
|
# 0xd893f4 github.com/couchbase/eventing/supervisor.glob..func8+0x194 /opt/build/goproj/src/github.com/couchbase/eventing/supervisor/cluster_ops.go:210
|
# 0x9176d3 github.com/couchbase/eventing/util.Retry+0x133 /opt/build/goproj/src/github.com/couchbase/eventing/util/retry.go:65
|
# 0xd85a0b github.com/couchbase/eventing/supervisor.initGocbBucketHandle+0x34b /opt/build/goproj/src/github.com/couchbase/eventing/supervisor/util.go:957
|
# 0xd84e11 github.com/couchbase/eventing/supervisor.(*gocbPool).insertBucket+0x171 /opt/build/goproj/src/github.com/couchbase/eventing/supervisor/util.go:893
|
# 0xd837d6 github.com/couchbase/eventing/supervisor.(*gocbGlobalConfig).maybeRegistergocbBucket+0x9b6 /opt/build/goproj/src/github.com/couchbase/eventing/supervisor/util.go:763
|
# 0xd81e56 github.com/couchbase/eventing/supervisor.(*SuperSupervisor).watchBucketWithGocb+0xd6 /opt/build/goproj/src/github.com/couchbase/eventing/supervisor/util.go:574
|
# 0xd79804 github.com/couchbase/eventing/supervisor.(*SuperSupervisor).spawnApp+0x6a4 /opt/build/goproj/src/github.com/couchbase/eventing/supervisor/super_supervisor.go:797
|
# 0xd73276 github.com/couchbase/eventing/supervisor.(*SuperSupervisor).SettingsChangeCallback+0x17b6 /opt/build/goproj/src/github.com/couchbase/eventing/supervisor/super_supervisor.go:304
|
# 0x7cbd3c github.com/couchbase/cbauth/metakv.RunObserveChildrenV2.func1+0x11c /opt/build/goproj/src/github.com/couchbase/cbauth/metakv/metakv.go:421
|
# 0x7c96b8 github.com/couchbase/cbauth/metakv.doRunObserveChildren+0x438 /opt/build/goproj/src/github.com/couchbase/cbauth/metakv/metakv.go:311
|
# 0x7c9b36 github.com/couchbase/cbauth/metakv.(*store).runObserveChildren+0x96 /opt/build/goproj/src/github.com/couchbase/cbauth/metakv/metakv.go:263
|
# 0x7c9adf github.com/couchbase/cb
|
The issue is similar to MB-45053
Eventing recently updated the gocb and gocbcore version to 2.2.3 and 9.1.4 respectively