Details
-
Bug
-
Resolution: Fixed
-
Blocker
-
7.1.0
-
Untriaged
-
-
1
-
Unknown
Description
Build : 7.1.0-2126
Test : -test tests/integration/neo/test_neo_couchstore_milestone4.yml -scope tests/integration/neo/scope_couchstore.yml
Scale : 3
Iteration : 1st
In the longevity test, after a few hrs of the test run, FTS on both nodes - 172.23.105.111 and 172.23.105.107 is crashing continuously. Here is the first crash seen on 172.23.105.111
panic: assignment to entry in nil map
|
|
|
goroutine 46854 [running]:
|
github.com/couchbase/cbft.(*BleveDocumentConfig).BuildDocumentEx(0xc03cec81b0, 0xc09f988e83, 0x24, 0x30, 0xc09f988ea7, 0x4, 0x4, 0xc1a4d31dc8, 0x8, 0x5, ...)
|
/home/couchbase/jenkins/workspace/couchbase-server-unix/cbft/pindex_bleve_doc.go:159 +0x223
|
github.com/couchbase/cbft.(*BleveDestPartition).DataUpdate(0xc0382a0d20, 0xc00029a208, 0x2, 0xc09f988e83, 0x24, 0x28, 0x1fc30, 0xc09f988ea7, 0x4, 0x4, ...)
|
/home/couchbase/jenkins/workspace/couchbase-server-unix/cbft/pindex_bleve.go:2075 +0x1de
|
github.com/couchbase/cbgt.(*DestForwarder).DataUpdate(0xc0ad819ca0, 0xc00029a208, 0x2, 0xc09f988e83, 0x24, 0x28, 0x1fc30, 0xc09f988ea7, 0x4, 0x4, ...)
|
/home/couchbase/jenkins/workspace/couchbase-server-unix/cbgt/dest_forwarder.go:54 +0x15b
|
github.com/couchbase/cbgt.(*GocbcoreDCPFeed).Mutation.func1(0x61ebdb12, 0x30810e6)
|
/home/couchbase/jenkins/workspace/couchbase-server-unix/cbgt/feed_dcp_gocbcore.go:1147 +0x78f
|
github.com/couchbase/cbgt.Timer.func1()
|
/home/couchbase/jenkins/workspace/couchbase-server-unix/cbgt/misc.go:275 +0x2f
|
github.com/rcrowley/go-metrics.(*StandardTimer).Time(0xc190080db0, 0xc0676993c0)
|
/home/couchbase/.cbdepscache/gomodcache/pkg/mod/github.com/rcrowley/go-metrics@v0.0.0-20190826022208-cac0b30c2563/timer.go:227 +0x50
|
github.com/couchbase/cbgt.Timer(0xc0428c9030, 0x139e8a0, 0xc190080db0, 0x0, 0x0)
|
/home/couchbase/jenkins/workspace/couchbase-server-unix/cbgt/misc.go:274 +0x91
|
github.com/couchbase/cbgt.(*GocbcoreDCPFeed).Mutation(0xc033b3f6c0, 0x1fc30, 0xcc1, 0x61ebe92100000000, 0xc000000000, 0x16cc90a30c3e0000, 0xc00190001, 0x19, 0xc09f988e83, 0x24, ...)
|
/home/couchbase/jenkins/workspace/couchbase-server-unix/cbgt/feed_dcp_gocbcore.go:1126 +0x26b
|
github.com/couchbase/gocbcore/v9.(*dcpComponent).OpenStream.func1(0xc0673a3f50, 0xc03cc9d6c0, 0x0, 0x0)
|
/home/couchbase/.cbdepscache/gomodcache/pkg/mod/github.com/couchbase/gocbcore/v9@v9.1.8/dcpcomponent.go:99 +0x4e2
|
github.com/couchbase/gocbcore/v9.(*memdQRequest).tryCallback(0xc03cc9d6c0, 0xc0673a3f50, 0x0, 0x0)
|
/home/couchbase/.cbdepscache/gomodcache/pkg/mod/github.com/couchbase/gocbcore/v9@v9.1.8/memdqpackets.go:187 +0xe8
|
github.com/couchbase/gocbcore/v9.(*memdClient).resolveRequest(0xc0351b4e00, 0xc0673a3f50)
|
/home/couchbase/.cbdepscache/gomodcache/pkg/mod/github.com/couchbase/gocbcore/v9@v9.1.8/memdclient.go:314 +0x4f6
|
github.com/couchbase/gocbcore/v9.(*memdClient).run.func1(0xc0a3f5b260, 0xc05f18c720, 0xc1ac9838dc, 0xc0351b4e00)
|
/home/couchbase/.cbdepscache/gomodcache/pkg/mod/github.com/couchbase/gocbcore/v9@v9.1.8/memdclient.go:347 +0x1a9
|
created by github.com/couchbase/gocbcore/v9.(*memdClient).run
|
/home/couchbase/.cbdepscache/gomodcache/pkg/mod/github.com/couchbase/gocbcore/v9@v9.1.8/memdclient.go:337 +0xbf
|
2022-01-22T02:23:15.975-08:00 [INFO] main: /opt/couchbase/bin/cbft started (v0.6.0/5.5.0)
|
Also on 172.23.105.107, the first crash was also around the same time.
2022-01-22T02:23:12.963-08:00 [INFO] audit: unable to get real userid from request: no web credentials found in request
|
panic: assignment to entry in nil map
|
|
|
goroutine 46712 [running]:
|
github.com/couchbase/cbft.(*BleveDocumentConfig).BuildDocumentEx(0xc03f4e9460, 0xc2020b21b3, 0x24, 0x30, 0xc2020b21d7, 0x4, 0x4, 0xc1add11180, 0x8, 0x5, ...)
|
/home/couchbase/jenkins/workspace/couchbase-server-unix/cbft/pindex_bleve_doc.go:159 +0x223
|
github.com/couchbase/cbft.(*BleveDestPartition).DataUpdate(0xc03f62e620, 0xc0002cf748, 0x3, 0xc2020b21b3, 0x24, 0x28, 0x27eb1, 0xc2020b21d7, 0x4, 0x4, ...)
|
/home/couchbase/jenkins/workspace/couchbase-server-unix/cbft/pindex_bleve.go:2075 +0x1de
|
github.com/couchbase/cbgt.(*DestForwarder).DataUpdate(0xc0354303b0, 0xc0002cf748, 0x3, 0xc2020b21b3, 0x24, 0x28, 0x27eb1, 0xc2020b21d7, 0x4, 0x4, ...)
|
/home/couchbase/jenkins/workspace/couchbase-server-unix/cbgt/dest_forwarder.go:54 +0x15b
|
github.com/couchbase/cbgt.(*GocbcoreDCPFeed).Mutation.func1(0x61ebdb12, 0x288c71f)
|
/home/couchbase/jenkins/workspace/couchbase-server-unix/cbgt/feed_dcp_gocbcore.go:1147 +0x78f
|
github.com/couchbase/cbgt.Timer.func1()
|
/home/couchbase/jenkins/workspace/couchbase-server-unix/cbgt/misc.go:275 +0x2f
|
github.com/rcrowley/go-metrics.(*StandardTimer).Time(0xc16c893b30, 0xc16278da40)
|
/home/couchbase/.cbdepscache/gomodcache/pkg/mod/github.com/rcrowley/go-metrics@v0.0.0-20190826022208-cac0b30c2563/timer.go:227 +0x50
|
github.com/couchbase/cbgt.Timer(0xc24b6c8fc0, 0x139e8a0, 0xc16c893b30, 0x0, 0x0)
|
/home/couchbase/jenkins/workspace/couchbase-server-unix/cbgt/misc.go:274 +0x91
|
github.com/couchbase/cbgt.(*GocbcoreDCPFeed).Mutation(0xc036d65860, 0x27eb1, 0x4c5, 0x61ebe92100000000, 0xc000000000, 0x16cc90a30bc70000, 0xc03500001, 0x1a, 0xc2020b21b3, 0x24, ...)
|
/home/couchbase/jenkins/workspace/couchbase-server-unix/cbgt/feed_dcp_gocbcore.go:1126 +0x26b
|
github.com/couchbase/gocbcore/v9.(*dcpComponent).OpenStream.func1(0xc16813d6e0, 0xc03f7b4380, 0x0, 0x0)
|
/home/couchbase/.cbdepscache/gomodcache/pkg/mod/github.com/couchbase/gocbcore/v9@v9.1.8/dcpcomponent.go:99 +0x4e2
|
github.com/couchbase/gocbcore/v9.(*memdQRequest).tryCallback(0xc03f7b4380, 0xc16813d6e0, 0x0, 0x0)
|
/home/couchbase/.cbdepscache/gomodcache/pkg/mod/github.com/couchbase/gocbcore/v9@v9.1.8/memdqpackets.go:187 +0xe8
|
github.com/couchbase/gocbcore/v9.(*memdClient).resolveRequest(0xc03791e2a0, 0xc16813d6e0)
|
/home/couchbase/.cbdepscache/gomodcache/pkg/mod/github.com/couchbase/gocbcore/v9@v9.1.8/memdclient.go:314 +0x4f6
|
github.com/couchbase/gocbcore/v9.(*memdClient).run.func1(0xc0c168c120, 0xc035302de0, 0xc04bab2b60, 0xc03791e2a0)
|
/home/couchbase/.cbdepscache/gomodcache/pkg/mod/github.com/couchbase/gocbcore/v9@v9.1.8/memdclient.go:347 +0x1a9
|
created by github.com/couchbase/gocbcore/v9.(*memdClient).run
|
/home/couchbase/.cbdepscache/gomodcache/pkg/mod/github.com/couchbase/gocbcore/v9@v9.1.8/memdclient.go:337 +0xbf
|
2022-01-22T02:23:15.634-08:00 [INFO] main: /opt/couchbase/bin/cbft started (v0.6.0/5.5.0)
|
The last time the longevity test was run was with build 7.1.0-2097. This issue was not seen in that run. Could this be a regression due to the following change ?
Commit: 466c88b6f029b8fed4d818d74e3f16e28356328b in build: couchbase-server-7.1.0-2102
|
MB-49307: Upgrade to gocbcore@v9.1.8
|
|
|
Change-Id: I2f46d7997e4b15d59c3b8448542339bc78fbd6b7
|
Reviewed-on: https://review.couchbase.org/c/cbft/+/169047
|
Well-Formed: Build Bot <build@couchbase.com>
|
Reviewed-by: Sitaram Vemulapalli <sitaram.vemulapalli@couchbase.com>
|
Tested-by: Abhinav Dangeti <abhinav@couchbase.com>
|
Author: Abhinav Dangeti <abhinav@couchbase.com>
|
Committer: Abhinav Dangeti <abhinav@couchbase.com>
|
Attachments
| For Gerrit Dashboard: MB-50559 | ||||||
|---|---|---|---|---|---|---|
| # | Subject | Branch | Project | Status | CR | V |
| 169368,3 | MB-50559: NULL document bytes does not cause json.Unmarshal to fail | master | cbft | Status: MERGED | +2 | +1 |