Details
-
Bug
-
Resolution: Fixed
-
Blocker
-
5.5.0
-
Cluster: atlas_setupA
OS: CentOS 7
CPU: E5-2680 v3 (48 vCPU)
Memory: 256 GB
Disk: Samsung PM863
-
Untriaged
-
Centos 64-bit
-
Yes
Description
Scorch only.
Works fine when single client thread is running FTS queries. But as soon as concurrency added cbft crashes with "Index out of range".
Detected in build 5.5.0-2653, list of changes for 2653:
godeps/src/github.com/blevesearch/bleve
[+] f83cd37 Merge pull request #916 from sreekanth-cb/dvreader_version_1 [+] a8bd3e6 refactoring the dvReader reuse
[+] b22d4fd Merge pull request #922 from abhinavdangeti/master
[+] 7c99671 Micro optimizations while estimating mem needed for search [+] 612a791 Merge pull request #921 from steveyen/optimize-scorch-ExternalID
[+] d784dda Merge pull request #920 from steveyen/optimize-via-vellum-FTS-Reader-API
[+] 23aaeb7 scorch optimize ExternalID() with Segment.DocID() API
[+] 0a3f3e4 scorch optimize via vellum.FST.Reader() API
[+] 96dad7f dvReader improvement
Logs from on of the failed tests:
https://s3-us-west-2.amazonaws.com/perf-artifacts/jenkins-atlas-sdk-2013/172.23.99.211.zip
The panic:
2018-05-21T09:25:31.346-07:00 [INFO] cfg_metakv_lean: getCurMetaKvPlanMeta, val: {"path":"/fts/cbgt/cfg/planPIndexesLean/planPIndexesLean-253becfc92c7dec139691c1fa8a15caf-1526919898522/","uuid":"550805f515a7904b","implVersion":"5.5.0"}
{{ 2018-05-21T09:25:36.347-07:00 [INFO] cfg_metakv_lean: getCurMetaKvPlanMeta, val: {"path":"/fts/cbgt/cfg/planPIndexesLean/planPIndexesLean-253becfc92c7dec139691c1fa8a15caf-1526919898522/","uuid":"550805f515a7904b","implVersion":"5.5.0"}}}
{{ panic: runtime error: index out of range}}
...
...
/home/couchbase/jenkins/workspace/couchbase-server-unix/godeps/src/github.com/blevesearch/bleve/index_impl.go:458 +0x2c2 fp=0xc42ec19e58 sp=0xc42ec19900 pc=0xa01b82
{{ github.com/couchbase/cbft.(*cacheBleveIndex).SearchInContext(0xc45fe92270, 0x12631c0, 0xc45fe92390, 0xc4400a2070, 0xc44d411ac0, 0xc444cd8800, 0x0)}}
/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/cbft/cache_bleve.go:110 +0x38a fp=0xc42ec19f18 sp=0xc42ec19e58 pc=0xb3f77a
{{ github.com/blevesearch/bleve.MultiSearch.func1(0x126d500, 0xc45fe92270, 0xc4400a2070)}}
/home/couchbase/jenkins/workspace/couchbase-server-unix/godeps/src/github.com/blevesearch/bleve/index_alias_impl.go:458 +0x152 fp=0xc42ec19fc8 sp=0xc42ec19f18 pc=0xa0a9d2
{{ runtime.goexit()}}
/home/couchbase/.cbdepscache/exploded/x86_64/go-1.9.6/go/src/runtime/asm_amd64.s:2337 +0x1 fp=0xc42ec19fd0 sp=0xc42ec19fc8 pc=0x45e8b1
{{ created by github.com/blevesearch/bleve.MultiSearch}}
/home/couchbase/jenkins/workspace/couchbase-server-unix/godeps/src/github.com/blevesearch/bleve/index_alias_impl.go:465 +0x19d