Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-49873

StreamReader Panic Err runtime error: index out of range [4] with length 4

    XMLWordPrintable

Details

    • Bug
    • Resolution: Not a Bug
    • Major
    • 7.1.0
    • 7.1.0
    • ns_server
    • None
    • Untriaged
    • 1
    • Unknown

    Description

      Build: 7.1.0 build 1624

      • Cluster with 3 nodes - kv, fts, index and n1ql
      • Created bucket and 100 scopes and overall 750 collections
      • Created a primary index with below:

        CREATE PRIMARY INDEX `idx1` ON `bucket1`.`scope_0`.`coll_0`
        

      Index not online. Seeing indexer crashing continuously and seeing below panic in the indexer log of node: 172.23.96.119

      2021-12-02T12:32:10.397-08:00 [Fatal] StreamReader Panic Err runtime error: index out of range [4] with length 4
      2021-12-02T12:32:10.397-08:00 [Info] TK SystemEvent INIT_STREAM bucket1:scope_0:coll_0 3 131860626422156 2 1 9 2
      2021-12-02T12:32:10.397-08:00 [Fatal]   /home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/indexing/secondary/logging/logging.go:312 +0x25
      github.com/couchbase/indexing/secondary/indexer.(*mutationStreamReader).panicHandler(0xc0072e0c30)
              /home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/indexing/secondary/indexer/stream_reader.go:391 +0x21d
      panic(0x1268d80, 0xc0061dd2c0)
              /home/couchbase/.cbdepscache/exploded/x86_64/go-1.16.5/go/src/runtime/panic.go:965 +0x1b9
      github.com/couchbase/indexing/secondary/indexer.(*streamWorker).updateVbuuidInFilter(0xc0059ab970, 0xc0072f8040)
              /home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/indexing/secondary/indexer/stream_reader.go:1403 +0x46a
      github.com/couchbase/indexing/secondary/indexer.(*streamWorker).handleSingleKeyVersion(0xc0059ab970, 0xc0061dd290, 0x16, 0x4, 0x5a2d5f29d263, 0x1, 0xc007f81bc0, 0xc006342905)
              /home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/indexing/secondary/indexer/stream_reader.go:820 +0xb9f
      github.com/couchbase/indexing/secondary/indexer.(*streamWorker).handleKeyVersions(0xc0059ab970, 0xc0061dd290, 0x16, 0x4, 0x5a2d5f29d263, 0x1, 0xc0006afba0, 0x3, 0x4, 0x448b05)
              /home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/indexing/secondary/indexer/stream_reader.go:697 +0xae
      github.com/couchbase/indexing/secondary/indexer.(*streamWorker).start(0xc0059ab970)
              /home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/indexing/secondary/indexer/stream_reader.go:680 +0xa8
      created by github.com/couchbase/indexing/secondary/indexer.CreateMutationStreamReader
              /home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/indexing/secondary/indexer/stream_reader.go:133 +0x60a
       
      2021-12-02T12:32:10.397-08:00 [Error] MutationMgr::listenWorkerMsgs Stream Reader Received for StreamId: INIT_STREAM, Err: runtime error: index out of range [4] with length 4
      2021-12-02T12:32:10.397-08:00 [Fatal] Indexer::handleWorkerMsgs Fatal Error On Worker Channel &{streamId:3 err:{code:9 severity:0 category:4 cause:{x:4 y:4 signed:true code:0} msg:}}
      panic: interface conversion: indexer.Message is *indexer.MsgStreamError, not *indexer.MsgError
       
      goroutine 1 [running]:
      github.com/couchbase/indexing/secondary/indexer.(*indexer).handleWorkerMsgs(0xc0001ac000, 0x15304a0, 0xc00734a480)
              /home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/indexing/secondary/indexer/indexer.go:1382 +0x1dd7
      github.com/couchbase/indexing/secondary/indexer.(*indexer).run(0xc0001ac000)
              /home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/indexing/secondary/indexer/indexer.go:1061 +0x773
      github.com/couchbase/indexing/secondary/indexer.NewIndexer(0xc0000df1d0, 0x4, 0x1381543, 0xf, 0xc0001d3e78)
              /home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/indexing/secondary/indexer/indexer.go:492 +0x2528
      main.main()
              goproj/src/github.com/couchbase/indexing/secondary/cmd/indexer/main.go:182 +0x1565
      Initializing write barrier = 8000
      

      Logs:

      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1638478288/collectinfo-2021-12-02T205130-ns_1%40172.23.96.117.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1638478288/collectinfo-2021-12-02T205130-ns_1%40172.23.96.118.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1638478288/collectinfo-2021-12-02T205130-ns_1%40172.23.96.119.zip

      Attachments

        For Gerrit Dashboard: MB-49873
        # Subject Branch Project Status CR V

        Activity

          People

            girish.benakappa Girish Benakappa
            girish.benakappa Girish Benakappa
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty