Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-34090

Panic in indexer logs while running GSI performance tests

    XMLWordPrintable

Details

    • Bug
    • Resolution: Duplicate
    • Critical
    • None
    • 6.5.0
    • secondary-index
    • Untriaged
    • Yes

    Description

      Build 6.5.0-3143

      Observed following panic in indexer logs while running scan throughput test with multiple indexes with plasma. Did not notice any panic in other tests.

      Job- http://perf.jenkins.couchbase.com/job/secondary-plasma/3577/
      Logs- https://s3-us-west-2.amazonaws.com/perf-artifacts/jenkins-secondary-plasma-3577/172.23.97.10.zip

      Panic-

      2019-05-05T01:15:28.903-07:00 [Info] MutationStreamReader::updateSnapInFilter Huge Snapshot Received for bucket-1 947 MAINT_STREAM 0-311222
      2019-05-05T01:15:28.923-07:00 [Fatal] IndexScanSource - panic detected while processing defnId:8372306296378590400, instId:18206720063615470450, index:bucket-1/myindex3, type:scan, partitions:[0], scans: <ud>([{["000015"] ["000028"] 3 range [] <nil>}])</ud>, consistency:any_consistency, requestId:./cbindexperf190542923
      panic: runtime error: invalid memory address or nil pointer dereference [recovered]
              panic: runtime error: invalid memory address or nil pointer dereference
      [signal SIGSEGV: segmentation violation code=0x1 addr=0x28 pc=0xafed76]
       
      goroutine 14933483 [running]:
      panic(0xf3b000, 0xc420016080)
              /home/couchbase/.cbdepscache/exploded/x86_64/go-1.7.6/go/src/runtime/panic.go:500 +0x1a1
      github.com/couchbase/indexing/secondary/indexer.(*IndexScanSource).Routine.func1(0xc51a113c70)
              goproj/src/github.com/couchbase/indexing/secondary/indexer/scan_pipeline.go:133 +0xcf
      panic(0xf3b000, 0xc420016080)
              /home/couchbase/.cbdepscache/exploded/x86_64/go-1.7.6/go/src/runtime/panic.go:458 +0x243
      github.com/couchbase/plasma.(*Iterator).Get(0xc42543d140, 0x0)
              goproj/src/github.com/couchbase/plasma/iterator.go:150 +0x26
      github.com/couchbase/plasma.(*MVCCIterator).Key(0xc42ac2f8c0, 0xc42826d7f8, 0xaff03d, 0xc516b68f80)
              goproj/src/github.com/couchbase/plasma/mvcc.go:232 +0x2f
      github.com/couchbase/plasma.(*MVCCIterator).refresh(0xc42ac2f8c0)
              goproj/src/github.com/couchbase/plasma/mvcc.go:191 +0x2f
      github.com/couchbase/plasma.(*MVCCIterator).refreshIfNecessary(0xc42ac2f8c0)
              goproj/src/github.com/couchbase/plasma/mvcc.go:208 +0x58
      github.com/couchbase/plasma.(*MVCCIterator).Next(0xc42ac2f8c0, 0x28, 0x28)
              goproj/src/github.com/couchbase/plasma/mvcc.go:216 +0x58
      github.com/couchbase/indexing/secondary/indexer.(*plasmaSnapshot).Iterate(0xc50738e460, 0x1a63220, 0xc49ab149c0, 0x1a676c0, 0xc49ab14a00, 0x1a676c0, 0xc49ab14a40, 0x3, 0x11a2510, 0xc503027d40, ...)
              goproj/src/github.com/couchbase/indexing/secondary/indexer/plasma_slice.go:2093 +0x322
      github.com/couchbase/indexing/secondary/indexer.(*plasmaSnapshot).Range(0xc50738e460, 0x1a63220, 0xc49ab149c0, 0x1a676c0, 0xc49ab14a00, 0x1a676c0, 0xc49ab14a40, 0x3, 0xc503027d40, 0xc42826db01, ...)
              goproj/src/github.com/couchbase/indexing/secondary/indexer/plasma_slice.go:2030 +0xb2
      github.com/couchbase/indexing/secondary/indexer.scanSingleSlice(0xc517f53900, 0x1a676c0, 0xc49ab14a00, 0x1a676c0, 0xc49ab14a40, 0x3, 0x10af94a, 0x5, 0x0, 0x0, ...)
              goproj/src/github.com/couchbase/indexing/secondary/indexer/scan_scatter.go:206 +0x41b
      github.com/couchbase/indexing/secondary/indexer.scanOne(0xc517f53900, 0x1a676c0, 0xc49ab14a00, 0x1a676c0, 0xc49ab14a40, 0x3, 0x10af94a, 0x5, 0x0, 0x0, ...)
              goproj/src/github.com/couchbase/indexing/secondary/indexer/scan_scatter.go:148 +0x11e
      github.com/couchbase/indexing/secondary/indexer.scatter(0xc517f53900, 0x1a676c0, 0xc49ab14a00, 0x1a676c0, 0xc49ab14a40, 0x3, 0x10af94a, 0x5, 0x0, 0x0, ...)
              goproj/src/github.com/couchbase/indexing/secondary/indexer/scan_scatter.go:55 +0xc6
      github.com/couchbase/indexing/secondary/indexer.(*IndexScanSource).Routine(0xc51a113c70, 0x0, 0x0)
              goproj/src/github.com/couchbase/indexing/secondary/indexer/scan_pipeline.go:336 +0x810
      github.com/couchbase/indexing/secondary/pipeline.(*Pipeline).runIt.func1(0xc49ab14a60, 0xc5032b6850)
              goproj/src/github.com/couchbase/indexing/secondary/pipeline/pipeline.go:75 +0x45
      created by github.com/couchbase/indexing/secondary/pipeline.(*Pipeline).runIt
              goproj/src/github.com/couchbase/indexing/secondary/pipeline/pipeline.go:80 +0x66
      

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              mahesh.mandhare Mahesh Mandhare (Inactive)
              mahesh.mandhare Mahesh Mandhare (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty