Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-33082

Panic in GSI performance tests

    XMLWordPrintable

Details

    • Untriaged
    • Yes

    Description

      Build 6.5.0-2360

      Observed following panic in scan latency plasma dgm case with 20% resident ratio hot workload. Rest of the DGM cases are still running.

      Last good build- 6.5.0-2282

      Panic-

      2019-02-17T17:37:15.191-08:00 [Info] [Queryport ":9101"] connection 172.23.97.8:58386 closed
      panic: close of closed channel
       
      goroutine 471214540 [running]:
      panic(0xee9940, 0xc4ffac9e80)
              /home/couchbase/.cbdepscache/exploded/x86_64/go-1.7.6/go/src/runtime/panic.go:500 +0x1a1
      github.com/couchbase/indexing/secondary/indexer.(*scanCoordinator).serverCallback.func7(0xc4acf23900, 0xc4ffac9e40, 0xc48121b860)
              goproj/src/github.com/couchbase/indexing/secondary/indexer/scan_coordinator.go:309 +0x192
      created by github.com/couchbase/indexing/secondary/indexer.(*scanCoordinator).serverCallback
              goproj/src/github.com/couchbase/indexing/secondary/indexer/scan_coordinator.go:312 +0x62a
      panic: close of closed channel
       
      goroutine 471214694 [running]:
      panic(0xee9940, 0xc4feada290)
              /home/couchbase/.cbdepscache/exploded/x86_64/go-1.7.6/go/src/runtime/panic.go:500 +0x1a1
      github.com/couchbase/indexing/secondary/indexer.(*scanCoordinator).serverCallback.func7(0xc4292cef00, 0xc4feada210, 0xc45fc265a0)
              goproj/src/github.com/couchbase/indexing/secondary/indexer/scan_coordinator.go:309 +0x192
      created by github.com/couchbase/indexing/secondary/indexer.(*scanCoordinator).serverCallback
              goproj/src/github.com/couchbase/indexing/secondary/indexer/scan_coordinator.go:312 +0x62a
      panic: close of closed channel
       
      goroutine 471214834 [running]:
      panic(0xee9940, 0xc4fef70b90)
              /home/couchbase/.cbdepscache/exploded/x86_64/go-1.7.6/go/src/runtime/panic.go:500 +0x1a1
      github.com/couchbase/indexing/secondary/indexer.(*scanCoordinator).serverCallback.func7(0xc42b8e3400, 0xc4fef70b50, 0xc4c59bd800)
              goproj/src/github.com/couchbase/indexing/secondary/indexer/scan_coordinator.go:309 +0x192
      created by github.com/couchbase/indexing/secondary/indexer.(*scanCoordinator).serverCallback
              goproj/src/github.com/couchbase/indexing/secondary/indexer/scan_coordinator.go:312 +0x62a
      panic: close of closed channel
       
      goroutine 471214883 [running]:
      panic(0xee9940, 0xc4ff5a1b00)
              /home/couchbase/.cbdepscache/exploded/x86_64/go-1.7.6/go/src/runtime/panic.go:500 +0x1a1
      github.com/couchbase/indexing/secondary/indexer.(*scanCoordinator).serverCallback.func7(0xc42b8e3900, 0xc4ff5a1ac0, 0xc4ed639f20)
              goproj/src/github.com/couchbase/indexing/secondary/indexer/scan_coordinator.go:309 +0x192
      created by github.com/couchbase/indexing/secondary/indexer.(*scanCoordinator).serverCallback
              goproj/src/github.com/couchbase/indexing/secondary/indexer/scan_coordinator.go:312 +0x62a
      2019-02-17T17:37:17.915-08:00 [Info] Indexer started with command line: [/opt/couchbase/bin/indexer -adminPort=9100 -scanPort=9101 -httpPort=9102 -streamInitPort=9103 -streamCatchupPort=9104 -streamMaintPort=9105 --httpsPort=19102 --certFile=/opt/couchbase/var/lib/couchbase/config/memcached-cert.pem --keyFile=/opt/couchbase/var/lib/couchbase/config/memcached-key.pem -vbuckets=1024 -cluster=127.0.0.1:8091 -storageDir=/data/@2i -diagDir=/opt/couchbase/var/lib/couchbase/crash -nodeUUID=c92ec00c12b2e8b56a80fbd2b3560ace -ipv6=false -isEnterprise=true]
      2019-02-17T17:37:17.966-08:00 [Info] Indexer::NewIndexer Status Warmup
      
      

      Job- http://perf.jenkins.couchbase.com/job/secondary-plasma-dgm/1818

      Logs-
      Indexer node- https://s3-us-west-2.amazonaws.com/perf-artifacts/jenkins-secondary-plasma-dgm-1818/172.23.97.10.zip
      KV node- https://s3-us-west-2.amazonaws.com/perf-artifacts/jenkins-secondary-plasma-dgm-1818/172.23.97.3.zip
      KV node- https://s3-us-west-2.amazonaws.com/perf-artifacts/jenkins-secondary-plasma-dgm-1818/172.23.97.4.zip
      KV node- https://s3-us-west-2.amazonaws.com/perf-artifacts/jenkins-secondary-plasma-dgm-1818/172.23.97.5.zip
      KV node- https://s3-us-west-2.amazonaws.com/perf-artifacts/jenkins-secondary-plasma-dgm-1818/172.23.97.6.zip
      KV node- https://s3-us-west-2.amazonaws.com/perf-artifacts/jenkins-secondary-plasma-dgm-1818/172.23.97.7.zip

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              mahesh.mandhare Mahesh Mandhare (Inactive)
              mahesh.mandhare Mahesh Mandhare (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty