Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-36868

[System Test] Seeing panic msgs in eventing logs while undeploying handlers- "addToAggChan: recover send on closed channel"

    XMLWordPrintable

Details

    Description

      Build : 5.5.6-4733
      Test : -test tests/integration/test_allFeatures_vulcan.yml -scope tests/integration/scope_Xattrs_Vulcan.yml
      Scale : 3
      Day : 1st
      Cluster : centos1

      Seeing these panic msgs on 172.23.96.148 in eventing logs when undeploying handler functions.

      2019-11-12T11:27:27.804-08:00 [Info] Consumer::addToAggChan [worker_bucket_op_complex_function_2:/tmp/127.0.0.1:8091_worker_bucket_op_complex_function_2.sock:13948] Closing dcp feed: {eventing:tbahHeMq-79:172.23.96.148:8096_172.23.106.100:11210_worker_bucket_op_complex_function_2}, count: 0 for bucket: default
      2019-11-12T11:27:27.804-08:00 [Info] Consumer::Stop [worker_bucket_op_complex_function_2:/tmp/127.0.0.1:8091_worker_bucket_op_complex_function_2.sock:13948] Closed all dcpfeed handles
      2019-11-12T11:27:27.804-08:00 [Info] Consumer::Stop [worker_bucket_op_complex_function_2:/tmp/127.0.0.1:8091_worker_bucket_op_complex_function_2.sock:13948] Closing up aggDcpFeed channel
      2019-11-12T11:27:27.804-08:00 [Info] Consumer::Stop [worker_bucket_op_complex_function_2:/tmp/127.0.0.1:8091_worker_bucket_op_complex_function_2.sock:13948] Requested to stop supervisor for Eventing.Consumer
      2019-11-12T11:27:27.805-08:00 [Info] Consumer::Stop [worker_bucket_op_complex_function_2:/tmp/127.0.0.1:8091_worker_bucket_op_complex_function_2.sock:13948] Exiting Consumer::Stop routine
      2019-11-12T11:27:27.805-08:00 [Info] Consumer::handlerFailoverLog [worker_bucket_op_complex_function_2:/tmp/127.0.0.1:8091_worker_bucket_op_complex_function_2.sock:13948] Exiting failover log handling routine
      2019-11-12T11:27:27.805-08:00 [Error] Consumer::addToAggChan [worker_bucket_op_complex_function_2:/tmp/127.0.0.1:8091_worker_bucket_op_complex_function_2.sock:13948] addToAggChan: recover send on closed channel stack trace: goroutine 513 [running]:
      runtime/debug.Stack(0xc421f79498, 0xc7f7e0, 0xc4265c43d0)
              /home/couchbase/.cbdepscache/exploded/x86_64/go-1.7.6/go/src/runtime/debug/stack.go:24 +0x79
      github.com/couchbase/eventing/consumer.(*Consumer).addToAggChan.func1.1(0xd9f797, 0x16, 0xc4204f9400)
              goproj/src/github.com/couchbase/eventing/consumer/process_events.go:775 +0x6e
      panic(0xc7f7e0, 0xc4265c43d0)
              /home/couchbase/.cbdepscache/exploded/x86_64/go-1.7.6/go/src/runtime/panic.go:458 +0x243
      github.com/couchbase/eventing/consumer.(*Consumer).addToAggChan.func1(0xd9f797, 0x16, 0xc4204f9400, 0xc4224e6750)
              goproj/src/github.com/couchbase/eventing/consumer/process_events.go:808 +0x145
      created by github.com/couchbase/eventing/consumer.(*Consumer).addToAggChan
              goproj/src/github.com/couchbase/eventing/consumer/process_events.go:812 +0x5e
      2019-11-12T11:27:27.805-08:00 [Error] Consumer::addToAggChan [worker_bucket_op_complex_function_2:/tmp/127.0.0.1:8091_worker_bucket_op_complex_function_2.sock:13948] addToAggChan: recover send on closed channel stack trace: goroutine 626 [running]:
      runtime/debug.Stack(0xc422238498, 0xc7f7e0, 0xc42c5ef270)
              /home/couchbase/.cbdepscache/exploded/x86_64/go-1.7.6/go/src/runtime/debug/stack.go:24 +0x79
      github.com/couchbase/eventing/consumer.(*Consumer).addToAggChan.func1.1(0xd9f797, 0x16, 0xc4204f9400)
              goproj/src/github.com/couchbase/eventing/consumer/process_events.go:775 +0x6e
      panic(0xc7f7e0, 0xc42c5ef270)
              /home/couchbase/.cbdepscache/exploded/x86_64/go-1.7.6/go/src/runtime/panic.go:458 +0x243
      github.com/couchbase/eventing/consumer.(*Consumer).addToAggChan.func1(0xd9f797, 0x16, 0xc4204f9400, 0xc4202b07e0)
              goproj/src/github.com/couchbase/eventing/consumer/process_events.go:808 +0x145
      created by github.com/couchbase/eventing/consumer.(*Consumer).addToAggChan
              goproj/src/github.com/couchbase/eventing/consumer/process_events.go:812 +0x5e
      
      

      This looks similar to https://issues.couchbase.com/browse/MB-29817

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            faiz.veettil Faiz Veettil (Inactive)
            mihir.kamdar Mihir Kamdar (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty