Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-57746

Panic seen in projector

    XMLWordPrintable

Details

    • Untriaged
    • 0
    • Unknown

    Description

      This was observed with the provisioned cluster system test run on an AWS cluster.

      The config used - 3 KV+ 2 GSI/Query (c5.4xlarge instances for all the nodes) which was then scaled to 4 KV+ 3 GSI/Query ( c5.4xlarge)

      There are quite a few panics. Not sure if all of them are because of the same reason.

       
      panic: runtime error: invalid memory address or nil pointer dereference
      [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x863afc]
       
      goroutine 13899 [running]:
      github.com/couchbase/indexing/secondary/dcp/transport/client.(*Client).SetWriteDeadline(...)
      	/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/indexing/secondary/dcp/transport/client/mc.go:90
      github.com/couchbase/indexing/secondary/dcp/transport/client.(*Client).SetMcdConnectionWriteDeadline(0x0)
      	/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/indexing/secondary/dcp/transport/client/mc.go:119 +0x5c
      github.com/couchbase/indexing/secondary/dcp/transport/client.(*DcpFeed).sendBufferAck.func1(0xc007050e40, 0x7f0d6025a410?, {0xc000333a00, 0x73}, 0xc0044e0f18, 0x84a3af0?)
      	/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/indexing/secondary/dcp/transport/client/dcp_feed.go:1275 +0x6f
      github.com/couchbase/indexing/secondary/dcp/transport/client.(*DcpFeed).sendBufferAck(0xc007050e40, 0xe0?, 0xc0?)
      	/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/indexing/secondary/dcp/transport/client/dcp_feed.go:1290 +0x1a9
      github.com/couchbase/indexing/secondary/dcp/transport/client.(*DcpFeed).DequeueMutations(0xc007050e40, 0xc003f427e0, 0xc007ce2788?)
      	/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/indexing/secondary/dcp/transport/client/dcp_feed.go:278 +0x185
      created by github.com/couchbase/indexing/secondary/dcp/transport/client.NewDcpFeed
      	/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/indexing/secondary/dcp/transport/client/dcp_feed.go:154 +0x865
      2023-07-05T21:08:17.467+00:00 [Info] Projector started with command line: [/opt/couchbase/bin/projector --httpsPort=9999 --certFile=/opt/couchbase/var/lib/couchbase/config/certs/chain.pem --keyFile=/opt/couchbase/var/lib/couchbase/config/certs/pkey.pem --caFile=/opt/couchbase/var/lib/couchbase/config/certs/ca.pem -ipv4=required -ipv6=off -kvaddrs=127.0.0.1:11210 -adminport=:9999 -diagDir=/opt/couchbase/var/lib/couchbase/crash 127.0.0.1:8091]
      

      cbcollect

      https://cb-engineering.s3.amazonaws.com/SysTest_6Jul_2/collectinfo-2023-07-06T164300-ns_1%40svc-d-node-001.gwn7sxdqdq5j7kve.sandbox.nonprod-project-avengers.com.zip
      https://cb-engineering.s3.amazonaws.com/SysTest_6Jul_2/collectinfo-2023-07-06T164300-ns_1%40svc-d-node-002.gwn7sxdqdq5j7kve.sandbox.nonprod-project-avengers.com.zip
      https://cb-engineering.s3.amazonaws.com/SysTest_6Jul_2/collectinfo-2023-07-06T164300-ns_1%40svc-d-node-003.gwn7sxdqdq5j7kve.sandbox.nonprod-project-avengers.com.zip
      https://cb-engineering.s3.amazonaws.com/SysTest_6Jul_2/collectinfo-2023-07-06T164300-ns_1%40svc-d-node-006.gwn7sxdqdq5j7kve.sandbox.nonprod-project-avengers.com.zip
      https://cb-engineering.s3.amazonaws.com/SysTest_6Jul_2/collectinfo-2023-07-06T164300-ns_1%40svc-qi-node-004.gwn7sxdqdq5j7kve.sandbox.nonprod-project-avengers.com.zip
      https://cb-engineering.s3.amazonaws.com/SysTest_6Jul_2/collectinfo-2023-07-06T164300-ns_1%40svc-qi-node-005.gwn7sxdqdq5j7kve.sandbox.nonprod-project-avengers.com.zip
      https://cb-engineering.s3.amazonaws.com/SysTest_6Jul_2/collectinfo-2023-07-06T164300-ns_1%40svc-qi-node-007.gwn7sxdqdq5j7kve.sandbox.nonprod-project-avengers.com.zip
      

      Also I'm unsure if this is related to a KV crash bug https://issues.couchbase.com/browse/MB-57745

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            varun.velamuri Varun Velamuri
            pavan.pb Pavan PB
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty