[BP 7.0.4] - Eventing: Handler stuck in pausing state after online upgrade and enforcing TLS

Description

STEPS

  1. Cluster consists of 3 nodes (data, data + eventing, index + query) running on 6.6.4-9961.

  2. Create 3 eventing handlers namely bucket_op, timers and n1ql.

  3. Deploy all 3 handlers, load docs into source bucket and verify mutations are processed or not.

  4. Perform online upgrade of the cluster to 6.6.5 - 10072.

  5. Create and deploy 4th handler.

  6. Disable auto failover, enable n2n encryption and Enforce TLS.

  7. Pause all 4 eventing handlers.

OBSERVATION
timers handler is stuck in pausing state.

curl -XGET -k https://Administrator:password@172.23.106.74:18096/api/v1/status { "apps": [ { "composite_status": "paused", "name": "bucket_op", "num_bootstrapping_nodes": 0, "num_deployed_nodes": 1, "deployment_status": true, "processing_status": false, "redeploy_required": false }, { "composite_status": "paused", "name": "n1ql", "num_bootstrapping_nodes": 0, "num_deployed_nodes": 1, "deployment_status": true, "processing_status": false, "redeploy_required": false }, { "composite_status": "pausing", "name": "timers", "num_bootstrapping_nodes": 0, "num_deployed_nodes": 1, "deployment_status": true, "processing_status": false, "redeploy_required": false }, { "composite_status": "paused", "name": "test", "num_bootstrapping_nodes": 0, "num_deployed_nodes": 1, "deployment_status": true, "processing_status": false, "redeploy_required": false } ], "num_eventing_nodes": 1 }

Components

Affects versions

Fix versions

Labels

Environment

Enterprise Edition 6.6.5 build 10072

Link to Log File, atop/blg, CBCollectInfo, Core dump

s3://cb-customers-secure/upgrade-issue/2022-01-07/collectinfo-2022-01-07t095401-ns_1@172.23.106.64.zip s3://cb-customers-secure/upgrade-issue/2022-01-07/collectinfo-2022-01-07t095401-ns_1@172.23.106.75.zip s3://cb-customers-secure/upgrade-issue/2022-01-07/collectinfo-2022-01-07t095401-ns_1@172.23.106.74.zip - eventing

Release Notes Description

None

Attachments

1
  • 11 Mar 2022, 08:39 AM

Activity

Sujay Gad April 8, 2022 at 11:33 AM

Verified the fix on Enterprise Edition 7.0.4 build 7247.

CB robot April 7, 2022 at 3:12 PM

Build couchbase-server-7.0.4-7245 contains eventing-ee commit 74480d3 with commit message:
https://couchbasecloud.atlassian.net/browse/MB-51390#icft=MB-51390 : Timer store change to facilitate lcb handle with early bailout support

CB robot April 7, 2022 at 3:12 PM

Build couchbase-server-7.0.4-7245 contains eventing commit ced088d with commit message:
https://couchbasecloud.atlassian.net/browse/MB-51390#icft=MB-51390 : Wrap timer lcb handle in early bailout wrapper

CB robot April 7, 2022 at 3:12 PM

Build couchbase-server-7.0.4-7245 contains eventing commit 4499af6 with commit message:
https://couchbasecloud.atlassian.net/browse/MB-51390#icft=MB-51390 : Proactively stream encryption level changes to C++ consumer

Fixed
Pinned fields
Click on the next to a field label to start pinning.

Details

Assignee

Reporter

Is this a Regression?

Unknown

Triage

Untriaged

Operating System

Centos 64-bit

Story Points

Priority

Instabug

Open Instabug

PagerDuty

Sentry

Zendesk Support

Created March 11, 2022 at 8:39 AM
Updated April 8, 2022 at 11:33 AM
Resolved April 7, 2022 at 2:07 PM
Instabug