Agent attempts to fallback to http polling when no http addresses exist

Description

On a CAO cluster with 1 node, using server 7.1, we're failing to connect to the bucket when adding documents. We receive this error from bucket.WaitUntilReady:

This only happens with 1 node (if we change to 3 nodes, the test passes fine), and also only happens on 7.1 (on 7.0.3, the test passes fine.). The bucket is responding to API calls from the operator via /pools/default/buckets... and also appears on the server UI.

I've attached cbopinfo, cbinfo, the operator logs, and the SDK logs from gocb.SetLogger(gocb.VerboseStdioLogger())

Environment

CAO 2.3 CB Server 7.1.0-2434 gocb 2.3

Gerrit Reviews

None

Release Notes Description

None

Attachments

4

Activity

Show:

Charles Dixon March 7, 2022 at 5:57 PM

I talked to and it looks like SRV records are being used. When this happens we only set the memd addresses for the gocbcore agent, we do no translation to also populate the http addresses. The issue here is that the server is replying to our SELECT BUCKET with an ENOENT so we automatically fallback to the http poller (this is required for bootstrapping against a non-kv node). However, the http poller has nothing to poll so just sits there and despite the underlying connections being built ok we can never fetch a cluster config.

Fixed
Pinned fields
Click on the next to a field label to start pinning.

Details

Assignee

Reporter

Story Points

Fix versions

Priority

Instabug

Open Instabug

PagerDuty

Sentry

Zendesk Support

Created March 7, 2022 at 1:47 PM
Updated March 8, 2022 at 8:57 AM
Resolved March 8, 2022 at 8:57 AM
Instabug