Skip to:

  1. Skip to Jira Navigation
  2. Skip to Side Navigation
  3. Skip to Main Content

[BP of MB-47666 to 7.0.2]- CBSE - Increase the poll time in pollForDeletedBuckets at projector

Description

Projector logs numDocsPending stats every 1min. To that extent, is spawns the go-routines to fetch seqnos from KV node. One go-routine queries ns_server every 10sec to validate the presence of a bucket in the cluster and clear the book-keeping if the bucket wa to be deleted.

In cases where there are large number of buckets, periodically querying all the bucket endpoints is adding significant load on ns_server i.e. in a 30 bucket setup, it would be 30*6 = 180 requests per min. These requests would be to "pools/default/buckets/<bucket_name>?uuid=<>" endpoint which is a relatively costly call.

We can safely increase the periodic poll to 10 min instead of 10 seconds as there will not be any functional impact to projector

Components

Affects versions

Fix versions

Labels

Environment

None

Link to Log File, atop/blg, CBCollectInfo, Core dump

None

Release Notes Description

None

Activity

Hemant Rajput September 3, 2021 at 8:43 AM

Validated on 7.0.2-6632. Seeing the increased in duration between periodic poll.

 

CB robot August 19, 2021 at 8:28 AM

Build couchbase-server-7.0.2-6537 contains indexing commit d99d3b7 with commit message:
: [BP of to 7.0.2] Increase polling interval in pollForDeletedBuckets

Jeelan Poola August 11, 2021 at 11:08 AM

FYI, the fix has no functional impact, as per Varun.

Varun Velamuri August 11, 2021 at 9:31 AM

, , I think we should take this issue for 7.0.2 as it is coming from CBSE's. Apologies for missing this out earlier. This issue reduces load on ns_server and would help in cases like rebalance. 

The tentative fix is to increase the timeout for "pollForDeletedBuckets" method in dcp_seqnos_local.go from 10 seconds to 10 minutes. Functionally, there will be no impact as this method is used in the context of "numDocsPending" stat at projector. In the worst case, the stat will not be logged. 

CC:

Resolved
Pinned fields
Click on the next to a field label to start pinning.

Details

Assignee

Reporter

Is this a Regression?

Unknown

Triage

Untriaged

Story Points

Priority

Instabug

Open Instabug

PagerDuty

Sentry

Zendesk Support

Created August 11, 2021 at 9:26 AM
Updated November 13, 2021 at 5:35 AM
Resolved August 19, 2021 at 12:57 PM
Instabug