Details
-
Bug
-
Status: Closed
-
Major
-
Resolution: Resolved
-
6.6.2, 7.0.0
-
Untriaged
-
1
-
Unknown
Description
Projector logs numDocsPending stats every 1min. To that extent, is spawns the go-routines to fetch seqnos from KV node. One go-routine queries ns_server every 10sec to validate the presence of a bucket in the cluster and clear the book-keeping if the bucket wa to be deleted.
In cases where there are large number of buckets, periodically querying all the bucket endpoints is adding significant load on ns_server i.e. in a 30 bucket setup, it would be 30*6 = 180 requests per min. These requests would be to "pools/default/buckets/<bucket_name>?uuid=<>" endpoint which is a relatively costly call.
We can safely increase the periodic poll to 10 min instead of 10 seconds as there will not be any functional impact to projector
Attachments
Issue Links
- is a backport of
-
-
- Closed
-
Activity
| Fix Version/s | 7.0.2 [ 18012 ] | |
| Fix Version/s | 6.6.4 [ 17614 ] | |
| Fix Version/s | Neo [ 17615 ] |
| Assignee | Varun Velamuri [ varun.velamuri ] | Sai Krishna Teja [ JIRAUSER25045 ] |
| Affects Version/s | 7.0.0 [ 17233 ] | |
| Affects Version/s | 7.0.1 [ 17104 ] | |
| Affects Version/s | Neo [ 17615 ] |
| Description |
Projector logs numDocsPending stats every 1min. To that extent, is spawns the go-routines to fetch seqnos from KV node. One go-routine queries ns_server every 10sec to validate the presence of a bucket in the cluster and clear the book-keeping if the bucket wa to be deleted.
In cases where there are large number of buckets, periodically querying all the bucket endpoints is adding significant load on ns_server i.e. in a 30 bucket setup, it would be 30*6 = 180 requests per min. These requests would be to "pools/default/buckets/<bucket_name>?uuid=<>" endpoint which is a relatively costly call. The goal of this ticket is to provide a configurable option to disable the logging of num_docs_pending stat. Logging this stat is disabled by default and it can be enabled only if some real need for it arises |
Projector logs numDocsPending stats every 1min. To that extent, is spawns the go-routines to fetch seqnos from KV node. One go-routine queries ns_server every 10sec to validate the presence of a bucket in the cluster and clear the book-keeping if the bucket wa to be deleted.
In cases where there are large number of buckets, periodically querying all the bucket endpoints is adding significant load on ns_server i.e. in a 30 bucket setup, it would be 30*6 = 180 requests per min. These requests would be to "pools/default/buckets/<bucket_name>?uuid=<>" endpoint which is a relatively costly call. We can safely increase the periodic poll to 10 min instead of 10 seconds as there will not be any functional impact to projector |
| Link | This issue blocks MB-46308 [ MB-46308 ] |
| Labels | approved-for-7.0.2 |
| Resolution | Resolved [ 10400 ] | |
| Status | Open [ 1 ] | Resolved [ 5 ] |
| Status | Resolved [ 5 ] | Closed [ 6 ] |