Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-47878

[BP of MB-47666 to 7.0.2]- CBSE - Increase the poll time in pollForDeletedBuckets at projector

    XMLWordPrintable

Details

    • Untriaged
    • 1
    • Unknown

    Description

      Projector logs numDocsPending stats every 1min. To that extent, is spawns the go-routines to fetch seqnos from KV node. One go-routine queries ns_server every 10sec to validate the presence of a bucket in the cluster and clear the book-keeping if the bucket wa to be deleted.

      In cases where there are large number of buckets, periodically querying all the bucket endpoints is adding significant load on ns_server i.e. in a 30 bucket setup, it would be 30*6 = 180 requests per min. These requests would be to "pools/default/buckets/<bucket_name>?uuid=<>" endpoint which is a relatively costly call.

      We can safely increase the periodic poll to 10 min instead of 10 seconds as there will not be any functional impact to projector

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            varun.velamuri Varun Velamuri created issue -
            varun.velamuri Varun Velamuri made changes -
            Field Original Value New Value
            Link This issue Clones MB-47666 [ MB-47666 ]
            varun.velamuri Varun Velamuri made changes -
            Fix Version/s 7.0.2 [ 18012 ]
            Fix Version/s 6.6.4 [ 17614 ]
            Fix Version/s Neo [ 17615 ]
            varun.velamuri Varun Velamuri made changes -
            Assignee Varun Velamuri [ varun.velamuri ] Sai Krishna Teja [ JIRAUSER25045 ]
            varun.velamuri Varun Velamuri made changes -
            Affects Version/s 7.0.0 [ 17233 ]
            Affects Version/s 7.0.1 [ 17104 ]
            Affects Version/s Neo [ 17615 ]
            varun.velamuri Varun Velamuri made changes -
            Summary [BP of MB-47666 to 7.0.2]- Provide configurable option to disbale logging numDocsPending stat for projector [BP of MB-47666 to 7.0.2]- Increase the poll time in pollForDeletedBuckets
            varun.velamuri Varun Velamuri made changes -
            Summary [BP of MB-47666 to 7.0.2]- Increase the poll time in pollForDeletedBuckets [BP of MB-47666 to 7.0.2]- Increase the poll time in pollForDeletedBuckets at projector
            varun.velamuri Varun Velamuri made changes -
            Description Projector logs numDocsPending stats every 1min. To that extent, is spawns the go-routines to fetch seqnos from KV node. One go-routine queries ns_server every 10sec to validate the presence of a bucket in the cluster and clear the book-keeping if the bucket wa to be deleted.

            In cases where there are large number of buckets, periodically querying all the bucket endpoints is adding significant load on ns_server i.e. in a 30 bucket setup, it would be 30*6 = 180 requests per min. These requests would be to "pools/default/buckets/<bucket_name>?uuid=<>" endpoint which is a relatively costly call.

            The goal of this ticket is to provide a configurable option to disable the logging of num_docs_pending stat. Logging this stat is disabled by default and it can be enabled only if some real need for it arises
            Projector logs numDocsPending stats every 1min. To that extent, is spawns the go-routines to fetch seqnos from KV node. One go-routine queries ns_server every 10sec to validate the presence of a bucket in the cluster and clear the book-keeping if the bucket wa to be deleted.

            In cases where there are large number of buckets, periodically querying all the bucket endpoints is adding significant load on ns_server i.e. in a 30 bucket setup, it would be 30*6 = 180 requests per min. These requests would be to "pools/default/buckets/<bucket_name>?uuid=<>" endpoint which is a relatively costly call.

            We can safely increase the periodic poll to 10 min instead of 10 seconds as there will not be any functional impact to projector
            jeelan.poola Jeelan Poola made changes -
            Summary [BP of MB-47666 to 7.0.2]- Increase the poll time in pollForDeletedBuckets at projector [BP of MB-47666 to 7.0.2]- CBSE - Increase the poll time in pollForDeletedBuckets at projector
            jeelan.poola Jeelan Poola made changes -
            Link This issue blocks MB-46308 [ MB-46308 ]
            jeelan.poola Jeelan Poola made changes -
            Labels approved-for-7.0.2
            sai.teja Sai Krishna Teja made changes -
            Resolution Resolved [ 10400 ]
            Status Open [ 1 ] Resolved [ 5 ]
            hemant.rajput Hemant Rajput made changes -
            Status Resolved [ 5 ] Closed [ 6 ]
            wayne Wayne Siu made changes -
            Link This issue is a backport of MB-47666 [ MB-47666 ]
            wayne Wayne Siu made changes -
            Link This issue Clones MB-47666 [ MB-47666 ]

            People

              sai.teja Sai Krishna Teja
              varun.velamuri Varun Velamuri
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty