Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-47674

[BP of MB-47666] Provide configurable option to disbale logging numDocsPending stat for projector

    XMLWordPrintable

Details

    • Untriaged
    • 1
    • Unknown

    Description

      Projector logs numDocsPending stats every 1min. To that extent, is spawns the go-routines to fetch seqnos from KV node. One go-routine queries ns_server every 10sec to validate the presence of a bucket in the cluster and clear the book-keeping if the bucket wa to be deleted.

      In cases where there are large number of buckets, periodically querying all the bucket endpoints is adding significant load on ns_server i.e. in a 30 bucket setup, it would be 30*6 = 180 requests per min. These requests would be to "pools/default/buckets/<bucket_name>?uuid=<>" endpoint which is a relatively costly call.

      The goal of this ticket is to provide a configurable option to disable the logging of num_docs_pending stat. Logging this stat is disabled by default and it can be enabled only if some real need for it arises

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            Wayne Siu, Mihir Kamdar Requesting your approval for the inclusion of this ticket to 6.6.3. The change would be a very localised one and it would disable a stat logging by default at projector. Having this would help reduce periodic load on ns_server.

            CC: Jeelan Poola, Deepkaran Salooja, Ritam Sharma

            varun.velamuri Varun Velamuri added a comment - Wayne Siu , Mihir Kamdar Requesting your approval for the inclusion of this ticket to 6.6.3. The change would be a very localised one and it would disable a stat logging by default at projector. Having this would help reduce periodic load on ns_server. CC: Jeelan Poola , Deepkaran Salooja , Ritam Sharma

            Hi Varun Velamuri I am afraid its too late for 6.6.3. We have identified the potentially final RC for 6.6.3. If there is any other issue that warrants a new build for 6.6.3, we will consider this.

            mihir.kamdar Mihir Kamdar (Inactive) added a comment - Hi Varun Velamuri I am afraid its too late for 6.6.3. We have identified the potentially final RC for 6.6.3. If there is any other issue that warrants a new build for 6.6.3, we will consider this.

            Wayne Siu , yes. We will BP the change today.

            varun.velamuri Varun Velamuri added a comment - Wayne Siu , yes. We will BP the change today.

            Build couchbase-server-6.6.4-9948 contains indexing commit 4c2a7fa with commit message:
            MB-47674 : Increase polling interval in pollForDeletedBuckets

            build-team Couchbase Build Team added a comment - Build couchbase-server-6.6.4-9948 contains indexing commit 4c2a7fa with commit message: MB-47674 : Increase polling interval in pollForDeletedBuckets

            Validated on Enterprise Edition 6.6.5 build 10067

             

             

            [root@node1-cb660-centos7 logs]# grep pools/default/buckets/travel-sample?bucket http_access_internal.log | grep @projector-cbauth |tail 
            127.0.0.1 - @projector-cbauth [22/Dec/2021:06:31:11 +0000] "GET /pools/default/buckets/travel-sample?bucket_uuid=d12bf81ec72a1b87d9e9f2bc3194bf9a HTTP/1.1" 200 10438 - Go-http-client/1.1
            127.0.0.1 - @projector-cbauth [22/Dec/2021:06:41:11 +0000] "GET /pools/default/buckets/travel-sample?bucket_uuid=d12bf81ec72a1b87d9e9f2bc3194bf9a HTTP/1.1" 200 10438 - Go-http-client/1.1
            127.0.0.1 - @projector-cbauth [22/Dec/2021:06:51:11 +0000] "GET /pools/default/buckets/travel-sample?bucket_uuid=d12bf81ec72a1b87d9e9f2bc3194bf9a HTTP/1.1" 200 10438 - Go-http-client/1.1
            127.0.0.1 - @projector-cbauth [22/Dec/2021:07:01:11 +0000] "GET /pools/default/buckets/travel-sample?bucket_uuid=d12bf81ec72a1b87d9e9f2bc3194bf9a HTTP/1.1" 200 10438 - Go-http-client/1.1
            127.0.0.1 - @projector-cbauth [22/Dec/2021:07:11:11 +0000] "GET /pools/default/buckets/travel-sample?bucket_uuid=d12bf81ec72a1b87d9e9f2bc3194bf9a HTTP/1.1" 200 10438 - Go-http-client/1.1
            127.0.0.1 - @projector-cbauth [22/Dec/2021:07:21:11 +0000] "GET /pools/default/buckets/travel-sample?bucket_uuid=d12bf81ec72a1b87d9e9f2bc3194bf9a HTTP/1.1" 200 10438 - Go-http-client/1.1
            127.0.0.1 - @projector-cbauth [22/Dec/2021:07:31:11 +0000] "GET /pools/default/buckets/travel-sample?bucket_uuid=d12bf81ec72a1b87d9e9f2bc3194bf9a HTTP/1.1" 200 10438 - Go-http-client/1.1
            127.0.0.1 - @projector-cbauth [22/Dec/2021:07:41:11 +0000] "GET /pools/default/buckets/travel-sample?bucket_uuid=d12bf81ec72a1b87d9e9f2bc3194bf9a HTTP/1.1" 200 10438 - Go-http-client/1.1

            hemant.rajput Hemant Rajput added a comment - Validated on Enterprise Edition 6.6.5 build 10067     [root@node1-cb660-centos7 logs] # grep pools/default/buckets/travel-sample?bucket http_access_internal.log | grep @projector-cbauth |tail  127.0.0.1 - @projector-cbauth [22/Dec/2021:06:31:11 +0000] "GET /pools/default/buckets/travel-sample?bucket_uuid=d12bf81ec72a1b87d9e9f2bc3194bf9a HTTP/1.1" 200 10438 - Go-http-client/1.1 127.0.0.1 - @projector-cbauth [22/Dec/2021:06:41:11 +0000] "GET /pools/default/buckets/travel-sample?bucket_uuid=d12bf81ec72a1b87d9e9f2bc3194bf9a HTTP/1.1" 200 10438 - Go-http-client/1.1 127.0.0.1 - @projector-cbauth [22/Dec/2021:06:51:11 +0000] "GET /pools/default/buckets/travel-sample?bucket_uuid=d12bf81ec72a1b87d9e9f2bc3194bf9a HTTP/1.1" 200 10438 - Go-http-client/1.1 127.0.0.1 - @projector-cbauth [22/Dec/2021:07:01:11 +0000] "GET /pools/default/buckets/travel-sample?bucket_uuid=d12bf81ec72a1b87d9e9f2bc3194bf9a HTTP/1.1" 200 10438 - Go-http-client/1.1 127.0.0.1 - @projector-cbauth [22/Dec/2021:07:11:11 +0000] "GET /pools/default/buckets/travel-sample?bucket_uuid=d12bf81ec72a1b87d9e9f2bc3194bf9a HTTP/1.1" 200 10438 - Go-http-client/1.1 127.0.0.1 - @projector-cbauth [22/Dec/2021:07:21:11 +0000] "GET /pools/default/buckets/travel-sample?bucket_uuid=d12bf81ec72a1b87d9e9f2bc3194bf9a HTTP/1.1" 200 10438 - Go-http-client/1.1 127.0.0.1 - @projector-cbauth [22/Dec/2021:07:31:11 +0000] "GET /pools/default/buckets/travel-sample?bucket_uuid=d12bf81ec72a1b87d9e9f2bc3194bf9a HTTP/1.1" 200 10438 - Go-http-client/1.1 127.0.0.1 - @projector-cbauth [22/Dec/2021:07:41:11 +0000] "GET /pools/default/buckets/travel-sample?bucket_uuid=d12bf81ec72a1b87d9e9f2bc3194bf9a HTTP/1.1" 200 10438 - Go-http-client/1.1

            People

              hemant.rajput Hemant Rajput
              varun.velamuri Varun Velamuri
              Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty