Uploaded image for project: 'Couchbase Gateway'
  1. Couchbase Gateway
  2. CBG-2179

[3.0.4 Backport] Periodic high response times on REST API due to persistent config polling

    XMLWordPrintable

Details

    • Bug
    • Resolution: Fixed
    • Major
    • 3.0.4
    • 3.0
    • SyncGateway
    • Security Level: Public
    • None
    • CBG Sprint 102
    • 1

    Description

      Overview

      The persistent config code that checks for config updates periodically (every 10 seconds by default) acquires a lock on ServerContext prior to discovering/fetching database configs from buckets, despite not needing the lock until SG is about to apply the configs.

      This can cause REST API response times to periodically spike up to a few seconds if the requests align with the ServerContext lock, whilst SG is fetching database configs from each bucket on server.

      Example

      2022-06-30T13:53:35.364+01:00 [INF] HTTP: c:#011 GET /db1.asdf.qwerty/<ud>doc1</ud> (as <ud>Administrator</ud> as ADMIN)
      2022-06-30T13:53:35.364+01:00 [INF] HTTP+: c:#011 #011:     --> 200   (0.8 ms)
      2022-06-30T13:53:35.465+01:00 [DBG] Config+: Fetching configs from buckets in cluster for group "default"
      2022-06-30T13:53:37.884+01:00 [DBG] Config+: Got config for group "default" from bucket "b1" with cas 1656592953873924096
      2022-06-30T13:53:39.157+01:00 [DBG] Config+: Bucket "b2" did not contain config for group "default"
      2022-06-30T13:53:39.157+01:00 [DBG] Config+: Database "db1" bucket "b1" config has not changed since last update
      2022-06-30T13:53:39.158+01:00 [INF] Auth: c:#012 #012: User <ud>Administrator</ud> was successfully authorized as an admin
      2022-06-30T13:53:39.158+01:00 [INF] HTTP: c:#012 GET /db1.asdf.qwerty/<ud>doc1</ud> (as <ud>Administrator</ud> as ADMIN)
      2022-06-30T13:53:39.158+01:00 [INF] HTTP+: c:#012 #012:     --> 200   (3304.3 ms)
      

      Proposed fix

      Move the lock in fetchAndLoadConfigs down to after we've fetched the configs from the bucket, but before we're about to apply them to the node.

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              isaac.lambat Isaac Lambat
              ben.brooks Ben Brooks
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty