Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-61717

[BP 7.6.2] - Avoid excessive delays with retries when rpc requests are throttled under disk saturation

    XMLWordPrintable

Details

    • Untriaged
    • 0
    • No

    Description

      I have observed this during code inspection.

      RPC server file channel timeout can happen when disk is saturated even with proactive rate control.  On such timeouts, server throttles incoming requests with minimum 30s delay. 
      Throttle requests are not bounded by retries as server is active and hence exponential backoff can add significant and unnecessary delays, slowing overall rebalance. Also in addition to  ew Open/Write requests, we throttle older SyncAndClose rpc requests. We can allow the latter since they are the terminal rpc during transfer for somewhat better progress.

      Please refer slack discussion: https://couchbase.slack.com/archives/GK1UHGFHU/p1707363622374929

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              srinath.duvuru Srinath Duvuru
              srinath.duvuru Srinath Duvuru
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty