Uploaded image for project: 'Couchbase Java Client'
  1. Couchbase Java Client
  2. JCBC-212

Add new throttling feature to allow us to keep from using all system memory

    XMLWordPrintable

Details

    • New Feature
    • Resolution: Fixed
    • Major
    • 1.1.1
    • None
    • None
    • Security Level: Public
    • None

    Description

      When doing things like bulk loading, there are times we have multiple workload profiles. Interactive applications using the same client and same server need "room to move" and should always be fast, while other workloads should throttle themselves.

      The intent here would be to add a feature such that:
      for every N operations, pull stats from the nodes
      if the amount of memory used is below high_wat, don't do anything
      else if the amount of memory used is 10% of the remaining memory above high_wat before max memory usage, insert a sleep of Y milliseconds before each operation, and log the backoff at a debug level, check stats again ever M operations
      else if the amount of memory used is greater than 10% above that level above high_wat, insert a sleep of Z milliseconds before each operation and log the backoff at an info level, check stats again every O operations
      Turning this on or off should be done via a properties file or just simple -D args to the JVM, either one is okay.

      N, M, O and Y, Z should all be tuneable, either via a properties file or a define. All of them should be optional.

      Default values should be:
      N = 10000
      M = N/100
      O = M/10

      Y = 1ms
      Z = Y * 3

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            ingenthr Matt Ingenthron
            ingenthr Matt Ingenthron
            Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty