Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-26267

Improve execution pipeline throughput

    XMLWordPrintable

Details

    Description

      Request execution involves a tree of operator executing request stages in step.
      Operator execution is controlled by the golang kernel - while each operator is executed in a timely manner, the whole pipeline executes synchronously and requests are completed in a predictable time.
      Sadly, the golang kernel has no visibility of the needs of the execution pipeline, and under load will probably allow a producer to proceed unhindered while a consumer is in fact struggling and a better approach would be to have the producer yield in favour of the consumer, so that its incoming queue can be drained, providing work for its own consumer.

      The upshot of this is that the value exchange will have to accumulate values while the consumer finally gets a chance to run, which in turn will mean increased memory consumption and reduced throughput because of garbage collector increased load.

      This MB keeps track of the need to have a different value exchange mechanism that has a wider view of the pipeline, so that it can self regulate to keep the whole of the pipeline active, so as to produce a more consistent throughput.
      This includes different strategies for different classes of operators.

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            marco.greco Marco Greco (Inactive)
            marco.greco Marco Greco (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty