Uploaded image for project: 'Couchbase Gateway'
  1. Couchbase Gateway
  2. CBG-3776

[3.1.4 Backport] Configurable revs parallelism limit

    XMLWordPrintable

Details

    • Improvement
    • Resolution: Fixed
    • Major
    • 3.1.4
    • None
    • SyncGateway
    • Security Level: Public
    • None
    • CBG Sprint 145
    • 1

    Description

      Changes batches can result in up to 200 simultaneous revisions being sent to clients.

      This paralellisation may help individual clients get some documents sooner than if they pulled all 200 sequentially (let's imagine a large slow to replicate document was the first in the batch).

       

      However, the websocket and message frames being sent to Couchbase Lite are still ultimately single threaded, so overall throughput is not improved with this parallelisation. It also has the potential to allow a single client to significantly impact Sync Gateway's memory usage and client replication fairness (a client pulling 200 revs gets as much resource as 200 clients pulling a single rev each - which seems unfair)

       

      Having a configurable limit to the number of in-flight revisions being sent for any given changes batch seems useful to limit the total resource used for a single client. This should be more than 1 to ensure SG is not stuck with the latency of fetching documents before being able to replicate between revisions, and also to ensure that a large/slow document can "block" small ones in a given changes batch.

       

      As a side effect of limiting the number of concurrent revisions, we're also able to reduce the number of instances a CBL client asks for the same attachment/blob digest within a changes batch, resulting in duplicate data being retrieved from the bucket and sent to clients.

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              gregory.newmansmith Gregory Newman-Smith
              ben.brooks Ben Brooks
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty