Uploaded image for project: 'Couchbase Gateway'
  1. Couchbase Gateway
  2. CBG-821

Limit on channel queries triggered by 2.x replication

    XMLWordPrintable

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0
    • Component/s: SyncGateway
    • Security Level: Public
    • Labels:
      None
    • Sprint:
      CBG Sprint 46, CBG Sprint 47
    • Story Points:
      5

      Description

      2.x replication doesn't set a limit value in changesOptions, as changes responses aren't batched in the same way 1.x responses are. However, this also means that queries triggered by changes requests aren't including a limit. This can result in query timeouts when backfill is required for very large channels.

      Sync Gateway should apply a query limit when not otherwise specified. The default limit can be relatively large (i.e. the size of the default channel cache), but may need to be configurable, so that customers can tune the limit to their channel sizes in scenarios where query performance supports larger queries.

        Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

          Hide
          adamf Adam Fraser added a comment -

          Even 1.x continuous replications don't currently support query limits - only one-shot and longpoll. To support query limits, the per-channel feeds will need to be modified to iterate over batches of query results. The feed structure is already set up to stream results, so it should be possible to trigger the next query once the feed has been drained, as opposed to triggering all queries at the start of the stream. This doesn't currently change the cache prepend criteria, which happens a few layers down the stack.

          Show
          adamf Adam Fraser added a comment - Even 1.x continuous replications don't currently support query limits - only one-shot and longpoll. To support query limits, the per-channel feeds will need to be modified to iterate over batches of query results. The feed structure is already set up to stream results, so it should be possible to trigger the next query once the feed has been drained, as opposed to triggering all queries at the start of the stream. This doesn't currently change the cache prepend criteria, which happens a few layers down the stack.
          Hide
          build-team Couchbase Build Team added a comment -

          Build sync_gateway-2.8.0-166 contains sync_gateway commit c2fbffd with commit message:
          CBG-821 Query limit for channel queries (#4557)

          Show
          build-team Couchbase Build Team added a comment - Build sync_gateway-2.8.0-166 contains sync_gateway commit c2fbffd with commit message: CBG-821 Query limit for channel queries (#4557)

            People

            Assignee:
            adamf Adam Fraser
            Reporter:
            adamf Adam Fraser
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved:

                Gerrit Reviews

                There are no open Gerrit changes

                  PagerDuty