Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-49838

Investigate session consistency scan latencies

    XMLWordPrintable

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • Neo
    • Neo
    • secondary-index
    • Untriaged
    • 1
    • Unknown

    Description

      In MB-46068, indexer code is enabled to use bucket seqnos by default for session consistent scans. This is done to ensure that indexer can support large concurrency in scans as the bottleneck was observed in fetching seqnos. from KV in earlier experiments.

      In the latest set of experiments, there seems to be no improvement in session consistent scan latencies with bucket seqnos. Rather, there seems to be a regression in performance.

      The goal of this ticket is to investigate if the scan latencies observed with bucket seqnos. for scans are expected or not. 

      Link to runs:

      http://perf.jenkins.couchbase.com/job/aether/1138/ -> 7.1.0 - 1805 -> 128 concurrency, Uses Collection seqnos, Latencies: 4.74, 6.36ms

      http://perf.jenkins.couchbase.com/job/aether/1140/ -> 7.1.0 - 1805, 1024 concurrency, Uses Collection seqnos, Latencies: 28.48, 35.97ms

      Toy build is based on 7.1.0-1805 with https://review.couchbase.org/c/indexing/+/166218

      http://perf.jenkins.couchbase.com/job/aether/1146 -> Toy build, 128 concurrency, Uses Bucket seqnos, Latencies: 20.37, 26.4 ms

      http://perf.jenkins.couchbase.com/job/aether/1144 -> Toy build, 1024 concurrency, Uses Bucket seqnos, Latencies: 43.69, 58.35ms

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          varun.velamuri Varun Velamuri added a comment - - edited

          Below are the scan stats between bucket and collection seqnos

          Concurrency = 128

          Seqnos avg_scan_request_init_latency (ns) avg_scan_wait_latency (ns)
          Bucket 910515 14691367
          Collection 1761521 868573

          Concurrency = 1024

          Seqnos avg_scan_request_init_latency (ns) avg_scan_wait_latency (ns)
          Bucket 2415271 27540105
          Collection 14885434 2815091

          While the init_latency is much less when using BucketSeqnos, the scan_wait_latency is much higher because of which the scans with bucket_seqnos are taking more time. I believe the default configuration should be to use collection_seqnos for session consistent scans and we can use bucket_seqnos if any issue is observed with high init latencies for scans.

           

          varun.velamuri Varun Velamuri added a comment - - edited Below are the scan stats between bucket and collection seqnos Concurrency = 128 Seqnos avg_scan_request_init_latency (ns) avg_scan_wait_latency (ns) Bucket 910515 14691367 Collection 1761521 868573 Concurrency = 1024 Seqnos avg_scan_request_init_latency (ns) avg_scan_wait_latency (ns) Bucket 2415271 27540105 Collection 14885434 2815091 While the init_latency is much less when using BucketSeqnos, the scan_wait_latency is much higher because of which the scans with bucket_seqnos are taking more time. I believe the default configuration should be to use collection_seqnos for session consistent scans and we can use bucket_seqnos if any issue is observed with high init latencies for scans.  

          Build couchbase-server-7.1.0-1853 contains indexing commit 08499e0 with commit message:
          MB-49838 Revert back to using collection seqnos for session consistent scans

          build-team Couchbase Build Team added a comment - Build couchbase-server-7.1.0-1853 contains indexing commit 08499e0 with commit message: MB-49838 Revert back to using collection seqnos for session consistent scans

          People

            varun.velamuri Varun Velamuri
            varun.velamuri Varun Velamuri
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty