Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-51492

10% higher latency observed in Hash Join tests

    XMLWordPrintable

Details

    • Untriaged
    • 1
    • Yes

    Attachments

      Issue Links

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          wayne Wayne Siu added a comment - - edited Running additional tests to narrow down the potential root cause. Looks like the regression was introduced in 7.1.0-2473. http://changelog.build.couchbase.com/?product=couchbase-server&fromVersion=7.1.0&fromBuild=2472&toVersion=7.1.0&toBuild=2473&f_backup=off&f_build=off&f_cbas-core=off&f_cbft=off&f_eventing=off&f_indexing=off&f_product-metadata=off&f_query=on&f_testrunner=off&f_asterixdb=off&f_kv_engine=off&f_magma=off&f_ns_server=off&f_platform=off&f_query-ui=off  

          Backing out MB-51372 and the figure is 664 (http://perf.jenkins.couchbase.com/job/arke/10253) for "90th percentile query latency (ms), Q26, Hash Join (1K -> 100K), not_bounded" which isn't quite back to 632 but still indicates the issue.

          But we can't not have this fix as it prevents a crash.

          The overhead comes from unavoidable additional allocations per covered value - and these tests rely on covered values; the tests have been falsely fast previously because of the bug.

           

          I will try another idea out that might be able to avoid the crash and the allocations and will update again later.

          Donald.haggart Donald Haggart added a comment - Backing out MB-51372 and the figure is 664 ( http://perf.jenkins.couchbase.com/job/arke/10253 ) for "90th percentile query latency (ms), Q26, Hash Join (1K -> 100K), not_bounded" which isn't quite back to 632 but still indicates the issue. But we can't not have this fix as it prevents a crash. The overhead comes from unavoidable additional allocations per covered value - and these tests rely on covered values; the tests have been falsely fast previously because of the bug.   I will try another idea out that might be able to avoid the crash and the allocations and will update again later.
          Donald.haggart Donald Haggart added a comment - - edited http://perf.jenkins.couchbase.com/job/arke/10254 - 656  # type http://perf.jenkins.couchbase.com/job/arke/10255 - 643  # break in fns http://perf.jenkins.couchbase.com/job/arke/10256 - 663  # copiedObject http://perf.jenkins.couchbase.com/job/arke/10257 - 913  # copy http://perf.jenkins.couchbase.com/job/arke/10259 - 659  # detect fields http://perf.jenkins.couchbase.com/job/arke/10260 - 653  # detect flag http://perf.jenkins.couchbase.com/job/arke/10261 - 734  # use NULL_VALUE http://perf.jenkins.couchbase.com/job/arke/10262 - 656  # copiedObject 2nd run http://perf.jenkins.couchbase.com/job/arke/10264 - 649  # aliased type Plain 2475 run - 785 (again)

          Build couchbase-server-7.1.0-2508 contains query commit 19e7ac8 with commit message:
          MB-51492 Revise fix.

          build-team Couchbase Build Team added a comment - Build couchbase-server-7.1.0-2508 contains query commit 19e7ac8 with commit message: MB-51492 Revise fix.

          There will be a small increase in latency with this revised fix in place, but it should be much closer to the base line.

          Donald.haggart Donald Haggart added a comment - There will be a small increase in latency with this revised fix in place, but it should be much closer to the base line.

          Build couchbase-server-7.2.0-1020 contains query commit 19e7ac8 with commit message:
          MB-51492 Revise fix.

          build-team Couchbase Build Team added a comment - Build couchbase-server-7.2.0-1020 contains query commit 19e7ac8 with commit message: MB-51492 Revise fix.
          wayne Wayne Siu added a comment -

          Verified with build 7.1.0-2508.

          The performance regression has been fixed.

          wayne Wayne Siu added a comment - Verified with build 7.1.0-2508. The performance regression has been fixed.

          People

            wayne Wayne Siu
            wayne Wayne Siu
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty