Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-50074

Avg. initial XDCR rate in XDCR rebalance test dropped from 216K to 160K on build 7.1.0-1885

    XMLWordPrintable

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 7.1.0
    • 7.1.0
    • XDCR
    • Untriaged
    • 1
    • Yes

    Description

      Avg. initial XDCR rate (items/sec) in XDCR rebalance test dropped from 216K to 160K on build 7.1.0-1885. The regression is reproducible.

      Avg. initial XDCR rate (items/sec), 1 -> 2 changes to 2 -> 2 (2 source nozzles, 4 target nozzles), 1 bucket x 200M x 1KB

       

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            neil.huang Neil Huang added a comment -

            With the checkpoint fix, things seem to look better as checkpoint resuming seems correct.

            neil.huang Neil Huang added a comment - With the checkpoint fix, things seem to look better as checkpoint resuming seems correct.
            neil.huang Neil Huang added a comment -

            We can see total cumulative docs_written for the 106 node now is less than before, as well as set_failed_cr_source

            neil.huang@NeilsMacbookPro:~/Downloads/perf/rebal_fix$ zipgrepCbCollectXdcr StatsMgr *105.zip | parseCumulativePipelineStats 5b5cee6447458bab7e20a97e17eb9412 bucket-1 bucket-1 "docs_written"
            59442680
            neil.huang@NeilsMacbookPro:~/Downloads/perf/rebal_fix$ zipgrepCbCollectXdcr StatsMgr *106.zip | parseCumulativePipelineStats 5b5cee6447458bab7e20a97e17eb9412 bucket-1 bucket-1 "set_failed_cr_source"
            13698040
            

            neil.huang Neil Huang added a comment - We can see total cumulative docs_written for the 106 node now is less than before, as well as set_failed_cr_source neil.huang@NeilsMacbookPro:~/Downloads/perf/rebal_fix$ zipgrepCbCollectXdcr StatsMgr *105.zip | parseCumulativePipelineStats 5b5cee6447458bab7e20a97e17eb9412 bucket-1 bucket-1 "docs_written" 59442680 neil.huang@NeilsMacbookPro:~/Downloads/perf/rebal_fix$ zipgrepCbCollectXdcr StatsMgr *106.zip | parseCumulativePipelineStats 5b5cee6447458bab7e20a97e17eb9412 bucket-1 bucket-1 "set_failed_cr_source" 13698040

            Build couchbase-server-7.1.0-2060 contains goxdcr commit 6881c5d with commit message:
            MB-50074: ensure combined checkpoint merge from peer sorts correctly

            build-team Couchbase Build Team added a comment - Build couchbase-server-7.1.0-2060 contains goxdcr commit 6881c5d with commit message: MB-50074 : ensure combined checkpoint merge from peer sorts correctly
            neil.huang Neil Huang added a comment -

            I compared the two profiles:

            7.1.0-1038: http://perf.jenkins.couchbase.com/job/titan/13148/
            Toy-Build reflecting 2060: http://perf.jenkins.couchbase.com/job/titan/13189/

            I did not find much differences between them.

            The number with fix is 212k.

            Per previous comment, build 1745 and 1787 is invalid... so comparing with 7.0.x numbers, we actually see an improvement of 9% or so.

            neil.huang Neil Huang added a comment - I compared the two profiles: 7.1.0-1038: http://perf.jenkins.couchbase.com/job/titan/13148/ Toy-Build reflecting 2060: http://perf.jenkins.couchbase.com/job/titan/13189/ I did not find much differences between them. The number with fix is 212k. Per previous comment, build 1745 and 1787 is invalid... so comparing with 7.0.x numbers, we actually see an improvement of 9% or so.
            bo-chun.wang Bo-Chun Wang added a comment -

            I finished a run with build 2061, and the performance is back to 210K. I close this issue.

            http://perf.jenkins.couchbase.com/job/titan/13221/

             

             

            bo-chun.wang Bo-Chun Wang added a comment - I finished a run with build 2061, and the performance is back to 210K. I close this issue. http://perf.jenkins.couchbase.com/job/titan/13221/    

            People

              bo-chun.wang Bo-Chun Wang
              bo-chun.wang Bo-Chun Wang
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty