Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-49902

py-tpcc performance runs show regression in 7.0.2 build

    XMLWordPrintable

Details

    • Untriaged
    • 1
    • Yes

    Description

      http://showfast.sc.couchbase.com/#/timeline/Linux/n1ql/pytpcc/Plasma

      Tpm-c (NewOrders/min), nodes 1, warehouse 100, replica 0, durability majority, scan_consistency request_plus, vCPU 16, s=1 c=9

       

      Tpm-c (NewOrders/min), nodes 2, warehouse 100, replica 1, durability majority, scan_consistency request_plus, vCPU 16, s=1 c=9

       

      Tpm-c (NewOrders/min), nodes 2, warehouse 100, replica 1, durability persistToMajority, scan_consistency request_plus, vCPU 16, s=1 c=9

       

      Tpm-c (NewOrders/min), nodes 4, warehouse 100, replica 2, durability majority, scan_consistency request_plus, vCPU 16, s=1 c=9

       

      Tpm-c (NewOrders/min), nodes 4, warehouse 100, replica 2, durability persistToMajority, scan_consistency request_plus, vCPU 16, s=1 c=9

       

      Tpm-c (NewOrders/min), nodes 6, warehouse 100, replica 2, durability majority, scan_consistency request_plus, vCPU 16, s=1 c=9

       

      Tpm-c (NewOrders/min), nodes 6, warehouse 100, replica 2, durability persistToMajority, scan_consistency request_plus, vCPU 16, s=1 c=9

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            kamini.jagtiani Kamini Jagtiani added a comment - - edited

            Hi Bo-Chun Wang 

            Thanks. Can you also add the latest 7.1.0 numbers?

            kamini.jagtiani Kamini Jagtiani added a comment - - edited Hi Bo-Chun Wang   Thanks. Can you also add the latest 7.1.0 numbers?
            Sitaram.Vemulapalli Sitaram Vemulapalli added a comment - - edited

            Nothing is changed in transactions after 7.0.0 release . 7.0.2 is released nothing can be done (Not sure why it did not catch before release). If 7.0.3 is planned we will take look at that time.

            Other way is pick one of config and do binary search narrow down build number

            Sitaram.Vemulapalli Sitaram Vemulapalli added a comment - - edited Nothing is changed in transactions after 7.0.0 release . 7.0.2 is released nothing can be done (Not sure why it did not catch before release). If 7.0.3 is planned we will take look at that time. Other way is pick one of config and do binary search narrow down build number

            When Bo-Chun ran, he found performance numbers are lower on 7.0.2 as compared to 7.0.0. The numbers are back to 7.0.0 or better in 7.1.0.

            So this is created for tracking purposes.

            Since there is no plan for 7.0.3 yet, he raised it for 7.0.2

             

            kamini.jagtiani Kamini Jagtiani added a comment - When Bo-Chun ran, he found performance numbers are lower on 7.0.2 as compared to 7.0.0. The numbers are back to 7.0.0 or better in 7.1.0. So this is created for tracking purposes. Since there is no plan for 7.0.3 yet, he raised it for 7.0.2  
            Sitaram.Vemulapalli Sitaram Vemulapalli added a comment - - edited

            MB-48496 might be suspect (revised fix MB-48658 ) did not make into 7.0.2 try with 7.0.2-6682

            Adding a benchmark: {
            15:54:19     "build": "7.0.2-6682",
            15:54:19     "buildURL": "http://perf.jenkins.couchbase.com/job/hercules_nest_pytpcc_perfrunner/445/",
            15:54:19     "dateTime": "2021-12-04 15:54",
            15:54:19     "id": "e9ba9374212c41c09d411ccf3ec5bb6c",
            15:54:19     "metric": "n1ql_trans_1node_pytpcc_16core_100warehouse_40clients_replica1_dur_maj_scancons_requestplus_hercules_n1ql_trans",
            15:54:19     "snapshots": [
            15:54:19         "hercules_702-6682_run_tpcc_f1bd"
            15:54:19     ],
            15:54:19     "value": 7200
            15:54:19 }
            

            Adding a benchmark: {
            18:35:51     "build": "7.0.2-6683",
            18:35:51     "buildURL": "http://perf.jenkins.couchbase.com/job/hercules_nest_pytpcc_perfrunner/446/",
            18:35:51     "dateTime": "2021-12-04 18:35",
            18:35:51     "id": "5a83c4ef1b8447bcb49a14811c99e927",
            18:35:51     "metric": "n1ql_trans_1node_pytpcc_16core_100warehouse_40clients_replica1_dur_maj_scancons_requestplus_hercules_n1ql_trans",
            18:35:51     "snapshots": [
            18:35:51         "hercules_702-6683_run_tpcc_2123"
            18:35:51     ],
            18:35:51     "value": 5940
            18:35:51 }
            

            Sitaram.Vemulapalli Sitaram Vemulapalli added a comment - - edited MB-48496 might be suspect (revised fix MB-48658 ) did not make into 7.0.2 try with 7.0.2-6682 Adding a benchmark: { 15 : 54 : 19 "build" : "7.0.2-6682" , 15 : 54 : 19 "buildURL" : "http://perf.jenkins.couchbase.com/job/hercules_nest_pytpcc_perfrunner/445/" , 15 : 54 : 19 "dateTime" : "2021-12-04 15:54" , 15 : 54 : 19 "id" : "e9ba9374212c41c09d411ccf3ec5bb6c" , 15 : 54 : 19 "metric" : "n1ql_trans_1node_pytpcc_16core_100warehouse_40clients_replica1_dur_maj_scancons_requestplus_hercules_n1ql_trans" , 15 : 54 : 19 "snapshots" : [ 15 : 54 : 19 "hercules_702-6682_run_tpcc_f1bd" 15 : 54 : 19 ], 15 : 54 : 19 "value" : 7200 15 : 54 : 19 } Adding a benchmark: { 18 : 35 : 51 "build" : "7.0.2-6683" , 18 : 35 : 51 "buildURL" : "http://perf.jenkins.couchbase.com/job/hercules_nest_pytpcc_perfrunner/446/" , 18 : 35 : 51 "dateTime" : "2021-12-04 18:35" , 18 : 35 : 51 "id" : "5a83c4ef1b8447bcb49a14811c99e927" , 18 : 35 : 51 "metric" : "n1ql_trans_1node_pytpcc_16core_100warehouse_40clients_replica1_dur_maj_scancons_requestplus_hercules_n1ql_trans" , 18 : 35 : 51 "snapshots" : [ 18 : 35 : 51 "hercules_702-6683_run_tpcc_2123" 18 : 35 : 51 ], 18 : 35 : 51 "value" : 5940 18 : 35 : 51 }
            kamini.jagtiani Kamini Jagtiani added a comment - - edited

            So it is confirmed that MB-48496  has caused the regression.

            It is fixed by MB-48658 in 7.1.0

            kamini.jagtiani Kamini Jagtiani added a comment - - edited So it is confirmed that MB-48496   has caused the regression. It is fixed by MB-48658 in 7.1.0

            People

              kamini.jagtiani Kamini Jagtiani
              bo-chun.wang Bo-Chun Wang
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty