Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-37003

MH Build 6.5.0-4788 showing increase in XDCR cpu utilization

    XMLWordPrintable

Details

    • Untriaged
    • Yes

    Description

      Neil/John,

      I am seeing ~60% increase in xdcr cpu utilization for below test case .

      Also there is minimum ~10% increase in most of other scenarios in general .

      Attachments

        1. goxdcr_cpu_newtest.png
          goxdcr_cpu_newtest.png
          132 kB
        2. goxdcr_cpu_oldtest.png
          goxdcr_cpu_oldtest.png
          145 kB
        3. image-2019-11-20-13-32-48-748.png
          image-2019-11-20-13-32-48-748.png
          95 kB
        4. Memcached_cpu_newtest.png
          Memcached_cpu_newtest.png
          172 kB
        5. Memcached_cpu_newtest.png
          Memcached_cpu_newtest.png
          193 kB
        6. Memcached_cpu_oldtest.png
          Memcached_cpu_oldtest.png
          232 kB
        7. Memcached_cpu_oldtest.png
          Memcached_cpu_oldtest.png
          99 kB
        8. Screen Shot 2019-12-05 at 11.46.14.png
          Screen Shot 2019-12-05 at 11.46.14.png
          10 kB
        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          Ascending order - Click to sort in descending order
          jliang John Liang added a comment -

          Sharath SulochanaIs there any regression on DCP?

          jliang John Liang added a comment - Sharath Sulochana Is there any regression on DCP?
          neil.huang Neil Huang added a comment -

          I believe the test Sharath Sulochana mentions shows a final number of the avg CPU utilization regardless of processes. (cpu_utilization_rate)

          Looking deeper into the test results, I see that goxdcr CPU utilization rate stays relatively low and same across tests.

          (Old test)

          (New test)

          ns_server CPU utilization also stayed the relatively same between tests.

          However, memcached CPU increased between the tests:

          (Old test):

          (New test):

           

          This could explain why the overall CPU went up.

          I also noted that compaction progress looks a bit different between the tests as well, but cannot directly conclude that it has affect on overall CPU usage. But that does show how KV behaviors between the two tests are not identical between them.

          neil.huang Neil Huang added a comment - I believe the test Sharath Sulochana  mentions shows a final number of the avg CPU utilization regardless of processes. (cpu_utilization_rate) Looking deeper into the test results, I see that goxdcr CPU utilization rate stays relatively low and same across tests. (Old test) (New test) ns_server CPU utilization also stayed the relatively same between tests. However, memcached CPU increased between the tests: (Old test): (New test):   This could explain why the overall CPU went up. I also noted that compaction progress looks a bit different between the tests as well, but cannot directly conclude that it has affect on overall CPU usage. But that does show how KV behaviors between the two tests are not identical between them.
          wayne Wayne Siu added a comment - - edited

          This could be related to the change of the # of shards that we use.

          As Neil pointed it out, goxdcr CPU usage is about the same between the builds. But memcached CPU is about 60% higher.

          http://cbmonitor.sc.couchbase.com/reports/html/?snapshot=titan_c1_650-4754_access_1d49&snapshot=titan_c1_650-4788_access_f4b5

          wayne Wayne Siu added a comment - - edited This could be related to the change of the # of shards that we use. As Neil pointed it out, goxdcr CPU usage is about the same between the builds. But memcached CPU is about 60% higher. http://cbmonitor.sc.couchbase.com/reports/html/?snapshot=titan_c1_650-4754_access_1d49&snapshot=titan_c1_650-4788_access_f4b5
          drigby Dave Rigby added a comment -

          CPU utilisation has returned to previous values in latest build:

          As discussed above, issue has been resolved by changing the default number of KV-Engine Writer threads back to the 6.0 value (4).

          drigby Dave Rigby added a comment - CPU utilisation has returned to previous values in latest build: As discussed above, issue has been resolved by changing the default number of KV-Engine Writer threads back to the 6.0 value (4).
          sharath.sulochana Sharath Sulochana (Inactive) added a comment - - edited

          CPU utilization is back to expected numbers in 6.5.0-4908 .Closing the ticket. 

          sharath.sulochana Sharath Sulochana (Inactive) added a comment - - edited CPU utilization is back to expected numbers in 6.5.0-4908 .Closing the ticket. 

          People

            sharath.sulochana Sharath Sulochana (Inactive)
            sharath.sulochana Sharath Sulochana (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty