Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-30841

[Analytics] Ingestion rate is 2-3 times slower with DEBUG logging level (new default)

    XMLWordPrintable

Details

    • Untriaged
    • Unknown
    • CX Sprint 115

    Description

      Setting this ticket as Critical because DEBUG level is default logging level since build 1370.
      The rate of degradation:

      6.0.0-1388 and 6.0.0-1432 were tested with DEBUG logging level
      And for 6.0.0-1460 I've manually switched it back to WARN

      The degradation is all across the product. Query latency degraded as well

      Comparison between 6.0.0-1432  (DEBUG) and 6.0.0-1460 (WARN):
      Avg. ingestion rate (items/sec), 1 node, BigFUN 20M users (320M docs), 3 indexes, SSD:
      http://cbmonitor.sc.couchbase.com/reports/html/?snapshot=oceanus_600-1460_sync_aa3f&label=6.0.0-1460-WARN&snapshot=oceanus_600-1432_sync_2116&label=6.0.0-1432-DEBUG

      Unfortunately I didn't have IO stats working  when running the DEBUG tests. Its now fixed.
      I'll  rerun the DEBUG test with all stats in place and update the ticket.

      Logs:
      172.23.96.5:kv
      172.23.96.7:kv
      172.23.96.8:cbas
      [https://s3-us-west-2.amazonaws.com/perf-artifacts/jenkins-oceanus-766/172.23.96.5.zip
      ] [https://s3-us-west-2.amazonaws.com/perf-artifacts/jenkins-oceanus-766/172.23.96.7.zip
      ] https://s3-us-west-2.amazonaws.com/perf-artifacts/jenkins-oceanus-766/172.23.96.8.zip

       

       

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              michael.blow Michael Blow
              oleksandr.gyryk Alex Gyryk (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty