Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-27421

disk_commit and storage_age timing histograms have too coarse buckets

    XMLWordPrintable

Details

    Description

      The histogram bucketing for the disk_commit and storage_age histograms are incorrectly defined so all samples were frequently placed in a single bucket:

       storage_age (65700 total)
          0 - 1s              : ( 98.35%) 64615 ####################################
          1s - 2s             : (100.00%)  1085 
          Avg                 : (   16ms)
       
       disk_commit (6273 total)
          0 - 1s              : (100.00%) 6273 ######################################
          Avg                 : (      0)
      

      This can also be seen in showfast / cbmonitor graphs - note how avg_disk_commit_time is constantly 0.5s:

      Also visible in the UI:

      This is caused by the histograms being constructed with a poor choice of bucket sizes:

                dirtyAgeHisto(GrowingWidthGenerator<UnsignedMicroseconds,
                                                    cb::duration_limits>(
                                      ONE_SECOND.zero(), ONE_SECOND, 1.4),
                              25),
                diskCommitHisto(GrowingWidthGenerator<UnsignedMicroseconds,
                                                      cb::duration_limits>(
                                        ONE_SECOND.zero(), ONE_SECOND, 1.4),
                                25),
      

      The same is seen in the UI - you'll typically never see better number than 0.5s.

      In other words, starting with a bucket at (0,1.0]s, then (1.0, 2.4], (2.4, 4.36], ... Given that disk_commit times are typically in the order of microseconds; these arn't particulary suitable buckets.

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            drigby Dave Rigby (Inactive)
            drigby Dave Rigby (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty