Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-61960

kv_vb_queue_age_seconds metric can underflow

    XMLWordPrintable

Details

    • Bug
    • Resolution: Fixed
    • Minor
    • Morpheus
    • 7.6.0, 7.0.0, 7.1.1, 7.2.0, 7.2.4, 7.1.6, 7.2.5, 7.6.2, 7.6.1
    • couchbase-bucket
    • None
    • Untriaged
    • 0
    • No

    Description

      This metric is calculated based on counters from the flusher in KV. When a mutation comes in, we increment dirtyQueueSize by 1 and dirtyQueueAge by the queue time of the mutation. When the mutation is written to disk, we decrement these by the same amounts.

      Metrics collection runs on a separate thread from the FlusherTask (which writes to disk), so a metrics scrape can observe an incremented dirtyQueueAge and non-incremented dirtyQueueSize (same in the decrement case).

      The kv_vb_queue_age_seconds is computed as:

      now() * dirtyQueueSize - dirtyQueueAge

      Where dirtyQueueAge is loaded from memory first, we can see a dirtyQueueAge that is small at the end of the flush batch and a dirtyQueueSize which is larger if new mutations just came in, resulting in underflow which wraps around.

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            vesko.karaganev Vesko Karaganev
            vesko.karaganev Vesko Karaganev
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty