Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-44056

Standardize on metric names or labelling for KV stats

    XMLWordPrintable

Details

    • 1

    Description

      A bit of feedback on the KV stats exposed by Prometheus, there's a fair bit of superfluous labelling going on.

      From the Prometheus docs (https://prometheus.io/docs/practices/naming/):

      Do not put the label names in the metric name, as this introduces redundancy and will cause confusion if the respective labels are aggregated away.

      As a rule of thumb, either the sum() or the avg() over all dimensions of a given metric should be meaningful (though not necessarily useful).

      Generally labels are useful for aggregation, so I could for example sum up memory usage across collections, or find the average item count across my buckets using the labels.

      An example of KV using these labels in an ideal way:

      kv_num_high_pri_requests{bucket="travel-sample",state="active"} 0.000000
      kv_num_high_pri_requests{bucket="travel-sample",state="replica"} 0.000000
      kv_num_high_pri_requests{bucket="travel-sample",state="pending"} 0.000000
      

      Here, I could sum across the different states to produce meaningful values.

      However there are a whole bunch of examples where the labels are not being used correctly:

      kv_vb_replica_rollback_item_count{bucket="travel-sample",state="replica"} 0.000000
      

      In this example, the replica part is in the metric name, so the label is not adding any value.
      I understand that some metrics may be named a specific way (i.e. include 'replica' or 'active' in the metric name) for backwards compatiblity so you can't change the metric name, but for everywhere that's the case these shouldn't be duplicated in the labels as there becomes slight extra storage overhead for any consumers that isn't adding much value.

      Attachments

        Issue Links

          For Gerrit Dashboard: MB-44056
          # Subject Branch Project Status CR V

          Activity

            People

              timofey.barmin Timofey Barmin
              matt.carabine Matt Carabine (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              12 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                PagerDuty