Details
-
Improvement
-
Resolution: Fixed
-
Major
-
Cheshire-Cat
-
1
Description
A bit of feedback on the KV stats exposed by Prometheus, there's a fair bit of superfluous labelling going on.
From the Prometheus docs (https://prometheus.io/docs/practices/naming/):
Do not put the label names in the metric name, as this introduces redundancy and will cause confusion if the respective labels are aggregated away.
As a rule of thumb, either the sum() or the avg() over all dimensions of a given metric should be meaningful (though not necessarily useful).
Generally labels are useful for aggregation, so I could for example sum up memory usage across collections, or find the average item count across my buckets using the labels.
An example of KV using these labels in an ideal way:
kv_num_high_pri_requests{bucket="travel-sample",state="active"} 0.000000
|
kv_num_high_pri_requests{bucket="travel-sample",state="replica"} 0.000000
|
kv_num_high_pri_requests{bucket="travel-sample",state="pending"} 0.000000
|
Here, I could sum across the different states to produce meaningful values.
However there are a whole bunch of examples where the labels are not being used correctly:
kv_vb_replica_rollback_item_count{bucket="travel-sample",state="replica"} 0.000000
|
In this example, the replica part is in the metric name, so the label is not adding any value.
I understand that some metrics may be named a specific way (i.e. include 'replica' or 'active' in the metric name) for backwards compatiblity so you can't change the metric name, but for everywhere that's the case these shouldn't be duplicated in the labels as there becomes slight extra storage overhead for any consumers that isn't adding much value.
Attachments
Issue Links
- relates to
-
MB-44462 Make operations metrics consistent
- Closed
For Gerrit Dashboard: MB-44056 | ||||||
---|---|---|---|---|---|---|
# | Subject | Branch | Project | Status | CR | V |
145065,2 | MB-44056: Mark VBucketCountVisitor getters as const | master | kv_engine | Status: MERGED | +2 | +1 |
145066,2 | MB-44056: Rename metrics in KVBucket::appendAggregatedVBucketStats | master | kv_engine | Status: MERGED | +2 | +1 |
145067,2 | MB-44056: Rename metrics in Ephemeral appendAggregatedVBucketStats | master | kv_engine | Status: MERGED | +2 | +1 |
145078,2 | MB-44056: Default Prometheus metrics to enum key not cbstats key | master | kv_engine | Status: MERGED | +2 | +1 |
145178,2 | MB-44056: Remove scope and collection IDs from Prometheus metrics | master | kv_engine | Status: MERGED | +2 | +1 |
145471,3 | Revert "MB-44056: Update kv stats mapping file" | master | ns_server | Status: MERGED | +2 | +1 |
145494,3 | MB-44056: Update kv stats mapping file | master | ns_server | Status: MERGED | +2 | +1 |
145605,3 | MB-44056: Update kv stats mapping file | master | ns_server | Status: MERGED | +2 | +1 |
145938,19 | MB-44056: labelling for KV stats | master | ns_server | Status: MERGED | +2 | +1 |
149889,3 | MB-44056: Expose HLC stats with vbucket state label | master | kv_engine | Status: MERGED | +2 | +1 |
150023,4 | MB-44056: More kv stat names updates | master | ns_server | Status: MERGED | +2 | +1 |
150483,5 | MB-44056: rename kv_ep_ahead_exceptions | master | ns_server | Status: MERGED | +2 | +1 |