Description
It looks like the json_rpc.log is causing some issues with its multiline parsing. Occasionally this seems to get out of sync so we keep ending up with stdout lines that are from a multiline message but just as individual randoms.
It may also be related to embedding new line characters in the log message as well, ideally we should remove these.
This was discovered when deploying the Loki stack which allowed us to filter the logs and visualise them a lot more: https://grafana.com/docs/loki/latest/installation/helm/#deploy-loki-stack-loki-fluent-bit-grafana-prometheus
helm repo add grafana https://grafana.github.io/helm-charts |
kubectl create namespace logging
|
helm upgrade --install loki --namespace=logging grafana/loki-stack --set fluent-bit.enabled=false,promtail.enabled=true,grafana.enabled=true,prometheus.enabled=true,prometheus.alertmanager.persistentVolume.enabled=false,prometheus.server.persistentVolume.enabled=false |
Attachments
Issue Links
- relates to
-
K8S-2108 Fluent bit improvements
-
- Closed
-