Description
Various improvements found that may be useful during testing:
Include cluster name when enriching the log dataReduce the tail refresh interval (currently 60 seconds) so it picks up logs sooner - the container starts quickly but if the log directory or rebalance is not present then can take a while once server starts so we may lose logs on a quick failure.Provide full integration tests with CIDefault Loki output - need to make sure no impact on customer usage, ideally a simple method to enable during testing but with a managed config. Relates toK8S-2112Look to see if we can provide counters for various errors and/or prometheus metrics (optionally): coming soon in FB but also see https://github.com/neiman-marcus/fluent-bit-out-prometheus-metricsAdd Docker-compose stack as an example for local usage.Reduce cyclometric complexity and refactor watcher to simplify, standardise logging as per operator too.Add unit tests for watcher functionality - all covered by integration tests currently so shift left if possible.Document GKE set up - issues with Autopilot and Promtail. Stalling of loki input. Working now it seems without PV: https://github.com/patrick-stephens/couchbase-gitops/tree/main/gkeRewrite the 4 letter level names for java logs. Good example of sorting case out as well: https://github.com/sassoftware/viya4-monitoring-kubernetes/blob/eaaf0498f835cbabbcf9f55715ddeafae2d68ca5/logging/fb/fluent-bit_config.configmap_open.yaml#L731- Ensure we test mount path changes, i.e. that we pick up the config from there/mount it in and can watch for changes:
K8S-2324
Attachments
Issue Links
- relates to
-
K8S-2324 Changing mount path for logging sidecar prevents dynamic reload
- Closed
-
K8S-2147 Log parsing issues
- Closed
-
K8S-2112 Fluent bit parsing error with json_rpc logs
- Resolved
-
K8S-2171 Add worker thread and LUA filter for common info
- Closed
-
K8S-2172 Add Mem_Buf_Limit setting for each input
- Closed