Description
Various improvements found that may be useful during testing:
Include cluster name when enriching the log dataReduce the tail refresh interval (currently 60 seconds) so it picks up logs sooner - the container starts quickly but if the log directory or rebalance is not present then can take a while once server starts so we may lose logs on a quick failure.Provide full integration tests with CIDefault Loki output - need to make sure no impact on customer usage, ideally a simple method to enable during testing but with a managed config. Relates toK8S-2112Look to see if we can provide counters for various errors and/or prometheus metrics (optionally): coming soon in FB but also see https://github.com/neiman-marcus/fluent-bit-out-prometheus-metricsAdd Docker-compose stack as an example for local usage.Reduce cyclometric complexity and refactor watcher to simplify, standardise logging as per operator too.Add unit tests for watcher functionality - all covered by integration tests currently so shift left if possible.Document GKE set up - issues with Autopilot and Promtail. Stalling of loki input. Working now it seems without PV: https://github.com/patrick-stephens/couchbase-gitops/tree/main/gkeRewrite the 4 letter level names for java logs. Good example of sorting case out as well: https://github.com/sassoftware/viya4-monitoring-kubernetes/blob/eaaf0498f835cbabbcf9f55715ddeafae2d68ca5/logging/fb/fluent-bit_config.configmap_open.yaml#L731- Ensure we test mount path changes, i.e. that we pick up the config from there/mount it in and can watch for changes:
K8S-2324
Attachments
Issue Links
- relates to
-
K8S-2324 Changing mount path for logging sidecar prevents dynamic reload
-
- Closed
-
-
K8S-2147 Log parsing issues
-
- Closed
-
-
K8S-2112 Fluent bit parsing error with json_rpc logs
-
- Resolved
-
-
K8S-2171 Add worker thread and LUA filter for common info
-
- Closed
-
-
K8S-2172 Add Mem_Buf_Limit setting for each input
-
- Closed
-
It would be nice to be able to move certain bits of the user documentation to the couchbase-fluent-bit repo. This would Couchbase-specific documentation (i.e., not directed at the open source audience) that would be pulled into the Operator docs. It will likely be mostly reference-type documentation, such as compatibility tables, marked-up snippets of config files, etc.
Our docs site generator — Antora — supports using tags instead of branches for version control of the docs. So if we want to move some doc content to the fluent bit repo, then we need to make sure that the docs are considered when tagging a release. We can control how the semver appears as in the docs, but we need to make sure that docs updates go in before the release is tagged, since there will eventually be releases that are incompatible with older/newer versions of the Operator, and we need to be able to pull relevant docs into each version of the Operator docs.