Based on the recommendations and feedback from FluentCon 2021, make the following updates:
Add a dedicated worker for the `stdout` plugin to ensure we maximise throughput. In fact indicate this is a recommendation for all output plugins (it helps to quickly drop the out-of-order retries in Loki for example). For Loki, probably want to set `retry_limit 0` as well to immediately drop: https://docs.fluentbit.io/manual/administration/scheduling-and-retries#configuring-retries
Use a LUA filter to pick up any and all environment variables matching annotations/labels applied by CAO. Current approach uses a modify-nest filter pipeline but modify only works for valid variables (if not present it exits as a failure) and we just want to add any/all available (if not available then not an issue) plus ensure future-proofing. All the feedback from yesterday shows LUA is both powerful and performant although we likely want to monitor as well.
|For Gerrit Dashboard: K8S-2171|
|155168,4||K8S-2171: Resolve issues with missing variables in non CAO use cases||main||couchbase-fluent-bit||Status: MERGED||+2||+1|
|155183,4||K8S-2171: Extend tests to check for extra info||master||couchbase-operator||Status: MERGED||+2||+1|