Uploaded image for project: 'Couchbase Kubernetes'
  1. Couchbase Kubernetes
  2. K8S-2171

Add worker thread and LUA filter for common info

    XMLWordPrintable

Details

    • Improvement
    • Status: Closed
    • Minor
    • Resolution: Fixed
    • None
    • 2.3.0
    • logging
    • None
    • 1

    Description

      Based on the recommendations and feedback from FluentCon 2021, make the following updates:

      Add a dedicated worker for the `stdout` plugin to ensure we maximise throughput. In fact indicate this is a recommendation for all output plugins (it helps to quickly drop the out-of-order retries in Loki for example). For Loki, probably want to set `retry_limit 0` as well to immediately drop: https://docs.fluentbit.io/manual/administration/scheduling-and-retries#configuring-retries 

      Use a LUA filter to pick up any and all environment variables matching annotations/labels applied by CAO. Current approach uses a modify-nest filter pipeline but modify only works for valid variables (if not present it exits as a failure) and we just want to add any/all available (if not available then not an issue) plus ensure future-proofing. All the feedback from yesterday shows LUA is both powerful and performant although we likely want to monitor as well.

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            No need for LUA and actually LUA does not give you an easy way to get a list of all environment variables: you can get named ones with a simple call but to get the full environment requires some C-shims which is not ideal particularly for a container.

            The `record_modifier` filter actually allows you to specify a superset of those variables you want: like the `modify` one it reports an error for any missing but carries on working rather than triggering an exit. This seems a better option as long as we document that the errors for missing variables can be ignored when they're not defined (e.g. when run outside kubernetes or for those configurations missing some of the services on a node - as per best practice).

            patrick.stephens Patrick Stephens (Inactive) added a comment - No need for LUA and actually LUA does not give you an easy way to get a list of all environment variables: you can get named ones with a simple call but to get the full environment requires some C-shims which is not ideal particularly for a container. The `record_modifier` filter actually allows you to specify a superset of those variables you want: like the `modify` one it reports an error for any missing but carries on working rather than triggering an exit. This seems a better option as long as we document that the errors for missing variables can be ignored when they're not defined (e.g. when run outside kubernetes or for those configurations missing some of the services on a node - as per best practice).

            Update default image used by operator to latest fluent bit release.

            patrick.stephens Patrick Stephens (Inactive) added a comment - Update default image used by operator to latest fluent bit release.

            Released in couchbase/fluent-bit:1.0.4

            patrick.stephens Patrick Stephens (Inactive) added a comment - Released in couchbase/fluent-bit:1.0.4

            People

              patrick.stephens Patrick Stephens (Inactive)
              patrick.stephens Patrick Stephens (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty