Uploaded image for project: 'Couchbase Kubernetes'
  1. Couchbase Kubernetes
  2. K8S-2171

Add worker thread and LUA filter for common info

    XMLWordPrintable

Details

    • Improvement
    • Resolution: Fixed
    • Minor
    • 2.3.0
    • None
    • logging
    • None
    • 1

    Description

      Based on the recommendations and feedback from FluentCon 2021, make the following updates:

      Add a dedicated worker for the `stdout` plugin to ensure we maximise throughput. In fact indicate this is a recommendation for all output plugins (it helps to quickly drop the out-of-order retries in Loki for example). For Loki, probably want to set `retry_limit 0` as well to immediately drop: https://docs.fluentbit.io/manual/administration/scheduling-and-retries#configuring-retries 

      Use a LUA filter to pick up any and all environment variables matching annotations/labels applied by CAO. Current approach uses a modify-nest filter pipeline but modify only works for valid variables (if not present it exits as a failure) and we just want to add any/all available (if not available then not an issue) plus ensure future-proofing. All the feedback from yesterday shows LUA is both powerful and performant although we likely want to monitor as well.

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              patrick.stephens Patrick Stephens (Inactive)
              patrick.stephens Patrick Stephens (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty