Uploaded image for project: 'Couchbase Kubernetes'
  1. Couchbase Kubernetes
  2. K8S-2171

Add worker thread and LUA filter for common info

    XMLWordPrintable

Details

    • Improvement
    • Resolution: Fixed
    • Minor
    • 2.3.0
    • None
    • logging
    • None
    • 1

    Description

      Based on the recommendations and feedback from FluentCon 2021, make the following updates:

      Add a dedicated worker for the `stdout` plugin to ensure we maximise throughput. In fact indicate this is a recommendation for all output plugins (it helps to quickly drop the out-of-order retries in Loki for example). For Loki, probably want to set `retry_limit 0` as well to immediately drop: https://docs.fluentbit.io/manual/administration/scheduling-and-retries#configuring-retries 

      Use a LUA filter to pick up any and all environment variables matching annotations/labels applied by CAO. Current approach uses a modify-nest filter pipeline but modify only works for valid variables (if not present it exits as a failure) and we just want to add any/all available (if not available then not an issue) plus ensure future-proofing. All the feedback from yesterday shows LUA is both powerful and performant although we likely want to monitor as well.

      Attachments

        Issue Links

          For Gerrit Dashboard: K8S-2171
          # Subject Branch Project Status CR V

          Activity

            People

              patrick.stephens Patrick Stephens (Inactive)
              patrick.stephens Patrick Stephens (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty