Uploaded image for project: 'Couchbase Kubernetes'
  1. Couchbase Kubernetes
  2. K8S-2493

Update log forwarding tutorial for Loki simplification

    XMLWordPrintable

Details

    • Page
    • Resolution: Resolved
    • Minor
    • 2.3.0
    • None
    • documentation
    • None
    • 5: Helm, backup, Marketplace, 1: Recovery to productivity, 3: SBEE, Multi-Cert
    • 1

    Description

      With Fluent Bit 1.1.2, there is no need to use a custom secret to control forwarding to Loki - this can be achieved by updating the pod template spec in the cluster config to provide some special annotations: https://docs-staging.couchbase.com/operator/current/tutorial-couchbase-log-forwarding.html#example-loki-stack

      As a simple example: https://github.com/couchbase/couchbase-fluent-bit/blob/3c10f93ef47443957edd738bfff72eb9e25ceaef/tools/loki-k8s-stack/values.yaml#L12-L18

            pod:
              metadata:
                annotations:
                  # Match all logs
                  fluentbit.couchbase.com/loki_match: "*"
                  # Send to this SVC
                  fluentbit.couchbase.com/loki_host: loki.monitoring
      

      This basically says match all logs (change that wildcard to just what you want, e.g. couchbase.log.audit) and send to that hostname (in cluster but separate namespace here).

      Here is an example that creates everything from scratch including the Grafana-Loki stack in a separate namespace: https://github.com/couchbase/couchbase-fluent-bit/tree/main/tools/loki-k8s-stack

      You can of course still do a full custom configuration via a K8S Secret. However this approach works with the default configuration and means people don’t need to manage Secrets. It also means we can easily integrate CMOS by just setting those annotations – potentially we could even do this when we deploy CMOS to update the pod specs automatically.

      We should highlight this example more heavily across the board as it will be the best way to support CMOS integration as well.

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              roo.thorp Roo Thorp
              patrick.stephens Patrick Stephens (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty