Uploaded image for project: 'Couchbase Kubernetes'
  1. Couchbase Kubernetes
  2. K8S-2108

Fluent bit improvements

    XMLWordPrintable

Details

    • Improvement
    • Status: Closed
    • Minor
    • Resolution: Done
    • None
    • 2.3.0
    • logging, operator
    • None
    • 1

    Description

      Various improvements found that may be useful during testing:

      1. Include cluster name when enriching the log data
      2. Reduce the tail refresh interval (currently 60 seconds) so it picks up logs sooner - the container starts quickly but if the log directory or rebalance is not present then can take a while once server starts so we may lose logs on a quick failure.
      3. Provide full integration tests with CI
      4. Default Loki output - need to make sure no impact on customer usage, ideally a simple method to enable during testing but with a managed config. Relates to K8S-2112
      5. Look to see if we can provide counters for various errors and/or prometheus metrics (optionally): coming soon in FB but also see https://github.com/neiman-marcus/fluent-bit-out-prometheus-metrics 
      6. Add Docker-compose stack as an example for local usage.
      7. Reduce cyclometric complexity and refactor watcher to simplify, standardise logging as per operator too.
      8. Add unit tests for watcher functionality - all covered by integration tests currently so shift left if possible.
      9. Document GKE set up - issues with Autopilot and Promtail. Stalling of loki input. Working now it seems without PV: https://github.com/patrick-stephens/couchbase-gitops/tree/main/gke 
      10. Rewrite the 4 letter level names for java logs. Good example of sorting case out as well: https://github.com/sassoftware/viya4-monitoring-kubernetes/blob/eaaf0498f835cbabbcf9f55715ddeafae2d68ca5/logging/fb/fluent-bit_config.configmap_open.yaml#L731 
      11. Ensure we test mount path changes, i.e. that we pick up the config from there/mount it in and can watch for changes: K8S-2324

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              roo.thorp Roo Thorp
              patrick.stephens Patrick Stephens (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                PagerDuty