Uploaded image for project: 'Couchbase Kubernetes'
  1. Couchbase Kubernetes
  2. K8S-2203

Retain log offset in DB for restarts

    XMLWordPrintable

Details

    • Improvement
    • Resolution: Unresolved
    • Minor
    • .maintenance
    • None
    • logging
    • None
    • 1

    Description

      Fluent Bit provides an option to store where we are up to in a log file within a DB: https://docs.fluentbit.io/manual/pipeline/inputs/tail#config 

      DB
      Specify the database file to keep track of monitored files and offsets.

      DB.sync
      Set a default synchronization (I/O) method. Values: Extra, Full, Normal, Off. This flag affects how the internal SQLite engine do synchronization to disk, for more details about each option please refer to this section. Most of workload scenarios will be fine with normal mode, but if you really need full synchronization after every write operation you should set full mode. Note that full has a high I/O performance cost.

      DB.locking
      Specify that the database will be accessed only by Fluent Bit. Enabling this feature helps to increase performance when accessing the database but it restrict any external tool to query the content.

      This can then be used on restart to start from that point.

      Need to confirm the impact and overhead of this.

      Attachments

        Activity

          People

            justin.ashworth Justin Ashworth
            patrick.stephens Patrick Stephens (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:

              PagerDuty