Uploaded image for project: 'Couchbase Kubernetes'
  1. Couchbase Kubernetes
  2. K8S-2172

Add Mem_Buf_Limit setting for each input

    XMLWordPrintable

Details

    • Improvement
    • Status: Closed
    • Minor
    • Resolution: Resolved
    • None
    • 2.3.0
    • logging
    • None
    • 5: Helm, backup, Marketplace, 50: Validation/Enforcement, 1: Recovery to productivity, 3: SBEE, Multi-Cert, 7: ARM arch? CMOS?
    • 3

    Description

      As per recommendations from AWS at Fluent Con, determine the `Mem_Buf_Limit` to best use for each input configuration: https://docs.fluentbit.io/manual/pipeline/inputs/tail 

      We probably want to get this from the resource limits on the container (if defined) via the DownwardAPI route - provide as an env var and use that (or default it if not defined).

      Other recommendations are to enable the filesystem buffering as well, limited by any limits we can discover too. We should make sure the metrics for all this are also enabled.

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            patrick.stephens Patrick Stephens (Inactive) created issue -
            patrick.stephens Patrick Stephens (Inactive) made changes -
            Field Original Value New Value
            Link This issue relates to K8S-2108 [ K8S-2108 ]
            patrick.stephens Patrick Stephens (Inactive) made changes -
            Link This issue relates to K8S-2171 [ K8S-2171 ]
            patrick.stephens Patrick Stephens (Inactive) made changes -
            Description As per recommendations from AWS at Fluent Con, determine the `Mem_Buf_Limit` to best use for each input configuration: [https://docs.fluentbit.io/manual/pipeline/inputs/tail As per recommendations from AWS at Fluent Con, determine the `Mem_Buf_Limit` to best use for each input configuration: [https://docs.fluentbit.io/manual/pipeline/inputs/tail

            We probably want to get this from the resource limits on the container (if defined) via the DownwardAPI route - provide as an env var and use that (or default it if not defined).
            patrick.stephens Patrick Stephens (Inactive) made changes -
            Description As per recommendations from AWS at Fluent Con, determine the `Mem_Buf_Limit` to best use for each input configuration: [https://docs.fluentbit.io/manual/pipeline/inputs/tail

            We probably want to get this from the resource limits on the container (if defined) via the DownwardAPI route - provide as an env var and use that (or default it if not defined).
            As per recommendations from AWS at Fluent Con, determine the `Mem_Buf_Limit` to best use for each input configuration: [https://docs.fluentbit.io/manual/pipeline/inputs/tail

            We probably want to get this from the resource limits on the container (if defined) via the DownwardAPI route - provide as an env var and use that (or default it if not defined).

            Other recommendations are to enable the filesystem buffering as well, limited by any limits we can discover too. We should make sure the metrics for all this are also enabled.
            simon.murray Simon Murray made changes -
            Fix Version/s not-targeted [ 16613 ]
            simon.murray Simon Murray made changes -
            Rank Ranked higher
            simon.murray Simon Murray made changes -
            Rank Ranked higher
            patrick.stephens Patrick Stephens (Inactive) made changes -
            Fix Version/s 2.3.0 [ 17600 ]
            Fix Version/s not-targeted [ 16613 ]
            simon.murray Simon Murray made changes -
            Rank Ranked lower
            simon.murray Simon Murray made changes -
            Rank Ranked higher
            simon.murray Simon Murray made changes -
            Rank Ranked higher
            simon.murray Simon Murray made changes -
            Rank Ranked higher
            simon.murray Simon Murray made changes -
            Story Points 1 3
            ingenthr Matt Ingenthron made changes -
            Fix Version/s 2.3.0-beta [ 18040 ]
            Fix Version/s 2.3.0 [ 17600 ]
            ingenthr Matt Ingenthron made changes -
            Fix Version/s 2.3.0 [ 17600 ]
            Fix Version/s 2.3.0-beta [ 18040 ]
            simon.murray Simon Murray made changes -
            Rank Ranked higher
            simon.murray Simon Murray made changes -
            Rank Ranked lower
            simon.murray Simon Murray made changes -
            Rank Ranked lower
            patrick.stephens Patrick Stephens (Inactive) made changes -
            Assignee Patrick Stephens [ JIRAUSER25332 ] Alex Emery [ JIRAUSER26037 ]
            ingenthr Matt Ingenthron made changes -
            Sprint 50 [ 1890 ]
            Alex.emery Alex Emery made changes -
            Status Open [ 1 ] In Progress [ 3 ]
            ingenthr Matt Ingenthron made changes -
            Sprint 50: Validation/Enforcement [ 1890 ] 50: Validation/Enforcement, 1 [ 1890, 1891 ]
            simon.murray Simon Murray made changes -
            Sprint 50: Validation/Enforcement, 1: Recovery to productivity [ 1890, 1891 ] 5: Automation, upgrades 2, 50: Validation/Enforcement, 1: Recovery to productivity [ 1534, 1890, 1891 ]
            simon.murray Simon Murray made changes -
            Sprint 5: Automation, upgrades 2, 50: Validation/Enforcement, 1: Recovery to productivity [ 1534, 1890, 1891 ] 50: Validation/Enforcement, 1: Recovery to productivity, 3: SBEE, Multi-Cert [ 1890, 1891, 1951 ]
            simon.murray Simon Murray made changes -
            Rank Ranked lower
            ingenthr Matt Ingenthron made changes -
            Sprint 50: Validation/Enforcement, 1: Recovery to productivity, 3: SBEE, Multi-Cert [ 1890, 1891, 1951 ] 5: Automation, upgrades 2, 50: Validation/Enforcement, 1: Recovery to productivity, 3: SBEE, Multi-Cert [ 1534, 1890, 1891, 1951 ]
            simon.murray Simon Murray made changes -
            Sprint 5: Helm, backup, Marketplace, 50: Validation/Enforcement, 1: Recovery to productivity, 3: SBEE, Multi-Cert [ 1534, 1890, 1891, 1951 ] 5: Helm, backup, Marketplace, 50: Validation/Enforcement, 1: Recovery to productivity, 3: SBEE, Multi-Cert, 7: ARM arch? CMOS? [ 1534, 1890, 1891, 1951, 2023 ]
            simon.murray Simon Murray made changes -
            Rank Ranked lower
            simon.murray Simon Murray made changes -
            Rank Ranked higher
            simon.murray Simon Murray made changes -
            Rank Ranked lower

            Build couchbase-fluent-bit-1.1.3-110 contains couchbase-fluent-bit commit b66da5a with commit message:
            K8S-2172: Added mem buf limits for inputs

            build-team Couchbase Build Team added a comment - Build couchbase-fluent-bit-1.1.3-110 contains couchbase-fluent-bit commit b66da5a with commit message: K8S-2172 : Added mem buf limits for inputs
            Alex.emery Alex Emery made changes -
            Resolution Resolved [ 10400 ]
            Status In Progress [ 3 ] Resolved [ 5 ]

            Build couchbase-operator-2.3.0-259 contains couchbase-operator commit fee2a4e with commit message:
            K8S-2172: Updated mem buf limits env var

            build-team Couchbase Build Team added a comment - Build couchbase-operator-2.3.0-259 contains couchbase-operator commit fee2a4e with commit message: K8S-2172 : Updated mem buf limits env var

            Verified in logging regression

            arunkumar Arunkumar Senthilnathan (Inactive) added a comment - Verified in logging regression
            arunkumar Arunkumar Senthilnathan (Inactive) made changes -
            Status Resolved [ 5 ] Closed [ 6 ]

            People

              Alex.emery Alex Emery
              patrick.stephens Patrick Stephens (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty