Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-7427

Compress logs to allow for much longer time period to be captured

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Duplicate
    • Affects Version/s: 2.0, 2.0.1
    • Fix Version/s: 3.0
    • Component/s: ns_server
    • Security Level: Public
    • Labels:

      Description

      With the new method of splitting out the log messages for all different levels, it makes it particularly difficult to correlate past log events since the logs rollover at different times.

      Suggestion is to compress the logs to keep much more data under the 100mb limit.

      Would it make sense to only have 2 log files for each type? 1 "in use" log file that's not compressed, and then roll it over into a single aggregate historical log file that is compressed? That would make it a little easier to look at the log files while on the node itself rather than using the collect_info to aggregate them together...

        Issue Links

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

          Hide
          alkondratenko Aleksey Kondratenko (Inactive) added a comment -

          We decided to put different types of log messages to different log files exactly because we want them to be rotated independently.

          I.e. to be able to preserve log error messages longer than debug.

          Show
          alkondratenko Aleksey Kondratenko (Inactive) added a comment - We decided to put different types of log messages to different log files exactly because we want them to be rotated independently. I.e. to be able to preserve log error messages longer than debug.
          Hide
          perry Perry Krug added a comment -

          Sorry Alk, but the larger problem is that with the current system of rolling over separately, it makes it very hard to correlate messages across log files (or impossible).

          The point in my mind of separating them by file type was actually more so that we didn't have to parse through debug messages when looking for errors...not for the preservation of log messages for different periods of time.

          Show
          perry Perry Krug added a comment - Sorry Alk, but the larger problem is that with the current system of rolling over separately, it makes it very hard to correlate messages across log files (or impossible). The point in my mind of separating them by file type was actually more so that we didn't have to parse through debug messages when looking for errors...not for the preservation of log messages for different periods of time.
          Hide
          alkondratenko Aleksey Kondratenko (Inactive) added a comment -

          Yes. Expect some tools for that soon.

          Show
          alkondratenko Aleksey Kondratenko (Inactive) added a comment - Yes. Expect some tools for that soon.
          Hide
          perry Perry Krug added a comment -

          But we need to change the way the product works as well. When stats logs roll over at 2x the speed of debug logs, you can't look back for the stats when a problem happened...

          Show
          perry Perry Krug added a comment - But we need to change the way the product works as well. When stats logs roll over at 2x the speed of debug logs, you can't look back for the stats when a problem happened...
          Hide
          perry Perry Krug added a comment -

          But we need to change the way the product works as well. When stats logs roll over at 2x the speed of debug logs, you can't look back for the stats when a problem happened...

          Show
          perry Perry Krug added a comment - But we need to change the way the product works as well. When stats logs roll over at 2x the speed of debug logs, you can't look back for the stats when a problem happened...
          Hide
          alkondratenko Aleksey Kondratenko (Inactive) added a comment -

          Yes, but the alternative is having less debug logs. IMHO it's fair.

          Show
          alkondratenko Aleksey Kondratenko (Inactive) added a comment - Yes, but the alternative is having less debug logs. IMHO it's fair.
          Hide
          perry Perry Krug added a comment -

          Sorry, I disagree..."fair" doesn't help support when we need it and the logs are already gone.

          Having all logs last for 2 weeks regardless of size (within reason) is really what I think helps.

          Show
          perry Perry Krug added a comment - Sorry, I disagree..."fair" doesn't help support when we need it and the logs are already gone. Having all logs last for 2 weeks regardless of size (within reason) is really what I think helps.
          Hide
          siri Sriram Melkote added a comment -

          Summarizing email thread:

          (a) We should compress log files, as they compress well (10x) and will mean fewer rollovers for a given time period and fixed log space
          (b) Logs should have an uniform timestamp line periodically (irrespective of rest of their content or format) so they can be aligned easily visually or by scripts

          Show
          siri Sriram Melkote added a comment - Summarizing email thread: (a) We should compress log files, as they compress well (10x) and will mean fewer rollovers for a given time period and fixed log space (b) Logs should have an uniform timestamp line periodically (irrespective of rest of their content or format) so they can be aligned easily visually or by scripts
          Hide
          maria Maria McDuff (Inactive) added a comment -

          closing as dupes.

          Show
          maria Maria McDuff (Inactive) added a comment - closing as dupes.

            People

            • Assignee:
              alkondratenko Aleksey Kondratenko (Inactive)
              Reporter:
              perry Perry Krug
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Gerrit Reviews

                There are no open Gerrit changes