Details
-
Improvement
-
Resolution: Fixed
-
Major
-
None
-
Security Level: Public
-
None
-
CBG Sprint 44
-
3
Description
When we extract rotated log files, we extract them to perform log redaction on them, and leave them in an extracted state lying around in /tmp for the duration of the sgcollect_info run before putting them into the final zip.
This seems to be counter to what was done in CBG-244, but here's an example with around 6GB of pre-compression logs:
There are a couple of things we can do to optimise the collection of rotated logs:
- If we're not performing log redaction, skip the extraction step completely. Put the .gzip files straight into the zip file as-is.
- If we are performing redaction:
- extract
- redact
- re-compress
- put the newly redacted/compressed file into the zip file
- Make sure the extracted files are cleaned up as we iterate over each log file, to reduce the overall disk requirements.