Description
What's the issue?
Once our archive gets quite large, the staging directory population process becomes quite time consuming.
I think there's a couple of things we can do to help improve this:
1) We should inflate archive metadata using a worker pool (if possible)
2) We should avoid compressing metadata that might already be compressed (e.g. collect the existing range files vs re-compressing them)