Details
-
Improvement
-
Resolution: Done
-
Major
-
Cheshire-Cat
Description
The upload/download pools for the AWS client are thin wrappers around the multipart downloaders provided by the SDK. Looking at the Azure SDK it's clear that they provider similar functionality, however, some notes indicate that they may not be idea due to high memory allocation with large files. We should look into/test this, and implement our own if required; these upload/download pools are important for staging directory population/syncing. Note that the more important use case is downloading Rift indexes during a restore, which can become quite large (hence the standard implementation may not be ideal).