Description
What's the issue?
We're currently performing bulk delete operations using a worker pool go goroutines, this is significantly faster than sequentially deleting data, however, the request overhead is still significant.
Most object store providers implement batch operations for this where multiple operations can be batched into a single HTTP request, this will significantly improve the performance of the remove sub-command along with the '--purge' flag.
AWS
S3 exposes this functionality in the SDK already so we could take advantage of this immediately.
Azure
Although the HTTP API exposes these batch operations, the SDK does not implement this functionality, however, it does appear to be on the developers radar.
Attachments
Issue Links
- relates to
-
MB-46642 Backup Service: Cannot delete, remove cloud storage backups
- Closed