Details
-
Bug
-
Resolution: Fixed
-
Major
-
7.1.0
-
Untriaged
-
1
-
Unknown
Description
What's the issue?
The GCP SDK is allocating an excessive amount of memory to upload files/objects to GCP storage. Considering we're usually passing the SDK buffers which are already allocated, this newly allocated memory is entirely unnecessary.
From my understanding the chunk size is set to allow resumable uploads, however, this type of upload is still done sequentially (See MB-49640). If we set the chunk size to zero, this disables this behavior (please note, this behavior is already disabled in tools-common).
What's the fix?
We should set the upload chunking size to zero (instead of the default 16MiB).