Major Description
What's the issue?
Using EC2 metadata is disabled by default (MB-39829) it must be enabled by the user; currently this is only possible via the 'CB_AWS_ENABLE_EC2_METADATA' environment variable. This isn't ideal for the backup service.
What's the fix?
We should add a flag to enable fetching credentials via the internal EC2 endpoint. Note that we should check other cloud providers to determine if we need a shared flag or not.
UPDATE:
Having had a look at three of the major cloud providers (the ones we initially intend to support) it looks like they all support some form of authentication via instance metadata.
1) For AWS we simply need to enable EC2 instance metadata (no further configuration is required)
2) Azure is slightly more complicated in that it users Azure Active Directory with both system/user roles; getting an access token optionally requires additional arguments depending on whether the user has multiple user-assigned identities (see the SDK support for more details).
3) GCE we would need to hit an endpoint to fetch a token (that would also need to be periodically refreshed since they expire after a predefined period of time) note that this may well be handled by the SDK (we haven't started looking into the GCE SDK yet). See this example in Python.
This leads me to believe we could have a common flag such as '--obj-auth-by-inst-md' or '--obj-auth-by-instance-metdata' which enables this type of authentication; for cases such as Azure, additional (Azure specific) flags will have to be added for the client id/token id and client secret (these may well only have to be optional).