cbbackupmgr archive locking in AWS S3

This is a deceptively difficult problem; AWS S3 is eventually consistent, however, in certain scenarios we might be able to get away with "good enough" locking.

How will we (attempt) to "ensure" exclusive access to the archive?
Before answering this question it's best to understand the S3 consistency model. S3 provides read-after-write consistency (with a few caveats).

First scenario (no-head/no-get before write)
PUT archive/lock.lk 200
GET archive/lock.lk 200

Second scenario (head/get before write)
GET archive/lock.lk 404
PUT archive/lock.lk 200
GET archive/lock.lk 404

Third scenario (update before get)
PUT archive/lock.lk 200
PUT archive/lock.lk 200 (new content)
GET archive/lock.lk 200

As we can see, the creates a few complications when it comes to creating a lockfile in AWS. We are unable to check whether another process already has
exclusive access to the archive because then we have eventual consistency on the put (meaning two processes could race to create the lockfile; both
continue believing that they have exclusive access to the archive)

Locking in S3 in planned to behave as follows:
1) Create a lockfile in the archive directory in the format 'lock-$uuid.lk'
2) List the contents of the archive with the prefix 'lock-'
3) If more than one lockfile exists, we remove the one we created and error out
3a) We will log which logfiles existed at the time that we failed to lock the archive.
4) If there was only our lockfile, we have exclusive access to the archive and can continue.
5) We remove our lockfile upon completed (excluding panics where they will have to be manually removed)

Why do we create lockfiles with a uuid posfix?
This allows us to always bypass the eventual consistency and have read-after-write consistency

Does the list contents causes any issues?
When we list the contents of the archive we will cause other put operations (for other lockfiles) to be eventually consistent we need to take this into account since it will
likely be possible for cbbackupmgr to create a lockfile, list the contents and see a single lockfile (which doesn't belong to that process). This edge case should be handled
gracefully (including cleaning up the lockfile it created)