Details
-
Bug
-
Resolution: Unresolved
-
Test Blocker
-
7.6.4
-
Untriaged
-
0
-
Unknown
Description
I'm attempting to restore a 100M vector dataset from an S3 AWS account, for which I have access. However, the process is failing with the latest builds that is on 7.7.0-1086.
Command I am running:
Running: ./opt/couchbase/bin/cbbackupmgr restore --force-updates --archive s3://fts-vector-100m-sift-128-dims --repo fts_vector_100m_sift_128_dims --cluster http://172.23.110.53 --username Administrator --password password --threads 50 --obj-region us-east-1 --obj-staging-dir /stage --map-data bucket-1._default._default=bucket-1.scope-1.collection-1 --no-progress-bar --purge |
Error:
14:37:56 Error restoring cluster: failed to lock remote archive: failed to create lockfile object with key 'lock-9d7d0e78-8c97-4fc4-8545-6374e4128f91.lk': operation error S3: PutObject, exceeded maximum number of attempts, 5, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "https://fts-vector-100m-sift-128-dims./lock-9d7d0e78-8c97-4fc4-8545-6374e4128f91.lk?x-id=PutObject": dial tcp: lookup fts-vector-100m-sift-128-dims. on 127.0.0.53:53: server misbehaving = |
I'm running the same test for both of the Jenkins links provided below:
Success build with build - 7.7.0-1070 - https://perf.jenkins.couchbase.com/job/aether_with_remote/131/
Failed build with older build - 7.7.0-1086 - https://perf.jenkins.couchbase.com/job/aether_with_remote/130
https://perf.jenkins.couchbase.com/job/aether_with_remote/128/console (It is reproducible)
There's no issue with locking the file on S3, I manually verified multiple files with the latest build.
The locking problem only occurs with latest builds.
Additionally, I confirmed that the file was not already locked when attempting to build with the latest versions.
Attachments
Gerrit Reviews
| For Gerrit Dashboard: MB-63191 | ||||||
|---|---|---|---|---|---|---|
| # | Subject | Branch | Project | Status | CR | V |
| 214688,2 | MB-63191 Don't add scheme for empty endpoint | trinity | backup | Status: MERGED | +2 | +1 |
| 214824,1 | Merge branch 'trinity' into 'master' | master | backup | Status: ABANDONED | 0 | +1 |
| 216141,2 | Merge branch 'trinity' into 'master' | master | backup | Status: MERGED | +2 | +1 |