Details
-
Bug
-
Resolution: Fixed
-
Major
-
None
-
None
-
None
-
1
Description
Steps to recreate:
1) Using the image that is the fix for K8S-1679.
Type Reason Age From Message
|
---- ------ ---- ---- -------
|
Normal Scheduled 104s default-scheduler Successfully assigned default/my-backup-full-1601335800-cb8dg to minikube |
Normal Pulled 103s kubelet, minikube Container image "couchbase/operator-backup:6.5.1-111" already present on machine |
Normal Created 103s kubelet, minikube Created container cbbackupmgr-full
|
Normal Started 103s kubelet, minikube Started container cbbackupmgr-full
|
2) Back up was created successfully:
2020-09-28 23:30:10,855 - root - INFO - epoch: 1601335810 |
2020-09-28 23:30:10,856 - root - INFO - timestamp: 2020-09-28T23_30_10 |
2020-09-28 23:30:10,856 - root - INFO - Namespace(backup_ret='24.00', cacert=None, cluster='cb-example', config= |
'true', end=None, log_ret='24.00', mode='backup', repo=None, start=None, strategy='full_incremental', verbosity= |
'INFO') |
2020-09-28 23:30:10,856 - root - INFO - start logRetention check |
2020-09-28 23:30:10,856 - root - INFO - removed 0 logs in /data/scriptlogs/incremental |
2020-09-28 23:30:10,856 - root - INFO - mode: BACKUP |
2020-09-28 23:30:10,856 - root - INFO - Perform CONFIG: new Repo to be created |
2020-09-28 23:30:10,856 - root - INFO - Strategy: full_incremental |
2020-09-28 23:30:10,856 - root - INFO - Perform FULL BACKUP |
mkdir: cannot create directory ‘/data/backups’: File exists
|
rm: cannot remove '/data/backups/lock.lk': No such file or directory |
2020-09-28 23:30:10,861 - root - INFO - backup archive /data/backups already exists |
2020-09-28 23:30:10,861 - root - INFO - config true, config needs to be performed |
2020-09-28 23:30:10,861 - root - INFO - attempting to create repo: cb-example-2020-09-28T23_30_10 |
2020-09-28 23:30:10,878 - root - INFO - b'Backup repository `cb-example-2020-09-28T23_30_10` created successfull |
y in archive `/data/backups`\n'
|
2020-09-28 23:30:10,879 - root - INFO - attempt to write status |
2020-09-28 23:30:10,879 - root - INFO - status should have been hopefully written to |
2020-09-28 23:30:10,879 - root - INFO - attempting to query K8S objects |
2020-09-28 23:30:10,880 - root - INFO - k8s config loaded |
2020-09-28 23:30:10,880 - root - INFO - working in namespace: default |
2020-09-28 23:30:10,923 - root - INFO - BACKUP start |
2020-09-28 23:30:10,923 - root - INFO - backing up to: cb-example-2020-09-28T23_30_10 |
2020-09-28 23:30:14,985 - root - INFO - Starting Backup cleanup |
2020-09-28 23:30:14,985 - root - INFO - Time 2020-09-28 23:30:14.985481, Retention 1 day, 0:00:00, Threshold 202 |
0-09-27 23:30:14.985481 |
2020-09-28 23:30:14,985 - root - INFO - Considering repository cb-example-2020-09-28T23_30_10 ... |
2020-09-28 23:30:14,985 - root - INFO - Creation time within threshold, continuing |
2020-09-28 23:30:15,010 - root - INFO - Exiting Script, success. Output: {"location": "2020-09-28T23_30_10.97088 |
4413Z", "duration_seconds": "3.976066463", "avg_data_transfer_rate_bytes_sec": 762839, "total_items": 7303, "tot |
al_items_size_bytes": 3030384, "buckets": {"beer-sample": {"mutations_backedup": "7303", "mutations_failed": "0" |
, "deletions_backedup": "0", "deletions_failed": "0"}}} |
(422) |
Reason: Unprocessable Entity
|
HTTP response headers: HTTPHeaderDict({'Cache-Control': 'no-cache, private', 'Content-Type': 'application/json', |
'Date': 'Mon, 28 Sep 2020 23:30:10 GMT', 'Content-Length': '383'}) |
HTTP response body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"CouchbaseBack |
up.couchbase.com \"my-backup\" is invalid: status.backups.full: Required value","reason":"Invalid","details":{"n |
ame":"my-backup","group":"couchbase.com","kind":"CouchbaseBackup","causes":[{"reason":"FieldValueRequired","mess |
age":"Required value","field":"status.backups.full"}]},"code":422} |
3) We see the following error when creating a restore, this is with specifying a specific repo in the restore resource yaml:
2020-09-28 23:32:55,423 - root - INFO - epoch: 1601335975 |
2020-09-28 23:32:55,423 - root - INFO - timestamp: 2020-09-28T23_32_55 |
2020-09-28 23:32:55,423 - root - INFO - Namespace(backup_ret=None, cacert=None, cluster='cb-example', config=False, end='oldest', log_ret='24.00', mode='restore', repo='cb-example-2020-09-28T23_30_10', start='oldest', strategy=None, verbosity='INFO') |
2020-09-28 23:32:55,423 - root - INFO - start logRetention check |
2020-09-28 23:32:55,423 - root - INFO - removed 0 logs in /data/scriptlogs/restore |
2020-09-28 23:32:55,424 - root - INFO - mode: RESTORE |
mkdir: cannot create directory ‘/data/backups’: File exists
|
rm: cannot remove '/data/backups/lock.lk': No such file or directory |
2020-09-28 23:32:55,428 - root - INFO - attempting to query K8S objects |
2020-09-28 23:32:55,428 - root - INFO - k8s config loaded |
2020-09-28 23:32:55,429 - root - INFO - working in namespace: default |
Traceback (most recent call last):
|
File "/opt/couchbase/bin/backup_script", line 1125, in <module> |
main()
|
File "/opt/couchbase/bin/backup_script", line 166, in main |
k8s_setup(args)
|
File "/opt/couchbase/bin/backup_script", line 455, in k8s_setup |
cbrestore = currentpod.metadata.labels[RESTORE_LABEL]
|
KeyError: 'couchbase_restore' |
Strangely, repeating the process with the following image, restore was sucessful:
couchbase/operator-backup:6.5.0 |
2020-09-28 23:20:25,983 - root - INFO - epoch: 1601335225 |
2020-09-28 23:20:25,983 - root - INFO - timestamp: 2020-09-28T23_20_25 |
2020-09-28 23:20:25,983 - root - INFO - Namespace(backup_ret=None, cacert=None, cluster='cb-example', config=Fal |
se, end='oldest', log_ret='24.00', mode='restore', repo='cb-example-2020-09-28T23_15_41', start='oldest', strate |
gy=None, verbosity='INFO') |
2020-09-28 23:20:25,983 - root - INFO - start logRetention check |
2020-09-28 23:20:25,983 - root - INFO - removed 0 logs in /data/scriptlogs/restore |
2020-09-28 23:20:25,983 - root - INFO - mode: RESTORE |
mkdir: cannot create directory ‘/data/backups’: File exists
|
rm: cannot remove '/data/backups/lock.lk': No such file or directory |
2020-09-28 23:20:25,988 - root - INFO - attempting to query K8S objects |
NoneType: None
|
2020-09-28 23:20:25,989 - root - INFO - k8s config loaded |
2020-09-28 23:20:25,989 - root - INFO - working in namespace: default |
2020-09-28 23:20:26,014 - root - INFO - running on POD[my-restore-n5htm] for JOB[my-restore] for RESTORE[my-rest |
ore]
|
2020-09-28 23:20:26,014 - root - INFO - current CouchbaseBackupRestore object: my-restore |
2020-09-28 23:20:26,101 - root - INFO - rfoldersepos |
2020-09-28 23:20:26,101 - root - INFO - ['/data/backups/cb-example-2020-09-28T23_15_41/2020-09-28T23_15_41.68796 |
0388Z/']
|
2020-09-28 23:20:26,192 - root - INFO - RESTORE start |
2020-09-28 23:20:26,192 - root - INFO - restoring from: cb-example-2020-09-28T23_15_41 |
2020-09-28 23:20:29,499 - root - INFO - rfoldersepos |
2020-09-28 23:20:29,500 - root - INFO - ['/data/backups/cb-example-2020-09-28T23_15_41/2020-09-28T23_15_41.68796 |
0388Z/']
|
2020-09-28 23:20:29,545 - root - INFO - Exiting Script, success. Output: {"buckets": {"beer-sample": {"mutations |
": {"succeeded": "0", "failed": "0", "skipped": "7303"}, "deletions": {"succeeded": "0", "failed": "0", "skipped |
": "0"}}}, "backups_restored": "1", "duration_seconds": "3.235310705", "avg_data_transfer_rate_bytes_sec": 91471 |
8, "total_items": "7303", "total_items_size_bytes": 2956984} |
Logs are attached