Details
Description
Steps to reproduce:
- Load any sample bucket, say beer-sample.
- Take backup using cbbackup locally via
./cbbackup http://localhost:8091 ~/backup1 -u Administrator -p couchbase -vvv
|
- Delete all design documents for the bucket.
- Run cbrestore with or without design_doc_only option:
./cbrestore ~/backup1 http://127.0.0.1:8091 -x design_doc_only=1 -b beer-sample -u Administrator -p couchbase -vvv
|
- Cannot see any design docs in the UI. Neither are the corresponding files created on the server.
—
Following are the logs generated for the above cbrestore command for 6.0.0:
./cbrestore ~/backup1 http://127.0.0.1:8091 -x design_doc_only=1 -b beer-sample -u Administrator -p couchbase -vvv
|
2018-12-20 05:15:37,883: mt cbrestore...
|
2018-12-20 05:15:37,884: mt source : /home/vagrant/backup1
|
2018-12-20 05:15:37,884: mt sink : http://127.0.0.1:8091
|
2018-12-20 05:15:37,884: mt opts : {'username': '<xxx>', 'verbose': 3, 'extra': {'max_retry': 10.0, 'rehash': 0.0, 'dcp_consumer_queue_length': 1000.0, 'data_only': 0.0, 'uncompress': 0.0, 'nmv_retry': 1.0, 'conflict_resolve': 1.0, 'cbb_max_mb': 100000.0, 'report': 5.0, 'mcd_compatible': 1.0, 'try_xwm': 1.0, 'backoff_cap': 10.0, 'batch_max_bytes': 400000.0, 'report_full': 2000.0, 'flow_control': 1.0, 'batch_max_size': 1000.0, 'seqno': 0.0, 'design_doc_only': 1.0, 'allow_recovery_vb_remap': 0.0, 'recv_min_bytes': 4096.0}, 'collection': None, 'ssl': False, 'threads': 4, 'to_date': None, 'key': None, 'password': '<xxx>', 'id': None, 'bucket_source': 'beer-sample', 'silent': False, 'dry_run': False, 'from_date': None, 'bucket_destination': None, 'add': False, 'vbucket_list': None, 'separator': '::'}
|
2018-12-20 05:15:37,884: mt source_class: <class 'pump_bfd.BFDSource'>
|
2018-12-20 05:15:37,885: mt sink_class: <class 'pump_cb.CBSink'>
|
2018-12-20 05:15:37,896: mt Starting new HTTP connection (1): 127.0.0.1
|
2018-12-20 05:15:37,900: mt "GET /pools/default/buckets HTTP/1.1" 200 10043
|
2018-12-20 05:15:37,902: mt source_bucket: beer-sample
|
2018-12-20 05:15:37,902: mt sink_bucket: beer-sample
|
2018-12-20 05:15:37,902: mt source_buckets: beer-sample
|
2018-12-20 05:15:37,902: mt bucket_source: beer-sample
|
2018-12-20 05:15:37,902: mt source_buckets filtered: beer-sample
|
2018-12-20 05:15:37,902: mt bucket: beer-sample
|
transfer design doc only. bucket msgs will be skipped.
|
done
|
And following are the logs generated for a successful run where data was backed up and restored from an earlier version, 5.1.1:
vagrant@node1-cb511-ubuntu14:/tmp/backup/bucket-travel-sample$ /opt/couchbase/bin/cbrestore /tmp/backup http://127.0.0.1:8091 -x design_doc_only=1 -b travel-sample -u Administrator -p password -vvv
|
2018-12-20 04:22:58,732: mt cbrestore...
|
2018-12-20 04:22:58,732: mt source : /tmp/backup
|
2018-12-20 04:22:58,733: mt sink : http://127.0.0.1:8091
|
2018-12-20 04:22:58,734: mt opts : {'username': '<xxx>', 'verbose': 3, 'extra': {'max_retry': 10.0, 'rehash': 0.0, 'dcp_consumer_queue_length': 1000.0, 'data_only': 0.0, 'uncompress': 0.0, 'nmv_retry': 1.0, 'conflict_resolve': 1.0, 'cbb_max_mb': 100000.0, 'report': 5.0, 'mcd_compatible': 1.0, 'try_xwm': 1.0, 'backoff_cap': 10.0, 'batch_max_bytes': 400000.0, 'report_full': 2000.0, 'flow_control': 1.0, 'batch_max_size': 1000.0, 'seqno': 0.0, 'design_doc_only': 1.0, 'allow_recovery_vb_remap': 0.0, 'recv_min_bytes': 4096.0}, 'collection': None, 'ssl': False, 'threads': 4, 'to_date': None, 'key': None, 'password': '<xxx>', 'id': None, 'bucket_source': 'travel-sample', 'silent': False, 'dry_run': False, 'from_date': None, 'bucket_destination': None, 'add': False, 'vbucket_list': None, 'separator': '::'}
|
2018-12-20 04:22:58,735: mt source_class: <class 'pump_bfd.BFDSource'>
|
2018-12-20 04:22:58,735: mt sink_class: <class 'pump_cb.CBSink'>
|
2018-12-20 04:22:58,741: mt Starting new HTTP connection (1): 127.0.0.1
|
2018-12-20 04:22:58,745: mt "GET /pools/default/buckets HTTP/1.1" 200 10023
|
2018-12-20 04:22:58,747: mt source_bucket: travel-sample
|
2018-12-20 04:22:58,747: mt sink_bucket: travel-sample
|
2018-12-20 04:22:58,747: mt source_buckets: travel-sample
|
2018-12-20 04:22:58,748: mt bucket_source: travel-sample
|
2018-12-20 04:22:58,748: mt source_buckets filtered: travel-sample
|
2018-12-20 04:22:58,748: mt bucket: travel-sample
|
transfer design doc only. bucket msgs will be skipped.
|
2018-12-20 04:22:58,749: mt design_doc row: {u'controllers': {u'compact': u'/pools/default/buckets/travel-sample/ddocs/_design%2Fdev_a/controller/compactView', u'setUpdateMinChanges': u'/pools/default/buckets/travel-sample/ddocs/_design%2Fdev_a/controller/setUpdateMinChanges'}, u'doc': {u'json': {u'views': {u'a': {u'map': u'function (doc, meta) {\n emit(meta.id, null);\n}'}}}, u'meta': {u'rev': u'2-c362e223', u'id': u'_design/dev_a'}}}
|
2018-12-20 04:22:58,749: mt design_doc: {"views": {"a": {"map": "function (doc, meta) {\n emit(meta.id, null);\n}"}}}
|
2018-12-20 04:22:58,750: mt design_doc id: _design/dev_a at: /travel-sample%2B84a31a2dae6f5d1b9c8cbdb93a09cd37/_design/dev_a
|
2018-12-20 04:22:58,750: mt rest_request: Administrator@127.0.0.1:8092/travel-sample%2B84a31a2dae6f5d1b9c8cbdb93a09cd37/_design/dev_a; reason: consume_design
|
2018-12-20 04:22:58,853: mt design_doc created at: /travel-sample%2B84a31a2dae6f5d1b9c8cbdb93a09cd37/_design/dev_a
|
2018-12-20 04:22:58,857: mt Starting new HTTP connection (1): 127.0.0.1
|
2018-12-20 04:22:58,859: mt "GET /pools/default/nodeServices HTTP/1.1" 200 337
|
2018-12-20 04:22:58,861: mt Starting new HTTP connection (1): 127.0.0.1
|
2018-12-20 04:22:58,960: mt "POST /restoreIndexMetadata?bucket=travel-sample HTTP/1.1" 200 18
|
done
|
Note that the log statement in the successful run, printing the contents of the ddoc.json file created by cbbackup, which is not present in the unsuccessful run.
2018-12-20 04:22:58,749: mt design_doc row: {u'controllers': {u'compact': u'/pools/default/buckets/travel-sample/ddocs/_design%2Fdev_a/controller/compactView', u'setUpdateMinChanges': u'/pools/default/buckets/travel-sample/ddocs/_design%2Fdev_a/controller/setUpdateMinChanges'}, u'doc': {u'json': {u'views': {u'a': {u'map': u'function (doc, meta) {\n emit(meta.id, null);\n}'}}}, u'meta': {u'rev': u'2-c362e223', u'id': u'_design/dev_a'}}}
|
Attachments
Issue Links
- backports to
-
MB-32626
[BP 5.5.x] cbrestore doesn't restore view definitions taken via cbbackup
-
- Closed
-
Gerrit Reviews
Activity
Build couchbase-server-6.5.0-2078 contains couchbase-cli commit bd24e5e with commit message:
MB-32419 Simplify transfering of design doc
Build couchbase-server-6.5.0-2078 contains couchbase-cli commit 25816fd with commit message:
MB-32419 Simplify transfering of design doc
Verified in 5.5.4-4321:
[root@node1-mad-hatter-testing-centos7 ~]# /opt/couchbase/bin/cbrestore ~/backup1 http://127.0.0.1:8091 -x design_doc_only=1 -b beer-sample -u Administrator -p password -vvv
2019-02-28 03:25:58,710: mt cbrestore...
2019-02-28 03:25:58,711: mt source : /root/backup1
2019-02-28 03:25:58,711: mt sink : http://127.0.0.1:8091
2019-02-28 03:25:58,711: mt opts : {'username': '<xxx>', 'verbose': 3, 'extra':
, 'collection': None, 'ssl': False, 'threads': 4, 'to_date': None, 'key': None, 'password': '<xxx>', 'id': None, 'bucket_source': 'beer-sample', 'silent': False, 'dry_run': False, 'from_date': None, 'bucket_destination': None, 'add': False, 'vbucket_list': None, 'separator': '::'}
2019-02-28 03:25:58,711: mt source_class: <class 'pump_bfd.BFDSource'>
2019-02-28 03:25:58,711: mt sink_class: <class 'pump_cb.CBSink'>
2019-02-28 03:25:58,723: mt Starting new HTTP connection (1): 127.0.0.1
2019-02-28 03:25:58,727: mt "GET /pools/default/buckets HTTP/1.1" 200 10065
2019-02-28 03:25:58,734: mt source_bucket: beer-sample
2019-02-28 03:25:58,734: mt sink_bucket: beer-sample
2019-02-28 03:25:58,734: mt source_buckets: beer-sample
2019-02-28 03:25:58,734: mt bucket_source: beer-sample
2019-02-28 03:25:58,734: mt source_buckets filtered: beer-sample
2019-02-28 03:25:58,735: mt bucket: beer-sample
transfer design doc only. bucket msgs will be skipped.
2019-02-28 03:25:58,735: mt design_doc row: {u'controllers':
, u'doc': {u'json': {u'_id': u'_design/beer', u'language': u'javascript', u'views': {u'brewery_beers': {u'map': u'function(doc, meta) {\n switch(doc.type) {\n case "brewery":\n emit([meta.id]);\n break;\n case "beer":\n if (doc.brewery_id)
{\n emit([doc.brewery_id, meta.id]);\n }\n break;\n }\n}\n'}, u'by_location': {u'map': u'function (doc, meta) {\n if (doc.country, doc.state, doc.city) {\n emit([doc.country, doc.state, doc.city], 1);\n } else if (doc.country, doc.state) {\n emit([doc.country, doc.state], 1);\n } else if (doc.country) {\n emit([doc.country], 1);\n }\n}', u'reduce': u'_count'}}}, u'meta': {u'rev': u'1-bd8521cf', u'id': u'_design/beer'}}}2019-02-28 03:25:58,735: mt design_doc: {"_id": "_design/beer", "language": "javascript", "views": {"brewery_beers": {"map": "function(doc, meta) {\n switch(doc.type) {\n case \"brewery\":\n emit([meta.id]);\n break;\n case \"beer\":\n if (doc.brewery_id) {n emit([doc.brewery_id, meta.id]);n }
\n break;\n }\n}\n"}, "by_location": {"map": "function (doc, meta) {\n if (doc.country, doc.state, doc.city)
{\n emit([doc.country, doc.state, doc.city], 1);\n }else if (doc.country, doc.state)
{\n emit([doc.country, doc.state], 1);\n }else if (doc.country)
{\n emit([doc.country], 1);\n }\n}", "reduce": "_count"}}}
2019-02-28 03:25:58,735: mt design_doc id: _design/beer at: /beer-sample%2B5407dd72f4e8f9aa1f16aa8fd422356c/_design/beer
2019-02-28 03:25:58,735: mt rest_request: <ud>Administrator</ud>@127.0.0.1:8092/beer-sample%2B5407dd72f4e8f9aa1f16aa8fd422356c/_design/beer; reason: consume_design
2019-02-28 03:25:58,782: mt design_doc created at: /beer-sample%2B5407dd72f4e8f9aa1f16aa8fd422356c/_design/beer
2019-02-28 03:25:58,783: mt Starting new HTTP connection (1): 127.0.0.1
2019-02-28 03:25:58,785: mt "GET /pools/default/nodeServices HTTP/1.1" 200 363
2019-02-28 03:25:58,786: mt Starting new HTTP connection (1): 127.0.0.1
2019-02-28 03:25:58,879: mt "POST /restoreIndexMetadata?bucket=beer-sample HTTP/1.1" 200 18
done
Verified on build 6.5.0-3143, index defs are restored back to restore cluster.
Build couchbase-server-6.5.0-2078 contains couchbase-cli commit 0db2826 with commit message:
MB-32672Merge branch 'vulcan' into 'alice'