Description
I create 5 items in default bucket, each item is a str with 256 bytes long. Then I update 3 of them, delete 2 of them. So there are 3 items at the end. Then I use cbtransfer to use couchstore files to recover these items to another cluster. It doesn't work in build 1677. You can see the "byte" transfered is incorrect:
[2012-09-04 16:17:46,221] - [remote_util:1105] INFO - running command.raw sudo /opt/couchbase/bin/cbtransfer couchstore-files:///opt/couchbase/var/lib/couchbase/data/ http://Administrator:password@10.5.2.22:8091 -b default -B default -v -v -v
[2012-09-04 16:17:46,263] - [transport:1368] INFO - Secsh channel 7 opened.
[2012-09-04 16:17:46,556] - [remote_util:1134] INFO - command executed successfully
[2012-09-04 16:17:46,556] - [remote_util:1086] INFO - 2012-09-04 16:15:34,019: mt cbtransfer...
[2012-09-04 16:17:46,557] - [remote_util:1086] INFO - 2012-09-04 16:15:34,019: mt source : couchstore-files:///opt/couchbase/var/lib/couchbase/data/
[2012-09-04 16:17:46,557] - [remote_util:1086] INFO - 2012-09-04 16:15:34,019: mt sink : http://Administrator:password@10.5.2.22:8091
[2012-09-04 16:17:46,557] - [remote_util:1086] INFO - 2012-09-04 16:15:34,020: mt opts : {'username': None, 'source_vbucket_state': 'active', 'destination_vbucket_state': 'active', 'verbose': 3, 'dry_run': False, 'extra':
, 'single_node': False, 'bucket_destination': 'default', 'destination_operation': None, 'threads': 4, 'key': None, 'password': None, 'id': None, 'bucket_source': 'default'}
[2012-09-04 16:17:46,558] - [remote_util:1086] INFO - 2012-09-04 16:15:34,020: mt source_class: <class 'pump_sfd.SFDSource'>
[2012-09-04 16:17:46,558] - [remote_util:1086] INFO - 2012-09-04 16:15:34,057: mt sink_class: <class 'pump_cb.CBSink'>
[2012-09-04 16:17:46,558] - [remote_util:1086] INFO - 2012-09-04 16:15:34,058: mt rest_request: Administrator@10.5.2.22:8091/pools/default/buckets
[2012-09-04 16:17:46,559] - [remote_util:1086] INFO - 2012-09-04 16:15:34,071: mt source_bucket: default
[2012-09-04 16:17:46,559] - [remote_util:1086] INFO - 2012-09-04 16:15:34,071: mt sink_bucket: default
[2012-09-04 16:17:46,559] - [remote_util:1086] INFO - 2012-09-04 16:15:34,072: mt source_buckets: default
[2012-09-04 16:17:46,559] - [remote_util:1086] INFO - 2012-09-04 16:15:34,072: mt bucket_source: default
[2012-09-04 16:17:46,560] - [remote_util:1086] INFO - 2012-09-04 16:15:34,072: mt source_buckets filtered: default
[2012-09-04 16:17:46,560] - [remote_util:1086] INFO - 2012-09-04 16:15:34,073: mt bucket: default
[2012-09-04 16:17:46,560] - [remote_util:1086] INFO - 2012-09-04 16:15:34,073: mt source_nodes: N/A
[2012-09-04 16:17:46,561] - [remote_util:1086] INFO - 2012-09-04 16:15:34,074: mt enqueueing node: N/A
[2012-09-04 16:17:46,561] - [remote_util:1086] INFO - 2012-09-04 16:15:34,075: w0 node: N/A
[2012-09-04 16:17:46,561] - [remote_util:1086] INFO - 2012-09-04 16:15:34,113: w0 pump (couchstore-files:///opt/couchbase/var/lib/couchbase/data/(default@N/A)->http://Administrator:password@10.5.2.22:8091(default@N/A)) done.
[2012-09-04 16:17:46,561] - [remote_util:1086] INFO - 2012-09-04 16:15:34,114: w0 source : couchstore-files:///opt/couchbase/var/lib/couchbase/data/(default@N/A)
[2012-09-04 16:17:46,562] - [remote_util:1086] INFO - 2012-09-04 16:15:34,114: w0 sink : http://Administrator:password@10.5.2.22:8091(default@N/A)
[2012-09-04 16:17:46,562] - [remote_util:1086] INFO - 2012-09-04 16:15:34,114: w0 : total | last | per sec
[2012-09-04 16:17:46,562] - [remote_util:1086] INFO - 2012-09-04 16:15:34,115: w0 batch : 1 | 1 | 26.2
[2012-09-04 16:17:46,562] - [remote_util:1086] INFO - 2012-09-04 16:15:34,115: w0 byte : 85 | 85 | 2229.2
[2012-09-04 16:17:46,563] - [remote_util:1086] INFO - 2012-09-04 16:15:34,115: w0 msg : 5 | 5 | 131.1
[2012-09-04 16:17:46,563] - [remote_util:1086] INFO - 2012-09-04 16:15:34,116: w0 node: N/A, done; rv: 0
[2012-09-04 16:17:46,563] - [remote_util:1086] INFO - .
[2012-09-04 16:17:46,563] - [remote_util:1086] INFO - bucket: default, msgs transferred...
[2012-09-04 16:17:46,564] - [remote_util:1086] INFO - : total | last | per sec
[2012-09-04 16:17:46,564] - [remote_util:1086] INFO - batch : 1 | 1 | 16.5
[2012-09-04 16:17:46,564] - [remote_util:1086] INFO - byte : 85 | 85 | 1400.7
[2012-09-04 16:17:46,564] - [remote_util:1086] INFO - msg : 5 | 5 | 82.4
[2012-09-04 16:17:46,564] - [remote_util:1086] INFO - done
Then I use "print" to see the item value before and after recovery
The original item value is nosqlV1aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.
But when recovered, it becomes:
?nosqlV1a????
But this feature works in 1666 and previous build:
[2012-09-04 16:14:25,988] - [remote_util:1105] INFO - running command.raw sudo /opt/couchbase/bin/cbtransfer couchstore-files:///opt/couchbase/var/lib/couchbase/data/ http://Administrator:password@10.1.3.72:8091 -b default -B default -v -v -v
[2012-09-04 16:14:26,026] - [transport:1368] INFO - Secsh channel 7 opened.
[2012-09-04 16:14:26,286] - [remote_util:1134] INFO - command executed successfully
[2012-09-04 16:14:26,287] - [remote_util:1086] INFO - 2012-09-04 16:13:12,264: mt cbtransfer...
[2012-09-04 16:14:26,287] - [remote_util:1086] INFO - 2012-09-04 16:13:12,264: mt source : couchstore-files:///opt/couchbase/var/lib/couchbase/data/
[2012-09-04 16:14:26,288] - [remote_util:1086] INFO - 2012-09-04 16:13:12,264: mt sink : http://Administrator:password@10.1.3.72:8091
[2012-09-04 16:14:26,288] - [remote_util:1086] INFO - 2012-09-04 16:13:12,264: mt opts : {'username': None, 'source_vbucket_state': 'active', 'destination_vbucket_state': 'active', 'verbose': 3, 'dry_run': False, 'extra':
, 'single_node': False, 'bucket_destination': 'default', 'destination_operation': None, 'threads': 4, 'key': None, 'password': None, 'id': None, 'bucket_source': 'default'}
[2012-09-04 16:14:26,288] - [remote_util:1086] INFO - 2012-09-04 16:13:12,265: mt source_class: <class 'pump_sfd.SFDSource'>
[2012-09-04 16:14:26,289] - [remote_util:1086] INFO - 2012-09-04 16:13:12,293: mt sink_class: <class 'pump_cb.CBSink'>
[2012-09-04 16:14:26,289] - [remote_util:1086] INFO - 2012-09-04 16:13:12,294: mt rest_request: Administrator@10.1.3.72:8091/pools/default/buckets
[2012-09-04 16:14:26,289] - [remote_util:1086] INFO - 2012-09-04 16:13:12,302: mt source_bucket: default
[2012-09-04 16:14:26,290] - [remote_util:1086] INFO - 2012-09-04 16:13:12,302: mt sink_bucket: default
[2012-09-04 16:14:26,290] - [remote_util:1086] INFO - 2012-09-04 16:13:12,302: mt source_buckets: default
[2012-09-04 16:14:26,290] - [remote_util:1086] INFO - 2012-09-04 16:13:12,303: mt bucket_source: default
[2012-09-04 16:14:26,290] - [remote_util:1086] INFO - 2012-09-04 16:13:12,303: mt source_buckets filtered: default
[2012-09-04 16:14:26,291] - [remote_util:1086] INFO - 2012-09-04 16:13:12,303: mt bucket: default
[2012-09-04 16:14:26,291] - [remote_util:1086] INFO - 2012-09-04 16:13:12,303: mt source_nodes: N/A
[2012-09-04 16:14:26,291] - [remote_util:1086] INFO - 2012-09-04 16:13:12,304: mt enqueueing node: N/A
[2012-09-04 16:14:26,291] - [remote_util:1086] INFO - 2012-09-04 16:13:12,305: w0 node: N/A
[2012-09-04 16:14:26,292] - [remote_util:1086] INFO - 2012-09-04 16:13:12,337: w0 pump (couchstore-files:///opt/couchbase/var/lib/couchbase/data/(default@N/A)->http://Administrator:password@10.1.3.72:8091(default@N/A)) done.
[2012-09-04 16:14:26,292] - [remote_util:1086] INFO - 2012-09-04 16:13:12,338: w0 source : couchstore-files:///opt/couchbase/var/lib/couchbase/data/(default@N/A)
[2012-09-04 16:14:26,292] - [remote_util:1086] INFO - 2012-09-04 16:13:12,338: w0 sink : http://Administrator:password@10.1.3.72:8091(default@N/A)
[2012-09-04 16:14:26,293] - [remote_util:1086] INFO - 2012-09-04 16:13:12,338: w0 : total | last | per sec
[2012-09-04 16:14:26,293] - [remote_util:1086] INFO - 2012-09-04 16:13:12,338: w0 batch : 1 | 1 | 30.4
[2012-09-04 16:14:26,293] - [remote_util:1086] INFO - 2012-09-04 16:13:12,338: w0 byte : 768 | 768 | 23361.2
[2012-09-04 16:14:26,293] - [remote_util:1086] INFO - 2012-09-04 16:13:12,338: w0 msg : 5 | 5 | 152.1
[2012-09-04 16:14:26,294] - [remote_util:1086] INFO - 2012-09-04 16:13:12,339: w0 node: N/A, done; rv: 0
[2012-09-04 16:14:26,294] - [remote_util:1086] INFO - .
[2012-09-04 16:14:26,294] - [remote_util:1086] INFO - bucket: default, msgs transferred...
[2012-09-04 16:14:26,294] - [remote_util:1086] INFO - : total | last | per sec
[2012-09-04 16:14:26,294] - [remote_util:1086] INFO - batch : 1 | 1 | 16.5
[2012-09-04 16:14:26,295] - [remote_util:1086] INFO - byte : 768 | 768 | 12689.0
[2012-09-04 16:14:26,295] - [remote_util:1086] INFO - msg : 5 | 5 | 82.6
[2012-09-04 16:14:26,295] - [remote_util:1086] INFO - done
Original Value:
nosql-1aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaAaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
Recovered Value:
nosql-1aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaAaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa