Compact command removes deletes in cbbackupmgr

Description

None

Components

Affects versions

Fix versions

Labels

Environment

None

Link to Log File, atop/blg, CBCollectInfo, Core dump

None

Release Notes Description

None

Attachments

5

Activity

Ian McCloy February 24, 2017 at 12:14 PM

Reopening - this is not fixed in a GA version of 4.5.1, fix version should be 4.6

Arunkumar Senthilnathan December 22, 2016 at 8:22 PM

Verified in 4.5.1-2586 as per Mike's comment above:

1. Configure backup in backup host
2. Create a document in cluster host
3. Take backup - compact the backup
4. Delete the document in cluster host
5. Take backup - compact the backup
5. Merge the two compacted backups
6. List the merged backup - it shows 0 docs as expected
7. Use this merged backup to restore on the restore host - no documents restored on target - used get meta to retrieve metadata - source: (1, 0, 1482437881, 10, 1482437881631145984) target: (1, 0, 1482437983, 10, 1482437881631145984)

closing this defect

Mike Wiederhold December 22, 2016 at 5:38 PM

The behavior on both of these builds is actually correct. Keep in mind that the problem was that the delete was getting dropped from the backup file during compaction. On 4.5.1-2855 you're seeing the deleted document restored with an empty body due to . While the overall behavior is incorrect because of the bug we can see that this issue is fixed because it is clear that the delete was not dropped from the backup file.

On 4.5.1-2856 you are seeing the correct overall behavior since is fixed on that build. The problem with you testing as I pointed out on the other issue is this comment:

"get failed on the document with key not found error and metadata could not be retrieved using views"

Verification of backups should always be done using get meta and not get because get meta will retrieve delete item tombstones and in general provides more information about the restored data since it includes the items metadata. I also don't trust views to be a source for verification of backups.

I just re-ran your test case and am seeing the delete in the couchstore files.

Arunkumar Senthilnathan December 21, 2016 at 5:30 AM
Edited

Behavior in 4.5.1-2855:

1. Configure backup in backup host
2. Create a document in cluster host
3. Take backup - compact the backup
4. Delete the document in cluster host
5. Take backup - compact the backup
5. Merge the two compacted backups
6. List the merged backup - it shows 0 docs as expected
7. Use this merged backup to restore on the restore host - deleted document is restored on the restore host without any content - get failed on the document with a python sdk error but metadata could be retrieved using views - meta data had "att_reason":"invalid_json" field set

Behavior in 4.5.1-2856:

1. Configure backup in backup host
2. Create a document in cluster host
3. Take backup - compact the backup
4. Delete the document in cluster host
5. Take backup - compact the backup
5. Merge the two compacted backups
6. List the merged backup - it shows 0 docs as expected
7. Use this merged backup to restore on the restore host - no documents restored on target - get failed on the document with key not found error and metadata could not be retrieved using views

screenshots attached

please confirm if the behaviour seen in 4.5.1-2856 is correct

Mike Wiederhold December 20, 2016 at 6:15 PM
Edited

Your comment above is correct that this is due to the fix missing in MB-21924. Also, the deleted document should be restored, but it should be in deleted state. Gets should fail, but get metas should succeed. Assigning to you since this is fixed and needs to be retested once you get the missing change.

Fixed
Pinned fields
Click on the next to a field label to start pinning.

Details

Assignee

Reporter

Priority

Instabug

Open Instabug

PagerDuty

Sentry

Zendesk Support

Created December 13, 2016 at 1:31 AM
Updated February 1, 2018 at 11:49 AM
Resolved February 24, 2017 at 7:17 PM
Instabug