Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-21962

Compact command removes deletes in cbbackupmgr

    XMLWordPrintable

Details

    • Bug
    • Status: Closed
    • Blocker
    • Resolution: Fixed
    • 4.5.0, 4.5.1, 4.6.0
    • 4.6.0
    • tools

    Attachments

      Issue Links

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          Your comment above is correct that this is due to the fix missing in MB-21924. Also, the deleted document should be restored, but it should be in deleted state. Gets should fail, but get metas should succeed. Assigning to you since this is fixed and needs to be retested once you get the missing change.

          mikew Mike Wiederhold [X] (Inactive) added a comment - - edited Your comment above is correct that this is due to the fix missing in MB-21924 . Also, the deleted document should be restored, but it should be in deleted state. Gets should fail, but get metas should succeed. Assigning to you since this is fixed and needs to be retested once you get the missing change.
          arunkumar Arunkumar Senthilnathan added a comment - - edited

          Behavior in 4.5.1-2855:

          1. Configure backup in backup host
          2. Create a document in cluster host
          3. Take backup - compact the backup
          4. Delete the document in cluster host
          5. Take backup - compact the backup
          5. Merge the two compacted backups
          6. List the merged backup - it shows 0 docs as expected
          7. Use this merged backup to restore on the restore host - deleted document is restored on the restore host without any content - get failed on the document with a python sdk error but metadata could be retrieved using views - meta data had "att_reason":"invalid_json" field set

          Behavior in 4.5.1-2856:

          1. Configure backup in backup host
          2. Create a document in cluster host
          3. Take backup - compact the backup
          4. Delete the document in cluster host
          5. Take backup - compact the backup
          5. Merge the two compacted backups
          6. List the merged backup - it shows 0 docs as expected
          7. Use this merged backup to restore on the restore host - no documents restored on target - get failed on the document with key not found error and metadata could not be retrieved using views

          screenshots attached

          Mike Wiederhold [X] please confirm if the behaviour seen in 4.5.1-2856 is correct

          arunkumar Arunkumar Senthilnathan added a comment - - edited Behavior in 4.5.1-2855: 1. Configure backup in backup host 2. Create a document in cluster host 3. Take backup - compact the backup 4. Delete the document in cluster host 5. Take backup - compact the backup 5. Merge the two compacted backups 6. List the merged backup - it shows 0 docs as expected 7. Use this merged backup to restore on the restore host - deleted document is restored on the restore host without any content - get failed on the document with a python sdk error but metadata could be retrieved using views - meta data had "att_reason":"invalid_json" field set Behavior in 4.5.1-2856: 1. Configure backup in backup host 2. Create a document in cluster host 3. Take backup - compact the backup 4. Delete the document in cluster host 5. Take backup - compact the backup 5. Merge the two compacted backups 6. List the merged backup - it shows 0 docs as expected 7. Use this merged backup to restore on the restore host - no documents restored on target - get failed on the document with key not found error and metadata could not be retrieved using views screenshots attached Mike Wiederhold [X] please confirm if the behaviour seen in 4.5.1-2856 is correct

          The behavior on both of these builds is actually correct. Keep in mind that the problem was that the delete was getting dropped from the backup file during compaction. On 4.5.1-2855 you're seeing the deleted document restored with an empty body due to MB-21924. While the overall behavior is incorrect because of the bug we can see that this issue is fixed because it is clear that the delete was not dropped from the backup file.

          On 4.5.1-2856 you are seeing the correct overall behavior since MB-21924 is fixed on that build. The problem with you testing as I pointed out on the other issue is this comment:

          "get failed on the document with key not found error and metadata could not be retrieved using views"

          Verification of backups should always be done using get meta and not get because get meta will retrieve delete item tombstones and in general provides more information about the restored data since it includes the items metadata. I also don't trust views to be a source for verification of backups.

          I just re-ran your test case and am seeing the delete in the couchstore files.

          mikew Mike Wiederhold [X] (Inactive) added a comment - The behavior on both of these builds is actually correct. Keep in mind that the problem was that the delete was getting dropped from the backup file during compaction. On 4.5.1-2855 you're seeing the deleted document restored with an empty body due to MB-21924 . While the overall behavior is incorrect because of the bug we can see that this issue is fixed because it is clear that the delete was not dropped from the backup file. On 4.5.1-2856 you are seeing the correct overall behavior since MB-21924 is fixed on that build. The problem with you testing as I pointed out on the other issue is this comment: "get failed on the document with key not found error and metadata could not be retrieved using views" Verification of backups should always be done using get meta and not get because get meta will retrieve delete item tombstones and in general provides more information about the restored data since it includes the items metadata. I also don't trust views to be a source for verification of backups. I just re-ran your test case and am seeing the delete in the couchstore files.

          Verified in 4.5.1-2586 as per Mike's comment above:

          1. Configure backup in backup host
          2. Create a document in cluster host
          3. Take backup - compact the backup
          4. Delete the document in cluster host
          5. Take backup - compact the backup
          5. Merge the two compacted backups
          6. List the merged backup - it shows 0 docs as expected
          7. Use this merged backup to restore on the restore host - no documents restored on target - used get meta to retrieve metadata - source: (1, 0, 1482437881, 10, 1482437881631145984) target: (1, 0, 1482437983, 10, 1482437881631145984)

          closing this defect

          arunkumar Arunkumar Senthilnathan added a comment - Verified in 4.5.1-2586 as per Mike's comment above: 1. Configure backup in backup host 2. Create a document in cluster host 3. Take backup - compact the backup 4. Delete the document in cluster host 5. Take backup - compact the backup 5. Merge the two compacted backups 6. List the merged backup - it shows 0 docs as expected 7. Use this merged backup to restore on the restore host - no documents restored on target - used get meta to retrieve metadata - source: (1, 0, 1482437881, 10, 1482437881631145984) target: (1, 0, 1482437983, 10, 1482437881631145984) closing this defect
          ianmccloy Ian McCloy added a comment -

          Reopening - this is not fixed in a GA version of 4.5.1, fix version should be 4.6

          ianmccloy Ian McCloy added a comment - Reopening - this is not fixed in a GA version of 4.5.1, fix version should be 4.6

          People

            arunkumar Arunkumar Senthilnathan
            mikew Mike Wiederhold [X] (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              PagerDuty