Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-8377

CBBackup - Need to ignore deleted items which are unecessary in backup

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Critical
    • Resolution: Duplicate
    • Affects Version/s: 2.0, 2.0.1, 2.1.0
    • Fix Version/s: 2.2.0
    • Component/s: tools
    • Security Level: Public
    • Labels:
      None
    • Sprint:
      PCI Team - Sprint 10, PCI Team - Sprint 11

      Description

      http://www.couchbase.com/issues/browse/MB-7149

      In case of Deletes on items – tool currently only captures the snapshot of 'active items' and doesn't consider any items getting deleted. Hence when it transfers it not only transfers current active items but also any deleted items which is unnecessary. To fix this we require some changes in EP-Engine side to provide stats on deleted items so that tool can smartly ignore those. Considering the timeframe for release this won't make it for 2.1.0 but we will have documentation explaining this to users.

      [Bala]:
      There are 2 issues here. One has been addressed in 2.1 (considering DGM in backup % calculation). But the other part (ignoring tombstones in backup) is not yet addressed. I see Anil's comment that it will not make to 2.1. But should the ticket still be kept open to avoid loosing visibility ?
      Also while ignoring tombstones in backup may be involved; how involved is just considering the # of tombstones in the % calculation ? Atleast we will not confuse the customers if we consider that in the denominator and completion displayed is never more than 100% (or max marginially above 100%).

        Issue Links

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

          Hide
          maria Maria McDuff (Inactive) added a comment -
          Show
          maria Maria McDuff (Inactive) added a comment - MB-9075 .
          Hide
          maria Maria McDuff (Inactive) added a comment -
          Show
          maria Maria McDuff (Inactive) added a comment - MB-9075 .
          Hide
          maria Maria McDuff (Inactive) added a comment -

          closing as dupe.

          Show
          maria Maria McDuff (Inactive) added a comment - closing as dupe.
          Hide
          bcui Bin Cui (Inactive) added a comment -

          That's the bug you filed as MB-9075. A fix is pushed for review. It should be part of 2.2.1 hot fix release.

          Show
          bcui Bin Cui (Inactive) added a comment - That's the bug you filed as MB-9075 . A fix is pushed for review. It should be part of 2.2.1 hot fix release.
          Hide
          shashank Shashank Gupta added a comment -

          I tried the following scenario:

          1. Loaded 199982 items in a bucket.
          2. Deleted some items. Items remaining : 163757

          3. Took backup :

          a) With build 2.1.1 : output :

          1. ./cbbackup http://10.3.3.66:8091 /tmp/backup/ -b default -u Administrator -p password

          ####################### 122.1% (199982/163757 msgs)
          bucket: default, msgs transferred...
          : total | last | per sec
          batch : 2712 | 2712 | 83.7
          byte : 204864653 | 204864653 | 6320891.2
          msg : 199982 | 199982 | 6170.2

          b) With build 2.2.0-821 : output :

          1. ./cbbackup http://10.3.3.66:8091 /tmp/backup/ -b default -u Administrator -p password

          ############################ 144.2% (236207/163757 msgs)
          bucket: default, msgs transferred...
          : total | last | per sec
          batch : 2729 | 2729 | 88.4
          byte : 204864653 | 204864653 | 6634171.2
          msg : 236207 | 236207 | 7649.1

          So the only difference found I found is that with 2.1.1, cbbackup ran upto 122.1% and then terminated successfully, but with 2.2.0, cbbackup ran upto 144.2% and then got terminated successfully. So, now also the user will not be able to predict that upto what extend the process will run.

          Show
          shashank Shashank Gupta added a comment - I tried the following scenario: 1. Loaded 199982 items in a bucket. 2. Deleted some items. Items remaining : 163757 3. Took backup : a) With build 2.1.1 : output : ./cbbackup http://10.3.3.66:8091 /tmp/backup/ -b default -u Administrator -p password ####################### 122.1% (199982/163757 msgs) bucket: default, msgs transferred... : total | last | per sec batch : 2712 | 2712 | 83.7 byte : 204864653 | 204864653 | 6320891.2 msg : 199982 | 199982 | 6170.2 b) With build 2.2.0-821 : output : ./cbbackup http://10.3.3.66:8091 /tmp/backup/ -b default -u Administrator -p password ############################ 144.2% (236207/163757 msgs) bucket: default, msgs transferred... : total | last | per sec batch : 2729 | 2729 | 88.4 byte : 204864653 | 204864653 | 6634171.2 msg : 236207 | 236207 | 7649.1 So the only difference found I found is that with 2.1.1, cbbackup ran upto 122.1% and then terminated successfully, but with 2.2.0, cbbackup ran upto 144.2% and then got terminated successfully. So, now also the user will not be able to predict that upto what extend the process will run.
          Show
          bcui Bin Cui (Inactive) added a comment - http://review.couchbase.org/#/c/27920/
          Hide
          perry Perry Krug added a comment -

          Agree definitely with the last sentence here...it's better to show too many items need to be transferred than to go over 100% where the user would have no idea or expectation of when it will stop.

          Show
          perry Perry Krug added a comment - Agree definitely with the last sentence here...it's better to show too many items need to be transferred than to go over 100% where the user would have no idea or expectation of when it will stop.

            People

            • Assignee:
              shashank Shashank Gupta
              Reporter:
              anil Anil Kumar
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Agile

                  Gerrit Reviews

                  There are no open Gerrit changes