Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-20213

cbrestore without Admin password silently fails to create GSI indexes

    XMLWordPrintable

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 4.1.0
    • 5.0.0
    • tools

    Description

      Using the beer-sample bucket with 6 indexes defined.
      If I take a full backup with cbbackup an index.json is created with the backup.

      If I drop all the indexes, delete the views then restore the backup.
      If I do not provide the administrator username/password the data and views get restored but the GSI indexes fail silently. If the administrator details are provided then the indexes get restored.

      w/o Admin authentication given, the views are recreated but no GSI indexes are created..
      No errors are shown to the user even in verbose mode.

       
      $ /opt/couchbase/bin/cbrestore /backups/2016-07-20T085549Z/2016-07-20T085549Z-full/ http://localhost:8091 --bucket-source=beer-sample -v
      2016-07-20 11:24:49,449: mt cbrestore...
      2016-07-20 11:24:49,449: mt  source : /backups/2016-07-20T085549Z/2016-07-20T085549Z-full/
      2016-07-20 11:24:49,449: mt  sink   : http://localhost:8091
      2016-07-20 11:24:49,449: mt  opts   : {'username': None, 'verbose': 1, 'extra': {'max_retry': 10.0, 'rehash': 0.0, 'dcp_consumer_queue_length': 1000.0, 'data_only': 0.0, 'uncompress': 0.0, 'nmv_retry': 1.0, 'cbb_max_mb': 100000.0, 'report': 5.0, 'mcd_compatible': 1.0, 'try_xwm': 1.0, 'backoff_cap': 10.0, 'batch_max_bytes': 400000.0, 'report_full': 2000.0, 'flow_control': 1.0, 'batch_max_size': 1000.0, 'seqno': 0.0, 'design_doc_only': 0.0, 'recv_min_bytes': 4096.0}, 'ssl': False, 'threads': 4, 'to_date': None, 'key': None, 'password': None, 'id': None, 'bucket_source': 'beer-sample', 'silent': False, 'dry_run': False, 'from_date': None, 'bucket_destination': None, 'add': False, 'vbucket_list': None}
      2016-07-20 11:24:49,468: mt bucket: beer-sample
      2016-07-20 11:24:50,008: w0   source : /backups/2016-07-20T085549Z/2016-07-20T085549Z-full/(beer-sample@10.111.110.101:8091)
      2016-07-20 11:24:50,008: w0   sink   : http://localhost:8091(beer-sample@10.111.110.101:8091)
      2016-07-20 11:24:50,008: w0          :                total |       last |    per sec
      2016-07-20 11:24:50,008: w0    batch :                    4 |          4 |        7.4
      2016-07-20 11:24:50,008: w0    byte  :              1299874 |    1299874 |  2413700.5
      2016-07-20 11:24:50,008: w0    msg   :                 3711 |       3711 |     6890.9
      2016-07-20 11:24:50,271: w1   source : /backups/2016-07-20T085549Z/2016-07-20T085549Z-full/(beer-sample@10.111.110.102:8091)
      2016-07-20 11:24:50,271: w1   sink   : http://localhost:8091(beer-sample@10.111.110.102:8091)
      2016-07-20 11:24:50,271: w1          :                total |       last |    per sec
      2016-07-20 11:24:50,271: w1    batch :                    4 |          4 |        6.6
      2016-07-20 11:24:50,271: w1    byte  :              1241675 |    1241675 |  2035686.7
      2016-07-20 11:24:50,271: w1    msg   :                 3592 |       3592 |     5889.0
        [####################] 100.0% (7303/estimated 7303 msgs)
      bucket: beer-sample, msgs transferred...
             :                total |       last |    per sec
       batch :                    8 |          8 |        9.6
       byte  :              2541549 |    2541549 |  3051624.5
       msg   :                 7303 |       7303 |     8768.7
      done
      
      

      With Admin details provided, the GSI indexes are created.

       
      $ /opt/couchbase/bin/cbrestore /backups/2016-07-20T085549Z/2016-07-20T085549Z-full/ http://Administrator:password@localhost:8091 --bucket-source=beer-sample -v
      2016-07-20 11:27:53,334: mt cbrestore...
      2016-07-20 11:27:53,334: mt  source : /backups/2016-07-20T085549Z/2016-07-20T085549Z-full/
      2016-07-20 11:27:53,338: mt  sink   : http://Administrator:password@localhost:8091
      2016-07-20 11:27:53,338: mt  opts   : {'username': None, 'verbose': 1, 'extra': {'max_retry': 10.0, 'rehash': 0.0, 'dcp_consumer_queue_length': 1000.0, 'data_only': 0.0, 'uncompress': 0.0, 'nmv_retry': 1.0, 'cbb_max_mb': 100000.0, 'report': 5.0, 'mcd_compatible': 1.0, 'try_xwm': 1.0, 'backoff_cap': 10.0, 'batch_max_bytes': 400000.0, 'report_full': 2000.0, 'flow_control': 1.0, 'batch_max_size': 1000.0, 'seqno': 0.0, 'design_doc_only': 0.0, 'recv_min_bytes': 4096.0}, 'ssl': False, 'threads': 4, 'to_date': None, 'key': None, 'password': None, 'id': None, 'bucket_source': 'beer-sample', 'silent': False, 'dry_run': False, 'from_date': None, 'bucket_destination': None, 'add': False, 'vbucket_list': None}
      2016-07-20 11:27:53,381: mt bucket: beer-sample
      2016-07-20 11:27:53,966: w0   source : /backups/2016-07-20T085549Z/2016-07-20T085549Z-full/(beer-sample@10.111.110.101:8091)
      2016-07-20 11:27:53,966: w0   sink   : http://Administrator:password@localhost:8091(beer-sample@10.111.110.101:8091)
      2016-07-20 11:27:53,966: w0          :                total |       last |    per sec
      2016-07-20 11:27:53,966: w0    batch :                    4 |          4 |        6.9
      2016-07-20 11:27:53,966: w0    byte  :              1299874 |    1299874 |  2230677.9
      2016-07-20 11:27:53,966: w0    msg   :                 3711 |       3711 |     6368.3
      2016-07-20 11:27:54,283: w1   source : /backups/2016-07-20T085549Z/2016-07-20T085549Z-full/(beer-sample@10.111.110.102:8091)
      2016-07-20 11:27:54,283: w1   sink   : http://Administrator:password@localhost:8091(beer-sample@10.111.110.102:8091)
      2016-07-20 11:27:54,284: w1          :                total |       last |    per sec
      2016-07-20 11:27:54,284: w1    batch :                    4 |          4 |        6.0
      2016-07-20 11:27:54,284: w1    byte  :              1241675 |    1241675 |  1848058.0
      2016-07-20 11:27:54,284: w1    msg   :                 3592 |       3592 |     5346.2
        [####################] 100.0% (7303/estimated 7303 msgs)
      bucket: beer-sample, msgs transferred...
             :                total |       last |    per sec
       batch :                    8 |          8 |        8.0
       byte  :              2541549 |    2541549 |  2527275.2
       msg   :                 7303 |       7303 |     7262.0
      done
      

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          I'll provide an error message in the logs, but I don't want to error out the backup since we didn't do this in the past for cbrestore. Also, usernames and passwords will be required in Spock.

          mikew Mike Wiederhold [X] (Inactive) added a comment - I'll provide an error message in the logs, but I don't want to error out the backup since we didn't do this in the past for cbrestore. Also, usernames and passwords will be required in Spock.

          Build 5.0.0-2735 contains couchbase-cli commit e88fb70fc04d652600dd99d54160dbe19ad66791 with commit message:
          MB-20213: Require a username and password when backing up and restoring
          https://github.com/couchbase/couchbase-cli/commit/e88fb70fc04d652600dd99d54160dbe19ad66791

          build-team Couchbase Build Team added a comment - Build 5.0.0-2735 contains couchbase-cli commit e88fb70fc04d652600dd99d54160dbe19ad66791 with commit message: MB-20213 : Require a username and password when backing up and restoring https://github.com/couchbase/couchbase-cli/commit/e88fb70fc04d652600dd99d54160dbe19ad66791

          Thuan Nguyen please verify and close this for spock

          arunkumar Arunkumar Senthilnathan (Inactive) added a comment - Thuan Nguyen please verify and close this for spock
          thuan Thuan Nguyen added a comment -

          Verified on build 5.0.0-3450. I could not reproduce this issue.
          Bucket in Spock needs authentication with correct role to do restore.

          [root@s44015 ~]# /opt/couchbase/bin/cbrestore /tmp/backup/ http://172.23.121.224:8091 -b beer-sample -B beer-sample -u nobucket -p password 
          2017-08-03 14:25:20,287: s0 error: CBSink.connect() for send: error: Bucket select failed: 172.23.121.224:11210 beer-sample, Memcached error #36
          2017-08-03 14:25:20,288: s0 error: async operation: error: Bucket select failed: 172.23.121.224:11210 beer-sample, Memcached error #36 on sink: http://172.23.121.224:8091(beer-sample@172.23.121.224:8091)
          2017-08-03 14:25:20,332: s1 error: CBSink.connect() for send: error: Bucket select failed: 172.23.121.225:11210 beer-sample, Memcached error #36
          2017-08-03 14:25:20,333: s1 error: async operation: error: Bucket select failed: 172.23.121.225:11210 beer-sample, Memcached error #36 on sink: http://172.23.121.224:8091(beer-sample@172.23.121.225:8091)
          error: Bucket select failed: 172.23.121.224:11210 beer-sample, Memcached error #36
          [root@s44015 ~]# 
          [root@s44015 ~]# 
          [root@s44015 ~]# /opt/couchbase/bin/cbrestore /tmp/backup/ http://172.23.121.224:8091 -b beer-sample -B beer-sample -u nobucket -p password 
          2017-08-03 14:25:55,546: s0 error: async operation: Authorization failure: can't execute SET_WITH_META operation without the MetaWrite privilege on sink: http://172.23.121.224:8091(beer-sample@172.23.121.224:8091)
          2017-08-03 14:25:55,678: s1 error: async operation: Authorization failure: can't execute SET_WITH_META operation without the MetaWrite privilege on sink: http://172.23.121.224:8091(beer-sample@172.23.121.225:8091)
          Authorization failure: can't execute SET_WITH_META operation without the MetaWrite privilege
          [root@s44015 ~]# 
          [root@s44015 ~]# 
          [root@s44015 ~]# /opt/couchbase/bin/cbrestore /tmp/backup/ http://172.23.121.224:8091 -b beer-sample -B beer-sample -u nobucket -p password 
            [####################] 100.0% (7303/estimated 7303 msgs)
          bucket: beer-sample, msgs transferred...
                 :                total |       last |    per sec
           byte  :              2542924 |    2542924 |  1935186.6
          done
          [root@s44015 ~]# 
          

          thuan Thuan Nguyen added a comment - Verified on build 5.0.0-3450. I could not reproduce this issue. Bucket in Spock needs authentication with correct role to do restore. [root@s44015 ~]# /opt/couchbase/bin/cbrestore /tmp/backup/ http://172.23.121.224:8091 -b beer-sample -B beer-sample -u nobucket -p password 2017-08-03 14:25:20,287: s0 error: CBSink.connect() for send: error: Bucket select failed: 172.23.121.224:11210 beer-sample, Memcached error #36 2017-08-03 14:25:20,288: s0 error: async operation: error: Bucket select failed: 172.23.121.224:11210 beer-sample, Memcached error #36 on sink: http://172.23.121.224:8091(beer-sample@172.23.121.224:8091) 2017-08-03 14:25:20,332: s1 error: CBSink.connect() for send: error: Bucket select failed: 172.23.121.225:11210 beer-sample, Memcached error #36 2017-08-03 14:25:20,333: s1 error: async operation: error: Bucket select failed: 172.23.121.225:11210 beer-sample, Memcached error #36 on sink: http://172.23.121.224:8091(beer-sample@172.23.121.225:8091) error: Bucket select failed: 172.23.121.224:11210 beer-sample, Memcached error #36 [root@s44015 ~]# [root@s44015 ~]# [root@s44015 ~]# /opt/couchbase/bin/cbrestore /tmp/backup/ http://172.23.121.224:8091 -b beer-sample -B beer-sample -u nobucket -p password 2017-08-03 14:25:55,546: s0 error: async operation: Authorization failure: can't execute SET_WITH_META operation without the MetaWrite privilege on sink: http://172.23.121.224:8091(beer-sample@172.23.121.224:8091) 2017-08-03 14:25:55,678: s1 error: async operation: Authorization failure: can't execute SET_WITH_META operation without the MetaWrite privilege on sink: http://172.23.121.224:8091(beer-sample@172.23.121.225:8091) Authorization failure: can't execute SET_WITH_META operation without the MetaWrite privilege [root@s44015 ~]# [root@s44015 ~]# [root@s44015 ~]# /opt/couchbase/bin/cbrestore /tmp/backup/ http://172.23.121.224:8091 -b beer-sample -B beer-sample -u nobucket -p password [####################] 100.0% (7303/estimated 7303 msgs) bucket: beer-sample, msgs transferred... : total | last | per sec byte : 2542924 | 2542924 | 1935186.6 done [root@s44015 ~]#

          People

            thuan Thuan Nguyen
            ianmccloy Ian McCloy
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty