Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-20213

cbrestore without Admin password silently fails to create GSI indexes

    XMLWordPrintable

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 4.1.0
    • 5.0.0
    • tools

    Description

      Using the beer-sample bucket with 6 indexes defined.
      If I take a full backup with cbbackup an index.json is created with the backup.

      If I drop all the indexes, delete the views then restore the backup.
      If I do not provide the administrator username/password the data and views get restored but the GSI indexes fail silently. If the administrator details are provided then the indexes get restored.

      w/o Admin authentication given, the views are recreated but no GSI indexes are created..
      No errors are shown to the user even in verbose mode.

       
      $ /opt/couchbase/bin/cbrestore /backups/2016-07-20T085549Z/2016-07-20T085549Z-full/ http://localhost:8091 --bucket-source=beer-sample -v
      2016-07-20 11:24:49,449: mt cbrestore...
      2016-07-20 11:24:49,449: mt  source : /backups/2016-07-20T085549Z/2016-07-20T085549Z-full/
      2016-07-20 11:24:49,449: mt  sink   : http://localhost:8091
      2016-07-20 11:24:49,449: mt  opts   : {'username': None, 'verbose': 1, 'extra': {'max_retry': 10.0, 'rehash': 0.0, 'dcp_consumer_queue_length': 1000.0, 'data_only': 0.0, 'uncompress': 0.0, 'nmv_retry': 1.0, 'cbb_max_mb': 100000.0, 'report': 5.0, 'mcd_compatible': 1.0, 'try_xwm': 1.0, 'backoff_cap': 10.0, 'batch_max_bytes': 400000.0, 'report_full': 2000.0, 'flow_control': 1.0, 'batch_max_size': 1000.0, 'seqno': 0.0, 'design_doc_only': 0.0, 'recv_min_bytes': 4096.0}, 'ssl': False, 'threads': 4, 'to_date': None, 'key': None, 'password': None, 'id': None, 'bucket_source': 'beer-sample', 'silent': False, 'dry_run': False, 'from_date': None, 'bucket_destination': None, 'add': False, 'vbucket_list': None}
      2016-07-20 11:24:49,468: mt bucket: beer-sample
      2016-07-20 11:24:50,008: w0   source : /backups/2016-07-20T085549Z/2016-07-20T085549Z-full/(beer-sample@10.111.110.101:8091)
      2016-07-20 11:24:50,008: w0   sink   : http://localhost:8091(beer-sample@10.111.110.101:8091)
      2016-07-20 11:24:50,008: w0          :                total |       last |    per sec
      2016-07-20 11:24:50,008: w0    batch :                    4 |          4 |        7.4
      2016-07-20 11:24:50,008: w0    byte  :              1299874 |    1299874 |  2413700.5
      2016-07-20 11:24:50,008: w0    msg   :                 3711 |       3711 |     6890.9
      2016-07-20 11:24:50,271: w1   source : /backups/2016-07-20T085549Z/2016-07-20T085549Z-full/(beer-sample@10.111.110.102:8091)
      2016-07-20 11:24:50,271: w1   sink   : http://localhost:8091(beer-sample@10.111.110.102:8091)
      2016-07-20 11:24:50,271: w1          :                total |       last |    per sec
      2016-07-20 11:24:50,271: w1    batch :                    4 |          4 |        6.6
      2016-07-20 11:24:50,271: w1    byte  :              1241675 |    1241675 |  2035686.7
      2016-07-20 11:24:50,271: w1    msg   :                 3592 |       3592 |     5889.0
        [####################] 100.0% (7303/estimated 7303 msgs)
      bucket: beer-sample, msgs transferred...
             :                total |       last |    per sec
       batch :                    8 |          8 |        9.6
       byte  :              2541549 |    2541549 |  3051624.5
       msg   :                 7303 |       7303 |     8768.7
      done
      
      

      With Admin details provided, the GSI indexes are created.

       
      $ /opt/couchbase/bin/cbrestore /backups/2016-07-20T085549Z/2016-07-20T085549Z-full/ http://Administrator:password@localhost:8091 --bucket-source=beer-sample -v
      2016-07-20 11:27:53,334: mt cbrestore...
      2016-07-20 11:27:53,334: mt  source : /backups/2016-07-20T085549Z/2016-07-20T085549Z-full/
      2016-07-20 11:27:53,338: mt  sink   : http://Administrator:password@localhost:8091
      2016-07-20 11:27:53,338: mt  opts   : {'username': None, 'verbose': 1, 'extra': {'max_retry': 10.0, 'rehash': 0.0, 'dcp_consumer_queue_length': 1000.0, 'data_only': 0.0, 'uncompress': 0.0, 'nmv_retry': 1.0, 'cbb_max_mb': 100000.0, 'report': 5.0, 'mcd_compatible': 1.0, 'try_xwm': 1.0, 'backoff_cap': 10.0, 'batch_max_bytes': 400000.0, 'report_full': 2000.0, 'flow_control': 1.0, 'batch_max_size': 1000.0, 'seqno': 0.0, 'design_doc_only': 0.0, 'recv_min_bytes': 4096.0}, 'ssl': False, 'threads': 4, 'to_date': None, 'key': None, 'password': None, 'id': None, 'bucket_source': 'beer-sample', 'silent': False, 'dry_run': False, 'from_date': None, 'bucket_destination': None, 'add': False, 'vbucket_list': None}
      2016-07-20 11:27:53,381: mt bucket: beer-sample
      2016-07-20 11:27:53,966: w0   source : /backups/2016-07-20T085549Z/2016-07-20T085549Z-full/(beer-sample@10.111.110.101:8091)
      2016-07-20 11:27:53,966: w0   sink   : http://Administrator:password@localhost:8091(beer-sample@10.111.110.101:8091)
      2016-07-20 11:27:53,966: w0          :                total |       last |    per sec
      2016-07-20 11:27:53,966: w0    batch :                    4 |          4 |        6.9
      2016-07-20 11:27:53,966: w0    byte  :              1299874 |    1299874 |  2230677.9
      2016-07-20 11:27:53,966: w0    msg   :                 3711 |       3711 |     6368.3
      2016-07-20 11:27:54,283: w1   source : /backups/2016-07-20T085549Z/2016-07-20T085549Z-full/(beer-sample@10.111.110.102:8091)
      2016-07-20 11:27:54,283: w1   sink   : http://Administrator:password@localhost:8091(beer-sample@10.111.110.102:8091)
      2016-07-20 11:27:54,284: w1          :                total |       last |    per sec
      2016-07-20 11:27:54,284: w1    batch :                    4 |          4 |        6.0
      2016-07-20 11:27:54,284: w1    byte  :              1241675 |    1241675 |  1848058.0
      2016-07-20 11:27:54,284: w1    msg   :                 3592 |       3592 |     5346.2
        [####################] 100.0% (7303/estimated 7303 msgs)
      bucket: beer-sample, msgs transferred...
             :                total |       last |    per sec
       batch :                    8 |          8 |        8.0
       byte  :              2541549 |    2541549 |  2527275.2
       msg   :                 7303 |       7303 |     7262.0
      done
      

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          thuan Thuan Nguyen added a comment -

          Verified on build 5.0.0-3450. I could not reproduce this issue.
          Bucket in Spock needs authentication with correct role to do restore.

          [root@s44015 ~]# /opt/couchbase/bin/cbrestore /tmp/backup/ http://172.23.121.224:8091 -b beer-sample -B beer-sample -u nobucket -p password 
          2017-08-03 14:25:20,287: s0 error: CBSink.connect() for send: error: Bucket select failed: 172.23.121.224:11210 beer-sample, Memcached error #36
          2017-08-03 14:25:20,288: s0 error: async operation: error: Bucket select failed: 172.23.121.224:11210 beer-sample, Memcached error #36 on sink: http://172.23.121.224:8091(beer-sample@172.23.121.224:8091)
          2017-08-03 14:25:20,332: s1 error: CBSink.connect() for send: error: Bucket select failed: 172.23.121.225:11210 beer-sample, Memcached error #36
          2017-08-03 14:25:20,333: s1 error: async operation: error: Bucket select failed: 172.23.121.225:11210 beer-sample, Memcached error #36 on sink: http://172.23.121.224:8091(beer-sample@172.23.121.225:8091)
          error: Bucket select failed: 172.23.121.224:11210 beer-sample, Memcached error #36
          [root@s44015 ~]# 
          [root@s44015 ~]# 
          [root@s44015 ~]# /opt/couchbase/bin/cbrestore /tmp/backup/ http://172.23.121.224:8091 -b beer-sample -B beer-sample -u nobucket -p password 
          2017-08-03 14:25:55,546: s0 error: async operation: Authorization failure: can't execute SET_WITH_META operation without the MetaWrite privilege on sink: http://172.23.121.224:8091(beer-sample@172.23.121.224:8091)
          2017-08-03 14:25:55,678: s1 error: async operation: Authorization failure: can't execute SET_WITH_META operation without the MetaWrite privilege on sink: http://172.23.121.224:8091(beer-sample@172.23.121.225:8091)
          Authorization failure: can't execute SET_WITH_META operation without the MetaWrite privilege
          [root@s44015 ~]# 
          [root@s44015 ~]# 
          [root@s44015 ~]# /opt/couchbase/bin/cbrestore /tmp/backup/ http://172.23.121.224:8091 -b beer-sample -B beer-sample -u nobucket -p password 
            [####################] 100.0% (7303/estimated 7303 msgs)
          bucket: beer-sample, msgs transferred...
                 :                total |       last |    per sec
           byte  :              2542924 |    2542924 |  1935186.6
          done
          [root@s44015 ~]# 
          

          thuan Thuan Nguyen added a comment - Verified on build 5.0.0-3450. I could not reproduce this issue. Bucket in Spock needs authentication with correct role to do restore. [root@s44015 ~]# /opt/couchbase/bin/cbrestore /tmp/backup/ http://172.23.121.224:8091 -b beer-sample -B beer-sample -u nobucket -p password 2017-08-03 14:25:20,287: s0 error: CBSink.connect() for send: error: Bucket select failed: 172.23.121.224:11210 beer-sample, Memcached error #36 2017-08-03 14:25:20,288: s0 error: async operation: error: Bucket select failed: 172.23.121.224:11210 beer-sample, Memcached error #36 on sink: http://172.23.121.224:8091(beer-sample@172.23.121.224:8091) 2017-08-03 14:25:20,332: s1 error: CBSink.connect() for send: error: Bucket select failed: 172.23.121.225:11210 beer-sample, Memcached error #36 2017-08-03 14:25:20,333: s1 error: async operation: error: Bucket select failed: 172.23.121.225:11210 beer-sample, Memcached error #36 on sink: http://172.23.121.224:8091(beer-sample@172.23.121.225:8091) error: Bucket select failed: 172.23.121.224:11210 beer-sample, Memcached error #36 [root@s44015 ~]# [root@s44015 ~]# [root@s44015 ~]# /opt/couchbase/bin/cbrestore /tmp/backup/ http://172.23.121.224:8091 -b beer-sample -B beer-sample -u nobucket -p password 2017-08-03 14:25:55,546: s0 error: async operation: Authorization failure: can't execute SET_WITH_META operation without the MetaWrite privilege on sink: http://172.23.121.224:8091(beer-sample@172.23.121.224:8091) 2017-08-03 14:25:55,678: s1 error: async operation: Authorization failure: can't execute SET_WITH_META operation without the MetaWrite privilege on sink: http://172.23.121.224:8091(beer-sample@172.23.121.225:8091) Authorization failure: can't execute SET_WITH_META operation without the MetaWrite privilege [root@s44015 ~]# [root@s44015 ~]# [root@s44015 ~]# /opt/couchbase/bin/cbrestore /tmp/backup/ http://172.23.121.224:8091 -b beer-sample -B beer-sample -u nobucket -p password [####################] 100.0% (7303/estimated 7303 msgs) bucket: beer-sample, msgs transferred... : total | last | per sec byte : 2542924 | 2542924 | 1935186.6 done [root@s44015 ~]#

          Thuan Nguyen please verify and close this for spock

          arunkumar Arunkumar Senthilnathan (Inactive) added a comment - Thuan Nguyen please verify and close this for spock

          Build 5.0.0-2735 contains couchbase-cli commit e88fb70fc04d652600dd99d54160dbe19ad66791 with commit message:
          MB-20213: Require a username and password when backing up and restoring
          https://github.com/couchbase/couchbase-cli/commit/e88fb70fc04d652600dd99d54160dbe19ad66791

          build-team Couchbase Build Team added a comment - Build 5.0.0-2735 contains couchbase-cli commit e88fb70fc04d652600dd99d54160dbe19ad66791 with commit message: MB-20213 : Require a username and password when backing up and restoring https://github.com/couchbase/couchbase-cli/commit/e88fb70fc04d652600dd99d54160dbe19ad66791

          I'll provide an error message in the logs, but I don't want to error out the backup since we didn't do this in the past for cbrestore. Also, usernames and passwords will be required in Spock.

          mikew Mike Wiederhold [X] (Inactive) added a comment - I'll provide an error message in the logs, but I don't want to error out the backup since we didn't do this in the past for cbrestore. Also, usernames and passwords will be required in Spock.

          People

            thuan Thuan Nguyen
            ianmccloy Ian McCloy
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty