Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-47573

[BAR] udf is not remaped to different scope using --map-data

    XMLWordPrintable

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 7.0.1
    • 7.0.2
    • query
    • Centos7
      7.0.1-5956-enterprise
    • Untriaged
    • 1
    • Unknown

    Description

      To repro:

      • create bucket1 and bucket2
      • create scope/collections:
        • create scope bucket1.scope1;
        • create scope bucket2.scope2;
        • create collection bucket1.scope1.collection1;
        • create collection bucket2.scope2.collection2;
      • create UDF:
        • CREATE FUNCTION func1() { 0 };
        • CREATE FUNCTION default:bucket1.scope1.func1() { 0 };
        • CREATE FUNCTION default:bucket2.scope2.func1() { 0 };
      • Check system cat:
        • select f.identity.name, f.identity.type, f.identity.`scope` from system:functions as f;
      • Backup:
        • ./cbbackupmgr config -a /backup-1 -r my_backup
        • ./cbbackupmgr backup -a /backup-1 -r my_backup -c http://localhost:8091 -u Administrator -p password
      • Drop functions:
        • drop function default:bucket1.scope1.func1;
        • drop function default:bucket2.scope2.func1;
      • Restore with remap
        • ./cbbackupmgr restore -a /backup-1 -r my_backup -c http://localhost:8091 -u Administrator -p password --map-data bucket1.scope1.collection1=bucket1.scope3.collection3
        • or
        • ./cbbackupmgr restore -a /backup-1 -r my_backup -c http://localhost:8091 -u Administrator -p password --map-data bucket1.scope1=bucket1.scope3
      • Check system cat:
        • select f.identity.name, f.identity.type, f.identity.`scope` from system:functions as f;

      Post restore I still see both func1 for scope1 and scope2 present and not scope3:

      [root@cluster bin]# ./cbq -u Administrator -p password
       Connected to : http://localhost:8091/. Type Ctrl-D or \QUIT to exit.
       
       
       Path to history file for the shell : /root/.cbq_history 
      cbq> select f.identity.name, f.identity.type, f.identity.`scope` from system:functions as f;
      {
          "requestID": "7ff668f3-d201-4680-b768-7da78faaf4ee",
          "signature": {
              "name": "json",
              "scope": "json",
              "type": "json"
          },
          "results": [
          {
              "name": "func1",
              "scope": "scope2",
              "type": "scope"
          },
          {
              "name": "func1",
              "scope": "scope1",
              "type": "scope"
          },
          {
              "name": "func1",
              "type": "global"
          }
          ], 

       

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          james.lee James Lee added a comment -

          Hi Pierre Regazzoni,

          Please see my comment on MB-47572, I believe the same applies here (I'm also using a very similar method to reproduce).

          Reproducing this locally whilst getting a PCAP, I see the following:

          5218	3.607886	::1	::1	HTTP/JSON	569	39820	9499	POST /api/v1/bucket/travel-sample/backup?remap=inventory%3Anew_scope HTTP/1.1 , JavaScript Object Notation (application/json)
          5274	3.623678	::1	::1	HTTP/JSON	569	39822	9499	POST /api/v1/bucket/travel-sample/backup?remap=inventory%3Anew_scope HTTP/1.1 , JavaScript Object Notation (application/json)
          5286	3.624813	::1	::1	HTTP/JSON	569	39824	9499	POST /api/v1/bucket/travel-sample/backup?remap=inventory%3Anew_scope HTTP/1.1 , JavaScript Object Notation (application/json)
          

          In each case, we can see that we're correctly informing Query that we're going to be remapping the 'inventory' scope to 'new_scope'. With this reproduction, we see that the Query service attempts to create a function attached to the scope 'inventory' which wasn't created by 'cbbackupmgr' (since it wasn't used).

          Please note that:
          1) The reason for the three attempts remains the same as in MB-47572
          2) Remapping should be taking place at the service level (as with including/excluding data)

          Removing 'tools' and assigning to 'query'.

          Thanks,
          James

          james.lee James Lee added a comment - Hi Pierre Regazzoni , Please see my comment on MB-47572 , I believe the same applies here (I'm also using a very similar method to reproduce). Reproducing this locally whilst getting a PCAP, I see the following: 5218 3.607886 ::1 ::1 HTTP/JSON 569 39820 9499 POST /api/v1/bucket/travel-sample/backup?remap=inventory%3Anew_scope HTTP/1.1 , JavaScript Object Notation (application/json) 5274 3.623678 ::1 ::1 HTTP/JSON 569 39822 9499 POST /api/v1/bucket/travel-sample/backup?remap=inventory%3Anew_scope HTTP/1.1 , JavaScript Object Notation (application/json) 5286 3.624813 ::1 ::1 HTTP/JSON 569 39824 9499 POST /api/v1/bucket/travel-sample/backup?remap=inventory%3Anew_scope HTTP/1.1 , JavaScript Object Notation (application/json) In each case, we can see that we're correctly informing Query that we're going to be remapping the ' inventory ' scope to ' new_scope '. With this reproduction, we see that the Query service attempts to create a function attached to the scope ' inventory ' which wasn't created by ' cbbackupmgr ' (since it wasn't used). Please note that: 1) The reason for the three attempts remains the same as in MB-47572 2) Remapping should be taking place at the service level (as with including/excluding data) Removing ' tools ' and assigning to ' query '. Thanks, James

          Build couchbase-server-7.0.1-5963 contains query commit 0f7f28a with commit message:
          MB-47572 MB-47573 exclude and remap work incorrectly

          build-team Couchbase Build Team added a comment - Build couchbase-server-7.0.1-5963 contains query commit 0f7f28a with commit message: MB-47572 MB-47573 exclude and remap work incorrectly

          Build couchbase-server-7.1.0-1098 contains query commit 3dbc3bd with commit message:
          MB-47572 MB-47573 exclude and remap work incorrectly

          build-team Couchbase Build Team added a comment - Build couchbase-server-7.1.0-1098 contains query commit 3dbc3bd with commit message: MB-47572 MB-47573 exclude and remap work incorrectly

          Verfied on 7.0.1-5963 remapping bucket.scope and bucket.

          Marco Greco similar question to exclude, I also did --map-data bucket1.scope1.collection1=bucket1.scope111.collection111 ... in this case:

          • scope1.collection function is restored as is
          • scope1.collection1 is mapped to new collection scope111.collection111 but without udf
          pierre.regazzoni Pierre Regazzoni added a comment - Verfied on 7.0.1-5963 remapping bucket.scope and bucket. Marco Greco  similar question to exclude, I also did --map-data bucket1.scope1.collection1=bucket1.scope111.collection111 ... in this case: scope1.collection function is restored as is scope1.collection1 is mapped to new collection scope111.collection111 but without udf
          marco.greco Marco Greco added a comment -

          Remapping applies to collections, not functions, so if you restore remapping scope1.collection1 to scope2.collection2, we don't remap the function, but if you restore remapping scope1 to scope2 then the function would be remapped.

          marco.greco Marco Greco added a comment - Remapping applies to collections, not functions, so if you restore remapping scope1.collection1 to scope2.collection2, we don't remap the function, but if you restore remapping scope1 to scope2 then the function would be remapped.

          People

            pierre.regazzoni Pierre Regazzoni
            pierre.regazzoni Pierre Regazzoni
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty