James Lee added a comment - 26/Jul/21 2:00 AM Hi Pierre Regazzoni , I don't believe this is an issue with ' cbbackupmgr ', from what I can see (from my local reproduction) it's behaving as defined. By this I mean that we're ' POST 'ing the Query UDF payload to the Query service with the query parameters ' exclude=scope ' (please note that the filtering/remapping should be happening at the service level). The following steps are those which I used to reproduce: 1) Spin up a single node cluster running KV + Index + N1QL 2) Load the ' travel-sample ' bucket 3) Create a scope level function using e.g. '{{CREATE FUNCTION default:`travel-sample`.inventory.to_meters(...) { args[0] * 0.3048 } ;}}' 4) Perform a backup 4a) Validate that we see the function definition in the backup bucket directory (should be in a ' query.json ' file) 5) Delete the ' travel-sample ' bucket 6) Perform a restore using the ' --auto-create-buckets ' and ' --exclude-data travel-sample.inventory ' flags We should see a failure such as the one below: 500 Status Code Error restoring cluster: internal server error executing 'POST' request to '/api/v1/bucket/travel-sample/backup': {"caller":"admin_accounting_endpoint:1208","code":1040,"icause":"Invalid function name default:travel-sample.inventory.to_meters - cause: Scope not found in CB datastore default:travel-sample.inventory","key":"service.io.request.bad_value","message":"Error processing UDF restore body"} This is because Query is attempting to use a scope which ' cbbackupmgr ' didn't bother creating (since it wasn't going to be used). In theory this is the same behavior observed by Pierre Regazzoni i.e. no filtering is taking place. To validate that we're behaving as expected, I captured a PCAP of the HTTP traffic, in which we see the following requests: HTTP Requests 1 0.000000 ::1 ::1 HTTP/JSON 559 39800 9499 POST /api/v1/bucket/travel-sample/backup?exclude=inventory HTTP/1.1 , JavaScript Object Notation (application/json) 2 0.000942 ::1 ::1 HTTP/JSON 559 39802 9499 POST /api/v1/bucket/travel-sample/backup?exclude=inventory HTTP/1.1 , JavaScript Object Notation (application/json) 3 0.001604 ::1 ::1 HTTP/JSON 559 39804 9499 POST /api/v1/bucket/travel-sample/backup?exclude=inventory HTTP/1.1 , JavaScript Object Notation (application/json) We see three separate requests since each failed with a 500 status code (which is retried by our REST client). In each case, we can see we're correctly indicating that we're not transferring any data for the ' inventory ' scope. Removing ' tools ' and assigning to ' query '. Thanks, James

Couchbase Build Team added a comment - 27/Jul/21 11:20 AM Build couchbase-server-7.0.1-5963 contains query commit 0f7f28a with commit message: MB-47572 MB-47573 exclude and remap work incorrectly

Couchbase Build Team added a comment - 27/Jul/21 9:16 PM Build couchbase-server-7.1.0-1098 contains query commit 3dbc3bd with commit message: MB-47572 MB-47573 exclude and remap work incorrectly

Pierre Regazzoni added a comment - 28/Jul/21 12:55 AM Verified with exclude bucket.scope or exclude bucket seems to work ... Marco Greco  can you also confirm that when using  --exclude-data bucket2.scope2.collection2 it should have not effect on scope2 UDFs. # ./cbbackupmgr restore -a /backup- 1 -r my_backup -c http: //localhost:8091 -u Administrator -p password --exclude-data bucket2.scope2 Restoring backup '2021-07-28T00_25_28.520646058-07_00' Copied all data in 1 .187s (Avg. 84 .00KiB/Sec)                                                                                             0 items / 84 .00KiB [==================================================================================================================================================] 100.00 %     | Transfer | -------- | Status    | Avg Transfer Rate | Started At                        | Finished At                     | Duration | | Succeeded | 84 .00KiB          | Wed, 28 Jul 2021 00 : 31 : 19 - 0700 /s | Wed, 28 Jul 2021 00 : 31 : 20 - 0700 | 1 .238s   |     | Bucket | ------ | Name    | Status    | Transferred | Avg Transfer Rate | Started At                      | Finished At                     | Duration | | default | Succeeded | 28 .00KiB    | 28 .00KiB/s        | Wed, 28 Jul 2021 00 : 31 : 19 - 0700 | Wed, 28 Jul 2021 00 : 31 : 19 - 0700 | 53ms     | | | Mutations                    | Deletions                    | Expirations                  | | ---------                    | ---------                    | -----------                  | | Received | Errored | Skipped | Received | Errored | Skipped | Received | Errored | Skipped | | 0         | 0       | 0       | 0         | 0       | 0       | 0         | 0       | 0       |     | Bucket | ------ | Name    | Status    | Transferred | Avg Transfer Rate | Started At                      | Finished At                     | Duration | | bucket1 | Succeeded | 28 .00KiB    | 28 .00KiB/s        | Wed, 28 Jul 2021 00 : 31 : 20 - 0700 | Wed, 28 Jul 2021 00 : 31 : 20 - 0700 | 51ms     | | | Mutations                    | Deletions                    | Expirations                  | | ---------                    | ---------                    | -----------                  | | Received | Errored | Skipped | Received | Errored | Skipped | Received | Errored | Skipped | | 0         | 0       | 0       | 0         | 0       | 0       | 0         | 0       | 0       |     | Bucket | ------ | Name    | Status    | Transferred | Avg Transfer Rate | Started At                      | Finished At                     | Duration | | bucket2 | Succeeded | 28 .00KiB    | 28 .00KiB/s        | Wed, 28 Jul 2021 00 : 31 : 20 - 0700 | Wed, 28 Jul 2021 00 : 31 : 20 - 0700 | 51ms     | | | Mutations                    | Deletions                    | Expirations                  | | ---------                    | ---------                    | -----------                  | | Received | Errored | Skipped | Received | Errored | Skipped | Received | Errored | Skipped | | 0         | 0       | 0       | 0         | 0       | 0       | 0         | 0       | 0       |     Restore completed successfully [root @cluster bin]# ./cbq -u Administrator -p password  Connected to : http: //localhost:8091/. Type Ctrl-D or \QUIT to exit.      Path to history file for the shell : /root/.cbq_history  cbq> select f.identity.name, f.identity.type, f.identity.`scope`, definition.`#language` from system:functions as f; {     "requestID" : "db4d2457-f2b1-40cd-8fa4-cde09db9c078" ,     "signature" : {         "#language" : "json" ,         "name" : "json" ,         "scope" : "json" ,         "type" : "json"     },     "results" : [     {         "#language" : "inline" ,         "name" : "func1" ,         "type" : "global"     },     {         "#language" : "inline" ,         "name" : "func1" ,         "scope" : "scope1" ,         "type" : "scope"     }     ],     "status" : "success" ,     "metrics" : {         "elapsedTime" : "10.888534ms" ,         "executionTime" : "10.663115ms" ,         "resultCount" : 2 ,         "resultSize" : 202 ,         "serviceLoad" : 6     } }

Marco Greco added a comment - 28/Jul/21 1:25 AM yes, excluding a collection does not affect the functions in the scope.

Pierre Regazzoni added a comment - 28/Jul/21 1:48 AM Verified on 7.0.1-5963