Details
-
Bug
-
Status: Closed
-
Major
-
Resolution: Fixed
-
7.0.1
-
Centos7
7.0.1-5956-enterprise
-
Untriaged
-
1
-
Unknown
Description
To repro:
- create bucket1 and bucket2
- create scope/collections:
- create scope bucket1.scope1;
- create scope bucket2.scope2;
- create collection bucket1.scope1.collection1;
- create collection bucket2.scope2.collection2;
- create UDF:
- CREATE FUNCTION func1() { 0 };
- CREATE FUNCTION default:bucket1.scope1.func1() { 0 };
- CREATE FUNCTION default:bucket2.scope2.func1() { 0 };
- Check system cat:
- select f.identity.name, f.identity.type, f.identity.`scope` from system:functions as f;
- Backup:
- ./cbbackupmgr config -a /backup-1 -r my_backup
- ./cbbackupmgr backup -a /backup-1 -r my_backup -c http://localhost:8091 -u Administrator -p password
- Drop functions:
- drop function default:bucket1.scope1.func1;
- drop function default:bucket2.scope2.func1;
- Restore with exclude
- ./cbbackupmgr restore -a /backup-1 -r my_backup -c http://localhost:8091 -u Administrator -p password --exclude-data bucket2.scope2.collection2
- or
- ./cbbackupmgr restore -a /backup-1 -r my_backup -c http://localhost:8091 -u Administrator -p password --exclude-data bucket2.scope2
- Check system cat:
- select f.identity.name, f.identity.type, f.identity.`scope` from system:functions as f;
Post restore I still see both func1 for scope1 and scope2 present:
cbq> select f.identity.name, f.identity.type, f.identity.`scope` from system:functions as f;
|
{
|
"requestID": "f5a86000-9744-4410-ac02-881a9ecfacc4", |
"signature": { |
"name": "json", |
"scope": "json", |
"type": "json" |
},
|
"results": [ |
{
|
"name": "func1", |
"scope": "scope2", |
"type": "scope" |
},
|
{
|
"name": "func1", |
"scope": "scope1", |
"type": "scope" |
},
|
{
|
"name": "func1", |
"type": "global" |
}
|
Hi Pierre Regazzoni,
I don't believe this is an issue with 'cbbackupmgr', from what I can see (from my local reproduction) it's behaving as defined. By this I mean that we're 'POST'ing the Query UDF payload to the Query service with the query parameters 'exclude=scope' (please note that the filtering/remapping should be happening at the service level).
The following steps are those which I used to reproduce:
{ args[0] * 0.3048 }1) Spin up a single node cluster running KV + Index + N1QL
2) Load the 'travel-sample' bucket
3) Create a scope level function using e.g. '{{CREATE FUNCTION default:`travel-sample`.inventory.to_meters(...)
;}}'
4) Perform a backup
4a) Validate that we see the function definition in the backup bucket directory (should be in a 'query.json' file)
5) Delete the 'travel-sample' bucket
6) Perform a restore using the '--auto-create-buckets' and '--exclude-data travel-sample.inventory' flags
We should see a failure such as the one below:
500 Status Code
Error restoring cluster: internal server error executing 'POST' request to '/api/v1/bucket/travel-sample/backup': {"caller":"admin_accounting_endpoint:1208","code":1040,"icause":"Invalid function name default:travel-sample.inventory.to_meters - cause: Scope not found in CB datastore default:travel-sample.inventory","key":"service.io.request.bad_value","message":"Error processing UDF restore body"}
This is because Query is attempting to use a scope which 'cbbackupmgr' didn't bother creating (since it wasn't going to be used). In theory this is the same behavior observed by Pierre Regazzoni i.e. no filtering is taking place.
To validate that we're behaving as expected, I captured a PCAP of the HTTP traffic, in which we see the following requests:
HTTP Requests
1 0.000000 ::1 ::1 HTTP/JSON 559 39800 9499 POST /api/v1/bucket/travel-sample/backup?exclude=inventory HTTP/1.1 , JavaScript Object Notation (application/json)
2 0.000942 ::1 ::1 HTTP/JSON 559 39802 9499 POST /api/v1/bucket/travel-sample/backup?exclude=inventory HTTP/1.1 , JavaScript Object Notation (application/json)
3 0.001604 ::1 ::1 HTTP/JSON 559 39804 9499 POST /api/v1/bucket/travel-sample/backup?exclude=inventory HTTP/1.1 , JavaScript Object Notation (application/json)
We see three separate requests since each failed with a 500 status code (which is retried by our REST client). In each case, we can see we're correctly indicating that we're not transferring any data for the 'inventory' scope.
Removing 'tools' and assigning to 'query'.
Thanks,
James