Details
-
Bug
-
Status: Closed
-
Critical
-
Resolution: Fixed
-
5.5.0
-
Initial Version: 4.6.4-4590
Upgrade Version: 5.5.0-2907
Storage Mode: Memory Optimized
Cluster Config:
Node 1: kv (101)
Node 2: kv (102)
Node 3: Index, n1ql (103)
Node 4: Index (104)
-
Untriaged
-
Unknown
Description
[Please note this error occurs in two scenarios. First is in description. Second is mentioned in the comment]
- Create and configure cluster, Create and load buckets, Create indexes
- Failover cluster nodes one by one and upgrade and add back. Create equivalent indexes if failed over node is indexer to continue queries
- After adding back the node and rebalancing them in, remove the equivalent indexes created.
- Start the int 64 queries.
Failed Query:
SELECT sum(long_num) as long_num, name FROM default use index (int_64_936834_long_num_name) where long_num > 2147483600 group by name
|
Error Message:
{'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==\n'} error: 500 reason: unknown {
|
|
|
"requestID": "fd7307b6-261b-4baf-b188-b278889b946f",
|
|
|
"signature": {"long_num":"number","name":"json"},
|
|
|
"results": [
|
|
|
],
|
|
|
"errors": [{"code":5000,"msg":" strconv.ParseInt: parsing \"9223372036854776000\": value out of range from [10.111.171.103:9101] - cause: strconv.ParseInt: parsing \"9223372036854776000\": value out of range from [10.111.171.103:9101]"}],
|
|
|
"status": "errors",
|
|
|
"metrics": {"elapsedTime": "33.185783ms","executionTime": "33.134507ms","resultCount": 0,"resultSize": 0,"errorCount": 1}
|
|
|
} auth: Administrator:password
|
Similar query passes on various other tests like test_offline_upgrade (http://qa.sc.couchbase.com/job/cen7-2i-plasma-set5-job1-upgrade-4-6-4-int64/51/consoleFull)
Logs:
https://s3.amazonaws.com/bugdb/jira/failover_upgrade/collectinfo-2018-06-21T094601-ns_1%4010.111.171.101.zip
https://s3.amazonaws.com/bugdb/jira/failover_upgrade/collectinfo-2018-06-21T094601-ns_1%4010.111.171.102.zip
https://s3.amazonaws.com/bugdb/jira/failover_upgrade/collectinfo-2018-06-21T094601-ns_1%4010.111.171.103.zip
https://s3.amazonaws.com/bugdb/jira/failover_upgrade/collectinfo-2018-06-21T094601-ns_1%4010.111.171.104.zip
Attachments
Issue Links
- is a backport of
-
-
- Closed
-
Activity
Validation steps for the backport fix:
1. 4.6 cluster: 2 KV and 2 index
All types of data, with all types of index: simple, composite, array index
2. Upgrade to 5.0
Insert/Update mutations. Observe trailing zero bug
Existing indexed items remain as they are.
Array index insert / updates
3. Upgrade to 5.1.1
Insert/Update mutations. New mutations should go through FixEncode path.
Array index insert / updates
New mutations with > MaxInt64 get skipped in mixed mode ( Bug )
New mutations with > MaxInt64 get indexed after full upgrade
4. Upgrade to 5.5.3
Insert/Update mutations - Everything should work fine
Array index insert / updates
New mutations with > MaxInt64 get indexed in mixed mode as well as after full upgrade
Verification at each step:
ScanAll/Range scans
Lookup query for each value
Build couchbase-server-5.5.3-4015 contains indexing commit e083223 with commit message:
MB-31745: Fix SUM aggregate query for large integers
Tested on 5.5.3-4016 with the following steps
1. Setup a 5.0 cluster. 4 nodes - kv+query, kv+query, index, index
2. Created a bucket with documents of integer types of various sizes and formats as mentioned in the test data set. None beyond int64 boundaries yet.
3. Observe trailing zero bug
4. Upgraded the indexer nodes to 5.1.1
5. New mutations with integers with large values (int64 limits) were indexed properly and no precision loss was seen. Range scans and lookup queries performed. Tried with array indexes as well.
6. Tried adding documents with value > int64 max values, they were skipped. So issue was reproduced reliably.
7. Upgraded the indexer nodes to 5.5.3
8. New mutations with integers with large values (int64 limits) were indexed properly and no precision loss was seen. Range scans and lookup queries performed. Tried with array indexes as well.
9. Tried adding documents with value > int64 max values. This time they werent skipped, they were indexed. But there was a loss of precision which is expected.
10. Upgraded the KV nodes to 5.5.3
11. Repeated Steps 8 & 9. No issues seen
12. Tested with a fresh install with 5.5.3-4016 build. No issues seen.
Repro steps:
1. Create a new 5.0 cluster (or upgrade to 5.0 from 4.6) with documents with numbers greater than and less than MaxInt64.
2. Partially upgrade to 5.1.1 with projector still on 5.0 but indexer moved to 5.1.1
3. In this mixed mode setup, create insert/update mutations with numbers > MaxInt64 range. These document mutations encounter an error as they go through FixEncodedInt step which was introduced to fix format error in bug
MB-28956. The error causes the mutation to be skipped. The expected behavior is that document gets indexed with loss of precision.