Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-31745

[BP MB-30207] - [GSI] Error strconv.ParseInt: parsing "9223372036854776000": value out of range

    XMLWordPrintable

Details

    • Bug
    • Status: Closed
    • Critical
    • Resolution: Fixed
    • 5.5.0
    • 5.5.3
    • secondary-index
    • Initial Version: 4.6.4-4590
      Upgrade Version: 5.5.0-2907
      Storage Mode: Memory Optimized

      Cluster Config:
      Node 1: kv (101)
      Node 2: kv (102)
      Node 3: Index, n1ql (103)
      Node 4: Index (104)
    • Untriaged
    • Unknown

    Description

      [Please note this error occurs in two scenarios. First is in description. Second is mentioned in the comment]

      1. Create and configure cluster, Create and load buckets, Create indexes
      2. Failover cluster nodes one by one and upgrade and add back. Create equivalent indexes if failed over node is indexer to continue queries
      3. After adding back the node and rebalancing them in, remove the equivalent indexes created.
      4. Start the int 64 queries.

      Failed Query:

      SELECT sum(long_num) as long_num, name FROM default use index (int_64_936834_long_num_name) where long_num > 2147483600 group by name

      Error Message:

      {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==\n'} error: 500 reason: unknown {
       
      "requestID": "fd7307b6-261b-4baf-b188-b278889b946f",
       
      "signature": {"long_num":"number","name":"json"},
       
      "results": [
       
      ],
       
      "errors": [{"code":5000,"msg":" strconv.ParseInt: parsing \"9223372036854776000\": value out of range from [10.111.171.103:9101] - cause:  strconv.ParseInt: parsing \"9223372036854776000\": value out of range from [10.111.171.103:9101]"}],
       
      "status": "errors",
       
      "metrics": {"elapsedTime": "33.185783ms","executionTime": "33.134507ms","resultCount": 0,"resultSize": 0,"errorCount": 1}
       
      } auth: Administrator:password

      Similar query passes on various other tests like test_offline_upgrade (http://qa.sc.couchbase.com/job/cen7-2i-plasma-set5-job1-upgrade-4-6-4-int64/51/consoleFull)

      Logs: 
      https://s3.amazonaws.com/bugdb/jira/failover_upgrade/collectinfo-2018-06-21T094601-ns_1%4010.111.171.101.zip
      https://s3.amazonaws.com/bugdb/jira/failover_upgrade/collectinfo-2018-06-21T094601-ns_1%4010.111.171.102.zip
      https://s3.amazonaws.com/bugdb/jira/failover_upgrade/collectinfo-2018-06-21T094601-ns_1%4010.111.171.103.zip
      https://s3.amazonaws.com/bugdb/jira/failover_upgrade/collectinfo-2018-06-21T094601-ns_1%4010.111.171.104.zip
       

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            Repro steps:

            1. Create a new 5.0 cluster (or upgrade to 5.0 from 4.6) with documents with numbers greater than and less than MaxInt64.
            2. Partially upgrade to 5.1.1 with projector still on 5.0 but indexer moved to 5.1.1
            3. In this mixed mode setup, create insert/update mutations with numbers > MaxInt64 range. These document mutations encounter an error as they go through FixEncodedInt step which was introduced to fix format error in bug MB-28956. The error causes the mutation to be skipped. The expected behavior is that document gets indexed with loss of precision.

            prathibha Prathibha Bisarahalli (Inactive) added a comment - Repro steps: 1. Create a new 5.0 cluster (or upgrade to 5.0 from 4.6) with documents with numbers greater than and less than MaxInt64. 2. Partially upgrade to 5.1.1 with projector still on 5.0 but indexer moved to 5.1.1 3. In this mixed mode setup, create insert/update mutations with numbers > MaxInt64 range. These document mutations encounter an error as they go through FixEncodedInt step which was introduced to fix format error in bug MB-28956 . The error causes the mutation to be skipped. The expected behavior is that document gets indexed with loss of precision.

            Validation steps for the backport fix:

            1. 4.6 cluster: 2 KV and 2 index
            All types of data, with all types of index: simple, composite, array index

            2. Upgrade to 5.0
            Insert/Update mutations. Observe trailing zero bug
            Existing indexed items remain as they are.
            Array index insert / updates

            3. Upgrade to 5.1.1
            Insert/Update mutations. New mutations should go through FixEncode path.
            Array index insert / updates
            New mutations with > MaxInt64 get skipped in mixed mode ( Bug )
            New mutations with > MaxInt64 get indexed after full upgrade

            4. Upgrade to 5.5.3
            Insert/Update mutations - Everything should work fine
            Array index insert / updates
            New mutations with > MaxInt64 get indexed in mixed mode as well as after full upgrade

            Verification at each step:
            ScanAll/Range scans
            Lookup query for each value

            prathibha Prathibha Bisarahalli (Inactive) added a comment - - edited Validation steps for the backport fix: 1. 4.6 cluster: 2 KV and 2 index All types of data, with all types of index: simple, composite, array index 2. Upgrade to 5.0 Insert/Update mutations. Observe trailing zero bug Existing indexed items remain as they are. Array index insert / updates 3. Upgrade to 5.1.1 Insert/Update mutations. New mutations should go through FixEncode path. Array index insert / updates New mutations with > MaxInt64 get skipped in mixed mode ( Bug ) New mutations with > MaxInt64 get indexed after full upgrade 4. Upgrade to 5.5.3 Insert/Update mutations - Everything should work fine Array index insert / updates New mutations with > MaxInt64 get indexed in mixed mode as well as after full upgrade Verification at each step: ScanAll/Range scans Lookup query for each value

            Build couchbase-server-5.5.3-4015 contains indexing commit e083223 with commit message:
            MB-31745: Fix SUM aggregate query for large integers

            build-team Couchbase Build Team added a comment - Build couchbase-server-5.5.3-4015 contains indexing commit e083223 with commit message: MB-31745 : Fix SUM aggregate query for large integers

            Tested on 5.5.3-4016 with the following steps

            1. Setup a 5.0 cluster. 4 nodes - kv+query, kv+query, index, index
            2. Created a bucket with documents of integer types of various sizes and formats as mentioned in the test data set. None beyond int64 boundaries yet.
            3. Observe trailing zero bug
            4. Upgraded the indexer nodes to 5.1.1
            5. New mutations with integers with large values (int64 limits) were indexed properly and no precision loss was seen. Range scans and lookup queries performed. Tried with array indexes as well.
            6. Tried adding documents with value > int64 max values, they were skipped. So issue was reproduced reliably.
            7. Upgraded the indexer nodes to 5.5.3
            8. New mutations with integers with large values (int64 limits) were indexed properly and no precision loss was seen. Range scans and lookup queries performed. Tried with array indexes as well.
            9. Tried adding documents with value > int64 max values. This time they werent skipped, they were indexed. But there was a loss of precision which is expected.
            10. Upgraded the KV nodes to 5.5.3
            11. Repeated Steps 8 & 9. No issues seen
            12. Tested with a fresh install with 5.5.3-4016 build. No issues seen.

            mihir.kamdar Mihir Kamdar (Inactive) added a comment - Tested on 5.5.3-4016 with the following steps 1. Setup a 5.0 cluster. 4 nodes - kv+query, kv+query, index, index 2. Created a bucket with documents of integer types of various sizes and formats as mentioned in the test data set. None beyond int64 boundaries yet. 3. Observe trailing zero bug 4. Upgraded the indexer nodes to 5.1.1 5. New mutations with integers with large values (int64 limits) were indexed properly and no precision loss was seen. Range scans and lookup queries performed. Tried with array indexes as well. 6. Tried adding documents with value > int64 max values, they were skipped. So issue was reproduced reliably. 7. Upgraded the indexer nodes to 5.5.3 8. New mutations with integers with large values (int64 limits) were indexed properly and no precision loss was seen. Range scans and lookup queries performed. Tried with array indexes as well. 9. Tried adding documents with value > int64 max values. This time they werent skipped, they were indexed. But there was a loss of precision which is expected. 10. Upgraded the KV nodes to 5.5.3 11. Repeated Steps 8 & 9. No issues seen 12. Tested with a fresh install with 5.5.3-4016 build. No issues seen.

            Verified on 5.5.3-4016

            mihir.kamdar Mihir Kamdar (Inactive) added a comment - Verified on 5.5.3-4016

            People

              amit.kulkarni Amit Kulkarni
              jeelan.poola Jeelan Poola
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty