Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-52907

[System Test] Panic seen in query logs - invalid memory address or nil pointer dereference

    XMLWordPrintable

Details

    • Untriaged
    • 1
    • Unknown

    Description

      BUILD - 7.1.2-3345

       

      QE TEST

       

      -test tests/integration/neo/test_neo.yml -scope tests/integration/neo/scope_couchstore.yml -scale 3

      TEST STEP

      Running FTS queries
      [2022-07-08T02:17:52-07:00, sequoiatools/ftsindexmanager:8ee646] -n 172.23.96.148 -o 8091 -u Administrator -p password -b bucket5 --print_interval 600 -a run_flex_queries -t 0 -nq 1

       

      List of FTS nodes
                1. fts : 2 ===== > [172.23.105.111:8091 172.23.96.148:8091] ###########

       

      List of N1QL nodes
                1. n1ql : 2 ===== > [172.23.104.155:8091 172.23.104.157:8091] ###########
      Query logs indicating panic
      2022-07-08T02:30:06.484-07:00 [Info] GSIC[default/bucket7-scope_4-coll_3-1657126799221181406] 45e19121-06c4-4252-b731-3f4181fdf685 new temp file ... /opt/couchbase/var/lib/couchbase/tmp/scan-results113581637047521
      2022-07-08T02:30:06.490-07:00 [Info] GSIC[default/bucket7-scope_4-coll_3-1657126799221181406] request(45e19121-06c4-4252-b731-3f4181fdf685) removing temp file /opt/couchbase/var/lib/couchbase/tmp/scan-results113581637047521 ...
      2022-07-08T02:30:06.539-07:00 [Info] GSIC[default/bucket5-scope_3-coll_3-1657127227544237184] 7fa2d9b0-f8a2-4f79-baee-0b9afb41acba new temp file ... /opt/couchbase/var/lib/couchbase/tmp/scan-results113582254115112
      2022-07-08T02:30:06.704-07:00 [Info] GSIC[default/bucket4-scope_1-coll_4-1657127226802466097] 3370e34e-cb92-4898-b059-cb8c54d05806 new temp file ... /opt/couchbase/var/lib/couchbase/tmp/scan-results11358958178330
      2022-07-08T02:30:06.705-07:00 [Info] GSIC[default/bucket7-scope_0-coll_1-1657127226916249710] 8c35bf76-3a52-441e-b0cf-ec8cb3b6fb8e new temp file ... /opt/couchbase/var/lib/couchbase/tmp/scan-results113582078354664
      2022-07-08T02:30:06.728-07:00 [Info] GSIC[default/bucket7-scope_0-coll_1-1657127226916249710] request(8c35bf76-3a52-441e-b0cf-ec8cb3b6fb8e) removing temp file /opt/couchbase/var/lib/couchbase/tmp/scan-results113582078354664 ...
      2022-07-08T02:30:06.740-07:00 [SEVERE] panic: runtime error: invalid memory address or nil pointer dereference
      2022-07-08T02:30:06.740-07:00 [SEVERE] request text: <ud>INSERT INTO `default`:`bucket7`.`scope_4`.`coll_3` (KEY UUID(), VALUE price) SELECT price FROM `default`:`bucket7`.`scope_4`.`coll_3` WHERE type='Hotel' AND free_breakfast=True AND free_parking=True LIMIT 100</ud>
      2022-07-08T02:30:06.740-07:00 [SEVERE] query context: <ud></ud>
      2022-07-08T02:30:06.740-07:00 [SEVERE] stack: goroutine 236668762 [running]:
      github.com/couchbase/query/execution.(*Context).Recover(0xc046148000, 0xc004abac80)
          /home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/query/execution/context.go:1202 +0x8a
      panic({0x23eb200, 0x3f83ae0})
          /home/couchbase/.cbdepscache/exploded/x86_64/go-1.18.1/go/src/runtime/panic.go:838 +0x207
      github.com/couchbase/query/execution.(*base).runConsumer.func1.2()
          /home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/query/execution/base.go:895 +0xb6
      panic({0x23eb200, 0x3f83ae0})
          /home/couchbase/.cbdepscache/exploded/x86_64/go-1.18.1/go/src/runtime/panic.go:838 +0x207
      github.com/couchbase/query/value.(*annotatedValue).ResetCovers(0xc004abad48?, {0x0?, 0x0?})
          /home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/query/value/annotated.go:251 +0x36
      github.com/couchbase/query/execution.(*Fetch).processItem(0xc004abac80, {0x2d4dd10, 0x0}, 0xc046148000)
          /home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/query/execution/fetch.go:89 +0x4c
      github.com/couchbase/query/execution.(*base).runConsumer.func1()
          /home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/query/execution/base.go:930 +0x3cf
      github.com/couchbase/query/util.(*Once).Do(0x0?, 0x0?)
          /home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/query/util/sync.go:54 +0x31
      github.com/couchbase/query/execution.(*base).runConsumer(0x0?, {0x2cf3b58?, 0xc004abac80?}, 0x0?, {0x0?, 0x0?})
          /home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/query/execution/base.go:864 +0x85
      github.com/couchbase/query/execution.(*Fetch).RunOnce(0x1221401?, 0x2cf31f0?, {0x0?, 0x0?})
          /home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/query/execution/fetch.go:77 +0x38
      github.com/couchbase/query/execution.execOp({0x2d0dd50, 0xc004abac80}, 0x20c8800?, {0x0, 0x0})
          /home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/query/execution/base.go:526 +0x54
      created by github.com/couchbase/query/execution.(*base).fork
          /home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/query/execution/base.go:537 +0x125
      goroutine 236668762 [running]:
      github.com/couchbase/query/execution.(*Context).Recover(0xc046148000, 0xc004abac80)
          /home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/query/execution/context.go:1202 +0x8a
      panic({0x23eb200, 0x3f83ae0})
          /home/couchbase/.cbdepscache/exploded/x86_64/go-1.18.1/go/src/runtime/panic.go:838 +0x207
      github.com/couchbase/query/execution.(*base).runConsumer.func1.2()
          /home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/query/execution/base.go:895 +0xb6
      panic({0x23eb200, 0x3f83ae0})
          /home/couchbase/.cbdepscache/exploded/x86_64/go-1.18.1/go/src/runtime/panic.go:838 +0x207
      github.com/couchbase/query/value.(*annotatedValue).ResetCovers(0xc004abad48?, {0x0?, 0x0?})
          /home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/query/value/annotated.go:251 +0x36
      github.com/couchbase/query/execution.(*Fetch).processItem(0xc004abac80, {0x2d4dd10, 0x0}, 0xc046148000)
          /home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/query/execution/fetch.go:89 +0x4c
      github.com/couchbase/query/execution.(*base).runConsumer.func1()
          /home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/query/execution/base.go:930 +0x3cf
      github.com/couchbase/query/util.(*Once).Do(0x0?, 0x0?)
          /home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/query/util/sync.go:54 +0x31
      github.com/couchbase/query/execution.(*base).runConsumer(0x0?, {0x2cf3b58?, 0xc004abac80?}, 0x0?, {0x0?, 0x0?})
          /home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/query/execution/base.go:864 +0x85
      github.com/couchbase/query/execution.(*Fetch).RunOnce(0x1221401?, 0x2cf31f0?, {0x0?, 0x0?})
          /home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/query/execution/fetch.go:77 +0x38
      github.com/couchbase/query/execution.execOp({0x2d0dd50, 0xc004abac80}, 0x20c8800?, {0x0, 0x0})
          /home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/query/execution/base.go:526 +0x54
      created by github.com/couchbase/query/execution.(*base).fork
          /home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/query/execution/base.go:537 +0x125
      2022-07-08T02:30:06.777-07:00 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
      2022-07-08T02:30:06.777-07:00 [Info] Refreshed Indexer List: [172.23.104.137:9100 172.23.120.245:9100 172.23.96.251:9100 172.23.96.252:9100 172.23.96.253:9100 172.23.99.11:9100]
      2022-07-08T02:30:06.835-07:00 [Info] switched currmeta from 55713 -> 55713 force true 
      2022-07-08T02:30:06.999-07:00 [Info] GSIC[default/bucket7-scope_0-coll_4-1657127226903068983] request(c1747d70-bbda-49d2-8ad8-efb09d25313b) removing temp file /opt/couchbase/var/lib/couchbase/tmp/scan-results113581633834568 ...
      2022-07-08T02:30:07.000-07:00 [Info] GSIC[default/bucket7-scope_0-coll_4-1657127226903068983] request(c1747d70-bbda-49d2-8ad8-efb09d25313b) removing temp file /opt/couchbase/var/lib/couchbase/tmp/scan-results113582841266577 ...
      2022-07-08T02:30:07.008-07:00 [Info] GSIC[default/bucket5-scope_3-coll_3-1657127227544237184] request(7fa2d9b0-f8a2-4f79-baee-0b9afb41acba) removing temp file /opt/couchbase/var/lib/couchbase/tmp/scan-results113582254115112 ...
      2022-07-08T02:30:07.067-07:00 [Info] GsiClient::UpdateUsecjson: using collatejson as data format between indexer and GsiClient
      2022-07-08T02:30:07.077-07:00 [Info] GSIC[default/bucket7-scope_0-coll_1-1657127226916249710] 2712cda0-34b3-40c2-9437-8ec61a11cc0e new temp file ... /opt/couchbase/var/lib/couchbase/tmp/scan-results113583420166018
      2022-07-08T02:30:07.088-07:00 [Info] GSIC[default/bucket7-scope_0-coll_1-1657127226916249710] 2712cda0-34b3-40c2-9437-8ec61a11cc0e new temp file ... /opt/couchbase/var/lib/couchbase/tmp/scan-results113584030924204
      2022-07-08T02:30:07.144-07:00 [Info] GSIC[default/bucket5-scope_2-coll_1-1657127227526246997] request(f4aa7999-eb45-4745-bbfc-2b180a7368ca) removing temp file /opt/couchbase/var/lib/couchbase/tmp/scan-results113584003827262 ...

      cbcollect logs:

               url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1657282815/collectinfo-2022-07-08T122018-ns_1%40172.23.104.137.zip
               url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1657282815/collectinfo-2022-07-08T122018-ns_1%40172.23.104.155.zip
               url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1657282815/collectinfo-2022-07-08T122018-ns_1%40172.23.104.157.zip
               url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1657282815/collectinfo-2022-07-08T122018-ns_1%40172.23.104.5.zip
               url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1657282815/collectinfo-2022-07-08T122018-ns_1%40172.23.104.67.zip
               url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1657282815/collectinfo-2022-07-08T122018-ns_1%40172.23.104.69.zip
               url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1657282815/collectinfo-2022-07-08T122018-ns_1%40172.23.105.107.zip
               url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1657282815/collectinfo-2022-07-08T122018-ns_1%40172.23.105.111.zip
               url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1657282815/collectinfo-2022-07-08T122018-ns_1%40172.23.105.168.zip
               url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1657282815/collectinfo-2022-07-08T122018-ns_1%40172.23.106.100.zip
               url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1657282815/collectinfo-2022-07-08T122018-ns_1%40172.23.106.188.zip
               url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1657282815/collectinfo-2022-07-08T122018-ns_1%40172.23.108.103.zip
               url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1657282815/collectinfo-2022-07-08T122018-ns_1%40172.23.120.107.zip
               url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1657282815/collectinfo-2022-07-08T122018-ns_1%40172.23.120.245.zip
               url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1657282815/collectinfo-2022-07-08T122018-ns_1%40172.23.121.117.zip
               url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1657282815/collectinfo-2022-07-08T122018-ns_1%40172.23.123.28.zip
               url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1657282815/collectinfo-2022-07-08T122018-ns_1%40172.23.96.148.zip
               url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1657282815/collectinfo-2022-07-08T122018-ns_1%40172.23.96.192.zip
               url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1657282815/collectinfo-2022-07-08T122018-ns_1%40172.23.96.251.zip
               url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1657282815/collectinfo-2022-07-08T122018-ns_1%40172.23.96.252.zip
               url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1657282815/collectinfo-2022-07-08T122018-ns_1%40172.23.96.253.zip
               url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1657282815/collectinfo-2022-07-08T122018-ns_1%40172.23.97.119.zip
               url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1657282815/collectinfo-2022-07-08T122018-ns_1%40172.23.97.121.zip
               url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1657282815/collectinfo-2022-07-08T122018-ns_1%40172.23.97.122.zip
               url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1657282815/collectinfo-2022-07-08T122018-ns_1%40172.23.97.239.zip
               url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1657282815/collectinfo-2022-07-08T122018-ns_1%40172.23.99.11.zip
               url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1657282815/collectinfo-2022-07-08T122018-ns_1%40172.23.99.20.zip
               url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1657282815/collectinfo-2022-07-08T122018-ns_1%40172.23.99.21.zip
               url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1657282815/collectinfo-2022-07-08T122018-ns_1%40172.23.99.25.zip

       

      cc Pierre Regazzoni Ajay Bhullar 

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            pavan.pb Pavan PB
            pavan.pb Pavan PB
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty