Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-31990

[BP MB-30467 to 5.5.3] - Panic seen in query log while running a query on xattr

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 5.5.0
    • Fix Version/s: 5.5.3
    • Component/s: query
    • Labels:
    • Triage:
      Untriaged
    • Is this a Regression?:
      Unknown

      Description

      Build : 5.5.0-2958 (RC4)

      While the system was running, a query was run separately on the same cluster (query is not a part of the test). This query resulted in a panic on the query log. Upon retrying the query, it returned results and didnt result in an error. Re-ran twice again, it panic'd once more. So 2/4 times it panicked.

      Query
      select meta().xattrs._sync.sequence
      from default
      where meta().xattrs._sync.sequence is not null
      limit 10

      Indexes on xattr

      CREATE INDEX `sg_roleAccess_x1` ON `default`((all (array (`op`.`name`) for `op` in object_pairs((((meta().`xattrs`).`_sync`).`role_access`)) end))) WITH { "defer_build":true, "retain_deleted_xattr":true, "num_replica":1 }
       
      CREATE INDEX `sg_access_x1` ON `default`((all (array (`op`.`name`) for `op` in object_pairs((((meta().`xattrs`).`_sync`).`access`)) end))) WITH { "defer_build":true, "retain_deleted_xattr":true, "num_replica":1 }
       
      CREATE INDEX `sg_allDocs_x1` ON `default`((((meta().`xattrs`).`_sync`).`sequence`),(((meta().`xattrs`).`_sync`).`rev`),(((meta().`xattrs`).`_sync`).`flags`),(((meta().`xattrs`).`_sync`).`deleted`)) WHERE (not ((meta().`id`) like "\\_sync:%")) WITH { "defer_build":true, "retain_deleted_xattr":true, "num_replica":1 }
       
      CREATE INDEX `sg_channels_x1` ON `default`((all (array [(`op`.`name`), least((((meta().`xattrs`).`_sync`).`sequence`), ((`op`.`val`).`seq`)), ifmissing(((`op`.`val`).`rev`), null), ifmissing(((`op`.`val`).`del`), null)] for `op` in object_pairs((((meta().`xattrs`).`_sync`).`channels`)) end)),(((meta().`xattrs`).`_sync`).`rev`),(((meta().`xattrs`).`_sync`).`sequence`),(((meta().`xattrs`).`_sync`).`flags`)) WITH { "defer_build":true, "retain_deleted_xattr":true, "num_replica":1 }
       
      CREATE INDEX `sg_tombstones_x1` ON `default`((((meta().`xattrs`).`_sync`).`tombstoned_at`)) WITH { "defer_build":true, "retain_deleted_xattr":true, "num_replica":1 }
      

      Panic seen in query log on 172.23.104.61

      _time=2018-07-13T01:58:19.535-07:00 _level=SEVERE _msg=panic: runtime error: index out of range
       
      request text:
      <ud>select meta().xattrs._sync.sequence
      from default
      where meta().xattrs._sync.sequence is not null
      limit 10</ud>
       
      stack:
      goroutine 3792435 [running]:
      github.com/couchbase/query/execution.(*Context).Recover(0xc4219a5600)
              goproj/src/github.com/couchbase/query/execution/context.go:498 +0xbc
      panic(0xe37480, 0x186d160)
              /home/couchbase/.cbdepscache/exploded/x86_64/go-1.8.5/go/src/runtime/panic.go:489 +0x2cf
      github.com/couchbase/query/datastore/couchbase.getSubDocFetchResults(0xc425bcef20, 0x12, 0xc42f4fd020, 0xc424b53d20, 0x2, 0x2, 0x1, 0x18d1280, 0xc42e482ae0)
              goproj/src/github.com/couchbase/query/datastore/couchbase/couchbase.go:1096 +0xb81
      github.com/couchbase/query/datastore/couchbase.(*keyspace).Fetch(0xc4201a3090, 0xc42d309000, 0x10, 0x40, 0xc42df36ea0, 0x189a4a0, 0xc4219a5600, 0xc421ec1540, 0x1, 0x1, ...)
              goproj/src/github.com/couchbase/query/datastore/couchbase/couchbase.go:1044 +0x379
      github.com/couchbase/query/execution.(*Fetch).flushBatch(0xc42bb381e0, 0xc4219a5600, 0x1906600)
              goproj/src/github.com/couchbase/query/execution/fetch.go:108 +0x533
      github.com/couchbase/query/execution.(*base).enbatchSize(0xc42bb381e0, 0x18d1280, 0xc431086330, 0x189a500, 0xc42bb381e0, 0x10, 0xc4219a5600, 0xc431086330)
              goproj/src/github.com/couchbase/query/execution/base.go:707 +0x1a2
      github.com/couchbase/query/execution.(*Fetch).processItem(0xc42bb381e0, 0x18d1280, 0xc431086330, 0xc4219a5600, 0x1)
              goproj/src/github.com/couchbase/query/execution/fetch.go:53 +0x6a
      github.com/couchbase/query/execution.(*base).runConsumer.func1()
              goproj/src/github.com/couchbase/query/execution/base.go:547 +0x25d
      github.com/couchbase/query/util.(*Once).Do(0xc42bb382d8, 0xc4242f0738)
              goproj/src/github.com/couchbase/query/util/sync.go:51 +0x68
      github.com/couchbase/query/execution.(*base).runConsumer(0xc42bb381e0, 0x1897c40, 0xc42bb381e0, 0xc4219a5600, 0x0, 0x0)
              goproj/src/github.com/couchbase/query/execution/base.go:552 +0xaf
      github.com/couchbase/query/execution.(*Fetch).RunOnce(0xc42bb381e0, 0xc4219a5600, 0x0, 0x0)
              goproj/src/github.com/couchbase/query/execution/fetch.go:49 +0x5c
      created by github.com/couchbase/query/execution.(*base).runConsumer.func1
              goproj/src/github.com/couchbase/query/execution/base.go:537 +0x2f6
      

      Explain plan from one of the successful queries

      {
        "plan": {
          "#operator": "Sequence",
          "~children": [
            {
              "#operator": "Sequence",
              "~children": [
                {
                  "#operator": "PrimaryScan3",
                  "index": "#primary",
                  "index_projection": {
                    "primary_key": true
                  },
                  "keyspace": "default",
                  "namespace": "default",
                  "using": "gsi"
                },
                {
                  "#operator": "Fetch",
                  "keyspace": "default",
                  "namespace": "default",
                  "subpaths": [
                    "_sync"
                  ]
                },
                {
                  "#operator": "Parallel",
                  "~child": {
                    "#operator": "Sequence",
                    "~children": [
                      {
                        "#operator": "Filter",
                        "condition": "((((meta(`default`).`xattrs`).`_sync`).`sequence`) is not null)"
                      },
                      {
                        "#operator": "InitialProject",
                        "result_terms": [
                          {
                            "expr": "(((meta(`default`).`xattrs`).`_sync`).`sequence`)"
                          }
                        ]
                      },
                      {
                        "#operator": "FinalProject"
                      }
                    ]
                  }
                }
              ]
            },
            {
              "#operator": "Limit",
              "expr": "10"
            }
          ]
        },
        "text": "select meta().xattrs._sync.sequence\nfrom default \nwhere meta().xattrs._sync.sequence is not null\nlimit 10"
      }
      

        Attachments

          Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

            Activity

            Hide
            isha Isha Kandaswamy added a comment -

            The fix for https://issues.couchbase.com/browse/MB-31991 should have fixed this. 

            Build no is -  5.5.3-4023  and  6.0.1-1962

            Show
            isha Isha Kandaswamy added a comment - The fix for https://issues.couchbase.com/browse/MB-31991  should have fixed this.  Build no is -  5.5.3-4023  and  6.0.1-1962

              People

              • Assignee:
                mihir.kamdar Mihir Kamdar
                Reporter:
                mihir.kamdar Mihir Kamdar
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Gerrit Reviews

                  There are no open Gerrit changes

                    PagerDuty

                    Error rendering 'com.pagerduty.jira-server-plugin:PagerDuty'. Please contact your Jira administrators.