Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-39839

[IPv6] Query service crashing upon node initialization

    XMLWordPrintable

Details

    Description

      Build : 7.0.0-2278

      Steps to Reproduce :
      1. Setup 1 node cluster with data, index and n1ql services with ipv6 enabled.
      2. Check the query logs after successful initialization. There are be panic msgs in the query logs like the following -

      _time=2020-06-09T09:18:10.160-07:00 _level=INFO _msg= Initialization of cbauth succeeded
      _time=2020-06-09T09:18:10.161-07:00 _level=INFO _msg=New store created with url http://[::1]:8091
      _time=2020-06-09T09:18:10.161-07:00 _level=INFO _msg=pollEOF: About to start stdin polling
      panic: runtime error: index out of range [0] with length 0
       
      goroutine 1 [running]:
      github.com/couchbase/query/server.HostNameandPort(0x0, 0x0, 0xc000127680, 0x0, 0x0, 0x22312c6)
              /home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/query/server/server.go:1007 +0x1d4
      github.com/couchbase/query/clustering/couchbase.(*cbConfigStore).doNameState(0xc0002b20a0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
              /home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/query/clustering/couchbase/clustering_cb.go:415 +0x469
      github.com/couchbase/query/clustering/couchbase.(*cbConfigStore).State(0xc0002b20a0, 0xc000457300, 0x0, 0xc000748800, 0xc00074bc00)
              /home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/query/clustering/couchbase/clustering_cb.go:275 +0x2b
      github.com/couchbase/query/server/http.(*systemRemoteHttp).WhoAmI(0xc000105140, 0x10a21d2, 0x1f41760)
              /home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/query/server/http/system_remote.go:544 +0x88
      github.com/couchbase/query-ee/dictionary.dictMetaKVInit()
              /home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/query-ee/dictionary/metakv.go:38 +0x39
      github.com/couchbase/query-ee/dictionary.DictionaryCacheInit(0x4000)
              /home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/query-ee/dictionary/dictionary_cache.go:38 +0x4a
      github.com/couchbase/query/server.InitDictionaryCache(...)
              /home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/query/server/dictionary_ee.go:19
      main.main()
              goproj/src/github.com/couchbase/query/server/cbq-engine/main.go:237 +0x50a
      _time=2020-06-09T09:18:15.235-07:00 _level=INFO _msg= Initialization of cbauth succeeded
      _time=2020-06-09T09:18:15.238-07:00 _level=INFO _msg=New store created with url http://[::1]:8091
      _time=2020-06-09T09:18:15.238-07:00 _level=INFO _msg=pollEOF: About to start stdin polling
      

      A similar query crash was also seen with the same steps in 7.0.0-2017 & 7.0.0-2208, but only once. Here in 7.0.0-2278, we see this happening 12 times. Not sure if something has regressed since 7.0.0-2017 or 7.0.0-2208.

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          Isha Kandaswamy,

          Can you take a quick look? Is it because of fix of  MB-31105?

          kamini.jagtiani Kamini Jagtiani added a comment - Isha Kandaswamy , Can you take a quick look? Is it because of fix of   MB-31105 ?

          Still seeing the same issue with 7.0.0-2416. Here's the stack trace :

          2020-06-22T12:30:38.138-07:00 [INFO] Reponse : Message:"Evaluator initialized successfully"
          _time=2020-06-22T12:30:39.075-07:00 _level=INFO _msg=Starting audit update stream
          _time=2020-06-23T15:34:30.776-07:00 _level=INFO _msg= Initialization of cbauth succeeded
          _time=2020-06-23T15:34:30.777-07:00 _level=INFO _msg=New store created with url http://[::1]:8091
          _time=2020-06-23T15:34:30.778-07:00 _level=INFO _msg=pollEOF: About to start stdin polling
          panic: runtime error: index out of range [0] with length 0
           
          goroutine 1 [running]:
          github.com/couchbase/query/server.HostNameandPort(0x0, 0x0, 0xc0001279c0, 0x0, 0x0, 0x226560a)
                  /home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/query/server/server.go:1021 +0x1d4
          github.com/couchbase/query/clustering/couchbase.(*cbConfigStore).doNameState(0xc0000bce60, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
                  /home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/query/clustering/couchbase/clustering_cb.go:415 +0x469
          github.com/couchbase/query/clustering/couchbase.(*cbConfigStore).State(0xc0000bce60, 0xc00036ba80, 0x0, 0xc0006c6800, 0xc0006c9c00)
                  /home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/query/clustering/couchbase/clustering_cb.go:275 +0x2b
          github.com/couchbase/query/server/http.(*systemRemoteHttp).WhoAmI(0xc00036f170, 0x10adb62, 0x1f72040)
                  /home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/query/server/http/system_remote.go:544 +0x88
          github.com/couchbase/query-ee/dictionary.dictMetaKVInit()
                  /home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/query-ee/dictionary/metakv.go:38 +0x39
          github.com/couchbase/query-ee/dictionary.DictionaryCacheInit(0x4000)
                  /home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/query-ee/dictionary/dictionary_cache.go:39 +0x4a
          github.com/couchbase/query/server.InitDictionaryCache(...)
                  /home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/query/server/dictionary_ee.go:19
          main.main()
                  goproj/src/github.com/couchbase/query/server/cbq-engine/main.go:237 +0x50a
          _time=2020-06-23T15:34:35.856-07:00 _level=INFO _msg= Initialization of cbauth succeeded
          _time=2020-06-23T15:34:35.859-07:00 _level=INFO _msg=New store created with url http://[::1]:8091
          _time=2020-06-23T15:34:35.859-07:00 _level=INFO _msg=pollEOF: About to start stdin polling
          panic: runtime error: index out of range [0] with length 0
           
          goroutine 1 [running]:
          github.com/couchbase/query/server.HostNameandPort(0x0, 0x0, 0xc00008c640, 0x0, 0x0, 0x226560a)
                  /home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/query/server/server.go:1021 +0x1d4
          github.com/couchbase/query/clustering/couchbase.(*cbConfigStore).doNameState(0xc0006ea140, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
                  /home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/query/clustering/couchbase/clustering_cb.go:415 +0x469
          github.com/couchbase/query/clustering/couchbase.(*cbConfigStore).State(0xc0006ea140, 0xc0004ce520, 0x0, 0xc0006f8800, 0xc0006fbc00)
                  /home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/query/clustering/couchbase/clustering_cb.go:275 +0x2b
          github.com/couchbase/query/server/http.(*systemRemoteHttp).WhoAmI(0xc000513680, 0x10adb62, 0x1f72040)
                  /home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/query/server/http/system_remote.go:544 +0x88
          github.com/couchbase/query-ee/dictionary.dictMetaKVInit()
                  /home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/query-ee/dictionary/metakv.go:38 +0x39
          github.com/couchbase/query-ee/dictionary.DictionaryCacheInit(0x4000)
                  /home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/query-ee/dictionary/dictionary_cache.go:39 +0x4a
          github.com/couchbase/query/server.InitDictionaryCache(...)
                  /home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/query/server/dictionary_ee.go:19
          main.main()
                  goproj/src/github.com/couchbase/query/server/cbq-engine/main.go:237 +0x50a
          _time=2020-06-23T15:34:40.939-07:00 _level=INFO _msg= Initialization of cbauth succeeded
          _time=2020-06-23T15:34:40.942-07:00 _level=INFO _msg=New store created with url http://[::1]:8091
          

          mihir.kamdar Mihir Kamdar (Inactive) added a comment - Still seeing the same issue with 7.0.0-2416. Here's the stack trace : 2020-06-22T12:30:38.138-07:00 [INFO] Reponse : Message:"Evaluator initialized successfully" _time=2020-06-22T12:30:39.075-07:00 _level=INFO _msg=Starting audit update stream _time=2020-06-23T15:34:30.776-07:00 _level=INFO _msg= Initialization of cbauth succeeded _time=2020-06-23T15:34:30.777-07:00 _level=INFO _msg=New store created with url http://[::1]:8091 _time=2020-06-23T15:34:30.778-07:00 _level=INFO _msg=pollEOF: About to start stdin polling panic: runtime error: index out of range [0] with length 0   goroutine 1 [running]: github.com/couchbase/query/server.HostNameandPort(0x0, 0x0, 0xc0001279c0, 0x0, 0x0, 0x226560a) /home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/query/server/server.go:1021 +0x1d4 github.com/couchbase/query/clustering/couchbase.(*cbConfigStore).doNameState(0xc0000bce60, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0) /home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/query/clustering/couchbase/clustering_cb.go:415 +0x469 github.com/couchbase/query/clustering/couchbase.(*cbConfigStore).State(0xc0000bce60, 0xc00036ba80, 0x0, 0xc0006c6800, 0xc0006c9c00) /home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/query/clustering/couchbase/clustering_cb.go:275 +0x2b github.com/couchbase/query/server/http.(*systemRemoteHttp).WhoAmI(0xc00036f170, 0x10adb62, 0x1f72040) /home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/query/server/http/system_remote.go:544 +0x88 github.com/couchbase/query-ee/dictionary.dictMetaKVInit() /home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/query-ee/dictionary/metakv.go:38 +0x39 github.com/couchbase/query-ee/dictionary.DictionaryCacheInit(0x4000) /home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/query-ee/dictionary/dictionary_cache.go:39 +0x4a github.com/couchbase/query/server.InitDictionaryCache(...) /home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/query/server/dictionary_ee.go:19 main.main() goproj/src/github.com/couchbase/query/server/cbq-engine/main.go:237 +0x50a _time=2020-06-23T15:34:35.856-07:00 _level=INFO _msg= Initialization of cbauth succeeded _time=2020-06-23T15:34:35.859-07:00 _level=INFO _msg=New store created with url http://[::1]:8091 _time=2020-06-23T15:34:35.859-07:00 _level=INFO _msg=pollEOF: About to start stdin polling panic: runtime error: index out of range [0] with length 0   goroutine 1 [running]: github.com/couchbase/query/server.HostNameandPort(0x0, 0x0, 0xc00008c640, 0x0, 0x0, 0x226560a) /home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/query/server/server.go:1021 +0x1d4 github.com/couchbase/query/clustering/couchbase.(*cbConfigStore).doNameState(0xc0006ea140, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0) /home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/query/clustering/couchbase/clustering_cb.go:415 +0x469 github.com/couchbase/query/clustering/couchbase.(*cbConfigStore).State(0xc0006ea140, 0xc0004ce520, 0x0, 0xc0006f8800, 0xc0006fbc00) /home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/query/clustering/couchbase/clustering_cb.go:275 +0x2b github.com/couchbase/query/server/http.(*systemRemoteHttp).WhoAmI(0xc000513680, 0x10adb62, 0x1f72040) /home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/query/server/http/system_remote.go:544 +0x88 github.com/couchbase/query-ee/dictionary.dictMetaKVInit() /home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/query-ee/dictionary/metakv.go:38 +0x39 github.com/couchbase/query-ee/dictionary.DictionaryCacheInit(0x4000) /home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/query-ee/dictionary/dictionary_cache.go:39 +0x4a github.com/couchbase/query/server.InitDictionaryCache(...) /home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/query/server/dictionary_ee.go:19 main.main() goproj/src/github.com/couchbase/query/server/cbq-engine/main.go:237 +0x50a _time=2020-06-23T15:34:40.939-07:00 _level=INFO _msg= Initialization of cbauth succeeded _time=2020-06-23T15:34:40.942-07:00 _level=INFO _msg=New store created with url http://[::1]:8091

          The stack of the function shows some thing wrong. My guess is might be query nodeServices without fully initialized because node is "" https://github.com/couchbase/query/blob/master/server/server.go#L1021
          Once find a root cause also change above line strings.HasPrefix(node,"[")

          Sitaram.Vemulapalli Sitaram Vemulapalli added a comment - The stack of the function shows some thing wrong. My guess is might be query nodeServices without fully initialized because node is "" https://github.com/couchbase/query/blob/master/server/server.go#L1021 Once find a root cause also change above line strings.HasPrefix(node,"[")
          marco.greco Marco Greco added a comment - - edited

          My guess is that something is changed in ns_server whereby our own services is not yet listed by the time we try and devise our own name, hence in

          https://github.com/couchbase/query/blob/2ae78076eb9db9da0efa3befe01480ef8507eef6/clustering/couchbase/clustering_cb.go#L477

          we return a starting state and an empty hostname.
          There are several defensive fixes that are possible, and we should probably use more than one at the same time.
          Firstly, HostNameandPort() should handle empty strings.
          Secondly, it's probably a good idea that

          https://github.com/couchbase/query/blob/2ae78076eb9db9da0efa3befe01480ef8507eef6/server/cbq-engine/main.go#L237

          gets moved to after we have started the listener, next to

          https://github.com/couchbase/query/blob/2ae78076eb9db9da0efa3befe01480ef8507eef6/server/cbq-engine/main.go#L306

          Thirdly, functions/metakv/metakv.go:Init() and query_ee/dictionary/metakv.go:Init() should handle and retry empty hostnames.

          marco.greco Marco Greco added a comment - - edited My guess is that something is changed in ns_server whereby our own services is not yet listed by the time we try and devise our own name, hence in https://github.com/couchbase/query/blob/2ae78076eb9db9da0efa3befe01480ef8507eef6/clustering/couchbase/clustering_cb.go#L477 we return a starting state and an empty hostname. There are several defensive fixes that are possible, and we should probably use more than one at the same time. Firstly, HostNameandPort() should handle empty strings. Secondly, it's probably a good idea that https://github.com/couchbase/query/blob/2ae78076eb9db9da0efa3befe01480ef8507eef6/server/cbq-engine/main.go#L237 gets moved to after we have started the listener, next to https://github.com/couchbase/query/blob/2ae78076eb9db9da0efa3befe01480ef8507eef6/server/cbq-engine/main.go#L306 Thirdly, functions/metakv/metakv.go:Init() and query_ee/dictionary/metakv.go:Init() should handle and retry empty hostnames.
          isha Isha Kandaswamy added a comment - - edited

          Seeing a different crash upon implementing the fix. It looks like the cache never initialized and so we get the below panic while trying to access the contents of the cache.

          github.com/couchbase/query/server.(*Server).serviceRequest.func1()
          	/Users/isha/workspace/query/src/github.com/couchbase/query/server/server.go:628 +0xb8
          panic(0x5376180, 0x699b7a0)
          	/usr/local/Cellar/go/1.13.7/libexec/src/runtime/panic.go:679 +0x1b2
          github.com/couchbase/query/util.(*GenCache).Use(0x0, 0xc00048c470, 0xf, 0xc000b2abc0, 0x0, 0x0)
          	/Users/isha/workspace/query/src/github.com/couchbase/query/util/cache.go:266 +0x45
          github.com/couchbase/query-ee/dictionary.getCacheEntry(0xc00048c470, 0xf, 0xc000a26980, 0xa, 0x0, 0x0, 0x2)
          	/Users/isha/workspace/query/src/github.com/couchbase/query-ee/dictionary/dictionary_cache.go:396 +0xa5
          github.com/couchbase/query-ee/dictionary.GetHistogram(0xc00048c470, 0xf, 0xc000a26960, 0xc, 0x592c840)
          	/Users/isha/workspace/query/src/github.com/couchbase/query-ee/dictionary/dictionary_cache.go:360 +0xc1
          github.com/couchbase/query-ee/optutil.selec(0xc000a26941, 0x7, 0xc00048c470, 0xf, 0x592ba80, 0xc0000ef770, 0x0, 0xc0004bc000)
          	/Users/isha/workspace/query/src/github.com/couchbase/query-ee/optutil/selectivity.go:225 +0x133
          github.com/couchbase/query-ee/optutil.ExprSelec(0xc000a84f90, 0x592ba80, 0xc0000ef770, 0x7, 0xc000378cb8, 0x1)
          	/Users/isha/workspace/query/src/github.com/couchbase/query-ee/optutil/selectivity.go:77 +0x1e5
          github.com/couchbase/query/planner.optExprSelec(...)
          	/Users/isha/workspace/query/src/github.com/couchbase/query/planner/optutil_ee.go:39
          github.com/couchbase/query/planner.(*exprClassifier).visitDefault(0xc00010cf00, 0x592ba80, 0xc0000ef5e0, 0x0, 0x0, 0x0, 0x0)
          	/Users/isha/workspace/query/src/github.com/couchbase/query/planner/expr_classifier.go:454 +0xd33
          github.com/couchbase/query/planner.(*exprClassifier).VisitAnd(0xc00010cf00, 0xc0000ef6d0, 0x60, 0x55359a0, 0x5924d01, 0xc00010cf00)
          	/Users/isha/workspace/query/src/github.com/couchbase/query/planner/expr_classifier.go:98 +0xaf
          github.com/couchbase/query/expression.(*And).Accept(0xc0000ef6d0, 0x5959720, 0xc00010cf00, 0xc0000ef6d0, 0xc000121a60, 0xc00033dcc0, 0xc000121a70)
          	/Users/isha/workspace/query/src/github.com/couchbase/query/expression/logic_and.go:37 +0x3b
          github.com/couchbase/query/planner.ClassifyExpr(0x5924dc0, 0xc0000ef6d0, 0xc000a84b70, 0xc000a84e10, 0x100, 0x110, 0xc000c12000, 0xc0004bc000, 0x9901460)
          	/Users/isha/workspace/query/src/github.com/couchbase/query/planner/expr_classifier.go:30 +0xa9
          github.com/couchbase/query/planner.(*builder).processPredicate(0xc00030e6c0, 0x5924dc0, 0xc0000ef6d0, 0xc000378c00, 0x5335fe0, 0xc000378c30, 0xc000a84b70, 0x4010e83)
          	/Users/isha/workspace/query/src/github.com/couchbase/query/planner/build_scan.go:505 +0x51c
          github.com/couchbase/query/planner.(*builder).processWhere(0xc00030e6c0, 0x5924dc0, 0xc00033dcc0, 0x0, 0x0)
          	/Users/isha/workspace/query/src/github.com/couchbase/query/planner/build_scan.go:511 +0x48
          github.com/couchbase/query/planner.(*builder).visitFrom(0xc00030e6c0, 0xc0008dc900, 0x0, 0x0, 0x0)
          	/Users/isha/workspace/query/src/github.com/couchbase/query/planner/build_select_from.go:69 +0x11e3
          github.com/couchbase/query/planner.(*builder).VisitSubselect(0xc00030e6c0, 0xc0008dc900, 0x0, 0x0, 0x0, 0x0)
          	/Users/isha/workspace/query/src/github.com/couchbase/query/planner/build_select_sub.go:220 +0xae0
          github.com/couchbase/query/algebra.(*Subselect).Accept(0xc0008dc900, 0x5917d00, 0xc00030e6c0, 0x58d4ee0, 0xc0008dc900, 0x0, 0x0)
          	/Users/isha/workspace/query/src/github.com/couchbase/query/algebra/select_sub.go:59 +0x3e
          github.com/couchbase/query/planner.(*builder).VisitUnion(0xc00030e6c0, 0xc00000e7e0, 0x0, 0x0, 0x0, 0x0)
          	/Users/isha/workspace/query/src/github.com/couchbase/query/planner/build_set_op.go:32 +0x1a3
          github.com/couchbase/query/algebra.(*Union).Accept(0xc00000e7e0, 0x5917d00, 0xc00030e6c0, 0x9ba7d28, 0xc00000e7e0, 0xc0008a8250, 0x0)
          	/Users/isha/workspace/query/src/github.com/couchbase/query/algebra/set_op.go:213 +0x3e
          github.com/couchbase/query/planner.(*builder).VisitUnion(0xc00030e6c0, 0xc00000e860, 0x0, 0x0, 0x0, 0x0)
          	/Users/isha/workspace/query/src/github.com/couchbase/query/planner/build_set_op.go:32 +0x1a3
          github.com/couchbase/query/algebra.(*Union).Accept(0xc00000e860, 0x5917d00, 0xc00030e6c0, 0x9ba7d28, 0xc00000e860, 0x28, 0x5434e20)
          	/Users/isha/workspace/query/src/github.com/couchbase/query/algebra/set_op.go:213 +0x3e
          github.com/couchbase/query/planner.(*builder).VisitExcept(0xc00030e6c0, 0xc000121ba0, 0x0, 0x0, 0x0, 0x0)
          	/Users/isha/workspace/query/src/github.com/couchbase/query/planner/build_set_op.go:178 +0x1a3
          github.com/couchbase/query/algebra.(*Except).Accept(0xc000121ba0, 0x5917d00, 0xc00030e6c0, 0x0, 0x0, 0x0, 0x0)
          	/Users/isha/workspace/query/src/github.com/couchbase/query/algebra/set_op.go:377 +0x3b
          github.com/couchbase/query/planner.(*builder).VisitSelect(0xc00030e6c0, 0xc000037320, 0x0, 0x0, 0x0, 0x0)
          	/Users/isha/workspace/query/src/github.com/couchbase/query/planner/build_select.go:72 +0x328
          github.com/couchbase/query/algebra.(*Select).Accept(...)
          	/Users/isha/workspace/query/src/github.com/couchbase/query/algebra/select.go:59
          github.com/couchbase/query/planner.(*builder).VisitSubqueryTerm(0xc00030e6c0, 0xc000a4ecc0, 0x538c6e0, 0x5545960, 0xc00030e900, 0x9ba7d08)
          	/Users/isha/workspace/query/src/github.com/couchbase/query/planner/build_select_from.go:293 +0x5e
          github.com/couchbase/query/algebra.(*SubqueryTerm).Accept(0xc000a4ecc0, 0x5917d00, 0xc00030e6c0, 0x0, 0x0, 0x0, 0x0)
          	/Users/isha/workspace/query/src/github.com/couchbase/query/algebra/from_subquery.go:36 +0x3e
          github.com/couchbase/query/planner.(*builder).visitFrom(0xc00030e6c0, 0xc00013b380, 0xc000052880, 0x0, 0x0)
          	/Users/isha/workspace/query/src/github.com/couchbase/query/planner/build_select_from.go:139 +0xab0
          github.com/couchbase/query/planner.(*builder).VisitSubselect(0xc00030e6c0, 0xc00013b380, 0x0, 0x0, 0x0, 0x0)
          	/Users/isha/workspace/query/src/github.com/couchbase/query/planner/build_select_sub.go:220 +0xae0
          github.com/couchbase/query/algebra.(*Subselect).Accept(0xc00013b380, 0x5917d00, 0xc00030e6c0, 0x0, 0x0, 0x0, 0x0)
          	/Users/isha/workspace/query/src/github.com/couchbase/query/algebra/select_sub.go:59 +0x3e
          github.com/couchbase/query/planner.(*builder).VisitSelect(0xc00030e6c0, 0xc00010ce40, 0x0, 0x0, 0x0, 0x0)
          	/Users/isha/workspace/query/src/github.com/couchbase/query/planner/build_select.go:72 +0x328
          github.com/couchbase/query/algebra.(*Select).Accept(0xc00010ce40, 0x591d4c0, 0xc00030e6c0, 0x203000, 0x203000, 0x203000, 0x18)
          	/Users/isha/workspace/query/src/github.com/couchbase/query/algebra/select.go:59 +0x3e
          github.com/couchbase/query/planner.Build(0x5908180, 0xc00010ce40, 0x591a280, 0xc000352840, 0x591b240, 0xc00000f960, 0x5645592, 0x7, 0x4050100, 0xc000986070, ...)
          
          

          isha Isha Kandaswamy added a comment - - edited Seeing a different crash upon implementing the fix. It looks like the cache never initialized and so we get the below panic while trying to access the contents of the cache. github.com/couchbase/query/server.(*Server).serviceRequest.func1() /Users/isha/workspace/query/src/github.com/couchbase/query/server/server.go: 628 + 0xb8 panic( 0x5376180 , 0x699b7a0 ) /usr/local/Cellar/go/ 1.13 . 7 /libexec/src/runtime/panic.go: 679 + 0x1b2 github.com/couchbase/query/util.(*GenCache).Use( 0x0 , 0xc00048c470 , 0xf , 0xc000b2abc0 , 0x0 , 0x0 ) /Users/isha/workspace/query/src/github.com/couchbase/query/util/cache.go: 266 + 0x45 github.com/couchbase/query-ee/dictionary.getCacheEntry( 0xc00048c470 , 0xf , 0xc000a26980 , 0xa , 0x0 , 0x0 , 0x2 ) /Users/isha/workspace/query/src/github.com/couchbase/query-ee/dictionary/dictionary_cache.go: 396 + 0xa5 github.com/couchbase/query-ee/dictionary.GetHistogram( 0xc00048c470 , 0xf , 0xc000a26960 , 0xc , 0x592c840 ) /Users/isha/workspace/query/src/github.com/couchbase/query-ee/dictionary/dictionary_cache.go: 360 + 0xc1 github.com/couchbase/query-ee/optutil.selec( 0xc000a26941 , 0x7 , 0xc00048c470 , 0xf , 0x592ba80 , 0xc0000ef770 , 0x0 , 0xc0004bc000 ) /Users/isha/workspace/query/src/github.com/couchbase/query-ee/optutil/selectivity.go: 225 + 0x133 github.com/couchbase/query-ee/optutil.ExprSelec( 0xc000a84f90 , 0x592ba80 , 0xc0000ef770 , 0x7 , 0xc000378cb8 , 0x1 ) /Users/isha/workspace/query/src/github.com/couchbase/query-ee/optutil/selectivity.go: 77 + 0x1e5 github.com/couchbase/query/planner.optExprSelec(...) /Users/isha/workspace/query/src/github.com/couchbase/query/planner/optutil_ee.go: 39 github.com/couchbase/query/planner.(*exprClassifier).visitDefault( 0xc00010cf00 , 0x592ba80 , 0xc0000ef5e0 , 0x0 , 0x0 , 0x0 , 0x0 ) /Users/isha/workspace/query/src/github.com/couchbase/query/planner/expr_classifier.go: 454 + 0xd33 github.com/couchbase/query/planner.(*exprClassifier).VisitAnd( 0xc00010cf00 , 0xc0000ef6d0 , 0x60 , 0x55359a0 , 0x5924d01 , 0xc00010cf00 ) /Users/isha/workspace/query/src/github.com/couchbase/query/planner/expr_classifier.go: 98 + 0xaf github.com/couchbase/query/expression.(*And).Accept( 0xc0000ef6d0 , 0x5959720 , 0xc00010cf00 , 0xc0000ef6d0 , 0xc000121a60 , 0xc00033dcc0 , 0xc000121a70 ) /Users/isha/workspace/query/src/github.com/couchbase/query/expression/logic_and.go: 37 + 0x3b github.com/couchbase/query/planner.ClassifyExpr( 0x5924dc0 , 0xc0000ef6d0 , 0xc000a84b70 , 0xc000a84e10 , 0x100 , 0x110 , 0xc000c12000 , 0xc0004bc000 , 0x9901460 ) /Users/isha/workspace/query/src/github.com/couchbase/query/planner/expr_classifier.go: 30 + 0xa9 github.com/couchbase/query/planner.(*builder).processPredicate( 0xc00030e6c0 , 0x5924dc0 , 0xc0000ef6d0 , 0xc000378c00 , 0x5335fe0 , 0xc000378c30 , 0xc000a84b70 , 0x4010e83 ) /Users/isha/workspace/query/src/github.com/couchbase/query/planner/build_scan.go: 505 + 0x51c github.com/couchbase/query/planner.(*builder).processWhere( 0xc00030e6c0 , 0x5924dc0 , 0xc00033dcc0 , 0x0 , 0x0 ) /Users/isha/workspace/query/src/github.com/couchbase/query/planner/build_scan.go: 511 + 0x48 github.com/couchbase/query/planner.(*builder).visitFrom( 0xc00030e6c0 , 0xc0008dc900 , 0x0 , 0x0 , 0x0 ) /Users/isha/workspace/query/src/github.com/couchbase/query/planner/build_select_from.go: 69 + 0x11e3 github.com/couchbase/query/planner.(*builder).VisitSubselect( 0xc00030e6c0 , 0xc0008dc900 , 0x0 , 0x0 , 0x0 , 0x0 ) /Users/isha/workspace/query/src/github.com/couchbase/query/planner/build_select_sub.go: 220 + 0xae0 github.com/couchbase/query/algebra.(*Subselect).Accept( 0xc0008dc900 , 0x5917d00 , 0xc00030e6c0 , 0x58d4ee0 , 0xc0008dc900 , 0x0 , 0x0 ) /Users/isha/workspace/query/src/github.com/couchbase/query/algebra/select_sub.go: 59 + 0x3e github.com/couchbase/query/planner.(*builder).VisitUnion( 0xc00030e6c0 , 0xc00000e7e0 , 0x0 , 0x0 , 0x0 , 0x0 ) /Users/isha/workspace/query/src/github.com/couchbase/query/planner/build_set_op.go: 32 + 0x1a3 github.com/couchbase/query/algebra.(*Union).Accept( 0xc00000e7e0 , 0x5917d00 , 0xc00030e6c0 , 0x9ba7d28 , 0xc00000e7e0 , 0xc0008a8250 , 0x0 ) /Users/isha/workspace/query/src/github.com/couchbase/query/algebra/set_op.go: 213 + 0x3e github.com/couchbase/query/planner.(*builder).VisitUnion( 0xc00030e6c0 , 0xc00000e860 , 0x0 , 0x0 , 0x0 , 0x0 ) /Users/isha/workspace/query/src/github.com/couchbase/query/planner/build_set_op.go: 32 + 0x1a3 github.com/couchbase/query/algebra.(*Union).Accept( 0xc00000e860 , 0x5917d00 , 0xc00030e6c0 , 0x9ba7d28 , 0xc00000e860 , 0x28 , 0x5434e20 ) /Users/isha/workspace/query/src/github.com/couchbase/query/algebra/set_op.go: 213 + 0x3e github.com/couchbase/query/planner.(*builder).VisitExcept( 0xc00030e6c0 , 0xc000121ba0 , 0x0 , 0x0 , 0x0 , 0x0 ) /Users/isha/workspace/query/src/github.com/couchbase/query/planner/build_set_op.go: 178 + 0x1a3 github.com/couchbase/query/algebra.(*Except).Accept( 0xc000121ba0 , 0x5917d00 , 0xc00030e6c0 , 0x0 , 0x0 , 0x0 , 0x0 ) /Users/isha/workspace/query/src/github.com/couchbase/query/algebra/set_op.go: 377 + 0x3b github.com/couchbase/query/planner.(*builder).VisitSelect( 0xc00030e6c0 , 0xc000037320 , 0x0 , 0x0 , 0x0 , 0x0 ) /Users/isha/workspace/query/src/github.com/couchbase/query/planner/build_select.go: 72 + 0x328 github.com/couchbase/query/algebra.(*Select).Accept(...) /Users/isha/workspace/query/src/github.com/couchbase/query/algebra/select.go: 59 github.com/couchbase/query/planner.(*builder).VisitSubqueryTerm( 0xc00030e6c0 , 0xc000a4ecc0 , 0x538c6e0 , 0x5545960 , 0xc00030e900 , 0x9ba7d08 ) /Users/isha/workspace/query/src/github.com/couchbase/query/planner/build_select_from.go: 293 + 0x5e github.com/couchbase/query/algebra.(*SubqueryTerm).Accept( 0xc000a4ecc0 , 0x5917d00 , 0xc00030e6c0 , 0x0 , 0x0 , 0x0 , 0x0 ) /Users/isha/workspace/query/src/github.com/couchbase/query/algebra/from_subquery.go: 36 + 0x3e github.com/couchbase/query/planner.(*builder).visitFrom( 0xc00030e6c0 , 0xc00013b380 , 0xc000052880 , 0x0 , 0x0 ) /Users/isha/workspace/query/src/github.com/couchbase/query/planner/build_select_from.go: 139 + 0xab0 github.com/couchbase/query/planner.(*builder).VisitSubselect( 0xc00030e6c0 , 0xc00013b380 , 0x0 , 0x0 , 0x0 , 0x0 ) /Users/isha/workspace/query/src/github.com/couchbase/query/planner/build_select_sub.go: 220 + 0xae0 github.com/couchbase/query/algebra.(*Subselect).Accept( 0xc00013b380 , 0x5917d00 , 0xc00030e6c0 , 0x0 , 0x0 , 0x0 , 0x0 ) /Users/isha/workspace/query/src/github.com/couchbase/query/algebra/select_sub.go: 59 + 0x3e github.com/couchbase/query/planner.(*builder).VisitSelect( 0xc00030e6c0 , 0xc00010ce40 , 0x0 , 0x0 , 0x0 , 0x0 ) /Users/isha/workspace/query/src/github.com/couchbase/query/planner/build_select.go: 72 + 0x328 github.com/couchbase/query/algebra.(*Select).Accept( 0xc00010ce40 , 0x591d4c0 , 0xc00030e6c0 , 0x203000 , 0x203000 , 0x203000 , 0x18 ) /Users/isha/workspace/query/src/github.com/couchbase/query/algebra/select.go: 59 + 0x3e github.com/couchbase/query/planner.Build( 0x5908180 , 0xc00010ce40 , 0x591a280 , 0xc000352840 , 0x591b240 , 0xc00000f960 , 0x5645592 , 0x7 , 0x4050100 , 0xc000986070 , ...)

          Build couchbase-server-7.0.0-2734 contains query commit a2db218 with commit message:
          MB-39839 Address query panic due to empty hostname

          build-team Couchbase Build Team added a comment - Build couchbase-server-7.0.0-2734 contains query commit a2db218 with commit message: MB-39839 Address query panic due to empty hostname

          Build couchbase-server-7.0.0-2734 contains query-ee commit 3c2f4c3 with commit message:
          MB-39839 For panic

          build-team Couchbase Build Team added a comment - Build couchbase-server-7.0.0-2734 contains query-ee commit 3c2f4c3 with commit message: MB-39839 For panic

          Verified on 7.0.0-4023

          mihir.kamdar Mihir Kamdar (Inactive) added a comment - Verified on 7.0.0-4023

          People

            isha Isha Kandaswamy
            mihir.kamdar Mihir Kamdar (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty