Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-19699

[130-node] projector panic on node startup (CBAuth database is stale)

    XMLWordPrintable

Details

    • Bug
    • Resolution: Duplicate
    • Major
    • 5.0.0
    • 4.5.0
    • ns_server

    Description

      Cluster is 130 Data Nodes. 1 Index node, 1 query node, but no indexes are created.

      Some of the projector services exited on nodes in the cluster on startup and also after being rebalanced in. Could be MB-16128

      panic: Get http://127.0.0.1:8091/_metakv/indexing/settings/config: CBAuth database is stale. Was never updated yet.
       
      goroutine 1 [running]:
      panic(0xcff200, 0xc820123500)
              /home/couchbase/.cbdepscache/exploded/x86_64/go-1.6/go/src/runtime/panic.go:464 +0x3e6 fp=0xc820121a20 sp=0xc8201219a0
      github.com/couchbase/indexing/secondary/common.CrashOnError(0x7f0699d6cca8, 0xc820123500)
              /home/couchbase/jenkins/workspace/watson-unix/goproj/src/github.com/couchbase/indexing/secondary/common/util.go:368 +0x4b fp=0xc820
      121a48 sp=0xc820121a20
      github.com/couchbase/indexing/secondary/projector.NewProjector(0x400, 0xc820123410, 0xc820121e88)
              /home/couchbase/jenkins/workspace/watson-unix/goproj/src/github.com/couchbase/indexing/secondary/projector/projector.go:54 +0x17e f
      p=0xc820121d98 sp=0xc820121a48
      main.main()
              /home/couchbase/jenkins/workspace/watson-unix/goproj/src/github.com/couchbase/indexing/secondary/cmd/projector/main.go:106 +0xaef f
      p=0xc820121f40 sp=0xc820121d98
      runtime.main()
              /home/couchbase/.cbdepscache/exploded/x86_64/go-1.6/go/src/runtime/proc.go:188 +0x2b0 fp=0xc820121f90 sp=0xc820121f40
      runtime.goexit()
              /home/couchbase/.cbdepscache/exploded/x86_64/go-1.6/go/src/runtime/asm_amd64.s:1998 +0x1 fp=0xc820121f98 sp=0xc820121f90
       
      goroutine 17 [syscall, locked to thread]:
      runtime.goexit()
              /home/couchbase/.cbdepscache/exploded/x86_64/go-1.6/go/src/runtime/asm_amd64.s:1998 +0x1 fp=0xc820050fb8 sp=0xc820050fb0
      

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              artem Artem Stemkovski
              tommie Tommie McAfee (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty