Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-14594

[system test] panic: Post http://l...8091/_metakv: CBAuth database is stale: last reason: dial tcp 127.0.0.1:8091: connection refused

    XMLWordPrintable

Details

    • Bug
    • Resolution: Fixed
    • Critical
    • 4.0.0
    • 4.0.0
    • ns_server
    • Security Level: Public
    • 400-1879
      6 nodes, 10 Buckets, 10 view-indexes
      3 nodes have ( kv, index, query services enabled)
      Run time ~ 15 hours
    • Untriaged
    • Unknown

    Description

      1.Load 5M data on the cluster
      2. Keep querying on the system
      3. Reboot entire cluster –

      Seeing a number of panics on the system - projector logs ( which is strange considering there are no gsi indexes defined on the system)

      panic: Post http://127.0.0.1:8091/_metakv: CBAuth database is stale: last reason: dial tcp 127.0.0.1:8091: connection refused

      goroutine 16 [running]:
      runtime.panic(0xab9040, 0xc20807aba0)
      /usr/local/go/src/pkg/runtime/panic.c:279 +0xf5
      github.com/couchbase/indexing/secondary/common.CrashOnError(0x7fcd84e39530, 0xc20807aba0)
      /home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/indexing/secondary/common/util.go:352 +0x4f
      github.com/couchbase/indexing/secondary/projector.NewProjector(0x400, 0xc20807a9c0, 0x1f)
      /home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/indexing/secondary/projector/projector.go:49 +0x105
      main.main()
      /home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/indexing/secondary/cmd/projector/main.go:92 +0x5c7

      goroutine 19 [finalizer wait]:
      runtime.park(0x415070, 0x12d2b68, 0x12b1149)
      /usr/local/go/src/pkg/runtime/proc.c:1369 +0x89
      runtime.parkunlock(0x12d2b68, 0x12b1149)
      /usr/local/go/src/pkg/runtime/proc.c:1385 +0x3b
      runfinq()
      /usr/local/go/src/pkg/runtime/mgc0.c:2644 +0xcf
      runtime.goexit()
      /usr/local/go/src/pkg/runtime/proc.c:1445

      goroutine 20 [syscall]:
      os/signal.loop()
      /usr/local/go/src/pkg/os/signal/signal_unix.go:21 +0x1e
      created by os/signal.init·1
      /usr/local/go/src/pkg/os/signal/signal_unix.go:27 +0x32

      goroutine 24 [sleep]:
      time.Sleep(0x3b9aca00)
      /usr/local/go/src/pkg/runtime/time.goc:39 +0x31
      github.com/couchbase/cbauth/revrpc.(*DefaultErrorPolicy).try(0xc20803cc80, 0x7fcd84e39468, 0xc20804f400, 0x0, 0x0)
      /home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/revrpc/revrpc.go:205 +0x3ee
      github.com/couchbase/cbauth/revrpc.*DefaultErrorPolicy.(github.com/couchbase/cbauth/revrpc.try)·fm(0x7fcd84e39468, 0xc20804f400, 0x0, 0x0)
      /home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/revrpc/revrpc.go:217 +0x4c

      Attaching logs and detailed stack.

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              Aliaksey Artamonau Aliaksey Artamonau (Inactive)
              ketaki Ketaki Gangal (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty