Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-19350

Inconsistent Internal State reached in indexer during handlePrepareRecovery

    XMLWordPrintable

Details

    • Bug
    • Resolution: Fixed
    • Critical
    • 4.5.0
    • 4.5.0
    • secondary-index
    • None
    • cluster_run on mac
    • Untriaged
    • Unknown

    Description

      While testing out rollback to 0 scenario, in one of the trials, I encountered a panic in indexer after receiving rollback. From the indexer log -

      2016-04-19T10:59:07.754+05:30 [Error] MutationMgr::handleRemoveIndexListFromStream Stream Already Closed MAINT_STREAM
      2016-04-19T10:59:07.754+05:30 [Error] Indexer::sendStreamUpdateToWorker - Error received from MutationMgr processing Msg
      Message: MsgStreamUpdate
      Type: REMOVE_BUCKET_FROM_STREAM
      Stream: MAINT_STREAM
      Bucket: default
      BuildTS: []
      IndexList: []
      RestartTs: <nil> Err <nil>. Aborted.
      panic: Inconsistent Internal State

      goroutine 1 [running]:
      panic(0x4a99da0, 0xc8201457a0)
      /Users/prathibha-mac/.cbdepscache/exploded/x86_64/go-1.6/go/src/runtime/panic.go:464 +0x3e6 fp=0xc823192cd0 sp=0xc823192c50
      github.com/couchbase/indexing/secondary/common.CrashOnError(0x6340000, 0xc8201457a0)
      /Users/prathibha-mac/Documents/source/cb/goproj/src/github.com/couchbase/indexing/secondary/common/util.go:367 +0x4b fp=0xc823192cf8 sp=0xc823192cd0
      github.com/couchbase/indexing/secondary/indexer.(*indexer).stopBucketStream(0xc82007a6c0, 0x1, 0xc823a63785, 0x7)
      /Users/prathibha-mac/Documents/source/cb/goproj/src/github.com/couchbase/indexing/secondary/indexer/indexer.go:2680 +0x432 fp=0xc823192ea8 sp=0xc823192cf8
      github.com/couchbase/indexing/secondary/indexer.(*indexer).handlePrepareRecovery(0xc82007a6c0, 0x1ab9cd68, 0xc82c8ce780)
      /Users/prathibha-mac/Documents/source/cb/goproj/src/github.com/couchbase/indexing/secondary/indexer/indexer.go:1342 +0x20a fp=0xc823192f58 sp=0xc823192ea8
      github.com/couchbase/indexing/secondary/indexer.(*indexer).handleWorkerMsgs(0xc82007a6c0, 0x1ab9cd68, 0xc82c8ce780)
      /Users/prathibha-mac/Documents/source/cb/goproj/src/github.com/couchbase/indexing/secondary/indexer/indexer.go:699 +0x1551 fp=0xc823193188 sp=0xc823192f58
      github.com/couchbase/indexing/secondary/indexer.(*indexer).run(0xc82007a6c0)
      /Users/prathibha-mac/Documents/source/cb/goproj/src/github.com/couchbase/indexing/secondary/indexer/indexer.go:482 +0x15f fp=0xc8231932e0 sp=0xc823193188
      github.com/couchbase/indexing/secondary/indexer.NewIndexer(0xc820142390, 0x0, 0x0, 0x0, 0x0)
      /Users/prathibha-mac/Documents/source/cb/goproj/src/github.com/couchbase/indexing/secondary/indexer/indexer.go:364 +0x3f1b fp=0xc823193c30 sp=0xc8231932e0
      main.main()
      /Users/prathibha-mac/Documents/source/cb/goproj/src/github.com/couchbase/indexing/secondary/cmd/indexer/main.go:106 +0x14f8 fp=0xc823193f40 sp=0xc823193c30
      runtime.main()

      I have attached the entire logs of n_0 and n_1. This was tested on local cluster_run on mac.

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            deepkaran.salooja Deepkaran Salooja
            prathibha Prathibha Bisarahalli (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty