Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-13693

Indexer continues to connect to bucket even after bucket is deleted

    XMLWordPrintable

Details

    • Bug
    • Resolution: Fixed
    • Major
    • 4.0.0
    • 4.0.0
    • secondary-index
    • Security Level: Public
    • None
    • mac osx
    • Untriaged
    • No

    Description

      After a bucket is deleted, the indexer logs indicate that it continues to connect KV bucket even after it is deleted with logs:

      21:33:07.286718 [Error] ScanCoordinator: Unable compute index count for testbucket2/bucket2_city (Index not found)
      21:33:07.286743 [Error] ScanCoordinator: Unable compute index count for testbucket3/bucket3_gender (Index not found)
      21:33:07.286780 [Error] ScanCoordinator: Unable compute index count for testbucket4/bucket4_balance (Index not found)
      21:33:07.446768 [Error] Indexer::getCurrentKVTs Error Connecting to KV Cluster No bucket named testbucket2
      21:33:07.446775 [Error] Timekeeper::handleStats Error occured while obtaining KV seqnos - No bucket named testbucket2

      However this doesnt result in any failure, the indexer log file continues to register log msgs like above.

      Indexer log files attached. In the logs, the indexer is attempting to retry and error out for more than an hour.

      Related issue: Bucket delete is also causing TestBucketFlush test to fail when run as part of functional suite. The indexer logs indicate:
      02:55:38.028841 [Info] StorageMgr::handleRollback
      Rollback Index: 17906732742100798393 PartitionId: 0 SliceId: 0 To Zero
      02:55:38.030391 [Info] StorageMgr::handleRollback
      Rollback Index: 16626908543363530500 PartitionId: 0 SliceId: 0 To Zero
      02:55:38.031873 [Info] StorageMgr::handleRollback
      Rollback Index: 6915475666737963918 PartitionId: 0 SliceId: 0 To Zero
      02:55:38.033314 [Info] StorageMgr::updateIndexSnapMap IndexInst:12816879626837700754 Attempting to open snapshot (SnapshotInfo: seqnos: 2, 2, 1)
      02:55:38.033363 [Error] ForestDBSnapshot::Open
      Unexpected Error Opening Meta DB Snapshot (/Users/prathibha-mac/Documents/source/cb/ns_server/data/n_1/data/@2i/default_index_nationality_12816879626837700754_0.index) SeqNum 1 no db instance
      panic: Unable to open snapshot -no db instance

      goroutine 81 [running]:
      runtime.panic(0x46aea40, 0xc20e1b5ce0)
      /usr/local/Cellar/go/1.3.1/libexec/src/pkg/runtime/panic.c:279 +0xf5
      github.com/couchbase/indexing/secondary/indexer.(*storageMgr).updateIndexSnapMap(0xc2086c87d0, 0xc2099d9b00, 0xc20a290001, 0xc20a299090, 0x7)
      /Users/prathibha-mac/Documents/source/cb/goproj/src/github.com/couchbase/indexing/secondary/indexer/storage_manager.go:766 +0x621
      github.com/couchbase/indexing/secondary/indexer.(*storageMgr).handleRollback(0xc2086c87d0, 0xb2ecf40, 0xc2095739e0)
      /Users/prathibha-mac/Documents/source/cb/goproj/src/github.com/couchbase/indexing/secondary/indexer/storage_manager.go:473 +0x8d5
      github.com/couchbase/indexing/secondary/indexer.(*storageMgr).handleSupvervisorCommands(0xc2086c87d0, 0xb2ecf40, 0xc2095739e0)
      /Users/prathibha-mac/Documents/source/cb/goproj/src/github.com/couchbase/indexing/secondary/indexer/storage_manager.go:159 +0xdc
      github.com/couchbase/indexing/secondary/indexer.(*storageMgr).run(0xc2086c87d0)
      /Users/prathibha-mac/Documents/source/cb/goproj/src/github.com/couchbase/indexing/secondary/indexer/storage_manager.go:141 +0x16d
      created by github.com/couchbase/indexing/secondary/indexer.NewStorageManager
      /Users/prathibha-mac/Documents/source/cb/goproj/src/github.com/couchbase/indexing/secondary/indexer/storage_manager.go:119 +0x3e0

      Attached logs for TestBucketFlush test as well ( BucketFlushIndexerCrash_Run2.zip )

      Attachments

        1. BucketFlushIndexerCrash_Run2.zip
          9.55 MB
        2. indexer.log
          1.45 MB
        3. indexer.log.1.gz
          859 kB
        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            deepkaran.salooja Deepkaran Salooja
            prathibha Prathibha Bisarahalli (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty