Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-15793

Index build stuck in a cluster with two index nodes after indexer process kill in both nodes

    XMLWordPrintable

Details

    • Bug
    • Resolution: Fixed
    • Blocker
    • 4.0.0
    • 4.0.0
    • secondary-index
    • Security Level: Public
    • Centos
    • Untriaged
    • Yes

    Description

      Run: http://qa.sc.couchbase.com/job/cen006-2i-vset03-03-recoverytests-add-index/79/consoleFull

      Steps:

      1. Create 3 buckets: default, bucket0, standardBucket0; Load 100800 docs in each bucket
      2. Create and build 9 indexes in all 3 buckets (index_name_1 to index_name_9)
      3. Create 2 more indexes job_title and join_yr on each bucket in deferred mode
      4. Build above indexes created in #3
      5. While build of these 6 indexes is in progress, kill indexer process both the indexer nodes (node 144 and node 145)

      It is observed that two of the indexes default:employeee7c2f50f6799499abedd551cbfc99235job_title and default:employeee7c2f50f6799499abedd551cbfc99235join_yr are stuck at 86% each. Screenshot of index tab attached to the bug.

      Cluster: (The cluster is not live as it is used to run other jobs)

      [SERVER_1]
      ip:172.23.121.141
      services=n1ql,kv

      [SERVER_2]
      ip:172.23.121.142
      services=kv

      [SERVER_3]
      ip:172.23.121.143
      services=kv

      [SERVER_4]
      ip:172.23.121.144
      services=index

      [SERVER_5]
      ip:172.23.121.145
      services=index

      Logs:

      https://s3-ap-southeast-1.amazonaws.com/indexing-test/addIndex_test23/collectinfo-2015-07-22T121532-ns_1%40172.23.121.141.zip
      https://s3-ap-southeast-1.amazonaws.com/indexing-test/addIndex_test23/collectinfo-2015-07-22T121532-ns_1%40172.23.121.142.zip
      https://s3-ap-southeast-1.amazonaws.com/indexing-test/addIndex_test23/collectinfo-2015-07-22T121532-ns_1%40172.23.121.143.zip
      https://s3-ap-southeast-1.amazonaws.com/indexing-test/addIndex_test23/collectinfo-2015-07-22T121532-ns_1%40172.23.121.144.zip
      https://s3-ap-southeast-1.amazonaws.com/indexing-test/addIndex_test23/collectinfo-2015-07-22T121532-ns_1%40172.23.121.145.zip

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            deepkaran.salooja Deepkaran Salooja
            prathibha Prathibha Bisarahalli (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty