Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-51371

Index build is stuck at 100% when create index is done after kill on memcached on ephemeral bucket

    XMLWordPrintable

Details

    • Untriaged
    • 1
    • Yes

    Description

      Index build is stuck at 100% and not finishing.

      Steps to reproduce:

      1. Create a 5 node cluster with following configuration - kv-kv-kv-index-index:n1ql
      2. Create an ephemeral bucket and load data to it
      3. create few indexes (primary + secondary)

        CREATE PRIMARY INDEX ON default  USING GSI
         
        CREATE INDEX `index_name_0` ON default(VMs) WHERE  VMs IS NOT NULL  USING GSI  WITH {'defer_build': True}
         
        BUILD INDEX on default(`index_name_0`) USING GSI
         
        CREATE INDEX `index_name_1` ON default(VMs) WHERE  VMs IS NOT NULL  USING GSI  WITH {'defer_build': True}
         
        BUILD INDEX on default(`index_name_1`) USING GSI
         
        CREATE INDEX `index_name_2` ON default(VMs) WHERE  VMs IS NOT NULL  USING GSI  WITH {'defer_build': True}
         
        BUILD INDEX on default(`index_name_2`) USING GSI
         
        CREATE INDEX `index_name_3` ON default(VMs) WHERE  VMs IS NOT NULL  USING GSI  WITH {'defer_build': True}
         
        BUILD INDEX on default(`index_name_3`) USING GSI
         

      4. Kill memcached on second KV node and sleep for 60sec

       kill -9 $(ps aux | pgrep 'memcached')

      5. Run create index and doc mutation in parallel

      CREATE INDEX `employee48f0ecbfce7242e384fb816e8a526e52job_title` ON default(job_title) WHERE  job_title IS NOT NULL  USING GSI  WITH {'defer_build': True}
      CREATE INDEX `employee48f0ecbfce7242e384fb816e8a526e52join_yr` ON default(join_yr) WHERE  join_yr > 2010 and join_yr < 2014  USING GSI  WITH {'defer_build': True}
      BUILD INDEX on default(employee48f0ecbfce7242e384fb816e8a526e52job_title,employee48f0ecbfce7242e384fb816e8a526e52join_yr) USING GSI
        

      # Index building is stuck for above 2 indexes.

      http://qa.sc.couchbase.com/job/test_suite_executor/449067/consoleText - test1

      I'm also able to reproduce the test on local vagrant environment

      It was passing till 7.1.0-1831

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            pavan.pb Pavan PB
            hemant.rajput Hemant Rajput
            Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty