Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-55135

Index is missing post indexer rebalance failure. "indexer rebalance failure - cleanup pending from previous failed/aborted rebalance/failover/move index.

    XMLWordPrintable

Details

    • Bug
    • Resolution: Duplicate
    • Critical
    • Elixir
    • Elixir
    • secondary-index
    • 7.5.0-3529

    Description

      1. Create 19 databases to avoid kv/gsi auto-scaling due to num_tenants.
      2. Create 10 collections(VolumeCollection0, VolumeCollection1....VolumeCollection0) in each database
      3. Create 100 indexes on each database i.e. 10 on each collection. This results in total 1900 indexes.
      4. Build 10 indexes per database at a time. Example: 

        Bucket0:VolumeCollection0 - 10 indexes under build
        		 
        Bucket1:VolumeCollection0 - 10 indexes under build
        		 
        Bucket2:VolumeCollection0 - 10 indexes under build
        		 
        .
        		 
        .
        		 
        .
        		 
        Bucket19:VolumeCollection0 - 10 indexes under build

      5. Repeat the above step
      6. While building `volumetestbucket_15_b3fh42_idx_VolumeCollection0_80` it is found missing.

      There is a rebalance failure observed:

      Rebalance exited with reason {service_rebalance_failed,index,
      		 
      {worker_died,
      		 
      {'EXIT',<0.17932.39>,
      		 
      {{badmatch,
      		 
      {error,
      		 
      {bad_nodes,index,prepare_rebalance,
      		 
      [{'ns_1@rzl68xts2c4ndr0.ef7-grjfaa9k98s.sandbox.nonprod-project-avengers.com',
      		 
      {error,
      		 
      {unknown_error,
      		 
      <<"indexer rebalance failure - cleanup pending from previous failed/aborted rebalance/failover/move index. please retry the request later.">>}}}]}}},
      		 
      [{service_manager,rebalance_op,5,
      		 
      [{file,"src/service_manager.erl"},
      		 
      {line,322}]},
      		 
      {service_manager,do_run_op,1,
      		 
      [{file,"src/service_manager.erl"},
      		 
      {line,269}]},
      		 
      {proc_lib,init_p,3,
      		 
      [{file,"proc_lib.erl"},{line,211}]}]}}}}.
      		 
      Rebalance Operation Id = 0efc637ae232191f3ef61230a0cfd90b

      CC: Varun Velamuri

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            pavan.pb Pavan PB
            ritesh.agarwal Ritesh Agarwal
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty