Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-46309

[Upgrade] - 2i rebalance using online upgrade fails with Error :: code= 52, severity= Normal, category= Metadata Repo, reason= Index Definition 'o3_claims' already exists

    XMLWordPrintable

Details

    • Untriaged
    • Centos 64-bit
    • 1
    • Yes

    Description

      Steps:
      Same as MB-46301.
      Stopped gave, left the cluster alone for hours /restarted rebalance many times to work around MB-46301(2i), MB-46246(eventing) and MB-46274(2i). Could be a dup. Also it would be good to know how to work around these 2i upgrade issues. I have hit them more than 10 times so far.

      The retried rebalance failed as shown below.
      ns_1@172.23.105.10 27:27:03 AM   14 May, 2021

      Starting rebalance, KeepNodes = ['ns_1@172.23.104.15','ns_1@172.23.104.214',
      'ns_1@172.23.104.232','ns_1@172.23.104.244',
      'ns_1@172.23.104.245','ns_1@172.23.105.102',
      'ns_1@172.23.105.109','ns_1@172.23.105.112',
      'ns_1@172.23.105.118','ns_1@172.23.105.164',
      'ns_1@172.23.105.206','ns_1@172.23.105.210',
      'ns_1@172.23.105.25','ns_1@172.23.105.61',
      'ns_1@172.23.105.62','ns_1@172.23.105.86',
      'ns_1@172.23.105.90','ns_1@172.23.105.93',
      'ns_1@172.23.106.117','ns_1@172.23.106.191',
      'ns_1@172.23.106.207','ns_1@172.23.106.209',
      'ns_1@172.23.106.225','ns_1@172.23.106.232',
      'ns_1@172.23.106.239','ns_1@172.23.106.246',
      'ns_1@172.23.106.32','ns_1@172.23.106.37',
      'ns_1@172.23.106.54','ns_1@172.23.110.75',
      'ns_1@172.23.110.76'], EjectNodes = [], Failed over and being ejected nodes = []; no delta recovery nodes; Operation Id = 47809e0eb32dd57204db34b1a2185191
      

      ns_1@172.23.105.10 27:29:07 AM   14 May, 2021

      Rebalance exited with reason {service_rebalance_failed,index,
      {worker_died,
      {'EXIT',<0.19256.2236>,
      {rebalance_failed,
      {service_error,
      <<"Error :: code= 52, severity= Normal, category= Metadata Repo, reason= Index Definition 'o3_claims' already exists, cause= %!s(<nil>)">>}}}}}.
      Rebalance Operation Id = 47809e0eb32dd57204db34b1a2185191ns_orchestrator 000ns_1@172.23.105.1027:29:07 AM   14 May, 2021
      

      cbcollect_info attached. This was not seen in upgrade during 6.6.2->9588 to 7.0.0-5033.

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            Balakumaran.Gopal Balakumaran Gopal
            Balakumaran.Gopal Balakumaran Gopal
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty