Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-49079

CBSE: Do not fail Rebalance for merge replica counter failure

    XMLWordPrintable

Details

    • 1

    Description

      [Split out of MB-47873 per 2021-10-20 GSI scrum. Originally opened by Varun Velamuri from CBSE-10499.
      MB-47874 was also created for the same problem from same CBSE, aiming to find and fix the root cause, while current MB is aimed at making Rebalance not fail even if this bug gets hit.]

      If merging replica counter fails, indexer can avoid replica repair for the definition on which merge failed, rather than failing rebalance.

      From CBSE-10499 description, which is the source of the original error:

      Background and Analysis:

       Analytics Nodes were added on `2021-07-27`, Rebalance is failing since then:

      2021-07-27T10:08:18.443000+0100 10.114.141.4 added node 10.114.141.16
       
      2021-07-27T10:14:59.570000+0100 10.114.141.4 added node 10.114.141.17
       

       Rebalance is failing with below error:

      [ns_server:error,2021-07-27T13:19:53.907+01:00,ns_1@10.114.141.10:service_status_keeper-index<0.725.0>:service_status_keeper:handle_cast:119]Service service_index returned incorrect status [ns_server:error,2021-07-27T13:22:33.398+01:00,ns_1@10.114.141.10:service_agent-index<0.669.0>:service_agent:handle_info:287]Rebalancer <13523.17251.1> died unexpectedly: {worker_died, {'EXIT',<13523.17280.1>, {rebalance_failed, {service_error, <<"Unable to read index layout from cluster 127.0.0.1:8091. err = Cannot merge counter with different base values">>}}}} ..
       

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              kevin.cherkauer Kevin Cherkauer
              kevin.cherkauer Kevin Cherkauer
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty