Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-60587

[System Test Upgrade] : Post upgrade rebalance out fails in analytics service with - Rebalance 15fd496cbcb4b778bfb6d3f5f7c374cc failed: CBAS0001: Analytics collection dv_2.dv_2.ds_9 in different partitions has different DCP states. Seqno gap = 1694439.

    XMLWordPrintable

Details

    • Bug
    • Resolution: Cannot Reproduce
    • Major
    • None
    • 7.6.0
    • analytics
    • 7.2.4-7070 -> 7.6.0-2054
    • Untriaged
    • Centos 64-bit
    • 0
    • Unknown
    • Analytics Sprint 35, Analytics Sprint 36

    Description

      Steps to Repro
      1. Run 7.2 longevity on 7.2.4 for 4-5 days.
      2. Start upgrade of this longevity cluster using various online upgrade strategies.

      Did a bunch of rebalances post upgrades. One of the rebalance out scenarios failed.

      172.23.123.33 11:37:45 PM 29 Jan, 2024

      Starting rebalance, KeepNodes = ['ns_1@172.23.106.134','ns_1@172.23.120.58',
      'ns_1@172.23.120.73','ns_1@172.23.120.74',
      'ns_1@172.23.120.77','ns_1@172.23.120.86',
      'ns_1@172.23.123.25','ns_1@172.23.123.26',
      'ns_1@172.23.123.31','ns_1@172.23.123.32',
      'ns_1@172.23.123.33','ns_1@172.23.96.122',
      'ns_1@172.23.96.243','ns_1@172.23.96.48',
      'ns_1@172.23.97.105','ns_1@172.23.97.110',
      'ns_1@172.23.97.148','ns_1@172.23.97.149',
      'ns_1@172.23.97.150'], EjectNodes = ['ns_1@172.23.97.74',
      'ns_1@172.23.97.241',
      'ns_1@172.23.121.77'], Failed over and being ejected nodes = []; no delta recovery nodes; Operation Id = 4e8058b4ff28041abb7c38adbb691727
      

      172.23.123.33 12:15:08 AM 30 Jan, 2024

      Rebalance exited with reason {service_rebalance_failed,cbas,
      {worker_died,
      {'EXIT',<0.6298.2044>,
      {task_failed,rebalance,
      {service_error,
      <<"Rebalance 15fd496cbcb4b778bfb6d3f5f7c374cc failed: CBAS0001: Analytics collection dv_2.dv_2.ds_9 in different partitions has different DCP states. Seqno gap = 1694439. User action: Try again later">>}}}}}.
      Rebalance Operation Id = 4e8058b4ff28041abb7c38adbb691727
      

      cbcollect_info attached.

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            Balakumaran.Gopal Balakumaran Gopal
            Balakumaran.Gopal Balakumaran Gopal
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty