Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-46112

Rebalance failed with reason "agent died"

    XMLWordPrintable

Details

    Description

      Build: 7.0.0-5095

      Scenario:

      • Cluster with index,eventing,fts,backup services actively running
      • 3 FTS indexes created (2 with indexPartition=1 and 1 index with indexPartition=6)
      • Rebalance in extra FTS node into the cluster along with few other nodes with other services

      Observation:

      Fts rebalance failed with reason fts worker died,

       

      Rebalance exited with reason {service_rebalance_failed,fts,
      {worker_died,
      {'EXIT',<0.12316.37>,
      {rebalance_failed,inactivity_timeout}}}}.
      Rebalance Operation Id = b5ce1e972169e0377441008a937cb498
      ..
      ..
      Starting rebalance, KeepNodes = ['ns_1@172.23.105.175','ns_1@172.23.106.233',
      'ns_1@172.23.106.236','ns_1@172.23.106.238',
      'ns_1@172.23.106.250','ns_1@172.23.106.251',
      'ns_1@172.23.107.43','ns_1@172.23.107.44',
      'ns_1@172.23.107.45','ns_1@172.23.107.58',
      'ns_1@172.23.121.74','ns_1@172.23.121.78'], EjectNodes = [], Failed over and being ejected nodes = []; no delta recovery nodes; Operation Id = b5ce1e972169e0377441008a937cb49

      Snapshot: https://supportal.couchbase.com/snapshot/ff8cc1ad0a5b1fdc2c01fdd52cec4165::0

       

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            ashwin.govindarajulu Ashwin Govindarajulu added a comment - - edited

            Not seeing this issue on latest run using 7.0.0-5112-enterprise.

            Test has run for ~8hrs till now (will continue to run for some more time)

            Closing this ticket.

            ashwin.govindarajulu Ashwin Govindarajulu added a comment - - edited Not seeing this issue on latest run using 7.0.0-5112-enterprise . Test has run for ~8hrs till now (will continue to run for some more time) Closing this ticket.

            Hitting MB-46154 while trying to validate the fix.

            ashwin.govindarajulu Ashwin Govindarajulu added a comment - Hitting MB-46154 while trying to validate the fix.

            Build couchbase-server-7.0.0-5109 contains cbgt commit fa82fd7 with commit message:
            MB-46112: Log feed name when StartGocbcoreDCPFeed fails

            build-team Couchbase Build Team added a comment - Build couchbase-server-7.0.0-5109 contains cbgt commit fa82fd7 with commit message: MB-46112 : Log feed name when StartGocbcoreDCPFeed fails

            Build couchbase-server-7.0.0-5109 contains cbgt commit 52db00a with commit message:
            MB-46112: Notify manager only on agent error on feed initiation

            build-team Couchbase Build Team added a comment - Build couchbase-server-7.0.0-5109 contains cbgt commit 52db00a with commit message: MB-46112 : Notify manager only on agent error on feed initiation

            Hoping the above fixes will address this issue. A build should become available soon with them.

            abhinav Abhinav Dangeti added a comment - Hoping the above fixes will address this issue. A build should become available soon with them.

            People

              ashwin.govindarajulu Ashwin Govindarajulu
              ashwin.govindarajulu Ashwin Govindarajulu
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty