Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-59758

[System Test] :- Index swap rebalance fails with "service_rebalance_failed,index, {worker_died, {'EXIT',<0.16944.5441>, {task_failed,rebalance, {service_error, <<"Post \"http://127.0.0.1:9102/buildRecoveredIndexesRebalance\": EOF"

    XMLWordPrintable

Details

    • Untriaged
    • Centos 64-bit
    • 0
    • Unknown

    Description

      Script to Repro

      ./sequoia -client 172.23.104.168:2375 -provider file:centos_second_cluster.yml -test tests/integration/7.6/test_7.6.yml -scope tests/integration/7.6/scope_7.6_magma.yml -scale 1 -repeat 0 -log_level 0 -version 7.6.0-1820 -skip_setup=false -skip_test=false -skip_teardown=true -skip_cleanup=false -continue=false -collect_on_error=false -stop_on_error=false -duration=0 -show_topology=true
      

      Step where it failed.

      [2023-11-21T06:50:40-08:00, sequoiatools/couchbase-cli:7.6:184aac] server-add -c 172.23.97.74:8091 --server-add https://172.23.120.81 -u Administrator -p password --server-add-username Administrator --server-add-password password --services index
      [2023-11-21T06:50:56-08:00, sequoiatools/couchbase-cli:7.6:87c560] rebalance -c 172.23.97.74:8091 --server-remove 172.23.123.31 -u Administrator -p password
       
      Error occurred on container - sequoiatools/couchbase-cli:7.6:[rebalance -c 172.23.97.74:8091 --server-remove 172.23.123.31 -u Administrator -p password]
       
      docker logs 87c560
      docker start 87c560
       
      *Unable to display progress bar on this os
      JERROR: Rebalance failed. See logs for detailed reason. You can try again.
      

      172.23.123.33 6:50:57 AM 21 Nov, 2023

      Starting rebalance, KeepNodes = ['ns_1@172.23.106.134','ns_1@172.23.120.58',
      'ns_1@172.23.120.73','ns_1@172.23.120.74',
      'ns_1@172.23.120.75','ns_1@172.23.120.77',
      'ns_1@172.23.120.81','ns_1@172.23.120.86',
      'ns_1@172.23.121.77','ns_1@172.23.123.25',
      'ns_1@172.23.123.26','ns_1@172.23.123.32',
      'ns_1@172.23.123.33','ns_1@172.23.96.122',
      'ns_1@172.23.96.14','ns_1@172.23.96.243',
      'ns_1@172.23.96.254','ns_1@172.23.96.48',
      'ns_1@172.23.97.105','ns_1@172.23.97.110',
      'ns_1@172.23.97.112','ns_1@172.23.97.148',
      'ns_1@172.23.97.149','ns_1@172.23.97.150',
      'ns_1@172.23.97.241','ns_1@172.23.97.74'], EjectNodes = ['ns_1@172.23.123.31'], Failed over and being ejected nodes = []; no delta recovery nodes; Operation Id = 21fdc7636fd7e6e65f5bc41c9cf11681
      

      172.23.123.33 6:53:08 AM 21 Nov, 2023

      Rebalance exited with reason {service_rebalance_failed,index,
      {worker_died,
      {'EXIT',<0.16944.5441>,
      {task_failed,rebalance,
      {service_error,
      <<"Post \"http://127.0.0.1:9102/buildRecoveredIndexesRebalance\": EOF">>}}}}}.
      Rebalance Operation Id = 21fdc7636fd7e6e65f5bc41c9cf11681
      

      cbcollect_info attached.

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            Balakumaran.Gopal Balakumaran Gopal
            Balakumaran.Gopal Balakumaran Gopal
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty