Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-50489

[CBSE][BP MB-46320 to 6.6.6]: Rebalance in 9 indexer nodes failed. agent_died

    XMLWordPrintable

Details

    • Bug
    • Resolution: Won't Fix
    • Critical
    • 6.6.6
    • Cheshire-Cat
    • secondary-index
    • 7.0.0-5161

    Description

      1. Create a cluster with 40 kv nodes, 35 index nodes, 5 query nodes
      2. Create a bucket and 100 collections.
      3. Create 50 indexes
      4. Start expiry data load. Start a query thread sending 1 QPS(running).
      5. Rebalance in 9 more indexer nodes.
        Rebalance failed:

        Rebalance exited with reason {service_rebalance_failed,index,
        {agent_died,<29555.14851.2>,
        {linked_process_died,<29555.14852.2>,
        {'ns_1@172.23.123.9',
        {no_connection,"index-service_api"}}}}}.
        Rebalance Operation Id = fec176ed9bb798a3af7b651f7d7a8b87
        

      QE Test

      git fetch "http://review.couchbase.org/TAF" refs/changes/34/151634/69 && git checkout FETCH_HEAD
       
      guides/gradlew --refresh-dependencies testrunner -P jython=/opt/jython/bin/jython -P 'args=-i /tmp/magma_temp_job4.ini -p bucket_storage=couchstore,bucket_eviction_policy=fullEviction,rerun=False -t aGoodDoctor.Hospital.Murphy.test_rebalance,nodes_init=40,graceful=True,skip_cleanup=True,num_items=500000000,num_buckets=1,bucket_names=GleamBook,doc_size=256,bucket_type=membase,eviction_policy=fullEviction,iterations=100,batch_size=1000,sdk_timeout=60,log_level=debug,infra_log_level=debug,rerun=False,skip_cleanup=True,key_size=18,randomize_doc_size=False,randomize_value=True,assert_crashes_on_load=True,num_collections=100,maxttl=1000,num_indexes=50,pc=1,sdk_client_pool=True,max_clients=10,index_nodes=35,query_nodes=5,cbas_nodes=0,fts_nodes=0,rebl_nodes=9 -m rest'
      
      

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              varun.velamuri Varun Velamuri
              kevin.cherkauer Kevin Cherkauer (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty