Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-28995

Rebalance fail with {service_rebalance_failed,cbas, {linked_process_died}

    XMLWordPrintable

Details

    • Bug
    • Resolution: Fixed
    • Critical
    • 5.5.0
    • 5.5.0
    • analytics
    • component test cluster

    Description

      Rebalance in failed with the following error while running system test

      Rebalance exited with reason {service_rebalance_failed,cbas, {linked_process_died,<21349.12583.47>, {badmatch, {false, {topology,[], [<<"9f0884d4b3eb1e9b1a31ba60fa515ba2">>, <<"322452621c4f50bfcc213fe10b97429a">>], false,[]}, {topology,[], [<<"9f0884d4b3eb1e9b1a31ba60fa515ba2">>, <<"322452621c4f50bfcc213fe10b97429a">>], true,[]}}}}}

      We run following steps in loop for system test. Error occur after 17 hours of run 

      Steps:

      1. Create 4 node cluster with 2 KV,2 CBAS nodes
      2. Create 4 data buckets and 4 corresponding buckets in CBAS
      3. Create 4 data sets and index on each bucket
      4. Start populating data 
      5. Run SELECT query (Note : These queries are running at the rate of 12 queries/sec during whole test )
      6. Connect all buckets and wait for some time for injection 
      7. Disconnect all the bucket
      8. Now rebalance out 1 analytics node
      9. Connect all buckets and wait for some time for injection 
      10. Disconnect all the bucket
      11. Now rebalance in 1 analytics node
      12. Connect all buckets and wait for some time for injection 
      13. Now do swap rebalance for analytics node
      14. Wait for some time
      15. kill analytics node added in the step 13
      16. Stop data population and query running on analytics node
      17. Disconnect bucket, drop index , drop dataset and drop bucket
      18. Now do the swap rebalance of the node added in Step 13 (remove it) and bring node remove in Step 8

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              michael.blow Michael Blow
              vikas.chaudhary Vikas Chaudhary
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty