Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-29807

rebalance hang during upgrade from regular rebalance from 4.6.3-4136 to 5.1.1-5721

    XMLWordPrintable

Details

    • Bug
    • Resolution: Won't Fix
    • Major
    • 5.1.1
    • 5.1.1
    • couchbase-bucket, ns_server
    • None
    • 5.1.1-5721

    Description

      Script to Repro

      ./testrunner -i /tmp/upgrade1.ini -p get-cbcollect-info=True,upgrade_version=5.1.1-5721 -t newupgradetests.MultiNodesUpgradeTests.online_upgrade_with_regular_rebalance_with_pillowfight,initial_version=4.6.3-4136,nodes_init=3,run_with_views=True,run_view_query_iterations=1,num_items=1000000
      

      There was no extra quirks that were enabled.
      Log

      [2018-05-23 04:47:13,457] - [task:437] INFO - adding node 172.23.106.62:8091 to cluster
      [2018-05-23 04:47:13,458] - [rest_client:1236] INFO - adding remote node @172.23.106.62:8091 to this cluster @172.23.106.64:8091
      [2018-05-23 04:47:39,076] - [rest_client:1454] INFO - rebalance params : {'password': 'password', 'ejectedNodes': '', 'user': 'Administrator', 'knownNodes': u'ns_1@172.23.106.62,ns_1@172.23.106.75,ns_1@172.23.106.77,ns_1@172.23.106.64'}
      

      After rebalance in of 172.23.106.62, rebalance hung at 95.72% for more than 5 mins, thats when our automation test fails. Memcached being killed in the logs would be from framework.

      I tried to reproduce issue, was not successful.

      Attachments

        1. cpu.png
          cpu.png
          302 kB
        2. cpu on .64.png
          cpu on .64.png
          597 kB
        3. vbucket counts per node.png
          vbucket counts per node.png
          194 kB
        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            Aliaksey Artamonau Aliaksey Artamonau (Inactive)
            Balakumaran.Gopal Balakumaran Gopal
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty