Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-11543

rebalance out hang in centos 6.x 64-bit in build with openssl098

    XMLWordPrintable

Details

    • Bug
    • Resolution: Fixed
    • Critical
    • 3.0
    • 3.0
    • couchbase-bucket
    • Security Level: Public
    • None
    • Centos 6.4 64-bit
    • Untriaged
    • Centos 64-bit
    • Unknown

    Description

      Two centos 6.4 64-bit
      10.3.121.15
      10.3.121.16
      Install couchbase server 3.0.0-876
      Create default bucket on node 10.3.121.15
      Load 1K to default bucket
      Add node 10.3.121.16 to node 10.3.121.15
      Rebalance. Passed
      Kill erlang in both node to do warmup test. Passed
      Clean up. Remove node 10.3.121.16 out of cluster. Failed. Rebalance hang

      Cluster is in hang state now.

      Note:
      This bug may relate to bug MB-11459
      This warm up test passed in ubuntu 12.04 64-bit with build ubuntu_12_04_3.0.0-867 for openssl 1
      This warm up test also passed in ubuntu 12.04 64-bit with build 3.0.0-876 for openssl 0.9.8
      This warm up test passed in centos 6.4 64-bit with build centos6_3.0.0-858 for openssl 1 as show in log below

      [2014-06-23 12:14:39,097] - [cluster_helper:253] INFO - rebalancing all nodes in order to remove nodes
      [2014-06-23 12:14:39,104] - [rest_client:1087] INFO - rebalance params : password=password&ejectedNodes=ns_1%40172.23.105.182%2Cns_1%40172.23.105.210%2Cns_1%40172.23.105.132&user=Administrator&knownNodes=ns_1%40172.23.105.182%2Cns_1%40172.23.105.131%2Cns_1%40172.23.105.210%2Cns_1%40172.23.105.132
      [2014-06-23 12:14:39,123] - [rest_client:1091] INFO - rebalance operation started
      [2014-06-23 12:14:39,133] - [rest_client:1208] INFO - rebalance percentage : 0 %
      [2014-06-23 12:14:44,148] - [rest_client:1208] INFO - rebalance percentage : 7.1484375 %
      [2014-06-23 12:14:49,163] - [rest_client:1208] INFO - rebalance percentage : 12.0703125 %
      [2014-06-23 12:14:54,174] - [rest_client:1208] INFO - rebalance percentage : 14.8828125 %
      [2014-06-23 12:14:59,186] - [rest_client:1208] INFO - rebalance percentage : 22.265625 %
      [2014-06-23 12:15:04,210] - [rest_client:1208] INFO - rebalance percentage : 28.59375 %
      [2014-06-23 12:15:09,238] - [rest_client:1208] INFO - rebalance percentage : 30.8203125 %
      [2014-06-23 12:15:14,251] - [rest_client:1208] INFO - rebalance percentage : 37.03125 %
      [2014-06-23 12:15:19,265] - [rest_client:1208] INFO - rebalance percentage : 37.3828125 %
      [2014-06-23 12:15:24,281] - [rest_client:1208] INFO - rebalance percentage : 44.296875 %
      [2014-06-23 12:15:29,298] - [rest_client:1208] INFO - rebalance percentage : 50.0390625 %
      [2014-06-23 12:15:34,314] - [rest_client:1208] INFO - rebalance percentage : 52.5 %
      [2014-06-23 12:15:39,331] - [rest_client:1208] INFO - rebalance percentage : 59.765625 %
      [2014-06-23 12:15:44,349] - [rest_client:1208] INFO - rebalance percentage : 67.3828125 %
      [2014-06-23 12:15:49,366] - [rest_client:1208] INFO - rebalance percentage : 71.484375 %
      [2014-06-23 12:15:54,384] - [rest_client:1208] INFO - rebalance percentage : 80.859375 %
      [2014-06-23 12:15:59,400] - [rest_client:1208] INFO - rebalance percentage : 90.0390625 %
      [2014-06-23 12:16:04,416] - [rest_client:1208] INFO - rebalance percentage : 96.7578125 %
      [2014-06-23 12:16:09,432] - [rest_client:1208] INFO - rebalance percentage : 100 %
      [2014-06-23 12:16:14,461] - [rest_client:1208] INFO - rebalance percentage : 100 %
      [2014-06-23 12:16:19,478] - [rest_client:1164] INFO - rebalance progress took 100.354690075 seconds
      [2014-06-23 12:16:19,481] - [rest_client:1165] INFO - sleep for 10 seconds after rebalance...

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            mikew Mike Wiederhold [X] (Inactive)
            thuan Thuan Nguyen
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty