Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-7906

Rebalancing fails when adding node to cluster with one node and empty bucket

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Cannot Reproduce
    • Affects Version/s: 2.0
    • Fix Version/s: 2.1.0
    • Component/s: couchbase-bucket
    • Security Level: Public
    • Labels:
    • Environment:
      Two virtual Windows 2008 R2 hosted in Hyper-V on Windows 8

      Description

      This is for a clean setup.

      1) Create one node with default bucket or other bucket
      2) Create a second node and join the previous to form a cluster
      3) Hit rebalance.

      ==> Fails.

      Have been able to reproduce.

      "Workaround" (well not really applicable to have to delete buckets)

      1) Delete buckets
      2) Re-create bucket

      Attached the diagnostics file.

      //Daniel

      No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

        Hide
        dipti Dipti Borkar added a comment -

        Siri

        Our community user reported this issue on windows. Is this something you can look into when you get a chance?

        Show
        dipti Dipti Borkar added a comment - Siri Our community user reported this issue on windows. Is this something you can look into when you get a chance?
        Hide
        danielwertheim Daniel Wertheim added a comment -

        Hey,

        I can verify that I also get the same issue if virtualizing my Win2008R2 instances on VirtualBox instead of Hyper-V.

        //Daniel

        Show
        danielwertheim Daniel Wertheim added a comment - Hey, I can verify that I also get the same issue if virtualizing my Win2008R2 instances on VirtualBox instead of Hyper-V. //Daniel
        Hide
        danielwertheim Daniel Wertheim added a comment -

        Hi,

        I've added yet another issue (http://www.couchbase.com/issues/browse/MB-7977) which is about replication not working, but I guess it could be a related issue.

        //Daniel

        Show
        danielwertheim Daniel Wertheim added a comment - Hi, I've added yet another issue ( http://www.couchbase.com/issues/browse/MB-7977 ) which is about replication not working, but I guess it could be a related issue. //Daniel
        Hide
        maria Maria McDuff (Inactive) added a comment -

        per bug scrub: siri — this is actually a windows issue. is this addressed already in rc 2.0.1 windows build? if so, dipti can share with the community.
        thanks.

        Show
        maria Maria McDuff (Inactive) added a comment - per bug scrub: siri — this is actually a windows issue. is this addressed already in rc 2.0.1 windows build? if so, dipti can share with the community. thanks.
        Hide
        maria Maria McDuff (Inactive) added a comment -

        per bug scrub, pls repro and update this bug.

        Show
        maria Maria McDuff (Inactive) added a comment - per bug scrub, pls repro and update this bug.
        Hide
        thuan Thuan Nguyen added a comment -

        Build tested:
        couchbase server 2.0.0 GA community version on 2 windows server 2008 R2 (with windows update is up to date)

        Procedure:
        Follow steps above of user to reproduce the bug but I could not reproduce it.
        Create one node (called node A) with default bucket.
        Create another node (called node B) and add this node to node A
        Rebalance is finished without any failed

        Log from the run:

        Rebalance completed successfully. ns_orchestrator001 ns_1@10.3.2.143 15:39:45 - Thu Apr 18, 2013
        Bucket "default" rebalance does not seem to be swap rebalance ns_vbucket_mover000 ns_1@10.3.2.143 15:30:00 - Thu Apr 18, 2013
        Started rebalancing bucket default ns_rebalancer000 ns_1@10.3.2.143 15:29:57 - Thu Apr 18, 2013
        Starting rebalance, KeepNodes = ['ns_1@10.3.2.143','ns_1@10.3.3.211'], EjectNodes = [] ns_orchestrator004 ns_1@10.3.2.143 15:29:52 - Thu Apr 18, 2013
        Node 'ns_1@10.3.2.143' saw that node 'ns_1@10.3.3.211' came up. ns_node_disco004 ns_1@10.3.2.143 15:29:19 - Thu Apr 18, 2013
        Started node add transaction by adding node 'ns_1@10.3.3.211' to nodes_wanted ns_cluster000 ns_1@10.3.2.143 15:29:03 - Thu Apr 18, 2013
        Bucket "default" loaded on node 'ns_1@10.3.3.211' in 0 seconds. ns_memcached001 ns_1@10.3.3.211 15:27:09 - Thu Apr 18, 2013
        Deleting old data files of bucket "default" ns_storage_conf000 ns_1@10.3.3.211 15:27:04 - Thu Apr 18, 2013
        Node ns_1@10.3.3.211 joined cluster ns_cluster003 ns_1@10.3.3.211 15:26:31 - Thu Apr 18, 2013
        Couchbase Server has started on web port 8091 on node 'ns_1@10.3.3.211'. menelaus_sup001 ns_1@10.3.3.211 15:26:31 - Thu Apr 18, 2013
        Bucket "default" loaded on node 'ns_1@10.3.2.143' in 0 seconds. ns_memcached001 ns_1@10.3.2.143 15:15:11 - Thu Apr 18, 2013
        Created bucket "default" of type: membase
        [

        {num_replicas,1}

        ,

        {replica_index,false}

        ,

        {ram_quota,2576351232}

        ,

        {auth_type,sasl}

        ,

        {flush_enabled,false}

        ] menelaus_web012 ns_1@10.3.2.143 15:15:10 - Thu Apr 18, 2013
        Couchbase Server has started on web port 8091 on node 'ns_1@10.3.2.143'. menelaus_sup001 ns_1@10.3.2.143 15:15:05 - Thu Apr 18, 2013
        I'm the only node, so I'm the master. mb_master000 ns_1@10.3.2.143 15:15:05 - Thu Apr 18, 2013
        Couchbase Server has started on web port 8091 on node 'ns_1@10.3.2.143'. menelaus_sup001 ns_1@10.3.2.143 15:12:55 - Thu Apr 18, 2013
        I'm the only node, so I'm the master. mb_master000 ns_1@10.3.2.143 15:12:55 - Thu Apr 18, 2013

        Show
        thuan Thuan Nguyen added a comment - Build tested: couchbase server 2.0.0 GA community version on 2 windows server 2008 R2 (with windows update is up to date) Procedure: Follow steps above of user to reproduce the bug but I could not reproduce it. Create one node (called node A) with default bucket. Create another node (called node B) and add this node to node A Rebalance is finished without any failed Log from the run: Rebalance completed successfully. ns_orchestrator001 ns_1@10.3.2.143 15:39:45 - Thu Apr 18, 2013 Bucket "default" rebalance does not seem to be swap rebalance ns_vbucket_mover000 ns_1@10.3.2.143 15:30:00 - Thu Apr 18, 2013 Started rebalancing bucket default ns_rebalancer000 ns_1@10.3.2.143 15:29:57 - Thu Apr 18, 2013 Starting rebalance, KeepNodes = ['ns_1@10.3.2.143','ns_1@10.3.3.211'] , EjectNodes = [] ns_orchestrator004 ns_1@10.3.2.143 15:29:52 - Thu Apr 18, 2013 Node 'ns_1@10.3.2.143' saw that node 'ns_1@10.3.3.211' came up. ns_node_disco004 ns_1@10.3.2.143 15:29:19 - Thu Apr 18, 2013 Started node add transaction by adding node 'ns_1@10.3.3.211' to nodes_wanted ns_cluster000 ns_1@10.3.2.143 15:29:03 - Thu Apr 18, 2013 Bucket "default" loaded on node 'ns_1@10.3.3.211' in 0 seconds. ns_memcached001 ns_1@10.3.3.211 15:27:09 - Thu Apr 18, 2013 Deleting old data files of bucket "default" ns_storage_conf000 ns_1@10.3.3.211 15:27:04 - Thu Apr 18, 2013 Node ns_1@10.3.3.211 joined cluster ns_cluster003 ns_1@10.3.3.211 15:26:31 - Thu Apr 18, 2013 Couchbase Server has started on web port 8091 on node 'ns_1@10.3.3.211'. menelaus_sup001 ns_1@10.3.3.211 15:26:31 - Thu Apr 18, 2013 Bucket "default" loaded on node 'ns_1@10.3.2.143' in 0 seconds. ns_memcached001 ns_1@10.3.2.143 15:15:11 - Thu Apr 18, 2013 Created bucket "default" of type: membase [ {num_replicas,1} , {replica_index,false} , {ram_quota,2576351232} , {auth_type,sasl} , {flush_enabled,false} ] menelaus_web012 ns_1@10.3.2.143 15:15:10 - Thu Apr 18, 2013 Couchbase Server has started on web port 8091 on node 'ns_1@10.3.2.143'. menelaus_sup001 ns_1@10.3.2.143 15:15:05 - Thu Apr 18, 2013 I'm the only node, so I'm the master. mb_master000 ns_1@10.3.2.143 15:15:05 - Thu Apr 18, 2013 Couchbase Server has started on web port 8091 on node 'ns_1@10.3.2.143'. menelaus_sup001 ns_1@10.3.2.143 15:12:55 - Thu Apr 18, 2013 I'm the only node, so I'm the master. mb_master000 ns_1@10.3.2.143 15:12:55 - Thu Apr 18, 2013
        Hide
        siri Sriram Melkote added a comment -

        Daniel - we can't reproduce this on our environment. Can you please attach cbcollect_info from both nodes in your environment? Please see http://www.couchbase.com/docs/couchbase-manual-2.0/couchbase-admin-cmdline-cbcollect_info.html for more details on generating the info package.

        Show
        siri Sriram Melkote added a comment - Daniel - we can't reproduce this on our environment. Can you please attach cbcollect_info from both nodes in your environment? Please see http://www.couchbase.com/docs/couchbase-manual-2.0/couchbase-admin-cmdline-cbcollect_info.html for more details on generating the info package.

          People

          • Assignee:
            thuan Thuan Nguyen
            Reporter:
            danielwertheim Daniel Wertheim
          • Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Gerrit Reviews

              There are no open Gerrit changes