Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-40536

[Ephemeral] Rebalance out failed with reaon "unnsafe nodes"

    XMLWordPrintable

Details

    Description

      Build: 6.6.0-7883

      Scenario:

      1. 7 node cluster, Ephemral bucket(replica=1)
      2. Load 250000 docs into the bucket (SyncWrites)
      3. Rebalance out 3 nodes from the cluster

        +----------------+----------+--------------+
        | Nodes          | Services | Status       |
        +----------------+----------+--------------+
        | 172.23.121.215 | kv       | Cluster node |
        | 172.23.105.5   | [u'kv']  | --- OUT ---> |
        | 172.23.105.173 | [u'kv']  | --- OUT ---> |
        | 172.23.105.200 | [u'kv']  | --- OUT ---> |
        | 172.23.105.163 | kv       | Cluster node |
        | 172.23.123.143 | kv       | Cluster node |
        | 172.23.123.161 | kv       | Cluster node |
        +----------------+----------+--------------+

      Observation:

      Rebalance failed with reason with reason,

      Rebalance exited with reason {pre_rebalance_janitor_run_failed,"default",
      {error,unsafe_nodes,['ns_1@172.23.105.163']}}.
      Rebalance Operation Id = bcddb51f49060183600aa7f4eaba6286

      Test log: http://qa.sc.couchbase.com/job/test_suite_executor-TAF/44667/consoleText

      Test case:

      ./testrunner -i /tmp/testexec.18646.ini sdk_retries=10,num_items=250000,GROUP=P0;durability,EXCLUDE_GROUP=not_for_ephemeral,durability=MAJORITY,bucket_type=ephemeral,rerun=False,get-cbcollect-info=True,collect_pcaps=True,log_level=info,upgrade_version=6.6.0-7883 -t rebalance_new.rebalance_out.RebalanceOutTests.rebalance_out_with_warming_up,value_size=1024,bucket_type=ephemeral,upgrade_version=6.6.0-7883,rerun=False,sdk_retries=10,GROUP=P0;durability,nodes_out=3,EXCLUDE_GROUP=not_for_ephemeral,max_verify=100000,get-cbcollect-info=False,replicas=1,durability=MAJORITY,log_level=debug,nodes_init=7,num_items=250000,infra_log_level=critical
      

      Attachments

        Activity

          People

            ashwin.govindarajulu Ashwin Govindarajulu
            ashwin.govindarajulu Ashwin Govindarajulu
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              PagerDuty