Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-40536

[Ephemeral] Rebalance out failed with reaon "unnsafe nodes"

    XMLWordPrintable

    Details

      Description

      Build: 6.6.0-7883

      Scenario:

      1. 7 node cluster, Ephemral bucket(replica=1)
      2. Load 250000 docs into the bucket (SyncWrites)
      3. Rebalance out 3 nodes from the cluster

        +----------------+----------+--------------+
        | Nodes          | Services | Status       |
        +----------------+----------+--------------+
        | 172.23.121.215 | kv       | Cluster node |
        | 172.23.105.5   | [u'kv']  | --- OUT ---> |
        | 172.23.105.173 | [u'kv']  | --- OUT ---> |
        | 172.23.105.200 | [u'kv']  | --- OUT ---> |
        | 172.23.105.163 | kv       | Cluster node |
        | 172.23.123.143 | kv       | Cluster node |
        | 172.23.123.161 | kv       | Cluster node |
        +----------------+----------+--------------+

      Observation:

      Rebalance failed with reason with reason,

      Rebalance exited with reason {pre_rebalance_janitor_run_failed,"default",
      {error,unsafe_nodes,['ns_1@172.23.105.163']}}.
      Rebalance Operation Id = bcddb51f49060183600aa7f4eaba6286

      Test log: http://qa.sc.couchbase.com/job/test_suite_executor-TAF/44667/consoleText

      Test case:

      ./testrunner -i /tmp/testexec.18646.ini sdk_retries=10,num_items=250000,GROUP=P0;durability,EXCLUDE_GROUP=not_for_ephemeral,durability=MAJORITY,bucket_type=ephemeral,rerun=False,get-cbcollect-info=True,collect_pcaps=True,log_level=info,upgrade_version=6.6.0-7883 -t rebalance_new.rebalance_out.RebalanceOutTests.rebalance_out_with_warming_up,value_size=1024,bucket_type=ephemeral,upgrade_version=6.6.0-7883,rerun=False,sdk_retries=10,GROUP=P0;durability,nodes_out=3,EXCLUDE_GROUP=not_for_ephemeral,max_verify=100000,get-cbcollect-info=False,replicas=1,durability=MAJORITY,log_level=debug,nodes_init=7,num_items=250000,infra_log_level=critical
      

        Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

          ashwin.govindarajulu Ashwin Govindarajulu created issue -
          owend Daniel Owen made changes -
          Field Original Value New Value
          Assignee Daniel Owen [ owend ] Ashwin Govindarajulu [ ashwin.govindarajulu ]
          jwalker Jim Walker made changes -
          Assignee Ashwin Govindarajulu [ ashwin.govindarajulu ] Dave Finlay [ dfinlay ]
          ashwin.govindarajulu Ashwin Govindarajulu made changes -
          Assignee Dave Finlay [ dfinlay ] Ashwin Govindarajulu [ ashwin.govindarajulu ]
          ritam.sharma Ritam Sharma made changes -
          Priority Major [ 3 ] Critical [ 2 ]
          ashwin.govindarajulu Ashwin Govindarajulu made changes -
          Labels 6.6.0 durability functional-test 6.6.0 Testing durability functional-test
          ashwin.govindarajulu Ashwin Govindarajulu made changes -
          Resolution User Error [ 10100 ]
          Status Open [ 1 ] Closed [ 6 ]

            People

            Assignee:
            ashwin.govindarajulu Ashwin Govindarajulu
            Reporter:
            ashwin.govindarajulu Ashwin Govindarajulu
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved:

                Gerrit Reviews

                There are no open Gerrit changes

                  PagerDuty