./testrunner -i /tmp/testexec.2031.ini num_items=10000,doc_size=512,transaction_timeout=150,durability=PERSIST_TO_MAJORITY,atomicity=True,GROUP=P0_Set1,rerun=False,upgrade_version=6.6.0-7873 -t rebalance_new.swaprebalancetests.SwapRebalanceBasicTests.do_test,nodes_init=3,replicas=2,standard_buckets=3,num-swap=1,GROUP=P1;P0_Set1

Test Input params:
{'doc_size': '512', 'standard_buckets': '3', 'transaction_timeout': '150', 'conf_file': 'conf/Atomicity/rebalance/swap_rebalance.conf', 'upgrade_version': '6.6.0-7873', 'spec': 'swap_rebalance', 'rerun': 'False', 'num_nodes': 7, 'num-swap': '1', 'GROUP': 'P0_Set1', 'case_number': 1, 'cluster_name': 'testexec.2031', 'atomicity': 'True', 'ini': '/tmp/testexec.2031.ini', 'replicas': '2', 'durability': 'PERSIST_TO_MAJORITY', 'logs_folder': '/data/workspace/centos-p0-transaction-vset00-00-swap_rebalance_durability_persist_to_majority_6.5_P0_Set1/logs/testrunner-20-Jul-13_10-45-30/test_1', 'nodes_init': '3', 'num_items': '10000'}

2020-07-13 10:45:31,469 | root  | WARNING | MainThread | [data_helper:<module>:31] Can not import concurrent module. Data for each server will be loaded/retrieved sequentially
do_test (rebalance_new.swaprebalancetests.SwapRebalanceBasicTests) ... 2020-07-13 10:45:32,980 | test  | INFO    | MainThread | [basetestcase:_BaseTestCase__log_setup_status:412] ========= basetestcase setup started for test #1 do_test =========

2020-07-13 10:45:34,592 | test  | INFO    | MainThread | [basetestcase:_BaseTestCase__log_setup_status:412] ========= basetestcase setup started for test #1 do_test =========

2020-07-13 10:46:09,039 | test  | INFO    | MainThread | [basetestcase:setUp:266] Initializing cluster
2020-07-13 10:46:09,371 | test  | WARNING | MainThread | [basetestcase:_initialize_nodes:443] RAM quota was defined less than 100 MB:

2020-07-13 10:46:12,424 | test  | INFO    | MainThread | [basetestcase:setUp:288] Couchbase Cluster: default, Nodes: 172.23.98.215, 172.23.123.111, 172.23.123.116, 172.23.123.115, 172.23.121.214, 172.23.122.78, 172.23.121.237 initialized
2020-07-13 10:46:12,447 | test  | INFO    | MainThread | [basetestcase:_BaseTestCase__log_setup_status:412] ========= basetestcase setup finished for test #1 do_test =========

2020-07-13 10:46:13,000 | test  | INFO    | MainThread | [basetestcase:sleep:514] Reason: None. Sleep for 5 secs ...

2020-07-13 10:46:22,631 | test  | INFO    | pool-2-thread-8 | [table_view:display:72] Rebalance Overview
+----------------+----------+--------------+
| Nodes          | Services | Status       |
+----------------+----------+--------------+
| 172.23.98.215  | kv       | Cluster node |
| 172.23.123.111 | None     | <--- IN ---  |
| 172.23.123.116 | None     | <--- IN ---  |
+----------------+----------+--------------+

2020-07-13 10:46:22,661 | test  | INFO    | pool-2-thread-8 | [task:check:304] Rebalance - status: running, progress: 0

2020-07-13 10:46:37,743 | test  | INFO    | pool-2-thread-8 | [task:check:304] Rebalance - status: none, progress: 100
2020-07-13 10:46:37,770 | test  | INFO    | pool-2-thread-8 | [task:check:363] Rebalance completed with progress: 100% in 15.1410000324 sec

2020-07-13 10:46:54,039 | test  | INFO    | MainThread | [data_helper:direct_client:268] creating direct client 172.23.123.111:11210 bucket-1
2020-07-13 10:46:54,275 | test  | INFO    | MainThread | [data_helper:direct_client:268] creating direct client 172.23.123.116:11210 bucket-1

2020-07-13 10:46:54,572 | test  | INFO    | MainThread | [data_helper:direct_client:268] creating direct client 172.23.98.215:11210 bucket-1
2020-07-13 10:46:54,917 | test  | INFO    | MainThread | [data_helper:direct_client:268] creating direct client 172.23.123.111:11210 bucket-1
2020-07-13 10:46:55,132 | test  | INFO    | MainThread | [data_helper:direct_client:268] creating direct client 172.23.123.116:11210 bucket-1
2020-07-13 10:46:55,328 | test  | INFO    | MainThread | [data_helper:direct_client:268] creating direct client 172.23.98.215:11210 bucket-1

2020-07-13 10:46:57,223 | test  | INFO    | MainThread | [data_helper:direct_client:268] creating direct client 172.23.123.111:11210 bucket-2

2020-07-13 10:46:58,433 | test  | INFO    | MainThread | [data_helper:direct_client:268] creating direct client 172.23.123.116:11210 bucket-2
2020-07-13 10:46:58,769 | test  | INFO    | MainThread | [data_helper:direct_client:268] creating direct client 172.23.98.215:11210 bucket-2
2020-07-13 10:46:59,088 | test  | INFO    | MainThread | [data_helper:direct_client:268] creating direct client 172.23.123.111:11210 bucket-2
2020-07-13 10:46:59,275 | test  | INFO    | MainThread | [data_helper:direct_client:268] creating direct client 172.23.123.116:11210 bucket-2

2020-07-13 10:46:59,470 | test  | INFO    | MainThread | [data_helper:direct_client:268] creating direct client 172.23.98.215:11210 bucket-2

2020-07-13 10:47:01,084 | test  | INFO    | MainThread | [data_helper:direct_client:268] creating direct client 172.23.123.111:11210 bucket-0
2020-07-13 10:47:01,269 | test  | INFO    | MainThread | [data_helper:direct_client:268] creating direct client 172.23.123.116:11210 bucket-0
2020-07-13 10:47:01,470 | test  | INFO    | MainThread | [data_helper:direct_client:268] creating direct client 172.23.98.215:11210 bucket-0
2020-07-13 10:47:01,914 | test  | INFO    | MainThread | [data_helper:direct_client:268] creating direct client 172.23.123.111:11210 bucket-0
2020-07-13 10:47:02,105 | test  | INFO    | MainThread | [data_helper:direct_client:268] creating direct client 172.23.123.116:11210 bucket-0
2020-07-13 10:47:02,280 | test  | INFO    | MainThread | [data_helper:direct_client:268] creating direct client 172.23.98.215:11210 bucket-0

2020-07-13 10:47:07,404 | test  | INFO    | pool-2-thread-5 | [task:call:3695] transaction timeout is 150
2020-07-13 10:47:07,450 | test  | INFO    | pool-2-thread-5 | [task:call:3699] transaction is com.couchbase.transactions.Transactions@633eaee0
2020-07-13 10:47:07,530 | test  | INFO    | pool-2-thread-4 | [task:call:3803] Starting Atomicity load generation thread
2020-07-13 10:47:07,530 | test  | INFO    | pool-2-thread-1 | [task:call:3803] Starting Atomicity load generation thread
2020-07-13 10:47:07,530 | test  | INFO    | pool-2-thread-7 | [task:call:3803] Starting Atomicity load generation thread
2020-07-13 10:47:07,536 | test  | INFO    | pool-2-thread-3 | [task:call:3803] Starting Atomicity load generation thread
2020-07-13 10:47:07,536 | test  | INFO    | pool-2-thread-8 | [task:call:3803] Starting Atomicity load generation thread
2020-07-13 10:47:07,536 | test  | INFO    | pool-2-thread-6 | [task:call:3803] Starting Atomicity load generation thread
2020-07-13 10:47:07,536 | test  | INFO    | pool-2-thread-2 | [task:call:3803] Starting Atomicity load generation thread
2020-07-13 10:47:07,536 | test  | INFO    | pool-2-thread-10 | [task:call:3803] Starting Atomicity load generation thread
TransactionsStarted: Transactions successfully started, regular cleanup enabled=true, lost cleanup enabled=true

TransactionCleanupEndRunEvent: start={bucket-0/_default/clientId=2244b,index=0,numClients=1,ATRs={checking=1024,total=1024},runLength=60000millis},end={taken=60014millis,ATRs={read=1024,errored=0,present=0},entries={expired=0,failed=0,total=0},maxEntries=0}

2020-07-13 10:48:17,062 | test  | INFO    | pool-2-thread-3 | [task:call:3931] Atomicity Load generation thread completed
2020-07-13 10:48:17,065 | test  | INFO    | pool-2-thread-3 | [task:call:3937] Starting Atomicity Verification thread
2020-07-13 10:48:17,253 | test  | INFO    | pool-2-thread-7 | [task:call:3931] Atomicity Load generation thread completed
2020-07-13 10:48:17,256 | test  | INFO    | pool-2-thread-7 | [task:call:3937] Starting Atomicity Verification thread

2020-07-13 10:48:17,538 | test  | INFO    | pool-2-thread-6 | [task:call:3931] Atomicity Load generation thread completed
2020-07-13 10:48:17,539 | test  | INFO    | pool-2-thread-6 | [task:call:3937] Starting Atomicity Verification thread
2020-07-13 10:48:17,588 | test  | INFO    | pool-2-thread-1 | [task:call:3931] Atomicity Load generation thread completed
2020-07-13 10:48:17,588 | test  | INFO    | pool-2-thread-1 | [task:call:3937] Starting Atomicity Verification thread
2020-07-13 10:48:17,601 | test  | INFO    | pool-2-thread-8 | [task:call:3931] Atomicity Load generation thread completed
2020-07-13 10:48:17,604 | test  | INFO    | pool-2-thread-8 | [task:call:3937] Starting Atomicity Verification thread
2020-07-13 10:48:18,042 | test  | INFO    | pool-2-thread-4 | [task:call:3931] Atomicity Load generation thread completed
2020-07-13 10:48:18,042 | test  | INFO    | pool-2-thread-4 | [task:call:3937] Starting Atomicity Verification thread
2020-07-13 10:48:18,714 | test  | INFO    | pool-2-thread-2 | [task:call:3931] Atomicity Load generation thread completed
2020-07-13 10:48:18,716 | test  | INFO    | pool-2-thread-2 | [task:call:3937] Starting Atomicity Verification thread
2020-07-13 10:48:18,878 | test  | INFO    | pool-2-thread-7 | [task:call:3957] Completed Atomicity Verification thread
2020-07-13 10:48:18,895 | test  | INFO    | pool-2-thread-3 | [task:call:3957] Completed Atomicity Verification thread
2020-07-13 10:48:19,130 | test  | INFO    | pool-2-thread-6 | [task:call:3957] Completed Atomicity Verification thread
2020-07-13 10:48:19,148 | test  | INFO    | pool-2-thread-1 | [task:call:3957] Completed Atomicity Verification thread
2020-07-13 10:48:19,180 | test  | INFO    | pool-2-thread-8 | [task:call:3957] Completed Atomicity Verification thread
2020-07-13 10:48:19,200 | test  | INFO    | pool-2-thread-10 | [task:call:3931] Atomicity Load generation thread completed
2020-07-13 10:48:19,200 | test  | INFO    | pool-2-thread-10 | [task:call:3937] Starting Atomicity Verification thread

2020-07-13 10:48:19,364 | test  | INFO    | pool-2-thread-4 | [task:call:3957] Completed Atomicity Verification thread
2020-07-13 10:48:19,589 | test  | INFO    | pool-2-thread-2 | [task:call:3957] Completed Atomicity Verification thread
2020-07-13 10:48:19,891 | test  | INFO    | pool-2-thread-10 | [task:call:3957] Completed Atomicity Verification thread
2020-07-13 10:48:20,848 | test  | INFO    | MainThread | [table_view:display:72] Cluster statistics
+----------------+----------+-----------------+------------+------------+------------------------+-------------------+-----------------------+
| Node           | Services | CPU_utilization | Mem_total  | Mem_free   | Swap_mem_used          | Active / Replica  | Version               |
+----------------+----------+-----------------+------------+------------+------------------------+-------------------+-----------------------+
| 172.23.98.215  | kv       | 36.0655737705   | 4339630080 | 3580170240 | 79736832 / 3758092288  | 10214 / 20462     | 6.6.0-7873-enterprise |
| 172.23.123.111 | kv       | 18.6868686869   | 4201672704 | 3366232064 | 30670848 / 3758092288  | 10278 / 20473     | 6.6.0-7873-enterprise |
| 172.23.123.116 | kv       | 48.7046632124   | 4201672704 | 3422486528 | 116359168 / 3758092288 | 10020 / 20118     | 6.6.0-7873-enterprise |
+----------------+----------+-----------------+------------+------------+------------------------+-------------------+-----------------------+

2020-07-13 10:48:20,966 | test  | INFO    | MainThread | [table_view:display:72] Bucket statistics
+----------+---------+----------+------------+-----+-------+------------+----------+-----------+
| Bucket   | Type    | Replicas | Durability | TTL | Items | RAM Quota  | RAM Used | Disk Used |
+----------+---------+----------+------------+-----+-------+------------+----------+-----------+
| bucket-0 | membase | 2        | none       | 0   | 9920  | 1541406720 | 90595104 | 470784378 |
| bucket-1 | membase | 2        | none       | 0   | 10673 | 1541406720 | 95380544 | 443875257 |
| bucket-2 | membase | 2        | none       | 0   | 9919  | 1541406720 | 90737664 | 472684883 |
+----------+---------+----------+------------+-----+-------+------------+----------+-----------+

2020-07-13 10:48:20,967 | test  | INFO    | MainThread | [rebalance_base:setUp:85] ==========Finished rebalance base setup========
2020-07-13 10:48:20,967 | test  | INFO    | MainThread | [swaprebalancetests:setUp:21] === SwapRebalanceBase setup started for test #1 do_test ===
2020-07-13 10:48:20,990 | test  | INFO    | MainThread | [swaprebalancetests:setUp:44] picking server : ip:172.23.98.215 port:8091 ssh_username:root as the master
2020-07-13 10:48:21,450 | test  | INFO    | MainThread | [swaprebalancetests:enable_diag_eval_on_non_local_hosts:90] Enabled diag/eval for non-local hosts from 172.23.98.215
2020-07-13 10:48:21,493 | test  | INFO    | MainThread | [swaprebalancetests:setUp:69] === SwapRebalanceBase setup done for test #1 do_test ===

2020-07-13 10:48:23,153 | test  | INFO    | pool-2-thread-9 | [task:call:3695] transaction timeout is 150
2020-07-13 10:48:23,154 | test  | INFO    | pool-2-thread-9 | [task:call:3699] transaction is com.couchbase.transactions.Transactions@704af476
TransactionsStarted: Transactions successfully started, regular cleanup enabled=true, lost cleanup enabled=true
2020-07-13 10:48:23,194 | test  | INFO    | pool-2-thread-6 | [task:call:3803] Starting Atomicity load generation thread
2020-07-13 10:48:23,194 | test  | INFO    | pool-2-thread-8 | [task:call:3803] Starting Atomicity load generation thread
2020-07-13 10:48:23,194 | test  | INFO    | pool-2-thread-7 | [task:call:3803] Starting Atomicity load generation thread
2020-07-13 10:48:23,194 | test  | INFO    | pool-2-thread-3 | [task:call:3803] Starting Atomicity load generation thread
2020-07-13 10:48:23,194 | test  | INFO    | pool-2-thread-1 | [task:call:3803] Starting Atomicity load generation thread
2020-07-13 10:48:23,196 | test  | INFO    | pool-2-thread-4 | [task:call:3803] Starting Atomicity load generation thread
2020-07-13 10:48:23,196 | test  | INFO    | pool-2-thread-2 | [task:call:3803] Starting Atomicity load generation thread
2020-07-13 10:48:23,197 | test  | INFO    | pool-2-thread-10 | [task:call:3803] Starting Atomicity load generation thread
2020-07-13 10:48:23,239 | test  | INFO    | MainThread | [swaprebalancetests:_common_test_body_swap_rebalance:130] Current nodes: [u'ns_1@172.23.98.215', u'ns_1@172.23.123.111', u'ns_1@172.23.123.116']
2020-07-13 10:48:23,332 | test  | INFO    | MainThread | [swaprebalancetests:_common_test_body_swap_rebalance:145] removing node ns_1@172.23.123.111 and rebalance afterwards

2020-07-13 10:48:25,760 | test  | INFO    | MainThread | [swaprebalancetests:_common_test_body_swap_rebalance:181] Swap Rebalance phase

2020-07-13 10:48:35,874 | test  | ERROR   | MainThread | [rest_client:_rebalance_status_and_progress:1470] {u'errorMessage': u'Rebalance failed. See logs for detailed reason. You can try again.', u'status': u'none'} - rebalance failed
2020-07-13 10:48:35,908 | test  | INFO    | MainThread | [rest_client:print_UI_logs:2526] Latest logs from UI on 172.23.98.215:
2020-07-13 10:48:35,910 | test  | ERROR   | MainThread | [rest_client:print_UI_logs:2528] {u'code': 0, u'module': u'ns_orchestrator', u'type': u'critical', u'node': u'ns_1@172.23.98.215', u'tstamp': 1594662509126L, u'shortText': u'message', u'serverTime': u'2020-07-13T10:48:29.126Z', u'text': u'Rebalance exited with reason {mover_crashed,\n                              {unexpected_exit,\n                               {\'EXIT\',<0.13721.0>,\n                                {{wait_seqno_persisted_failed,"bucket-1",341,\n                                  92,\n                                  [{\'ns_1@172.23.123.115\',\n                                    {\'EXIT\',\n                                     {{{{{badmatch,\n                                          [{<25251.5355.0>,\n                                            {done,exit,\n                                             {socket_closed,\n                                              {gen_server,call,\n                                               [<25251.5288.0>,\n                                                {setup_streams,[339,340,341]},\n                                                infinity]}},\n                                             [{gen_server,call,3,\n                                               [{file,"gen_server.erl"},\n                                                {line,214}]},\n                                              {dcp_replicator,\n                                               \'-spawn_and_wait/1-fun-0-\',1,\n                                               [{file,\n                                                 "src/dcp_replicator.erl"},\n                                                {line,243}]}]}}]},\n                                         [{misc,\n                                           sync_shutdown_many_i_am_trapping_exits,\n                                           1,\n                                           [{file,"src/misc.erl"},\n                                            {line,1374}]},\n                                          {dcp_replicator,spawn_and_wait,1,\n                                           [{file,"src/dcp_replicator.erl"},\n                                            {line,265}]},\n                                          {dcp_replicator,handle_call,3,\n                                           [{file,"src/dcp_replicator.erl"},\n                                            {line,121}]},\n                                          {gen_server,try_handle_call,4,\n                                           [{file,"gen_server.erl"},\n                                            {line,636}]},\n                                          {gen_server,handle_msg,6,\n                                           [{file,"gen_server.erl"},\n                                            {line,665}]},\n                                          {proc_lib,init_p_do_apply,3,\n                                           [{file,"proc_lib.erl"},\n                                            {line,247}]}]},\n                                        {gen_server,call,\n                                         [<25251.5287.0>,get_partitions,\n                                          infinity]}},\n                                       {gen_server,call,\n                                        [\'dcp_replication_manager-bucket-1\',\n                                         {get_replicator_pid,340},\n                                         infinity]}},\n                                      {gen_server,call,\n                                       [{\'janitor_agent-bucket-1\',\n                                         \'ns_1@172.23.123.115\'},\n                                        {if_rebalance,<0.13522.0>,\n                                         {wait_seqno_persisted,341,92}},\n                                        infinity]}}}}]},\n                                 [{ns_single_vbucket_mover,\n                                   \'-wait_seqno_persisted_many/5-fun-2-\',5,\n                                   [{file,"src/ns_single_vbucket_mover.erl"},\n                                    {line,488}]},\n                                  {proc_lib,init_p,3,\n                                   [{file,"proc_lib.erl"},{line,232}]}]}}}}.\nRebalance Operation Id = 2c18c85d730aa4c35cdf7f72b414ddd4'}
2020-07-13 10:48:35,911 | test  | ERROR   | MainThread | [rest_client:print_UI_logs:2528] {u'code': 0, u'module': u'ns_vbucket_mover', u'type': u'critical', u'node': u'ns_1@172.23.98.215', u'tstamp': 1594662509116L, u'shortText': u'message', u'serverTime': u'2020-07-13T10:48:29.116Z', u'text': u'Worker <0.13559.0> (for action {move,{341,\n                                      [\'ns_1@172.23.123.111\',\n                                       \'ns_1@172.23.98.215\',\n                                       \'ns_1@172.23.123.116\'],\n                                      [\'ns_1@172.23.123.115\',\n                                       \'ns_1@172.23.98.215\',\n                                       \'ns_1@172.23.123.116\'],\n                                      []}}) exited with reason {unexpected_exit,\n                                                                {\'EXIT\',\n                                                                 <0.13721.0>,\n                                                                 {{wait_seqno_persisted_failed,\n                                                                   "bucket-1",\n                                                                   341,92,\n                                                                   [{\'ns_1@172.23.123.115\',\n                                                                     {\'EXIT\',\n                                                                      {{{{{badmatch,\n                                                                           [{<25251.5355.0>,\n                                                                             {done,\n                                                                              exit,\n                                                                              {socket_closed,\n                                                                               {gen_server,\n                                                                                call,\n                                                                                [<25251.5288.0>,\n                                                                                 {setup_streams,\n                                                                                  [339,\n                                                                                   340,\n                                                                                   341]},\n                                                                                 infinity]}},\n                                                                              [{gen_server,\n                                                                                call,\n                                                                                3,\n                                                                                [{file,\n                                                                                  "gen_server.erl"},\n                                                                                 {line,\n                                                                                  214}]},\n                                                                               {dcp_replicator,\n                                                                                \'-spawn_and_wait/1-fun-0-\',\n                                                                                1,\n                                                                                [{file,\n                                                                                  "src/dcp_replicator.erl"},\n                                                                                 {line,\n                                                                                  243}]}]}}]},\n                                                                          [{misc,\n                                                                            sync_shutdown_many_i_am_trapping_exits,\n                                                                            1,\n                                                                            [{file,\n                                                                              "src/misc.erl"},\n                                                                             {line,\n                                                                              1374}]},\n                                                                           {dcp_replicator,\n                                                                            spawn_and_wait,\n                                                                            1,\n                                                                            [{file,\n                                                                              "src/dcp_replicator.erl"},\n                                                                             {line,\n                                                                              265}]},\n                                                                           {dcp_replicator,\n                                                                            handle_call,\n                                                                            3,\n                                                                            [{file,\n                                                                              "src/dcp_replicator.erl"},\n                                                                             {line,\n                                                                              121}]},\n                                                                           {gen_server,\n                                                                            try_handle_call,\n                                                                            4,\n                                                                            [{file,\n                                                                              "gen_server.erl"},\n                                                                             {line,\n                                                                              636}]},\n                                                                           {gen_server,\n                                                                            handle_msg,\n                                                                            6,\n                                                                            [{file,\n                                                                              "gen_server.erl"},\n                                                                             {line,\n                                                                              665}]},\n                                                                           {proc_lib,\n                                                                            init_p_do_apply,\n                                                                            3,\n                                                                            [{file,\n                                                                              "proc_lib.erl"},\n                                                                             {line,\n                                                                              247}]}]},\n                                                                         {gen_server,\n                                                                          call,\n                                                                          [<25251.5287.0>,\n                                                                           get_partitions,\n                                                                           infinity]}},\n                                                                        {gen_server,\n                                                                         call,\n                                                                         [\'dcp_replication_manager-bucket-1\',\n                                                                          {get_replicator_pid,\n                                                                           340},\n                                                                          infinity]}},\n                                                                       {gen_server,\n                                                                        call,\n                                                                        [{\'janitor_agent-bucket-1\',\n                                                                          \'ns_1@172.23.123.115\'},\n                                                                         {if_rebalance,\n                                                                          <0.13522.0>,\n                                                                          {wait_seqno_persisted,\n                                                                           341,\n                                                                           92}},\n                                                                         infinity]}}}}]},\n                                                                  [{ns_single_vbucket_mover,\n                                                                    \'-wait_seqno_persisted_many/5-fun-2-\',\n                                                                    5,\n                                                                    [{file,\n                                                                      "src/ns_single_vbucket_mover.erl"},\n                                                                     {line,\n                                                                      488}]},\n                                                                   {proc_lib,\n                                                                    init_p,3,\n                                                                    [{file,\n                                                                      "proc_lib.erl"},\n                                                                     {line,\n                                                                      232}]}]}}}'}
2020-07-13 10:48:35,915 | test  | ERROR   | MainThread | [rest_client:print_UI_logs:2528] {u'code': 0, u'module': u'ns_vbucket_mover', u'type': u'info', u'node': u'ns_1@172.23.98.215', u'tstamp': 1594662508265L, u'shortText': u'message', u'serverTime': u'2020-07-13T10:48:28.265Z', u'text': u'Bucket "bucket-1" rebalance appears to be swap rebalance'}
2020-07-13 10:48:35,917 | test  | ERROR   | MainThread | [rest_client:print_UI_logs:2528] {u'code': 0, u'module': u'ns_memcached', u'type': u'info', u'node': u'ns_1@172.23.123.115', u'tstamp': 1594662507117L, u'shortText': u'message', u'serverTime': u'2020-07-13T10:48:27.117Z', u'text': u'Bucket "bucket-1" loaded on node \'ns_1@172.23.123.115\' in 0 seconds.'}
2020-07-13 10:48:35,917 | test  | ERROR   | MainThread | [rest_client:print_UI_logs:2528] {u'code': 0, u'module': u'ns_rebalancer', u'type': u'info', u'node': u'ns_1@172.23.98.215', u'tstamp': 1594662506965L, u'shortText': u'message', u'serverTime': u'2020-07-13T10:48:26.965Z', u'text': u'Started rebalancing bucket bucket-1'}
2020-07-13 10:48:35,918 | test  | ERROR   | MainThread | [rest_client:print_UI_logs:2528] {u'code': 0, u'module': u'ns_orchestrator', u'type': u'info', u'node': u'ns_1@172.23.98.215', u'tstamp': 1594662506606L, u'shortText': u'message', u'serverTime': u'2020-07-13T10:48:26.606Z', u'text': u"Starting rebalance, KeepNodes = ['ns_1@172.23.98.215','ns_1@172.23.123.115',\n                                 'ns_1@172.23.123.116'], EjectNodes = ['ns_1@172.23.123.111'], Failed over and being ejected nodes = []; no delta recovery nodes; Operation Id = 2c18c85d730aa4c35cdf7f72b414ddd4"}
2020-07-13 10:48:35,918 | test  | ERROR   | MainThread | [rest_client:print_UI_logs:2528] {u'code': 0, u'module': u'memcached_config_mgr', u'type': u'info', u'node': u'ns_1@172.23.123.115', u'tstamp': 1594662506581L, u'shortText': u'message', u'serverTime': u'2020-07-13T10:48:26.581Z', u'text': u'Hot-reloaded memcached.json for config change of the following keys: [<<"scramsha_fallback_salt">>]'}
2020-07-13 10:48:35,920 | test  | ERROR   | MainThread | [rest_client:print_UI_logs:2528] {u'code': 3, u'module': u'ns_cluster', u'type': u'info', u'node': u'ns_1@172.23.123.115', u'tstamp': 1594662506534L, u'shortText': u'message', u'serverTime': u'2020-07-13T10:48:26.534Z', u'text': u'Node ns_1@172.23.123.115 joined cluster'}
2020-07-13 10:48:35,921 | test  | ERROR   | MainThread | [rest_client:print_UI_logs:2528] {u'code': 1, u'module': u'menelaus_sup', u'type': u'info', u'node': u'ns_1@172.23.123.115', u'tstamp': 1594662506500L, u'shortText': u'web start ok', u'serverTime': u'2020-07-13T10:48:26.500Z', u'text': u'Couchbase Server has started on web port 8091 on node \'ns_1@172.23.123.115\'. Version: "6.6.0-7873-enterprise".'}
2020-07-13 10:48:35,921 | test  | ERROR   | MainThread | [rest_client:print_UI_logs:2528] {u'code': 4, u'module': u'ns_node_disco', u'type': u'info', u'node': u'ns_1@172.23.123.111', u'tstamp': 1594662504733L, u'shortText': u'node up', u'serverTime': u'2020-07-13T10:48:24.733Z', u'text': u"Node 'ns_1@172.23.123.111' saw that node 'ns_1@172.23.123.115' came up. Tags: []"}
ERROR
2020-07-13 10:48:35,982 | test  | INFO    | MainThread | [table_view:display:72] Cluster statistics
+----------------+----------+-----------------+------------+------------+------------------------+-------------------+-----------------------+
| Node           | Services | CPU_utilization | Mem_total  | Mem_free   | Swap_mem_used          | Active / Replica  | Version               |
+----------------+----------+-----------------+------------+------------+------------------------+-------------------+-----------------------+
| 172.23.98.215  | kv       | 42.0765027322   | 4339630080 | 3543969792 | 79736832 / 3758092288  | 10420 / 20874     | 6.6.0-7873-enterprise |
| 172.23.123.115 | kv       | 0.502512562814  | 4201672704 | 3456155648 | 79462400 / 3758092288  | 0 / 17            | 6.6.0-7873-enterprise |
| 172.23.123.111 | kv       | 7.84810126582   | 4201672704 | 3273207808 | 30670848 / 3758092288  | 10736 / 21427     | 6.6.0-7873-enterprise |
| 172.23.123.116 | kv       | 14.1772151899   | 4201672704 | 3307180032 | 116359168 / 3758092288 | 10733 / 21437     | 6.6.0-7873-enterprise |
+----------------+----------+-----------------+------------+------------+------------------------+-------------------+-----------------------+