./testrunner -i b/resources/5-nodes-template.ini -p get-cbcollect-info=True,GROUP=bucket_op,skip_log_scan=False,bucket_storage=couchstore -t eventing.eventing_rebalance_collection.EventingRebalanceCollection.test_eventing_rebalance_with_multiple_kv_nodes,doc-per-day=20,dataset=default,nodes_init=5,services_init=kv-kv-kv-eventing-index:n1ql,groups=simple,reset_services=True,GROUP=bucket_op,java_sdk_client=True,get-cbcollect-info=True,skip_log_scan=False,bucket_storage=couchstore INFO:root:__main__ *** TestRunner *** INFO:__main__:TestRunner: parsing args... INFO:__main__:Checking arguments... INFO:__main__:TestRunner: start... INFO:__main__:Global Test input params: {'GROUP': 'bucket_op', 'bucket_storage': 'couchstore', 'cluster_name': '5-nodes-template', 'get-cbcollect-info': 'True', 'ini': 'b/resources/5-nodes-template.ini', 'num_nodes': 5, 'skip_log_scan': 'False'} INFO:__main__: Number of tests initially selected before GROUP filters: 1 Only cases in GROUPs 'bucket_op' will be executed INFO:__main__:--> Running test: eventing.eventing_rebalance_collection.EventingRebalanceCollection.test_eventing_rebalance_with_multiple_kv_nodes,doc-per-day=20,dataset=default,nodes_init=5,services_init=kv-kv-kv-eventing-index:n1ql,groups=simple,reset_services=True,GROUP=bucket_op,java_sdk_client=True,get-cbcollect-info=True,skip_log_scan=False,bucket_storage=couchstore INFO:__main__:Logs folder: /Users/chanabasappaghali/testrunner_mad-hatter/testrunner_latest/testrunner/logs/testrunner-21-May-05_19-11-08/test_1 Logs will be stored at /Users/chanabasappaghali/testrunner_mad-hatter/testrunner_latest/testrunner/logs/testrunner-21-May-05_19-11-08/test_1 ./testrunner -i b/resources/5-nodes-template.ini -p get-cbcollect-info=True,GROUP=bucket_op,skip_log_scan=False,bucket_storage=couchstore -t eventing.eventing_rebalance_collection.EventingRebalanceCollection.test_eventing_rebalance_with_multiple_kv_nodes,doc-per-day=20,dataset=default,nodes_init=5,services_init=kv-kv-kv-eventing-index:n1ql,groups=simple,reset_services=True,GROUP=bucket_op,java_sdk_client=True,get-cbcollect-info=True,skip_log_scan=False,bucket_storage=couchstore Test Input params: {'doc-per-day': '20', 'dataset': 'default', 'nodes_init': '5', 'services_init': 'kv-kv-kv-eventing-index:n1ql', 'groups': 'simple', 'reset_services': 'True', 'GROUP': 'bucket_op', 'java_sdk_client': 'True', 'get-cbcollect-info': 'True', 'skip_log_scan': 'False', 'bucket_storage': 'couchstore', 'ini': 'b/resources/5-nodes-template.ini', 'cluster_name': '5-nodes-template', 'num_nodes': 5, 'case_number': 1, 'logs_folder': '/Users/chanabasappaghali/testrunner_mad-hatter/testrunner_latest/testrunner/logs/testrunner-21-May-05_19-11-08/test_1'} Run before suite setup for eventing.eventing_rebalance_collection.EventingRebalanceCollection.test_eventing_rebalance_with_multiple_kv_nodes -->before_suite_name:eventing.eventing_rebalance_collection.EventingRebalanceCollection.suite_setUp,suite: ]> suite_setUp (eventing.eventing_rebalance_collection.EventingRebalanceCollection) ... 2021-05-05 19:11:08 | INFO | MainProcess | MainThread | [eventing_base.setUp] ============== EventingBaseTest setup has started ============== 2021-05-05 19:11:08 | INFO | MainProcess | MainThread | [eventing_base.setUp] Starting Test: suite_setUp 2021-05-05 19:11:08 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.101 with username:root, attempt#1 of 5 2021-05-05 19:11:08 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.101 as root 2021-05-05 19:11:08 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:11:08 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:11:08 | INFO | MainProcess | MainThread | [rest_client.is_ns_server_running] -->is_ns_server_running? 2021-05-05 19:11:08 | INFO | MainProcess | MainThread | [rest_client.get_nodes_version] Node version in cluster 7.0.0-5102-enterprise 2021-05-05 19:11:08 | INFO | MainProcess | MainThread | [rest_client.get_nodes_versions] Node versions in cluster ['7.0.0-5102-enterprise', '7.0.0-5102-enterprise', '7.0.0-5102-enterprise', '7.0.0-5102-enterprise', '7.0.0-5102-enterprise'] 2021-05-05 19:11:08 | INFO | MainProcess | MainThread | [basetestcase.setUp] ============== basetestcase setup was started for test #1 suite_setUp============== 2021-05-05 19:11:09 | INFO | MainProcess | MainThread | [bucket_helper.delete_all_buckets_or_assert] deleting existing buckets ['dst_bucket', 'metadata', 'src_bucket'] on 10.112.206.101 2021-05-05 19:11:10 | INFO | MainProcess | MainThread | [bucket_helper.wait_for_bucket_deletion] waiting for bucket deletion to complete.... 2021-05-05 19:11:10 | INFO | MainProcess | MainThread | [rest_client.bucket_exists] node 10.112.206.101 existing buckets : ['metadata', 'src_bucket'] 2021-05-05 19:11:10 | INFO | MainProcess | MainThread | [bucket_helper.delete_all_buckets_or_assert] deleted bucket : dst_bucket from 10.112.206.101 2021-05-05 19:11:10 | INFO | MainProcess | MainThread | [bucket_helper.wait_for_bucket_deletion] waiting for bucket deletion to complete.... 2021-05-05 19:11:10 | INFO | MainProcess | MainThread | [rest_client.bucket_exists] node 10.112.206.101 existing buckets : ['src_bucket'] 2021-05-05 19:11:10 | INFO | MainProcess | MainThread | [bucket_helper.delete_all_buckets_or_assert] deleted bucket : metadata from 10.112.206.101 2021-05-05 19:11:12 | INFO | MainProcess | MainThread | [bucket_helper.wait_for_bucket_deletion] waiting for bucket deletion to complete.... 2021-05-05 19:11:12 | INFO | MainProcess | MainThread | [rest_client.bucket_exists] node 10.112.206.101 existing buckets : [] 2021-05-05 19:11:12 | INFO | MainProcess | MainThread | [bucket_helper.delete_all_buckets_or_assert] deleted bucket : src_bucket from 10.112.206.101 2021-05-05 19:11:12 | INFO | MainProcess | MainThread | [bucket_helper.delete_all_buckets_or_assert] Could not find any buckets for node 10.112.206.102, nothing to delete 2021-05-05 19:11:12 | INFO | MainProcess | MainThread | [bucket_helper.delete_all_buckets_or_assert] Could not find any buckets for node 10.112.206.103, nothing to delete 2021-05-05 19:11:12 | INFO | MainProcess | MainThread | [bucket_helper.delete_all_buckets_or_assert] Could not find any buckets for node 10.112.206.104, nothing to delete 2021-05-05 19:11:12 | INFO | MainProcess | MainThread | [bucket_helper.delete_all_buckets_or_assert] Could not find any buckets for node 10.112.206.105, nothing to delete 2021-05-05 19:11:12 | INFO | MainProcess | MainThread | [rest_client.is_ns_server_running] -->is_ns_server_running? 2021-05-05 19:11:12 | INFO | MainProcess | MainThread | [cluster_helper.cleanup_cluster] rebalancing all nodes in order to remove nodes 2021-05-05 19:11:12 | INFO | MainProcess | MainThread | [rest_client.rebalance] rebalance params : {'knownNodes': 'ns_1@10.112.206.101,ns_1@10.112.206.102,ns_1@10.112.206.103,ns_1@10.112.206.104,ns_1@10.112.206.105', 'ejectedNodes': 'ns_1@10.112.206.102,ns_1@10.112.206.103,ns_1@10.112.206.104,ns_1@10.112.206.105', 'user': 'Administrator', 'password': 'password'} 2021-05-05 19:11:12 | INFO | MainProcess | MainThread | [rest_client.rebalance] rebalance operation started 2021-05-05 19:11:12 | INFO | MainProcess | MainThread | [rest_client._rebalance_status_and_progress] rebalance percentage : 0.00 % 2021-05-05 19:11:22 | INFO | MainProcess | MainThread | [rest_client._rebalance_status_and_progress] rebalance percentage : 70.00 % 2021-05-05 19:11:32 | INFO | MainProcess | MainThread | [rest_client._rebalance_status_and_progress] rebalance percentage : 90.00 % 2021-05-05 19:11:52 | INFO | MainProcess | MainThread | [rest_client.monitorRebalance] rebalance progress took 40.02 seconds 2021-05-05 19:11:52 | INFO | MainProcess | MainThread | [rest_client.monitorRebalance] sleep for 10 seconds after rebalance... 2021-05-05 19:12:02 | INFO | MainProcess | MainThread | [cluster_helper.cleanup_cluster] removed all the nodes from cluster associated with ip:10.112.206.101 port:8091 ssh_username:root ? [('ns_1@10.112.206.102', 8091), ('ns_1@10.112.206.103', 8091), ('ns_1@10.112.206.104', 8091), ('ns_1@10.112.206.105', 8091)] 2021-05-05 19:12:02 | INFO | MainProcess | MainThread | [cluster_helper.wait_for_ns_servers_or_assert] waiting for ns_server @ 10.112.206.101:8091 2021-05-05 19:12:02 | INFO | MainProcess | MainThread | [rest_client.is_ns_server_running] -->is_ns_server_running? 2021-05-05 19:12:02 | INFO | MainProcess | MainThread | [cluster_helper.wait_for_ns_servers_or_assert] ns_server @ 10.112.206.101:8091 is running 2021-05-05 19:12:02 | INFO | MainProcess | MainThread | [cluster_helper.wait_for_ns_servers_or_assert] waiting for ns_server @ 10.112.206.102:8091 2021-05-05 19:12:02 | INFO | MainProcess | MainThread | [rest_client.is_ns_server_running] -->is_ns_server_running? 2021-05-05 19:12:02 | INFO | MainProcess | MainThread | [cluster_helper.wait_for_ns_servers_or_assert] ns_server @ 10.112.206.102:8091 is running 2021-05-05 19:12:02 | INFO | MainProcess | MainThread | [cluster_helper.wait_for_ns_servers_or_assert] waiting for ns_server @ 10.112.206.103:8091 2021-05-05 19:12:02 | INFO | MainProcess | MainThread | [rest_client.is_ns_server_running] -->is_ns_server_running? 2021-05-05 19:12:02 | INFO | MainProcess | MainThread | [cluster_helper.wait_for_ns_servers_or_assert] ns_server @ 10.112.206.103:8091 is running 2021-05-05 19:12:02 | INFO | MainProcess | MainThread | [cluster_helper.wait_for_ns_servers_or_assert] waiting for ns_server @ 10.112.206.104:8091 2021-05-05 19:12:02 | INFO | MainProcess | MainThread | [rest_client.is_ns_server_running] -->is_ns_server_running? 2021-05-05 19:12:02 | INFO | MainProcess | MainThread | [cluster_helper.wait_for_ns_servers_or_assert] ns_server @ 10.112.206.104:8091 is running 2021-05-05 19:12:02 | INFO | MainProcess | MainThread | [cluster_helper.wait_for_ns_servers_or_assert] waiting for ns_server @ 10.112.206.105:8091 2021-05-05 19:12:02 | INFO | MainProcess | MainThread | [rest_client.is_ns_server_running] -->is_ns_server_running? 2021-05-05 19:12:02 | INFO | MainProcess | MainThread | [cluster_helper.wait_for_ns_servers_or_assert] ns_server @ 10.112.206.105:8091 is running 2021-05-05 19:12:02 | INFO | MainProcess | MainThread | [basetestcase.setUp] Building docker image with java sdk client 2021-05-05 19:12:05 | INFO | MainProcess | MainThread | [basetestcase.get_nodes_from_services_map] cannot find service node eventing in cluster 2021-05-05 19:12:05 | INFO | MainProcess | MainThread | [eventing_base.tearDown] ============== EventingBaseTest tearDown has started ============== 2021-05-05 19:12:05 | INFO | MainProcess | MainThread | [basetestcase.get_nodes_from_services_map] cannot find service node eventing in cluster 2021-05-05 19:12:05 | INFO | MainProcess | MainThread | [basetestcase.print_cluster_stats] ------- Cluster statistics ------- 2021-05-05 19:12:05 | INFO | MainProcess | MainThread | [basetestcase.print_cluster_stats] 10.112.206.101:8091 => {'services': ['kv'], 'cpu_utilization': 1.634320735444331, 'mem_free': 1417535488, 'mem_total': 1930829824, 'swap_mem_used': 0, 'swap_mem_total': 1107292160} 2021-05-05 19:12:05 | INFO | MainProcess | MainThread | [basetestcase.print_cluster_stats] --- End of cluster statistics --- 2021-05-05 19:12:05 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.101 with username:root, attempt#1 of 5 2021-05-05 19:12:05 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.101 as root 2021-05-05 19:12:05 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:12:06 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:12:06 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.102 with username:root, attempt#1 of 5 2021-05-05 19:12:06 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.102 as root 2021-05-05 19:12:06 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:12:06 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:12:06 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.103 with username:root, attempt#1 of 5 2021-05-05 19:12:06 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.103 as root 2021-05-05 19:12:06 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:12:07 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:12:07 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.104 with username:root, attempt#1 of 5 2021-05-05 19:12:07 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.104 as root 2021-05-05 19:12:07 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:12:07 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:12:07 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.105 with username:root, attempt#1 of 5 2021-05-05 19:12:07 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.105 as root 2021-05-05 19:12:07 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:12:07 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:12:21 | INFO | MainProcess | MainThread | [basetestcase.tearDown] ============== basetestcase cleanup was started for test #1 suite_setUp ============== 2021-05-05 19:12:21 | WARNING | MainProcess | MainThread | [basetestcase.tearDown] Alerts were found: [{'msg': "The time on node 'ns_1@10.112.206.102' is not synchronized. Please ensure that NTP is set up correctly on all nodes and that clocks are synchronized.", 'serverTime': '2021-05-05T06:41:09.000Z', 'disableUIPopUp': False}, {'msg': "The time on node 'ns_1@10.112.206.103' is not synchronized. Please ensure that NTP is set up correctly on all nodes and that clocks are synchronized.", 'serverTime': '2021-05-05T06:41:36.000Z', 'disableUIPopUp': False}, {'msg': "The time on node 'ns_1@10.112.206.104' is not synchronized. Please ensure that NTP is set up correctly on all nodes and that clocks are synchronized.", 'serverTime': '2021-05-05T06:42:04.000Z', 'disableUIPopUp': False}, {'msg': "The time on node 'ns_1@10.112.206.105' is not synchronized. Please ensure that NTP is set up correctly on all nodes and that clocks are synchronized.", 'serverTime': '2021-05-05T06:42:32.000Z', 'disableUIPopUp': False}] 2021-05-05 19:12:21 | INFO | MainProcess | MainThread | [bucket_helper.delete_all_buckets_or_assert] Could not find any buckets for node 10.112.206.101, nothing to delete 2021-05-05 19:12:21 | INFO | MainProcess | MainThread | [bucket_helper.delete_all_buckets_or_assert] Could not find any buckets for node 10.112.206.102, nothing to delete 2021-05-05 19:12:21 | INFO | MainProcess | MainThread | [bucket_helper.delete_all_buckets_or_assert] Could not find any buckets for node 10.112.206.103, nothing to delete 2021-05-05 19:12:21 | INFO | MainProcess | MainThread | [bucket_helper.delete_all_buckets_or_assert] Could not find any buckets for node 10.112.206.104, nothing to delete 2021-05-05 19:12:21 | INFO | MainProcess | MainThread | [bucket_helper.delete_all_buckets_or_assert] Could not find any buckets for node 10.112.206.105, nothing to delete 2021-05-05 19:12:21 | INFO | MainProcess | MainThread | [rest_client.is_ns_server_running] -->is_ns_server_running? 2021-05-05 19:12:21 | INFO | MainProcess | MainThread | [cluster_helper.wait_for_ns_servers_or_assert] waiting for ns_server @ 10.112.206.101:8091 2021-05-05 19:12:21 | INFO | MainProcess | MainThread | [rest_client.is_ns_server_running] -->is_ns_server_running? 2021-05-05 19:12:21 | INFO | MainProcess | MainThread | [cluster_helper.wait_for_ns_servers_or_assert] ns_server @ 10.112.206.101:8091 is running 2021-05-05 19:12:21 | INFO | MainProcess | MainThread | [cluster_helper.wait_for_ns_servers_or_assert] waiting for ns_server @ 10.112.206.102:8091 2021-05-05 19:12:21 | INFO | MainProcess | MainThread | [rest_client.is_ns_server_running] -->is_ns_server_running? 2021-05-05 19:12:21 | INFO | MainProcess | MainThread | [cluster_helper.wait_for_ns_servers_or_assert] ns_server @ 10.112.206.102:8091 is running 2021-05-05 19:12:21 | INFO | MainProcess | MainThread | [cluster_helper.wait_for_ns_servers_or_assert] waiting for ns_server @ 10.112.206.103:8091 2021-05-05 19:12:21 | INFO | MainProcess | MainThread | [rest_client.is_ns_server_running] -->is_ns_server_running? 2021-05-05 19:12:21 | INFO | MainProcess | MainThread | [cluster_helper.wait_for_ns_servers_or_assert] ns_server @ 10.112.206.103:8091 is running 2021-05-05 19:12:21 | INFO | MainProcess | MainThread | [cluster_helper.wait_for_ns_servers_or_assert] waiting for ns_server @ 10.112.206.104:8091 2021-05-05 19:12:21 | INFO | MainProcess | MainThread | [rest_client.is_ns_server_running] -->is_ns_server_running? 2021-05-05 19:12:21 | INFO | MainProcess | MainThread | [cluster_helper.wait_for_ns_servers_or_assert] ns_server @ 10.112.206.104:8091 is running 2021-05-05 19:12:21 | INFO | MainProcess | MainThread | [cluster_helper.wait_for_ns_servers_or_assert] waiting for ns_server @ 10.112.206.105:8091 2021-05-05 19:12:21 | INFO | MainProcess | MainThread | [rest_client.is_ns_server_running] -->is_ns_server_running? 2021-05-05 19:12:21 | INFO | MainProcess | MainThread | [cluster_helper.wait_for_ns_servers_or_assert] ns_server @ 10.112.206.105:8091 is running 2021-05-05 19:12:21 | INFO | MainProcess | MainThread | [basetestcase.tearDown] ============== basetestcase cleanup was finished for test #1 suite_setUp ============== 2021-05-05 19:12:21 | INFO | MainProcess | MainThread | [basetestcase.tearDown] closing all ssh connections 2021-05-05 19:12:21 | INFO | MainProcess | MainThread | [basetestcase.tearDown] closing all memcached connections 2021-05-05 19:12:21 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.101 with username:root, attempt#1 of 5 2021-05-05 19:12:21 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.101 as root 2021-05-05 19:12:22 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:12:22 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:12:22 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.101 with username:root, attempt#1 of 5 2021-05-05 19:12:22 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.101 as root 2021-05-05 19:12:22 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:12:22 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:12:22 | INFO | MainProcess | MainThread | [remote_util.is_couchbase_installed] 10.112.206.101 **** The linux version file /opt/couchbase/ VERSION.txt exists 2021-05-05 19:12:22 | INFO | MainProcess | MainThread | [remote_util.stop_couchbase] Running systemd command on this server 2021-05-05 19:12:22 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] running command.raw on 10.112.206.101: systemctl stop couchbase-server.service 2021-05-05 19:12:24 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:12:24 | INFO | MainProcess | MainThread | [basetestcase.stop_server] Couchbase stopped 2021-05-05 19:12:24 | INFO | MainProcess | MainThread | [basetestcase.sleep] sleep for 5 secs. ... 2021-05-05 19:12:29 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] running command.raw on 10.112.206.101: rm -rf /opt/couchbase/var/lib/couchbase/data/* 2021-05-05 19:12:29 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:12:29 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] running command.raw on 10.112.206.101: rm -rf /opt/couchbase/var/lib/couchbase/config/* 2021-05-05 19:12:29 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:12:29 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.101 with username:root, attempt#1 of 5 2021-05-05 19:12:29 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.101 as root 2021-05-05 19:12:29 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:12:29 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:12:29 | INFO | MainProcess | MainThread | [remote_util.is_couchbase_installed] 10.112.206.101 **** The linux version file /opt/couchbase/ VERSION.txt exists 2021-05-05 19:12:29 | INFO | MainProcess | MainThread | [remote_util.is_process_running] Checking for process beam.smp on linux 2021-05-05 19:12:29 | INFO | MainProcess | MainThread | [remote_util.start_couchbase] Starting couchbase server 2021-05-05 19:12:29 | INFO | MainProcess | MainThread | [remote_util.start_couchbase] Running systemd command on this server 2021-05-05 19:12:29 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] running command.raw on 10.112.206.101: systemctl start couchbase-server.service 2021-05-05 19:12:30 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:12:30 | INFO | MainProcess | MainThread | [remote_util.sleep] 10.112.206.101:sleep for 5 secs. waiting for couchbase server to come up ... 2021-05-05 19:12:35 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] running command.raw on 10.112.206.101: systemctl status couchbase-server.service | grep ExecStop=/opt/couchbase/bin/couchbase-server 2021-05-05 19:12:35 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:12:35 | INFO | MainProcess | MainThread | [remote_util.start_couchbase] Couchbase server status: [' Process: 23696 ExecStop=/opt/couchbase/bin/couchbase-server -k (code=exited, status=0/SUCCESS)'] 2021-05-05 19:12:35 | INFO | MainProcess | MainThread | [remote_util.is_process_running] Checking for process beam.smp on linux 2021-05-05 19:12:35 | INFO | MainProcess | MainThread | [remote_util.is_process_running] process beam.smp is running on 10.112.206.101: with pid 23777 2021-05-05 19:12:35 | INFO | MainProcess | MainThread | [basetestcase.start_server] Couchbase started 2021-05-05 19:12:35 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.102 with username:root, attempt#1 of 5 2021-05-05 19:12:35 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.102 as root 2021-05-05 19:12:35 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:12:35 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:12:35 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.102 with username:root, attempt#1 of 5 2021-05-05 19:12:35 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.102 as root 2021-05-05 19:12:35 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:12:35 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:12:35 | INFO | MainProcess | MainThread | [remote_util.is_couchbase_installed] 10.112.206.102 **** The linux version file /opt/couchbase/ VERSION.txt exists 2021-05-05 19:12:36 | INFO | MainProcess | MainThread | [remote_util.stop_couchbase] Running systemd command on this server 2021-05-05 19:12:36 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] running command.raw on 10.112.206.102: systemctl stop couchbase-server.service 2021-05-05 19:12:37 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:12:37 | INFO | MainProcess | MainThread | [basetestcase.stop_server] Couchbase stopped 2021-05-05 19:12:37 | INFO | MainProcess | MainThread | [basetestcase.sleep] sleep for 5 secs. ... 2021-05-05 19:12:42 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] running command.raw on 10.112.206.102: rm -rf /opt/couchbase/var/lib/couchbase/data/* 2021-05-05 19:12:42 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:12:42 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] running command.raw on 10.112.206.102: rm -rf /opt/couchbase/var/lib/couchbase/config/* 2021-05-05 19:12:42 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:12:42 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.102 with username:root, attempt#1 of 5 2021-05-05 19:12:42 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.102 as root 2021-05-05 19:12:42 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:12:43 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:12:43 | INFO | MainProcess | MainThread | [remote_util.is_couchbase_installed] 10.112.206.102 **** The linux version file /opt/couchbase/ VERSION.txt exists 2021-05-05 19:12:43 | INFO | MainProcess | MainThread | [remote_util.is_process_running] Checking for process beam.smp on linux 2021-05-05 19:12:43 | INFO | MainProcess | MainThread | [remote_util.start_couchbase] Starting couchbase server 2021-05-05 19:12:43 | INFO | MainProcess | MainThread | [remote_util.start_couchbase] Running systemd command on this server 2021-05-05 19:12:43 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] running command.raw on 10.112.206.102: systemctl start couchbase-server.service 2021-05-05 19:12:43 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:12:43 | INFO | MainProcess | MainThread | [remote_util.sleep] 10.112.206.102:sleep for 5 secs. waiting for couchbase server to come up ... 2021-05-05 19:12:48 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] running command.raw on 10.112.206.102: systemctl status couchbase-server.service | grep ExecStop=/opt/couchbase/bin/couchbase-server 2021-05-05 19:12:48 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:12:48 | INFO | MainProcess | MainThread | [remote_util.start_couchbase] Couchbase server status: [' Process: 5868 ExecStop=/opt/couchbase/bin/couchbase-server -k (code=exited, status=0/SUCCESS)'] 2021-05-05 19:12:48 | INFO | MainProcess | MainThread | [remote_util.is_process_running] Checking for process beam.smp on linux 2021-05-05 19:12:48 | INFO | MainProcess | MainThread | [remote_util.is_process_running] process beam.smp is running on 10.112.206.102: with pid 5949 2021-05-05 19:12:48 | INFO | MainProcess | MainThread | [basetestcase.start_server] Couchbase started 2021-05-05 19:12:48 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.103 with username:root, attempt#1 of 5 2021-05-05 19:12:48 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.103 as root 2021-05-05 19:12:48 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:12:48 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:12:48 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.103 with username:root, attempt#1 of 5 2021-05-05 19:12:48 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.103 as root 2021-05-05 19:12:48 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:12:49 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:12:49 | INFO | MainProcess | MainThread | [remote_util.is_couchbase_installed] 10.112.206.103 **** The linux version file /opt/couchbase/ VERSION.txt exists 2021-05-05 19:12:49 | INFO | MainProcess | MainThread | [remote_util.stop_couchbase] Running systemd command on this server 2021-05-05 19:12:49 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] running command.raw on 10.112.206.103: systemctl stop couchbase-server.service 2021-05-05 19:12:50 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:12:50 | INFO | MainProcess | MainThread | [basetestcase.stop_server] Couchbase stopped 2021-05-05 19:12:50 | INFO | MainProcess | MainThread | [basetestcase.sleep] sleep for 5 secs. ... 2021-05-05 19:12:55 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] running command.raw on 10.112.206.103: rm -rf /opt/couchbase/var/lib/couchbase/data/* 2021-05-05 19:12:55 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:12:55 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] running command.raw on 10.112.206.103: rm -rf /opt/couchbase/var/lib/couchbase/config/* 2021-05-05 19:12:55 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:12:55 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.103 with username:root, attempt#1 of 5 2021-05-05 19:12:55 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.103 as root 2021-05-05 19:12:55 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:12:56 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:12:56 | INFO | MainProcess | MainThread | [remote_util.is_couchbase_installed] 10.112.206.103 **** The linux version file /opt/couchbase/ VERSION.txt exists 2021-05-05 19:12:56 | INFO | MainProcess | MainThread | [remote_util.is_process_running] Checking for process beam.smp on linux 2021-05-05 19:12:56 | INFO | MainProcess | MainThread | [remote_util.start_couchbase] Starting couchbase server 2021-05-05 19:12:56 | INFO | MainProcess | MainThread | [remote_util.start_couchbase] Running systemd command on this server 2021-05-05 19:12:56 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] running command.raw on 10.112.206.103: systemctl start couchbase-server.service 2021-05-05 19:12:56 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:12:56 | INFO | MainProcess | MainThread | [remote_util.sleep] 10.112.206.103:sleep for 5 secs. waiting for couchbase server to come up ... 2021-05-05 19:13:01 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] running command.raw on 10.112.206.103: systemctl status couchbase-server.service | grep ExecStop=/opt/couchbase/bin/couchbase-server 2021-05-05 19:13:01 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:13:01 | INFO | MainProcess | MainThread | [remote_util.start_couchbase] Couchbase server status: [' Process: 3551 ExecStop=/opt/couchbase/bin/couchbase-server -k (code=exited, status=0/SUCCESS)'] 2021-05-05 19:13:01 | INFO | MainProcess | MainThread | [remote_util.is_process_running] Checking for process beam.smp on linux 2021-05-05 19:13:01 | INFO | MainProcess | MainThread | [remote_util.is_process_running] process beam.smp is running on 10.112.206.103: with pid 3633 2021-05-05 19:13:01 | INFO | MainProcess | MainThread | [basetestcase.start_server] Couchbase started 2021-05-05 19:13:01 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.104 with username:root, attempt#1 of 5 2021-05-05 19:13:01 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.104 as root 2021-05-05 19:13:01 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:13:01 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:13:01 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.104 with username:root, attempt#1 of 5 2021-05-05 19:13:02 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.104 as root 2021-05-05 19:13:02 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:13:02 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:13:02 | INFO | MainProcess | MainThread | [remote_util.is_couchbase_installed] 10.112.206.104 **** The linux version file /opt/couchbase/ VERSION.txt exists 2021-05-05 19:13:02 | INFO | MainProcess | MainThread | [remote_util.stop_couchbase] Running systemd command on this server 2021-05-05 19:13:02 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] running command.raw on 10.112.206.104: systemctl stop couchbase-server.service 2021-05-05 19:13:03 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:13:03 | INFO | MainProcess | MainThread | [basetestcase.stop_server] Couchbase stopped 2021-05-05 19:13:03 | INFO | MainProcess | MainThread | [basetestcase.sleep] sleep for 5 secs. ... 2021-05-05 19:13:08 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] running command.raw on 10.112.206.104: rm -rf /opt/couchbase/var/lib/couchbase/data/* 2021-05-05 19:13:08 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:13:08 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] running command.raw on 10.112.206.104: rm -rf /opt/couchbase/var/lib/couchbase/config/* 2021-05-05 19:13:08 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:13:08 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.104 with username:root, attempt#1 of 5 2021-05-05 19:13:09 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.104 as root 2021-05-05 19:13:09 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:13:09 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:13:09 | INFO | MainProcess | MainThread | [remote_util.is_couchbase_installed] 10.112.206.104 **** The linux version file /opt/couchbase/ VERSION.txt exists 2021-05-05 19:13:09 | INFO | MainProcess | MainThread | [remote_util.is_process_running] Checking for process beam.smp on linux 2021-05-05 19:13:09 | INFO | MainProcess | MainThread | [remote_util.start_couchbase] Starting couchbase server 2021-05-05 19:13:09 | INFO | MainProcess | MainThread | [remote_util.start_couchbase] Running systemd command on this server 2021-05-05 19:13:09 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] running command.raw on 10.112.206.104: systemctl start couchbase-server.service 2021-05-05 19:13:09 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:13:09 | INFO | MainProcess | MainThread | [remote_util.sleep] 10.112.206.104:sleep for 5 secs. waiting for couchbase server to come up ... 2021-05-05 19:13:14 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] running command.raw on 10.112.206.104: systemctl status couchbase-server.service | grep ExecStop=/opt/couchbase/bin/couchbase-server 2021-05-05 19:13:14 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:13:14 | INFO | MainProcess | MainThread | [remote_util.start_couchbase] Couchbase server status: [' Process: 5041 ExecStop=/opt/couchbase/bin/couchbase-server -k (code=exited, status=0/SUCCESS)'] 2021-05-05 19:13:14 | INFO | MainProcess | MainThread | [remote_util.is_process_running] Checking for process beam.smp on linux 2021-05-05 19:13:14 | INFO | MainProcess | MainThread | [remote_util.is_process_running] process beam.smp is running on 10.112.206.104: with pid 5122 2021-05-05 19:13:14 | INFO | MainProcess | MainThread | [basetestcase.start_server] Couchbase started 2021-05-05 19:13:14 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.105 with username:root, attempt#1 of 5 2021-05-05 19:13:14 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.105 as root 2021-05-05 19:13:14 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:13:15 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:13:15 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.105 with username:root, attempt#1 of 5 2021-05-05 19:13:15 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.105 as root 2021-05-05 19:13:15 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:13:15 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:13:15 | INFO | MainProcess | MainThread | [remote_util.is_couchbase_installed] 10.112.206.105 **** The linux version file /opt/couchbase/ VERSION.txt exists 2021-05-05 19:13:15 | INFO | MainProcess | MainThread | [remote_util.stop_couchbase] Running systemd command on this server 2021-05-05 19:13:15 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] running command.raw on 10.112.206.105: systemctl stop couchbase-server.service 2021-05-05 19:13:17 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:13:17 | INFO | MainProcess | MainThread | [basetestcase.stop_server] Couchbase stopped 2021-05-05 19:13:17 | INFO | MainProcess | MainThread | [basetestcase.sleep] sleep for 5 secs. ... 2021-05-05 19:13:22 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] running command.raw on 10.112.206.105: rm -rf /opt/couchbase/var/lib/couchbase/data/* 2021-05-05 19:13:22 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:13:22 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] running command.raw on 10.112.206.105: rm -rf /opt/couchbase/var/lib/couchbase/config/* 2021-05-05 19:13:22 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:13:22 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.105 with username:root, attempt#1 of 5 2021-05-05 19:13:22 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.105 as root 2021-05-05 19:13:22 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:13:22 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:13:22 | INFO | MainProcess | MainThread | [remote_util.is_couchbase_installed] 10.112.206.105 **** The linux version file /opt/couchbase/ VERSION.txt exists 2021-05-05 19:13:22 | INFO | MainProcess | MainThread | [remote_util.is_process_running] Checking for process beam.smp on linux 2021-05-05 19:13:22 | INFO | MainProcess | MainThread | [remote_util.start_couchbase] Starting couchbase server 2021-05-05 19:13:22 | INFO | MainProcess | MainThread | [remote_util.start_couchbase] Running systemd command on this server 2021-05-05 19:13:22 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] running command.raw on 10.112.206.105: systemctl start couchbase-server.service 2021-05-05 19:13:22 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:13:22 | INFO | MainProcess | MainThread | [remote_util.sleep] 10.112.206.105:sleep for 5 secs. waiting for couchbase server to come up ... 2021-05-05 19:13:27 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] running command.raw on 10.112.206.105: systemctl status couchbase-server.service | grep ExecStop=/opt/couchbase/bin/couchbase-server 2021-05-05 19:13:27 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:13:27 | INFO | MainProcess | MainThread | [remote_util.start_couchbase] Couchbase server status: [' Process: 3859 ExecStop=/opt/couchbase/bin/couchbase-server -k (code=exited, status=0/SUCCESS)'] 2021-05-05 19:13:27 | INFO | MainProcess | MainThread | [remote_util.is_process_running] Checking for process beam.smp on linux 2021-05-05 19:13:27 | INFO | MainProcess | MainThread | [remote_util.is_process_running] process beam.smp is running on 10.112.206.105: with pid 3940 2021-05-05 19:13:27 | INFO | MainProcess | MainThread | [basetestcase.start_server] Couchbase started 2021-05-05 19:13:27 | INFO | MainProcess | MainThread | [basetestcase.sleep] sleep for 10 secs. ... Cluster instance shutdown with force 2021-05-05 19:13:37 | INFO | MainProcess | MainThread | [eventing_base.tearDown] ============== EventingBaseTest tearDown has completed ============== 2021-05-05 19:13:37 | INFO | MainProcess | MainThread | [basetestcase.setUp] initializing cluster 2021-05-05 19:13:37 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.101 with username:root, attempt#1 of 5 2021-05-05 19:13:38 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.101 as root 2021-05-05 19:13:38 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:13:38 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:13:38 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.101 with username:root, attempt#1 of 5 2021-05-05 19:13:38 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.101 as root 2021-05-05 19:13:38 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:13:38 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:13:38 | INFO | MainProcess | MainThread | [remote_util.is_couchbase_installed] 10.112.206.101 **** The linux version file /opt/couchbase/ VERSION.txt exists 2021-05-05 19:13:38 | INFO | MainProcess | MainThread | [remote_util.stop_couchbase] Running systemd command on this server 2021-05-05 19:13:38 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] running command.raw on 10.112.206.101: systemctl stop couchbase-server.service 2021-05-05 19:13:40 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:13:40 | INFO | MainProcess | MainThread | [basetestcase.stop_server] Couchbase stopped 2021-05-05 19:13:40 | INFO | MainProcess | MainThread | [basetestcase.sleep] sleep for 5 secs. ... 2021-05-05 19:13:45 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] running command.raw on 10.112.206.101: rm -rf /opt/couchbase/var/lib/couchbase/data/* 2021-05-05 19:13:45 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:13:45 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] running command.raw on 10.112.206.101: rm -rf /opt/couchbase/var/lib/couchbase/config/* 2021-05-05 19:13:45 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:13:45 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.101 with username:root, attempt#1 of 5 2021-05-05 19:13:45 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.101 as root 2021-05-05 19:13:45 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:13:45 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:13:45 | INFO | MainProcess | MainThread | [remote_util.is_couchbase_installed] 10.112.206.101 **** The linux version file /opt/couchbase/ VERSION.txt exists 2021-05-05 19:13:45 | INFO | MainProcess | MainThread | [remote_util.is_process_running] Checking for process beam.smp on linux 2021-05-05 19:13:46 | INFO | MainProcess | MainThread | [remote_util.start_couchbase] Starting couchbase server 2021-05-05 19:13:46 | INFO | MainProcess | MainThread | [remote_util.start_couchbase] Running systemd command on this server 2021-05-05 19:13:46 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] running command.raw on 10.112.206.101: systemctl start couchbase-server.service 2021-05-05 19:13:46 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:13:46 | INFO | MainProcess | MainThread | [remote_util.sleep] 10.112.206.101:sleep for 5 secs. waiting for couchbase server to come up ... 2021-05-05 19:13:51 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] running command.raw on 10.112.206.101: systemctl status couchbase-server.service | grep ExecStop=/opt/couchbase/bin/couchbase-server 2021-05-05 19:13:51 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:13:51 | INFO | MainProcess | MainThread | [remote_util.start_couchbase] Couchbase server status: [' Process: 24077 ExecStop=/opt/couchbase/bin/couchbase-server -k (code=exited, status=0/SUCCESS)'] 2021-05-05 19:13:51 | INFO | MainProcess | MainThread | [remote_util.is_process_running] Checking for process beam.smp on linux 2021-05-05 19:13:51 | INFO | MainProcess | MainThread | [remote_util.is_process_running] process beam.smp is running on 10.112.206.101: with pid 24159 2021-05-05 19:13:51 | INFO | MainProcess | MainThread | [basetestcase.start_server] Couchbase started 2021-05-05 19:13:51 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.102 with username:root, attempt#1 of 5 2021-05-05 19:13:51 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.102 as root 2021-05-05 19:13:51 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:13:51 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:13:51 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.102 with username:root, attempt#1 of 5 2021-05-05 19:13:51 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.102 as root 2021-05-05 19:13:51 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:13:52 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:13:52 | INFO | MainProcess | MainThread | [remote_util.is_couchbase_installed] 10.112.206.102 **** The linux version file /opt/couchbase/ VERSION.txt exists 2021-05-05 19:13:52 | INFO | MainProcess | MainThread | [remote_util.stop_couchbase] Running systemd command on this server 2021-05-05 19:13:52 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] running command.raw on 10.112.206.102: systemctl stop couchbase-server.service 2021-05-05 19:13:53 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:13:53 | INFO | MainProcess | MainThread | [basetestcase.stop_server] Couchbase stopped 2021-05-05 19:13:53 | INFO | MainProcess | MainThread | [basetestcase.sleep] sleep for 5 secs. ... 2021-05-05 19:13:58 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] running command.raw on 10.112.206.102: rm -rf /opt/couchbase/var/lib/couchbase/data/* 2021-05-05 19:13:58 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:13:58 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] running command.raw on 10.112.206.102: rm -rf /opt/couchbase/var/lib/couchbase/config/* 2021-05-05 19:13:58 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:13:58 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.102 with username:root, attempt#1 of 5 2021-05-05 19:13:58 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.102 as root 2021-05-05 19:13:58 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:13:59 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:13:59 | INFO | MainProcess | MainThread | [remote_util.is_couchbase_installed] 10.112.206.102 **** The linux version file /opt/couchbase/ VERSION.txt exists 2021-05-05 19:13:59 | INFO | MainProcess | MainThread | [remote_util.is_process_running] Checking for process beam.smp on linux 2021-05-05 19:13:59 | INFO | MainProcess | MainThread | [remote_util.start_couchbase] Starting couchbase server 2021-05-05 19:13:59 | INFO | MainProcess | MainThread | [remote_util.start_couchbase] Running systemd command on this server 2021-05-05 19:13:59 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] running command.raw on 10.112.206.102: systemctl start couchbase-server.service 2021-05-05 19:13:59 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:13:59 | INFO | MainProcess | MainThread | [remote_util.sleep] 10.112.206.102:sleep for 5 secs. waiting for couchbase server to come up ... 2021-05-05 19:14:04 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] running command.raw on 10.112.206.102: systemctl status couchbase-server.service | grep ExecStop=/opt/couchbase/bin/couchbase-server 2021-05-05 19:14:04 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:14:04 | INFO | MainProcess | MainThread | [remote_util.start_couchbase] Couchbase server status: [' Process: 6247 ExecStop=/opt/couchbase/bin/couchbase-server -k (code=exited, status=0/SUCCESS)'] 2021-05-05 19:14:04 | INFO | MainProcess | MainThread | [remote_util.is_process_running] Checking for process beam.smp on linux 2021-05-05 19:14:04 | INFO | MainProcess | MainThread | [remote_util.is_process_running] process beam.smp is running on 10.112.206.102: with pid 6328 2021-05-05 19:14:04 | INFO | MainProcess | MainThread | [basetestcase.start_server] Couchbase started 2021-05-05 19:14:04 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.103 with username:root, attempt#1 of 5 2021-05-05 19:14:04 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.103 as root 2021-05-05 19:14:04 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:14:04 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:14:04 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.103 with username:root, attempt#1 of 5 2021-05-05 19:14:04 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.103 as root 2021-05-05 19:14:04 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:14:05 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:14:05 | INFO | MainProcess | MainThread | [remote_util.is_couchbase_installed] 10.112.206.103 **** The linux version file /opt/couchbase/ VERSION.txt exists 2021-05-05 19:14:05 | INFO | MainProcess | MainThread | [remote_util.stop_couchbase] Running systemd command on this server 2021-05-05 19:14:05 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] running command.raw on 10.112.206.103: systemctl stop couchbase-server.service 2021-05-05 19:14:06 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:14:06 | INFO | MainProcess | MainThread | [basetestcase.stop_server] Couchbase stopped 2021-05-05 19:14:06 | INFO | MainProcess | MainThread | [basetestcase.sleep] sleep for 5 secs. ... 2021-05-05 19:14:11 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] running command.raw on 10.112.206.103: rm -rf /opt/couchbase/var/lib/couchbase/data/* 2021-05-05 19:14:11 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:14:11 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] running command.raw on 10.112.206.103: rm -rf /opt/couchbase/var/lib/couchbase/config/* 2021-05-05 19:14:11 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:14:11 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.103 with username:root, attempt#1 of 5 2021-05-05 19:14:11 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.103 as root 2021-05-05 19:14:12 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:14:12 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:14:12 | INFO | MainProcess | MainThread | [remote_util.is_couchbase_installed] 10.112.206.103 **** The linux version file /opt/couchbase/ VERSION.txt exists 2021-05-05 19:14:12 | INFO | MainProcess | MainThread | [remote_util.is_process_running] Checking for process beam.smp on linux 2021-05-05 19:14:12 | INFO | MainProcess | MainThread | [remote_util.start_couchbase] Starting couchbase server 2021-05-05 19:14:12 | INFO | MainProcess | MainThread | [remote_util.start_couchbase] Running systemd command on this server 2021-05-05 19:14:12 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] running command.raw on 10.112.206.103: systemctl start couchbase-server.service 2021-05-05 19:14:12 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:14:12 | INFO | MainProcess | MainThread | [remote_util.sleep] 10.112.206.103:sleep for 5 secs. waiting for couchbase server to come up ... 2021-05-05 19:14:17 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] running command.raw on 10.112.206.103: systemctl status couchbase-server.service | grep ExecStop=/opt/couchbase/bin/couchbase-server 2021-05-05 19:14:17 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:14:17 | INFO | MainProcess | MainThread | [remote_util.start_couchbase] Couchbase server status: [' Process: 3932 ExecStop=/opt/couchbase/bin/couchbase-server -k (code=exited, status=0/SUCCESS)'] 2021-05-05 19:14:17 | INFO | MainProcess | MainThread | [remote_util.is_process_running] Checking for process beam.smp on linux 2021-05-05 19:14:17 | INFO | MainProcess | MainThread | [remote_util.is_process_running] process beam.smp is running on 10.112.206.103: with pid 4013 2021-05-05 19:14:17 | INFO | MainProcess | MainThread | [basetestcase.start_server] Couchbase started 2021-05-05 19:14:17 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.104 with username:root, attempt#1 of 5 2021-05-05 19:14:17 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.104 as root 2021-05-05 19:14:17 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:14:17 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:14:17 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.104 with username:root, attempt#1 of 5 2021-05-05 19:14:18 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.104 as root 2021-05-05 19:14:18 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:14:18 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:14:18 | INFO | MainProcess | MainThread | [remote_util.is_couchbase_installed] 10.112.206.104 **** The linux version file /opt/couchbase/ VERSION.txt exists 2021-05-05 19:14:18 | INFO | MainProcess | MainThread | [remote_util.stop_couchbase] Running systemd command on this server 2021-05-05 19:14:18 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] running command.raw on 10.112.206.104: systemctl stop couchbase-server.service 2021-05-05 19:14:19 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:14:19 | INFO | MainProcess | MainThread | [basetestcase.stop_server] Couchbase stopped 2021-05-05 19:14:19 | INFO | MainProcess | MainThread | [basetestcase.sleep] sleep for 5 secs. ... 2021-05-05 19:14:24 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] running command.raw on 10.112.206.104: rm -rf /opt/couchbase/var/lib/couchbase/data/* 2021-05-05 19:14:24 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:14:24 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] running command.raw on 10.112.206.104: rm -rf /opt/couchbase/var/lib/couchbase/config/* 2021-05-05 19:14:24 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:14:24 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.104 with username:root, attempt#1 of 5 2021-05-05 19:14:25 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.104 as root 2021-05-05 19:14:25 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:14:25 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:14:25 | INFO | MainProcess | MainThread | [remote_util.is_couchbase_installed] 10.112.206.104 **** The linux version file /opt/couchbase/ VERSION.txt exists 2021-05-05 19:14:25 | INFO | MainProcess | MainThread | [remote_util.is_process_running] Checking for process beam.smp on linux 2021-05-05 19:14:25 | INFO | MainProcess | MainThread | [remote_util.start_couchbase] Starting couchbase server 2021-05-05 19:14:25 | INFO | MainProcess | MainThread | [remote_util.start_couchbase] Running systemd command on this server 2021-05-05 19:14:25 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] running command.raw on 10.112.206.104: systemctl start couchbase-server.service 2021-05-05 19:14:25 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:14:25 | INFO | MainProcess | MainThread | [remote_util.sleep] 10.112.206.104:sleep for 5 secs. waiting for couchbase server to come up ... 2021-05-05 19:14:30 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] running command.raw on 10.112.206.104: systemctl status couchbase-server.service | grep ExecStop=/opt/couchbase/bin/couchbase-server 2021-05-05 19:14:30 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:14:30 | INFO | MainProcess | MainThread | [remote_util.start_couchbase] Couchbase server status: [' Process: 5421 ExecStop=/opt/couchbase/bin/couchbase-server -k (code=exited, status=0/SUCCESS)'] 2021-05-05 19:14:30 | INFO | MainProcess | MainThread | [remote_util.is_process_running] Checking for process beam.smp on linux 2021-05-05 19:14:30 | INFO | MainProcess | MainThread | [remote_util.is_process_running] process beam.smp is running on 10.112.206.104: with pid 5502 2021-05-05 19:14:30 | INFO | MainProcess | MainThread | [basetestcase.start_server] Couchbase started 2021-05-05 19:14:30 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.105 with username:root, attempt#1 of 5 2021-05-05 19:14:30 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.105 as root 2021-05-05 19:14:30 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:14:31 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:14:31 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.105 with username:root, attempt#1 of 5 2021-05-05 19:14:31 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.105 as root 2021-05-05 19:14:31 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:14:31 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:14:31 | INFO | MainProcess | MainThread | [remote_util.is_couchbase_installed] 10.112.206.105 **** The linux version file /opt/couchbase/ VERSION.txt exists 2021-05-05 19:14:31 | INFO | MainProcess | MainThread | [remote_util.stop_couchbase] Running systemd command on this server 2021-05-05 19:14:31 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] running command.raw on 10.112.206.105: systemctl stop couchbase-server.service 2021-05-05 19:14:33 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:14:33 | INFO | MainProcess | MainThread | [basetestcase.stop_server] Couchbase stopped 2021-05-05 19:14:33 | INFO | MainProcess | MainThread | [basetestcase.sleep] sleep for 5 secs. ... 2021-05-05 19:14:38 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] running command.raw on 10.112.206.105: rm -rf /opt/couchbase/var/lib/couchbase/data/* 2021-05-05 19:14:38 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:14:38 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] running command.raw on 10.112.206.105: rm -rf /opt/couchbase/var/lib/couchbase/config/* 2021-05-05 19:14:38 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:14:38 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.105 with username:root, attempt#1 of 5 2021-05-05 19:14:38 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.105 as root 2021-05-05 19:14:38 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:14:38 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:14:38 | INFO | MainProcess | MainThread | [remote_util.is_couchbase_installed] 10.112.206.105 **** The linux version file /opt/couchbase/ VERSION.txt exists 2021-05-05 19:14:38 | INFO | MainProcess | MainThread | [remote_util.is_process_running] Checking for process beam.smp on linux 2021-05-05 19:14:38 | INFO | MainProcess | MainThread | [remote_util.start_couchbase] Starting couchbase server 2021-05-05 19:14:38 | INFO | MainProcess | MainThread | [remote_util.start_couchbase] Running systemd command on this server 2021-05-05 19:14:38 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] running command.raw on 10.112.206.105: systemctl start couchbase-server.service 2021-05-05 19:14:38 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:14:38 | INFO | MainProcess | MainThread | [remote_util.sleep] 10.112.206.105:sleep for 5 secs. waiting for couchbase server to come up ... 2021-05-05 19:14:43 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] running command.raw on 10.112.206.105: systemctl status couchbase-server.service | grep ExecStop=/opt/couchbase/bin/couchbase-server 2021-05-05 19:14:43 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:14:43 | INFO | MainProcess | MainThread | [remote_util.start_couchbase] Couchbase server status: [' Process: 4239 ExecStop=/opt/couchbase/bin/couchbase-server -k (code=exited, status=0/SUCCESS)'] 2021-05-05 19:14:43 | INFO | MainProcess | MainThread | [remote_util.is_process_running] Checking for process beam.smp on linux 2021-05-05 19:14:43 | INFO | MainProcess | MainThread | [remote_util.is_process_running] process beam.smp is running on 10.112.206.105: with pid 4320 2021-05-05 19:14:43 | INFO | MainProcess | MainThread | [basetestcase.start_server] Couchbase started 2021-05-05 19:14:43 | INFO | MainProcess | MainThread | [basetestcase.sleep] sleep for 10 secs. ... 2021-05-05 19:14:54 | INFO | MainProcess | Cluster_Thread | [task.execute] server: ip:10.112.206.101 port:8091 ssh_username:root, nodes/self 2021-05-05 19:14:54 | INFO | MainProcess | Cluster_Thread | [task.execute] {'uptime': '63', 'memoryTotal': 1930829824, 'memoryFree': 1544232960, 'mcdMemoryReserved': 1473, 'mcdMemoryAllocated': 1473, 'status': 'healthy', 'hostname': '10.112.206.101:8091', 'clusterCompatibility': 458752, 'clusterMembership': 'active', 'recoveryType': 'none', 'version': '7.0.0-5102-enterprise', 'os': 'x86_64-unknown-linux-gnu', 'ports': [], 'availableStorage': [], 'storage': [], 'memoryQuota': 256, 'moxi': 11211, 'memcached': 11210, 'id': 'ns_1@10.112.206.101', 'ip': '10.112.206.101', 'rest_username': '', 'rest_password': '', 'port': '8091', 'services': ['kv'], 'storageTotalRam': 1841, 'curr_items': 0} 2021-05-05 19:14:54 | ERROR | MainProcess | Cluster_Thread | [rest_client._http_request] GET http://10.112.206.101:8091/pools/default body: headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==', 'Accept': '*/*'} error: 404 reason: unknown b'"unknown pool"' auth: Administrator:password 2021-05-05 19:14:54 | INFO | MainProcess | Cluster_Thread | [rest_client.init_cluster_memoryQuota] pools/default params : memoryQuota=986 2021-05-05 19:14:54 | INFO | MainProcess | Cluster_Thread | [rest_client.init_node_services] --> init_node_services(Administrator,password,10.112.206.101,8091,['kv']) 2021-05-05 19:14:54 | INFO | MainProcess | Cluster_Thread | [rest_client.init_node_services] /node/controller/setupServices params on 10.112.206.101: 8091:hostname=10.112.206.101&user=Administrator&password=password&services=kv 2021-05-05 19:14:54 | INFO | MainProcess | Cluster_Thread | [rest_client.init_cluster] --> in init_cluster...Administrator,password,8091 2021-05-05 19:14:54 | INFO | MainProcess | Cluster_Thread | [rest_client.init_cluster] settings/web params on 10.112.206.101:8091:port=8091&username=Administrator&password=password 2021-05-05 19:14:54 | INFO | MainProcess | Cluster_Thread | [rest_client.init_cluster] --> status:True 2021-05-05 19:14:54 | INFO | MainProcess | Cluster_Thread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.101 with username:root, attempt#1 of 5 2021-05-05 19:14:54 | INFO | MainProcess | Cluster_Thread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.101 as root 2021-05-05 19:14:54 | INFO | MainProcess | Cluster_Thread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:14:54 | INFO | MainProcess | Cluster_Thread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:14:54 | INFO | MainProcess | Cluster_Thread | [remote_util.execute_command_raw] running command.raw on 10.112.206.101: curl --silent --show-error http://Administrator:password@localhost:8091/diag/eval -X POST -d 'ns_config:set(allow_nonlocal_eval, true).' 2021-05-05 19:14:55 | INFO | MainProcess | Cluster_Thread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:14:55 | INFO | MainProcess | Cluster_Thread | [remote_util.enable_diag_eval_on_non_local_hosts] ['ok'] 2021-05-05 19:14:55 | INFO | MainProcess | Cluster_Thread | [rest_client.diag_eval] /diag/eval status on 10.112.206.101:8091: True content: [7,0] command: cluster_compat_mode:get_compat_version(). 2021-05-05 19:14:55 | INFO | MainProcess | Cluster_Thread | [rest_client.diag_eval] /diag/eval status on 10.112.206.101:8091: True content: [7,0] command: cluster_compat_mode:get_compat_version(). 2021-05-05 19:14:55 | INFO | MainProcess | Cluster_Thread | [rest_client.set_indexer_storage_mode] settings/indexes params : storageMode=plasma 2021-05-05 19:14:55 | INFO | MainProcess | Cluster_Thread | [task.execute] server: ip:10.112.206.102 port:8091 ssh_username:root, nodes/self 2021-05-05 19:14:55 | INFO | MainProcess | Cluster_Thread | [task.execute] {'uptime': '53', 'memoryTotal': 1930829824, 'memoryFree': 1597353984, 'mcdMemoryReserved': 1473, 'mcdMemoryAllocated': 1473, 'status': 'healthy', 'hostname': '10.112.206.102:8091', 'clusterCompatibility': 458752, 'clusterMembership': 'active', 'recoveryType': 'none', 'version': '7.0.0-5102-enterprise', 'os': 'x86_64-unknown-linux-gnu', 'ports': [], 'availableStorage': [], 'storage': [], 'memoryQuota': 256, 'moxi': 11211, 'memcached': 11210, 'id': 'ns_1@10.112.206.102', 'ip': '10.112.206.102', 'rest_username': '', 'rest_password': '', 'port': '8091', 'services': ['kv'], 'storageTotalRam': 1841, 'curr_items': 0} 2021-05-05 19:14:55 | ERROR | MainProcess | Cluster_Thread | [rest_client._http_request] GET http://10.112.206.102:8091/pools/default body: headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==', 'Accept': '*/*'} error: 404 reason: unknown b'"unknown pool"' auth: Administrator:password 2021-05-05 19:14:55 | INFO | MainProcess | Cluster_Thread | [rest_client.init_cluster_memoryQuota] pools/default params : memoryQuota=986 2021-05-05 19:14:55 | INFO | MainProcess | Cluster_Thread | [rest_client.init_cluster] --> in init_cluster...Administrator,password,8091 2021-05-05 19:14:55 | INFO | MainProcess | Cluster_Thread | [rest_client.init_cluster] settings/web params on 10.112.206.102:8091:port=8091&username=Administrator&password=password 2021-05-05 19:14:55 | INFO | MainProcess | Cluster_Thread | [rest_client.init_cluster] --> status:True 2021-05-05 19:14:55 | INFO | MainProcess | Cluster_Thread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.102 with username:root, attempt#1 of 5 2021-05-05 19:14:55 | INFO | MainProcess | Cluster_Thread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.102 as root 2021-05-05 19:14:55 | INFO | MainProcess | Cluster_Thread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:14:55 | INFO | MainProcess | Cluster_Thread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:14:55 | INFO | MainProcess | Cluster_Thread | [remote_util.execute_command_raw] running command.raw on 10.112.206.102: curl --silent --show-error http://Administrator:password@localhost:8091/diag/eval -X POST -d 'ns_config:set(allow_nonlocal_eval, true).' 2021-05-05 19:14:56 | INFO | MainProcess | Cluster_Thread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:14:56 | INFO | MainProcess | Cluster_Thread | [remote_util.enable_diag_eval_on_non_local_hosts] ['ok'] 2021-05-05 19:14:56 | INFO | MainProcess | Cluster_Thread | [rest_client.diag_eval] /diag/eval status on 10.112.206.102:8091: True content: [7,0] command: cluster_compat_mode:get_compat_version(). 2021-05-05 19:14:56 | INFO | MainProcess | Cluster_Thread | [rest_client.diag_eval] /diag/eval status on 10.112.206.102:8091: True content: [7,0] command: cluster_compat_mode:get_compat_version(). 2021-05-05 19:14:56 | INFO | MainProcess | Cluster_Thread | [rest_client.set_indexer_storage_mode] settings/indexes params : storageMode=plasma 2021-05-05 19:14:56 | INFO | MainProcess | Cluster_Thread | [task.execute] server: ip:10.112.206.103 port:8091 ssh_username:root, nodes/self 2021-05-05 19:14:56 | INFO | MainProcess | Cluster_Thread | [task.execute] {'uptime': '38', 'memoryTotal': 1930829824, 'memoryFree': 1598218240, 'mcdMemoryReserved': 1473, 'mcdMemoryAllocated': 1473, 'status': 'healthy', 'hostname': '10.112.206.103:8091', 'clusterCompatibility': 458752, 'clusterMembership': 'active', 'recoveryType': 'none', 'version': '7.0.0-5102-enterprise', 'os': 'x86_64-unknown-linux-gnu', 'ports': [], 'availableStorage': [], 'storage': [], 'memoryQuota': 256, 'moxi': 11211, 'memcached': 11210, 'id': 'ns_1@10.112.206.103', 'ip': '10.112.206.103', 'rest_username': '', 'rest_password': '', 'port': '8091', 'services': ['kv'], 'storageTotalRam': 1841, 'curr_items': 0} 2021-05-05 19:14:56 | ERROR | MainProcess | Cluster_Thread | [rest_client._http_request] GET http://10.112.206.103:8091/pools/default body: headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==', 'Accept': '*/*'} error: 404 reason: unknown b'"unknown pool"' auth: Administrator:password 2021-05-05 19:14:56 | INFO | MainProcess | Cluster_Thread | [rest_client.init_cluster_memoryQuota] pools/default params : memoryQuota=986 2021-05-05 19:14:56 | INFO | MainProcess | Cluster_Thread | [rest_client.init_cluster] --> in init_cluster...Administrator,password,8091 2021-05-05 19:14:56 | INFO | MainProcess | Cluster_Thread | [rest_client.init_cluster] settings/web params on 10.112.206.103:8091:port=8091&username=Administrator&password=password 2021-05-05 19:14:56 | INFO | MainProcess | Cluster_Thread | [rest_client.init_cluster] --> status:True 2021-05-05 19:14:56 | INFO | MainProcess | Cluster_Thread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.103 with username:root, attempt#1 of 5 2021-05-05 19:14:56 | INFO | MainProcess | Cluster_Thread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.103 as root 2021-05-05 19:14:56 | INFO | MainProcess | Cluster_Thread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:14:56 | INFO | MainProcess | Cluster_Thread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:14:56 | INFO | MainProcess | Cluster_Thread | [remote_util.execute_command_raw] running command.raw on 10.112.206.103: curl --silent --show-error http://Administrator:password@localhost:8091/diag/eval -X POST -d 'ns_config:set(allow_nonlocal_eval, true).' 2021-05-05 19:14:56 | INFO | MainProcess | Cluster_Thread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:14:56 | INFO | MainProcess | Cluster_Thread | [remote_util.enable_diag_eval_on_non_local_hosts] ['ok'] 2021-05-05 19:14:56 | INFO | MainProcess | Cluster_Thread | [rest_client.diag_eval] /diag/eval status on 10.112.206.103:8091: True content: [7,0] command: cluster_compat_mode:get_compat_version(). 2021-05-05 19:14:56 | INFO | MainProcess | Cluster_Thread | [rest_client.diag_eval] /diag/eval status on 10.112.206.103:8091: True content: [7,0] command: cluster_compat_mode:get_compat_version(). 2021-05-05 19:14:56 | INFO | MainProcess | Cluster_Thread | [rest_client.set_indexer_storage_mode] settings/indexes params : storageMode=plasma 2021-05-05 19:14:56 | INFO | MainProcess | Cluster_Thread | [task.execute] server: ip:10.112.206.104 port:8091 ssh_username:root, nodes/self 2021-05-05 19:14:56 | INFO | MainProcess | Cluster_Thread | [task.execute] {'uptime': '28', 'memoryTotal': 1930829824, 'memoryFree': 1594929152, 'mcdMemoryReserved': 1473, 'mcdMemoryAllocated': 1473, 'status': 'healthy', 'hostname': '10.112.206.104:8091', 'clusterCompatibility': 458752, 'clusterMembership': 'active', 'recoveryType': 'none', 'version': '7.0.0-5102-enterprise', 'os': 'x86_64-unknown-linux-gnu', 'ports': [], 'availableStorage': [], 'storage': [], 'memoryQuota': 256, 'moxi': 11211, 'memcached': 11210, 'id': 'ns_1@10.112.206.104', 'ip': '10.112.206.104', 'rest_username': '', 'rest_password': '', 'port': '8091', 'services': ['kv'], 'storageTotalRam': 1841, 'curr_items': 0} 2021-05-05 19:14:56 | ERROR | MainProcess | Cluster_Thread | [rest_client._http_request] GET http://10.112.206.104:8091/pools/default body: headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==', 'Accept': '*/*'} error: 404 reason: unknown b'"unknown pool"' auth: Administrator:password 2021-05-05 19:14:56 | INFO | MainProcess | Cluster_Thread | [rest_client.init_cluster_memoryQuota] pools/default params : memoryQuota=986 2021-05-05 19:14:56 | INFO | MainProcess | Cluster_Thread | [rest_client.init_cluster] --> in init_cluster...Administrator,password,8091 2021-05-05 19:14:56 | INFO | MainProcess | Cluster_Thread | [rest_client.init_cluster] settings/web params on 10.112.206.104:8091:port=8091&username=Administrator&password=password 2021-05-05 19:14:57 | INFO | MainProcess | Cluster_Thread | [rest_client.init_cluster] --> status:True 2021-05-05 19:14:57 | INFO | MainProcess | Cluster_Thread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.104 with username:root, attempt#1 of 5 2021-05-05 19:14:57 | INFO | MainProcess | Cluster_Thread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.104 as root 2021-05-05 19:14:57 | INFO | MainProcess | Cluster_Thread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:14:57 | INFO | MainProcess | Cluster_Thread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:14:57 | INFO | MainProcess | Cluster_Thread | [remote_util.execute_command_raw] running command.raw on 10.112.206.104: curl --silent --show-error http://Administrator:password@localhost:8091/diag/eval -X POST -d 'ns_config:set(allow_nonlocal_eval, true).' 2021-05-05 19:14:57 | INFO | MainProcess | Cluster_Thread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:14:57 | INFO | MainProcess | Cluster_Thread | [remote_util.enable_diag_eval_on_non_local_hosts] ['ok'] 2021-05-05 19:14:57 | INFO | MainProcess | Cluster_Thread | [rest_client.diag_eval] /diag/eval status on 10.112.206.104:8091: True content: [7,0] command: cluster_compat_mode:get_compat_version(). 2021-05-05 19:14:57 | INFO | MainProcess | Cluster_Thread | [rest_client.diag_eval] /diag/eval status on 10.112.206.104:8091: True content: [7,0] command: cluster_compat_mode:get_compat_version(). 2021-05-05 19:14:57 | INFO | MainProcess | Cluster_Thread | [rest_client.set_indexer_storage_mode] settings/indexes params : storageMode=plasma 2021-05-05 19:14:57 | INFO | MainProcess | Cluster_Thread | [task.execute] server: ip:10.112.206.105 port:8091 ssh_username:root, nodes/self 2021-05-05 19:14:57 | INFO | MainProcess | Cluster_Thread | [task.execute] {'uptime': '13', 'memoryTotal': 1930829824, 'memoryFree': 1598754816, 'mcdMemoryReserved': 1473, 'mcdMemoryAllocated': 1473, 'status': 'healthy', 'hostname': '10.112.206.105:8091', 'clusterCompatibility': 458752, 'clusterMembership': 'active', 'recoveryType': 'none', 'version': '7.0.0-5102-enterprise', 'os': 'x86_64-unknown-linux-gnu', 'ports': [], 'availableStorage': [], 'storage': [], 'memoryQuota': 256, 'moxi': 11211, 'memcached': 11210, 'id': 'ns_1@10.112.206.105', 'ip': '10.112.206.105', 'rest_username': '', 'rest_password': '', 'port': '8091', 'services': ['kv'], 'storageTotalRam': 1841, 'curr_items': 0} 2021-05-05 19:14:57 | ERROR | MainProcess | Cluster_Thread | [rest_client._http_request] GET http://10.112.206.105:8091/pools/default body: headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==', 'Accept': '*/*'} error: 404 reason: unknown b'"unknown pool"' auth: Administrator:password 2021-05-05 19:14:57 | INFO | MainProcess | Cluster_Thread | [rest_client.init_cluster_memoryQuota] pools/default params : memoryQuota=986 2021-05-05 19:14:57 | INFO | MainProcess | Cluster_Thread | [rest_client.init_cluster] --> in init_cluster...Administrator,password,8091 2021-05-05 19:14:57 | INFO | MainProcess | Cluster_Thread | [rest_client.init_cluster] settings/web params on 10.112.206.105:8091:port=8091&username=Administrator&password=password 2021-05-05 19:14:57 | INFO | MainProcess | Cluster_Thread | [rest_client.init_cluster] --> status:True 2021-05-05 19:14:57 | INFO | MainProcess | Cluster_Thread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.105 with username:root, attempt#1 of 5 2021-05-05 19:14:58 | INFO | MainProcess | Cluster_Thread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.105 as root 2021-05-05 19:14:58 | INFO | MainProcess | Cluster_Thread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:14:58 | INFO | MainProcess | Cluster_Thread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:14:58 | INFO | MainProcess | Cluster_Thread | [remote_util.execute_command_raw] running command.raw on 10.112.206.105: curl --silent --show-error http://Administrator:password@localhost:8091/diag/eval -X POST -d 'ns_config:set(allow_nonlocal_eval, true).' 2021-05-05 19:14:58 | INFO | MainProcess | Cluster_Thread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:14:58 | INFO | MainProcess | Cluster_Thread | [remote_util.enable_diag_eval_on_non_local_hosts] ['ok'] 2021-05-05 19:14:58 | INFO | MainProcess | Cluster_Thread | [rest_client.diag_eval] /diag/eval status on 10.112.206.105:8091: True content: [7,0] command: cluster_compat_mode:get_compat_version(). 2021-05-05 19:14:58 | INFO | MainProcess | Cluster_Thread | [rest_client.diag_eval] /diag/eval status on 10.112.206.105:8091: True content: [7,0] command: cluster_compat_mode:get_compat_version(). 2021-05-05 19:14:58 | INFO | MainProcess | Cluster_Thread | [rest_client.set_indexer_storage_mode] settings/indexes params : storageMode=plasma 2021-05-05 19:14:58 | INFO | MainProcess | MainThread | [basetestcase.add_built_in_server_user] **** add built-in 'cbadminbucket' user to node 10.112.206.101 **** 2021-05-05 19:14:58 | ERROR | MainProcess | MainThread | [rest_client._http_request] DELETE http://10.112.206.101:8091/settings/rbac/users/local/cbadminbucket body: headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==', 'Accept': '*/*'} error: 404 reason: unknown b'"User was not found."' auth: Administrator:password 2021-05-05 19:14:58 | INFO | MainProcess | MainThread | [internal_user.delete_user] Exception while deleting user. Exception is -b'"User was not found."' 2021-05-05 19:14:58 | INFO | MainProcess | MainThread | [basetestcase.sleep] sleep for 5 secs. ... 2021-05-05 19:15:03 | INFO | MainProcess | MainThread | [basetestcase.add_built_in_server_user] **** add 'admin' role to 'cbadminbucket' user **** 2021-05-05 19:15:03 | INFO | MainProcess | MainThread | [basetestcase.setUp] done initializing cluster 2021-05-05 19:15:04 | INFO | MainProcess | Cluster_Thread | [task.add_nodes] adding node 10.112.206.102:8091 to cluster 2021-05-05 19:15:04 | INFO | MainProcess | Cluster_Thread | [rest_client.add_node] adding remote node @10.112.206.102:8091 to this cluster @10.112.206.101:8091 2021-05-05 19:15:14 | INFO | MainProcess | Cluster_Thread | [rest_client.monitorRebalance] rebalance progress took 10.01 seconds 2021-05-05 19:15:14 | INFO | MainProcess | Cluster_Thread | [rest_client.monitorRebalance] sleep for 10 seconds after rebalance... 2021-05-05 19:15:32 | INFO | MainProcess | Cluster_Thread | [task.add_nodes] adding node 10.112.206.103:8091 to cluster 2021-05-05 19:15:32 | INFO | MainProcess | Cluster_Thread | [rest_client.add_node] adding remote node @10.112.206.103:8091 to this cluster @10.112.206.101:8091 2021-05-05 19:15:42 | INFO | MainProcess | Cluster_Thread | [rest_client.monitorRebalance] rebalance progress took 10.00 seconds 2021-05-05 19:15:42 | INFO | MainProcess | Cluster_Thread | [rest_client.monitorRebalance] sleep for 10 seconds after rebalance... 2021-05-05 19:15:59 | INFO | MainProcess | Cluster_Thread | [task.add_nodes] adding node 10.112.206.104:8091 to cluster 2021-05-05 19:15:59 | INFO | MainProcess | Cluster_Thread | [rest_client.add_node] adding remote node @10.112.206.104:8091 to this cluster @10.112.206.101:8091 2021-05-05 19:16:09 | INFO | MainProcess | Cluster_Thread | [rest_client.monitorRebalance] rebalance progress took 10.01 seconds 2021-05-05 19:16:09 | INFO | MainProcess | Cluster_Thread | [rest_client.monitorRebalance] sleep for 10 seconds after rebalance... 2021-05-05 19:16:26 | INFO | MainProcess | Cluster_Thread | [task.add_nodes] adding node 10.112.206.105:8091 to cluster 2021-05-05 19:16:26 | INFO | MainProcess | Cluster_Thread | [rest_client.add_node] adding remote node @10.112.206.105:8091 to this cluster @10.112.206.101:8091 2021-05-05 19:16:36 | INFO | MainProcess | Cluster_Thread | [rest_client.monitorRebalance] rebalance progress took 10.01 seconds 2021-05-05 19:16:36 | INFO | MainProcess | Cluster_Thread | [rest_client.monitorRebalance] sleep for 10 seconds after rebalance... 2021-05-05 19:16:54 | INFO | MainProcess | Cluster_Thread | [rest_client.rebalance] rebalance params : {'knownNodes': 'ns_1@10.112.206.101,ns_1@10.112.206.102,ns_1@10.112.206.103,ns_1@10.112.206.104,ns_1@10.112.206.105', 'ejectedNodes': '', 'user': 'Administrator', 'password': 'password'} 2021-05-05 19:16:54 | INFO | MainProcess | Cluster_Thread | [rest_client.rebalance] rebalance operation started 2021-05-05 19:17:04 | INFO | MainProcess | Cluster_Thread | [task.check] Rebalance - status: none, progress: 100.00% 2021-05-05 19:17:04 | INFO | MainProcess | Cluster_Thread | [task.check] rebalancing was completed with progress: 100% in 10.013190984725952 sec 2021-05-05 19:17:04 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.101 with username:root, attempt#1 of 5 2021-05-05 19:17:04 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.101 as root 2021-05-05 19:17:05 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:17:05 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:17:05 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] running command.raw on 10.112.206.101: curl --silent --show-error http://Administrator:password@localhost:8091/diag/eval -X POST -d 'ns_config:set(allow_nonlocal_eval, true).' 2021-05-05 19:17:05 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:17:05 | INFO | MainProcess | MainThread | [remote_util.enable_diag_eval_on_non_local_hosts] ['ok'] 2021-05-05 19:17:05 | INFO | MainProcess | MainThread | [basetestcase.enable_diag_eval_on_non_local_hosts] Enabled diag/eval for non-local hosts from 10.112.206.101 2021-05-05 19:17:05 | INFO | MainProcess | MainThread | [basetestcase.setUp] ============== basetestcase setup was finished for test #1 suite_setUp ============== 2021-05-05 19:17:05 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.101 with username:root, attempt#1 of 5 2021-05-05 19:17:05 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.101 as root 2021-05-05 19:17:05 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:17:05 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:17:05 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.102 with username:root, attempt#1 of 5 2021-05-05 19:17:06 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.102 as root 2021-05-05 19:17:06 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:17:06 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:17:06 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.103 with username:root, attempt#1 of 5 2021-05-05 19:17:06 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.103 as root 2021-05-05 19:17:06 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:17:06 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:17:06 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.104 with username:root, attempt#1 of 5 2021-05-05 19:17:06 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.104 as root 2021-05-05 19:17:07 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:17:07 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:17:07 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.105 with username:root, attempt#1 of 5 2021-05-05 19:17:07 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.105 as root 2021-05-05 19:17:07 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:17:07 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:17:22 | INFO | MainProcess | MainThread | [basetestcase.print_cluster_stats] ------- Cluster statistics ------- 2021-05-05 19:17:22 | INFO | MainProcess | MainThread | [basetestcase.print_cluster_stats] 10.112.206.101:8091 => {'services': ['kv'], 'cpu_utilization': 25.79821200510856, 'mem_free': 1513390080, 'mem_total': 1930829824, 'swap_mem_used': 0, 'swap_mem_total': 1107292160} 2021-05-05 19:17:22 | INFO | MainProcess | MainThread | [basetestcase.print_cluster_stats] 10.112.206.102:8091 => {'services': ['kv'], 'cpu_utilization': 1.973001038421599, 'mem_free': 1564028928, 'mem_total': 1930829824, 'swap_mem_used': 0, 'swap_mem_total': 1107292160} 2021-05-05 19:17:22 | INFO | MainProcess | MainThread | [basetestcase.print_cluster_stats] 10.112.206.103:8091 => {'services': ['kv'], 'cpu_utilization': 1.752577319587629, 'mem_free': 1560686592, 'mem_total': 1930829824, 'swap_mem_used': 0, 'swap_mem_total': 1107292160} 2021-05-05 19:17:22 | INFO | MainProcess | MainThread | [basetestcase.print_cluster_stats] 10.112.206.104:8091 => {'services': ['eventing'], 'cpu_utilization': 1.34297520661157, 'mem_free': 1559941120, 'mem_total': 1930829824, 'swap_mem_used': 0, 'swap_mem_total': 1107292160} 2021-05-05 19:17:22 | INFO | MainProcess | MainThread | [basetestcase.print_cluster_stats] 10.112.206.105:8091 => {'services': ['index', 'n1ql'], 'cpu_utilization': 2.178423236514523, 'mem_free': 1511256064, 'mem_total': 1930829824, 'swap_mem_used': 0, 'swap_mem_total': 1107292160} 2021-05-05 19:17:22 | INFO | MainProcess | MainThread | [basetestcase.print_cluster_stats] --- End of cluster statistics --- 2021-05-05 19:17:22 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.101 with username:root, attempt#1 of 5 2021-05-05 19:17:22 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.101 as root 2021-05-05 19:17:22 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:17:22 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:17:24 | INFO | MainProcess | MainThread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:17:24 | INFO | MainProcess | MainThread | [basetestcase.get_nodes_from_services_map] list of eventing nodes in cluster: [ip:10.112.206.104 port:8091 ssh_username:root] 2021-05-05 19:17:24 | INFO | MainProcess | MainThread | [rest_client.set_indexer_storage_mode] settings/indexes params : storageMode=plasma 2021-05-05 19:17:24 | INFO | MainProcess | MainThread | [eventing_base.setUp] Setting the min possible memory quota so that adding mode nodes to the cluster wouldn't be a problem. 2021-05-05 19:17:24 | INFO | MainProcess | MainThread | [rest_client.set_service_memoryQuota] pools/default params : memoryQuota=330 2021-05-05 19:17:24 | INFO | MainProcess | MainThread | [rest_client.set_service_memoryQuota] pools/default params : indexMemoryQuota=256 2021-05-05 19:17:24 | INFO | MainProcess | MainThread | [rest_client.set_service_memoryQuota] pools/default params : eventingMemoryQuota=512 2021-05-05 19:17:24 | INFO | MainProcess | MainThread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:17:24 | INFO | MainProcess | MainThread | [eventing_base.setUp] ============== EventingBaseTest setup has completed ============== 2021-05-05 19:17:24 | INFO | MainProcess | MainThread | [rest_client.set_service_memoryQuota] pools/default params : memoryQuota=700 2021-05-05 19:17:24 | INFO | MainProcess | MainThread | [eventing_rebalance_collection.setUp] 200 2021-05-05 19:17:25 | INFO | MainProcess | Cluster_Thread | [rest_client.create_bucket] http://10.112.206.101:8091/pools/default/buckets with param: name=src_bucket&ramQuotaMB=200&replicaNumber=0&bucketType=membase&replicaIndex=1&threadsNumber=3&flushEnabled=1&evictionPolicy=valueOnly&compressionMode=passive&storageBackend=couchstore 2021-05-05 19:17:25 | INFO | MainProcess | Cluster_Thread | [rest_client.create_bucket] 0.03 seconds to create bucket src_bucket 2021-05-05 19:17:25 | INFO | MainProcess | Cluster_Thread | [bucket_helper.wait_for_memcached] waiting for memcached bucket : src_bucket in 10.112.206.101 to accept set ops 2021-05-05 19:17:26 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.101:11210 src_bucket 2021-05-05 19:17:26 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.102:11210 src_bucket 2021-05-05 19:17:26 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.103:11210 src_bucket 2021-05-05 19:17:26 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.101:11210 src_bucket 2021-05-05 19:17:26 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.102:11210 src_bucket 2021-05-05 19:17:26 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.103:11210 src_bucket 2021-05-05 19:17:26 | WARNING | MainProcess | Cluster_Thread | [task.check] vbucket map not ready after try 0 2021-05-05 19:17:26 | INFO | MainProcess | Cluster_Thread | [bucket_helper.wait_for_memcached] waiting for memcached bucket : src_bucket in 10.112.206.101 to accept set ops 2021-05-05 19:17:27 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.101:11210 src_bucket 2021-05-05 19:17:27 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.102:11210 src_bucket 2021-05-05 19:17:27 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.103:11210 src_bucket 2021-05-05 19:17:27 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.101:11210 src_bucket 2021-05-05 19:17:27 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.102:11210 src_bucket 2021-05-05 19:17:27 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.103:11210 src_bucket 2021-05-05 19:17:27 | WARNING | MainProcess | Cluster_Thread | [task.check] vbucket map not ready after try 1 2021-05-05 19:17:27 | INFO | MainProcess | Cluster_Thread | [bucket_helper.wait_for_memcached] waiting for memcached bucket : src_bucket in 10.112.206.101 to accept set ops 2021-05-05 19:17:27 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.101:11210 src_bucket 2021-05-05 19:17:28 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.102:11210 src_bucket 2021-05-05 19:17:28 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.103:11210 src_bucket 2021-05-05 19:17:28 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.101:11210 src_bucket 2021-05-05 19:17:28 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.102:11210 src_bucket 2021-05-05 19:17:28 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.103:11210 src_bucket 2021-05-05 19:17:28 | WARNING | MainProcess | Cluster_Thread | [task.check] vbucket map not ready after try 2 2021-05-05 19:17:28 | INFO | MainProcess | Cluster_Thread | [bucket_helper.wait_for_memcached] waiting for memcached bucket : src_bucket in 10.112.206.101 to accept set ops 2021-05-05 19:17:28 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.101:11210 src_bucket 2021-05-05 19:17:28 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.102:11210 src_bucket 2021-05-05 19:17:28 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.103:11210 src_bucket 2021-05-05 19:17:29 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.101:11210 src_bucket 2021-05-05 19:17:29 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.102:11210 src_bucket 2021-05-05 19:17:29 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.103:11210 src_bucket 2021-05-05 19:17:29 | WARNING | MainProcess | Cluster_Thread | [task.check] vbucket map not ready after try 3 2021-05-05 19:17:29 | INFO | MainProcess | Cluster_Thread | [bucket_helper.wait_for_memcached] waiting for memcached bucket : src_bucket in 10.112.206.101 to accept set ops 2021-05-05 19:17:29 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.101:11210 src_bucket 2021-05-05 19:17:29 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.102:11210 src_bucket 2021-05-05 19:17:29 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.103:11210 src_bucket 2021-05-05 19:17:29 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.101:11210 src_bucket 2021-05-05 19:17:29 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.102:11210 src_bucket 2021-05-05 19:17:29 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.103:11210 src_bucket 2021-05-05 19:17:30 | WARNING | MainProcess | Cluster_Thread | [task.check] vbucket map not ready after try 4 2021-05-05 19:17:30 | INFO | MainProcess | Cluster_Thread | [bucket_helper.wait_for_memcached] waiting for memcached bucket : src_bucket in 10.112.206.101 to accept set ops 2021-05-05 19:17:30 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.101:11210 src_bucket 2021-05-05 19:17:30 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.102:11210 src_bucket 2021-05-05 19:17:30 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.103:11210 src_bucket 2021-05-05 19:17:30 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.101:11210 src_bucket 2021-05-05 19:17:30 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.102:11210 src_bucket 2021-05-05 19:17:30 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.103:11210 src_bucket 2021-05-05 19:17:30 | WARNING | MainProcess | Cluster_Thread | [task.check] vbucket map not ready after try 5 2021-05-05 19:17:31 | INFO | MainProcess | Cluster_Thread | [rest_client.create_bucket] http://10.112.206.101:8091/pools/default/buckets with param: name=dst_bucket&ramQuotaMB=200&replicaNumber=0&bucketType=membase&replicaIndex=1&threadsNumber=3&flushEnabled=1&evictionPolicy=valueOnly&compressionMode=passive&storageBackend=couchstore 2021-05-05 19:17:31 | INFO | MainProcess | Cluster_Thread | [rest_client.create_bucket] 0.03 seconds to create bucket dst_bucket 2021-05-05 19:17:31 | INFO | MainProcess | Cluster_Thread | [bucket_helper.wait_for_memcached] waiting for memcached bucket : dst_bucket in 10.112.206.101 to accept set ops 2021-05-05 19:17:32 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.101:11210 dst_bucket 2021-05-05 19:17:33 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.102:11210 dst_bucket 2021-05-05 19:17:33 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.103:11210 dst_bucket 2021-05-05 19:17:33 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.101:11210 dst_bucket 2021-05-05 19:17:33 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.102:11210 dst_bucket 2021-05-05 19:17:33 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.103:11210 dst_bucket 2021-05-05 19:17:33 | WARNING | MainProcess | Cluster_Thread | [task.check] vbucket map not ready after try 0 2021-05-05 19:17:33 | INFO | MainProcess | Cluster_Thread | [bucket_helper.wait_for_memcached] waiting for memcached bucket : dst_bucket in 10.112.206.101 to accept set ops 2021-05-05 19:17:33 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.101:11210 dst_bucket 2021-05-05 19:17:33 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.102:11210 dst_bucket 2021-05-05 19:17:34 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.103:11210 dst_bucket 2021-05-05 19:17:34 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.101:11210 dst_bucket 2021-05-05 19:17:34 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.102:11210 dst_bucket 2021-05-05 19:17:34 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.103:11210 dst_bucket 2021-05-05 19:17:34 | WARNING | MainProcess | Cluster_Thread | [task.check] vbucket map not ready after try 1 2021-05-05 19:17:34 | INFO | MainProcess | Cluster_Thread | [bucket_helper.wait_for_memcached] waiting for memcached bucket : dst_bucket in 10.112.206.101 to accept set ops 2021-05-05 19:17:34 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.101:11210 dst_bucket 2021-05-05 19:17:34 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.102:11210 dst_bucket 2021-05-05 19:17:34 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.103:11210 dst_bucket 2021-05-05 19:17:34 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.101:11210 dst_bucket 2021-05-05 19:17:35 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.102:11210 dst_bucket 2021-05-05 19:17:35 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.103:11210 dst_bucket 2021-05-05 19:17:35 | WARNING | MainProcess | Cluster_Thread | [task.check] vbucket map not ready after try 2 2021-05-05 19:17:35 | INFO | MainProcess | Cluster_Thread | [bucket_helper.wait_for_memcached] waiting for memcached bucket : dst_bucket in 10.112.206.101 to accept set ops 2021-05-05 19:17:35 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.101:11210 dst_bucket 2021-05-05 19:17:35 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.102:11210 dst_bucket 2021-05-05 19:17:35 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.103:11210 dst_bucket 2021-05-05 19:17:35 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.101:11210 dst_bucket 2021-05-05 19:17:35 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.102:11210 dst_bucket 2021-05-05 19:17:35 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.103:11210 dst_bucket 2021-05-05 19:17:36 | WARNING | MainProcess | Cluster_Thread | [task.check] vbucket map not ready after try 3 2021-05-05 19:17:36 | INFO | MainProcess | Cluster_Thread | [bucket_helper.wait_for_memcached] waiting for memcached bucket : dst_bucket in 10.112.206.101 to accept set ops 2021-05-05 19:17:36 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.101:11210 dst_bucket 2021-05-05 19:17:36 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.102:11210 dst_bucket 2021-05-05 19:17:36 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.103:11210 dst_bucket 2021-05-05 19:17:36 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.101:11210 dst_bucket 2021-05-05 19:17:36 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.102:11210 dst_bucket 2021-05-05 19:17:36 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.103:11210 dst_bucket 2021-05-05 19:17:36 | WARNING | MainProcess | Cluster_Thread | [task.check] vbucket map not ready after try 4 2021-05-05 19:17:36 | INFO | MainProcess | Cluster_Thread | [bucket_helper.wait_for_memcached] waiting for memcached bucket : dst_bucket in 10.112.206.101 to accept set ops 2021-05-05 19:17:37 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.101:11210 dst_bucket 2021-05-05 19:17:37 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.102:11210 dst_bucket 2021-05-05 19:17:37 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.103:11210 dst_bucket 2021-05-05 19:17:37 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.101:11210 dst_bucket 2021-05-05 19:17:37 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.102:11210 dst_bucket 2021-05-05 19:17:37 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.103:11210 dst_bucket 2021-05-05 19:17:37 | WARNING | MainProcess | Cluster_Thread | [task.check] vbucket map not ready after try 5 2021-05-05 19:17:38 | INFO | MainProcess | Cluster_Thread | [rest_client.create_bucket] http://10.112.206.101:8091/pools/default/buckets with param: name=metadata&ramQuotaMB=200&replicaNumber=0&bucketType=membase&replicaIndex=1&threadsNumber=3&flushEnabled=1&evictionPolicy=valueOnly&compressionMode=passive&storageBackend=couchstore 2021-05-05 19:17:38 | INFO | MainProcess | Cluster_Thread | [rest_client.create_bucket] 0.04 seconds to create bucket metadata 2021-05-05 19:17:38 | INFO | MainProcess | Cluster_Thread | [bucket_helper.wait_for_memcached] waiting for memcached bucket : metadata in 10.112.206.101 to accept set ops 2021-05-05 19:17:39 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.101:11210 metadata 2021-05-05 19:17:40 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.102:11210 metadata 2021-05-05 19:17:40 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.103:11210 metadata 2021-05-05 19:17:40 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.101:11210 metadata 2021-05-05 19:17:40 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.102:11210 metadata 2021-05-05 19:17:40 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.103:11210 metadata 2021-05-05 19:17:40 | WARNING | MainProcess | Cluster_Thread | [task.check] vbucket map not ready after try 0 2021-05-05 19:17:40 | INFO | MainProcess | Cluster_Thread | [bucket_helper.wait_for_memcached] waiting for memcached bucket : metadata in 10.112.206.101 to accept set ops 2021-05-05 19:17:40 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.101:11210 metadata 2021-05-05 19:17:41 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.102:11210 metadata 2021-05-05 19:17:41 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.103:11210 metadata 2021-05-05 19:17:41 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.101:11210 metadata 2021-05-05 19:17:41 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.102:11210 metadata 2021-05-05 19:17:41 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.103:11210 metadata 2021-05-05 19:17:41 | WARNING | MainProcess | Cluster_Thread | [task.check] vbucket map not ready after try 1 2021-05-05 19:17:41 | INFO | MainProcess | Cluster_Thread | [bucket_helper.wait_for_memcached] waiting for memcached bucket : metadata in 10.112.206.101 to accept set ops 2021-05-05 19:17:41 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.101:11210 metadata 2021-05-05 19:17:41 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.102:11210 metadata 2021-05-05 19:17:41 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.103:11210 metadata 2021-05-05 19:17:42 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.101:11210 metadata 2021-05-05 19:17:42 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.102:11210 metadata 2021-05-05 19:17:42 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.103:11210 metadata 2021-05-05 19:17:42 | WARNING | MainProcess | Cluster_Thread | [task.check] vbucket map not ready after try 2 2021-05-05 19:17:42 | INFO | MainProcess | Cluster_Thread | [bucket_helper.wait_for_memcached] waiting for memcached bucket : metadata in 10.112.206.101 to accept set ops 2021-05-05 19:17:42 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.101:11210 metadata 2021-05-05 19:17:42 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.102:11210 metadata 2021-05-05 19:17:42 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.103:11210 metadata 2021-05-05 19:17:42 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.101:11210 metadata 2021-05-05 19:17:42 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.102:11210 metadata 2021-05-05 19:17:43 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.103:11210 metadata 2021-05-05 19:17:43 | WARNING | MainProcess | Cluster_Thread | [task.check] vbucket map not ready after try 3 2021-05-05 19:17:43 | INFO | MainProcess | Cluster_Thread | [bucket_helper.wait_for_memcached] waiting for memcached bucket : metadata in 10.112.206.101 to accept set ops 2021-05-05 19:17:43 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.101:11210 metadata 2021-05-05 19:17:43 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.102:11210 metadata 2021-05-05 19:17:43 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.103:11210 metadata 2021-05-05 19:17:43 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.101:11210 metadata 2021-05-05 19:17:43 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.102:11210 metadata 2021-05-05 19:17:43 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.103:11210 metadata 2021-05-05 19:17:43 | WARNING | MainProcess | Cluster_Thread | [task.check] vbucket map not ready after try 4 2021-05-05 19:17:43 | INFO | MainProcess | Cluster_Thread | [bucket_helper.wait_for_memcached] waiting for memcached bucket : metadata in 10.112.206.101 to accept set ops 2021-05-05 19:17:44 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.101:11210 metadata 2021-05-05 19:17:44 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.102:11210 metadata 2021-05-05 19:17:44 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.103:11210 metadata 2021-05-05 19:17:44 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.101:11210 metadata 2021-05-05 19:17:44 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.102:11210 metadata 2021-05-05 19:17:44 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.103:11210 metadata 2021-05-05 19:17:44 | WARNING | MainProcess | Cluster_Thread | [task.check] vbucket map not ready after try 5 2021-05-05 19:17:44 | INFO | MainProcess | MainThread | [rest_client.create_scope] http://10.112.206.104:8091/pools/default/buckets/dst_bucket/scopes with params: name=scope_0 2021-05-05 19:17:44 | INFO | MainProcess | MainThread | [rest_client.create_scope] Scope created dst_bucket->scope_0 {'uid': '1'} 2021-05-05 19:17:44 | INFO | MainProcess | MainThread | [rest_client.create_scope] http://10.112.206.104:8091/pools/default/buckets/dst_bucket/scopes with params: name=scope_1 2021-05-05 19:17:44 | INFO | MainProcess | MainThread | [rest_client.create_scope] Scope created dst_bucket->scope_1 {'uid': '2'} 2021-05-05 19:17:44 | INFO | MainProcess | MainThread | [rest_client.create_scope] http://10.112.206.104:8091/pools/default/buckets/dst_bucket/scopes with params: name=scope_2 2021-05-05 19:17:44 | INFO | MainProcess | MainThread | [rest_client.create_scope] Scope created dst_bucket->scope_2 {'uid': '3'} 2021-05-05 19:17:44 | INFO | MainProcess | MainThread | [rest_client.create_scope] http://10.112.206.104:8091/pools/default/buckets/dst_bucket/scopes with params: name=scope_3 2021-05-05 19:17:44 | INFO | MainProcess | MainThread | [rest_client.create_scope] Scope created dst_bucket->scope_3 {'uid': '4'} 2021-05-05 19:17:44 | INFO | MainProcess | MainThread | [rest_client.create_scope] http://10.112.206.104:8091/pools/default/buckets/dst_bucket/scopes with params: name=scope_4 2021-05-05 19:17:44 | INFO | MainProcess | MainThread | [rest_client.create_scope] Scope created dst_bucket->scope_4 {'uid': '5'} 2021-05-05 19:17:44 | INFO | MainProcess | MainThread | [rest_client.create_scope] http://10.112.206.104:8091/pools/default/buckets/src_bucket/scopes with params: name=scope_0 2021-05-05 19:17:44 | INFO | MainProcess | MainThread | [rest_client.create_scope] Scope created src_bucket->scope_0 {'uid': '1'} 2021-05-05 19:17:45 | INFO | MainProcess | MainThread | [rest_client.create_scope] http://10.112.206.104:8091/pools/default/buckets/src_bucket/scopes with params: name=scope_1 2021-05-05 19:17:45 | INFO | MainProcess | MainThread | [rest_client.create_scope] Scope created src_bucket->scope_1 {'uid': '2'} 2021-05-05 19:17:45 | INFO | MainProcess | MainThread | [rest_client.create_scope] http://10.112.206.104:8091/pools/default/buckets/src_bucket/scopes with params: name=scope_2 2021-05-05 19:17:45 | INFO | MainProcess | MainThread | [rest_client.create_scope] Scope created src_bucket->scope_2 {'uid': '3'} 2021-05-05 19:17:45 | INFO | MainProcess | MainThread | [rest_client.create_scope] http://10.112.206.104:8091/pools/default/buckets/src_bucket/scopes with params: name=scope_3 2021-05-05 19:17:45 | INFO | MainProcess | MainThread | [rest_client.create_scope] Scope created src_bucket->scope_3 {'uid': '4'} 2021-05-05 19:17:45 | INFO | MainProcess | MainThread | [rest_client.create_scope] http://10.112.206.104:8091/pools/default/buckets/src_bucket/scopes with params: name=scope_4 2021-05-05 19:17:45 | INFO | MainProcess | MainThread | [rest_client.create_scope] Scope created src_bucket->scope_4 {'uid': '5'} 2021-05-05 19:17:45 | INFO | MainProcess | MainThread | [rest_client._create_single_collection] http://10.112.206.104:8091/pools/default/buckets/dst_bucket/scopes/scope_1/collections with params: name=coll_0 2021-05-05 19:17:45 | INFO | MainProcess | MainThread | [rest_client.create_collection] Collection created dst_bucket->scope_1->coll_0 manifest:{'uid': '6'} 2021-05-05 19:17:45 | INFO | MainProcess | MainThread | [rest_client._create_single_collection] http://10.112.206.104:8091/pools/default/buckets/dst_bucket/scopes/scope_1/collections with params: name=coll_1 2021-05-05 19:17:45 | INFO | MainProcess | MainThread | [rest_client.create_collection] Collection created dst_bucket->scope_1->coll_1 manifest:{'uid': '7'} 2021-05-05 19:17:45 | INFO | MainProcess | MainThread | [rest_client._create_single_collection] http://10.112.206.104:8091/pools/default/buckets/dst_bucket/scopes/scope_1/collections with params: name=coll_2 2021-05-05 19:17:45 | INFO | MainProcess | MainThread | [rest_client.create_collection] Collection created dst_bucket->scope_1->coll_2 manifest:{'uid': '8'} 2021-05-05 19:17:45 | INFO | MainProcess | MainThread | [rest_client._create_single_collection] http://10.112.206.104:8091/pools/default/buckets/dst_bucket/scopes/scope_1/collections with params: name=coll_3 2021-05-05 19:17:45 | INFO | MainProcess | MainThread | [rest_client.create_collection] Collection created dst_bucket->scope_1->coll_3 manifest:{'uid': '9'} 2021-05-05 19:17:45 | INFO | MainProcess | MainThread | [rest_client._create_single_collection] http://10.112.206.104:8091/pools/default/buckets/dst_bucket/scopes/scope_1/collections with params: name=coll_4 2021-05-05 19:17:45 | INFO | MainProcess | MainThread | [rest_client.create_collection] Collection created dst_bucket->scope_1->coll_4 manifest:{'uid': 'a'} 2021-05-05 19:17:45 | INFO | MainProcess | MainThread | [rest_client._create_single_collection] http://10.112.206.104:8091/pools/default/buckets/src_bucket/scopes/scope_1/collections with params: name=coll_0 2021-05-05 19:17:45 | INFO | MainProcess | MainThread | [rest_client.create_collection] Collection created src_bucket->scope_1->coll_0 manifest:{'uid': '6'} 2021-05-05 19:17:45 | INFO | MainProcess | MainThread | [rest_client._create_single_collection] http://10.112.206.104:8091/pools/default/buckets/src_bucket/scopes/scope_1/collections with params: name=coll_1 2021-05-05 19:17:45 | INFO | MainProcess | MainThread | [rest_client.create_collection] Collection created src_bucket->scope_1->coll_1 manifest:{'uid': '7'} 2021-05-05 19:17:45 | INFO | MainProcess | MainThread | [rest_client._create_single_collection] http://10.112.206.104:8091/pools/default/buckets/src_bucket/scopes/scope_1/collections with params: name=coll_2 2021-05-05 19:17:45 | INFO | MainProcess | MainThread | [rest_client.create_collection] Collection created src_bucket->scope_1->coll_2 manifest:{'uid': '8'} 2021-05-05 19:17:45 | INFO | MainProcess | MainThread | [rest_client._create_single_collection] http://10.112.206.104:8091/pools/default/buckets/src_bucket/scopes/scope_1/collections with params: name=coll_3 2021-05-05 19:17:45 | INFO | MainProcess | MainThread | [rest_client.create_collection] Collection created src_bucket->scope_1->coll_3 manifest:{'uid': '9'} 2021-05-05 19:17:45 | INFO | MainProcess | MainThread | [rest_client._create_single_collection] http://10.112.206.104:8091/pools/default/buckets/src_bucket/scopes/scope_1/collections with params: name=coll_4 2021-05-05 19:17:45 | INFO | MainProcess | MainThread | [rest_client.create_collection] Collection created src_bucket->scope_1->coll_4 manifest:{'uid': 'a'} 2021-05-05 19:17:45 | INFO | MainProcess | MainThread | [basetestcase.get_nodes_from_services_map] list of eventing nodes in cluster: [ip:10.112.206.104 port:8091 ssh_username:root] 2021-05-05 19:17:45 | INFO | MainProcess | MainThread | [eventing_base.print_eventing_stats_from_all_eventing_nodes] Stats for Node 10.112.206.104 is [] 2021-05-05 19:17:45 | INFO | MainProcess | MainThread | [eventing_base.tearDown] ============== EventingBaseTest tearDown has started ============== 2021-05-05 19:17:45 | INFO | MainProcess | MainThread | [basetestcase.get_nodes_from_services_map] list of eventing nodes in cluster: [ip:10.112.206.104 port:8091 ssh_username:root] 2021-05-05 19:17:45 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.104 with username:root, attempt#1 of 5 2021-05-05 19:17:45 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.104 as root 2021-05-05 19:17:45 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:17:46 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:17:46 | INFO | MainProcess | MainThread | [rest_client.diag_eval] /diag/eval status on 10.112.206.104:8091: True content: "/opt/couchbase/var/lib/couchbase/logs" command: filename:absname(element(2, application:get_env(ns_server,error_logger_mf_dir))). 2021-05-05 19:17:46 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] running command.raw on 10.112.206.104: zgrep "panic" "/opt/couchbase/var/lib/couchbase/logs"/eventing.log* | wc -l 2021-05-05 19:17:46 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:17:46 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] running command.raw on 10.112.206.104: ls "/opt/couchbase/var/lib/couchbase/logs"/../crash/| wc -l 2021-05-05 19:17:46 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:17:46 | INFO | MainProcess | MainThread | [eventing_base.check_eventing_logs_for_panic] ===== CORE DUMPS SEEN ON EVENTING NODES, SERVER 10.112.206.104 : 1 crashes seen ===== 2021-05-05 19:17:46 | INFO | MainProcess | MainThread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:17:46 | INFO | MainProcess | MainThread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+metadata 2021-05-05 19:17:46 | INFO | MainProcess | MainThread | [eventing_base.tearDown] number of documents in metadata bucket 0 2021-05-05 19:17:46 | INFO | MainProcess | MainThread | [basetestcase.print_cluster_stats] ------- Cluster statistics ------- 2021-05-05 19:17:46 | INFO | MainProcess | MainThread | [basetestcase.print_cluster_stats] 10.112.206.101:8091 => {'services': ['kv'], 'cpu_utilization': 58.83069427527406, 'mem_free': 1412931584, 'mem_total': 1930829824, 'swap_mem_used': 0, 'swap_mem_total': 1107292160} 2021-05-05 19:17:46 | INFO | MainProcess | MainThread | [basetestcase.print_cluster_stats] 10.112.206.102:8091 => {'services': ['kv'], 'cpu_utilization': 45.56354916067146, 'mem_free': 1465577472, 'mem_total': 1930829824, 'swap_mem_used': 0, 'swap_mem_total': 1107292160} 2021-05-05 19:17:46 | INFO | MainProcess | MainThread | [basetestcase.print_cluster_stats] 10.112.206.103:8091 => {'services': ['kv'], 'cpu_utilization': 35.05882352941177, 'mem_free': 1463611392, 'mem_total': 1930829824, 'swap_mem_used': 0, 'swap_mem_total': 1107292160} 2021-05-05 19:17:46 | INFO | MainProcess | MainThread | [basetestcase.print_cluster_stats] 10.112.206.104:8091 => {'services': ['eventing'], 'cpu_utilization': 4.931794333683106, 'mem_free': 1549725696, 'mem_total': 1930829824, 'swap_mem_used': 0, 'swap_mem_total': 1107292160} 2021-05-05 19:17:46 | INFO | MainProcess | MainThread | [basetestcase.print_cluster_stats] 10.112.206.105:8091 => {'services': ['index', 'n1ql'], 'cpu_utilization': 8.577633007600435, 'mem_free': 1445560320, 'mem_total': 1930829824, 'swap_mem_used': 0, 'swap_mem_total': 1107292160} 2021-05-05 19:17:46 | INFO | MainProcess | MainThread | [basetestcase.print_cluster_stats] --- End of cluster statistics --- 2021-05-05 19:17:46 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.101 with username:root, attempt#1 of 5 2021-05-05 19:17:46 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.101 as root 2021-05-05 19:17:46 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:17:47 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:17:47 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.102 with username:root, attempt#1 of 5 2021-05-05 19:17:47 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.102 as root 2021-05-05 19:17:47 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:17:47 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:17:47 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.103 with username:root, attempt#1 of 5 2021-05-05 19:17:47 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.103 as root 2021-05-05 19:17:47 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:17:48 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:17:48 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.104 with username:root, attempt#1 of 5 2021-05-05 19:17:48 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.104 as root 2021-05-05 19:17:48 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:17:48 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:17:48 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.105 with username:root, attempt#1 of 5 2021-05-05 19:17:48 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.105 as root 2021-05-05 19:17:48 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:17:48 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:18:04 | INFO | MainProcess | MainThread | [basetestcase.tearDown] ============== basetestcase cleanup was started for test #1 suite_setUp ============== 2021-05-05 19:18:04 | WARNING | MainProcess | MainThread | [basetestcase.tearDown] Alerts were found: [{'msg': "The time on node 'ns_1@10.112.206.102' is not synchronized. Please ensure that NTP is set up correctly on all nodes and that clocks are synchronized.", 'serverTime': '2021-05-05T06:48:10.000Z', 'disableUIPopUp': False}, {'msg': "The time on node 'ns_1@10.112.206.103' is not synchronized. Please ensure that NTP is set up correctly on all nodes and that clocks are synchronized.", 'serverTime': '2021-05-05T06:48:38.000Z', 'disableUIPopUp': False}, {'msg': "The time on node 'ns_1@10.112.206.104' is not synchronized. Please ensure that NTP is set up correctly on all nodes and that clocks are synchronized.", 'serverTime': '2021-05-05T06:49:06.000Z', 'disableUIPopUp': False}, {'msg': "The time on node 'ns_1@10.112.206.105' is not synchronized. Please ensure that NTP is set up correctly on all nodes and that clocks are synchronized.", 'serverTime': '2021-05-05T06:49:34.000Z', 'disableUIPopUp': False}] 2021-05-05 19:18:04 | INFO | MainProcess | MainThread | [bucket_helper.delete_all_buckets_or_assert] deleting existing buckets ['dst_bucket', 'metadata', 'src_bucket'] on 10.112.206.101 2021-05-05 19:18:04 | INFO | MainProcess | MainThread | [bucket_helper.wait_for_bucket_deletion] waiting for bucket deletion to complete.... 2021-05-05 19:18:04 | INFO | MainProcess | MainThread | [rest_client.bucket_exists] node 10.112.206.101 existing buckets : ['metadata', 'src_bucket'] 2021-05-05 19:18:04 | INFO | MainProcess | MainThread | [bucket_helper.delete_all_buckets_or_assert] deleted bucket : dst_bucket from 10.112.206.101 2021-05-05 19:18:04 | INFO | MainProcess | MainThread | [bucket_helper.wait_for_bucket_deletion] waiting for bucket deletion to complete.... 2021-05-05 19:18:04 | INFO | MainProcess | MainThread | [rest_client.bucket_exists] node 10.112.206.101 existing buckets : ['src_bucket'] 2021-05-05 19:18:04 | INFO | MainProcess | MainThread | [bucket_helper.delete_all_buckets_or_assert] deleted bucket : metadata from 10.112.206.101 2021-05-05 19:18:05 | INFO | MainProcess | MainThread | [bucket_helper.wait_for_bucket_deletion] waiting for bucket deletion to complete.... 2021-05-05 19:18:05 | INFO | MainProcess | MainThread | [rest_client.bucket_exists] node 10.112.206.101 existing buckets : [] 2021-05-05 19:18:05 | INFO | MainProcess | MainThread | [bucket_helper.delete_all_buckets_or_assert] deleted bucket : src_bucket from 10.112.206.101 2021-05-05 19:18:05 | INFO | MainProcess | MainThread | [bucket_helper.delete_all_buckets_or_assert] Could not find any buckets for node 10.112.206.102, nothing to delete 2021-05-05 19:18:05 | INFO | MainProcess | MainThread | [bucket_helper.delete_all_buckets_or_assert] Could not find any buckets for node 10.112.206.103, nothing to delete 2021-05-05 19:18:05 | INFO | MainProcess | MainThread | [bucket_helper.delete_all_buckets_or_assert] Could not find any buckets for node 10.112.206.104, nothing to delete 2021-05-05 19:18:05 | INFO | MainProcess | MainThread | [bucket_helper.delete_all_buckets_or_assert] Could not find any buckets for node 10.112.206.105, nothing to delete 2021-05-05 19:18:05 | INFO | MainProcess | MainThread | [rest_client.is_ns_server_running] -->is_ns_server_running? 2021-05-05 19:18:05 | INFO | MainProcess | MainThread | [cluster_helper.cleanup_cluster] rebalancing all nodes in order to remove nodes 2021-05-05 19:18:05 | INFO | MainProcess | MainThread | [rest_client.rebalance] rebalance params : {'knownNodes': 'ns_1@10.112.206.101,ns_1@10.112.206.102,ns_1@10.112.206.103,ns_1@10.112.206.104,ns_1@10.112.206.105', 'ejectedNodes': 'ns_1@10.112.206.102,ns_1@10.112.206.103,ns_1@10.112.206.104,ns_1@10.112.206.105', 'user': 'Administrator', 'password': 'password'} 2021-05-05 19:18:05 | INFO | MainProcess | MainThread | [rest_client.rebalance] rebalance operation started 2021-05-05 19:18:05 | INFO | MainProcess | MainThread | [rest_client._rebalance_status_and_progress] rebalance percentage : 0.00 % 2021-05-05 19:18:15 | INFO | MainProcess | MainThread | [rest_client._rebalance_status_and_progress] rebalance percentage : 70.00 % 2021-05-05 19:18:25 | INFO | MainProcess | MainThread | [rest_client._rebalance_status_and_progress] rebalance percentage : 70.00 % 2021-05-05 19:18:45 | INFO | MainProcess | MainThread | [rest_client.monitorRebalance] rebalance progress took 40.02 seconds 2021-05-05 19:18:45 | INFO | MainProcess | MainThread | [rest_client.monitorRebalance] sleep for 10 seconds after rebalance... 2021-05-05 19:18:55 | INFO | MainProcess | MainThread | [cluster_helper.cleanup_cluster] removed all the nodes from cluster associated with ip:10.112.206.101 port:8091 ssh_username:root ? [('ns_1@10.112.206.102', 8091), ('ns_1@10.112.206.103', 8091), ('ns_1@10.112.206.104', 8091), ('ns_1@10.112.206.105', 8091)] 2021-05-05 19:18:55 | INFO | MainProcess | MainThread | [cluster_helper.wait_for_ns_servers_or_assert] waiting for ns_server @ 10.112.206.101:8091 2021-05-05 19:18:55 | INFO | MainProcess | MainThread | [rest_client.is_ns_server_running] -->is_ns_server_running? 2021-05-05 19:18:55 | INFO | MainProcess | MainThread | [cluster_helper.wait_for_ns_servers_or_assert] ns_server @ 10.112.206.101:8091 is running 2021-05-05 19:18:55 | INFO | MainProcess | MainThread | [cluster_helper.wait_for_ns_servers_or_assert] waiting for ns_server @ 10.112.206.102:8091 2021-05-05 19:18:55 | INFO | MainProcess | MainThread | [rest_client.is_ns_server_running] -->is_ns_server_running? 2021-05-05 19:18:55 | INFO | MainProcess | MainThread | [cluster_helper.wait_for_ns_servers_or_assert] ns_server @ 10.112.206.102:8091 is running 2021-05-05 19:18:55 | INFO | MainProcess | MainThread | [cluster_helper.wait_for_ns_servers_or_assert] waiting for ns_server @ 10.112.206.103:8091 2021-05-05 19:18:55 | INFO | MainProcess | MainThread | [rest_client.is_ns_server_running] -->is_ns_server_running? 2021-05-05 19:18:55 | INFO | MainProcess | MainThread | [cluster_helper.wait_for_ns_servers_or_assert] ns_server @ 10.112.206.103:8091 is running 2021-05-05 19:18:55 | INFO | MainProcess | MainThread | [cluster_helper.wait_for_ns_servers_or_assert] waiting for ns_server @ 10.112.206.104:8091 2021-05-05 19:18:55 | INFO | MainProcess | MainThread | [rest_client.is_ns_server_running] -->is_ns_server_running? 2021-05-05 19:18:55 | INFO | MainProcess | MainThread | [cluster_helper.wait_for_ns_servers_or_assert] ns_server @ 10.112.206.104:8091 is running 2021-05-05 19:18:55 | INFO | MainProcess | MainThread | [cluster_helper.wait_for_ns_servers_or_assert] waiting for ns_server @ 10.112.206.105:8091 2021-05-05 19:18:55 | INFO | MainProcess | MainThread | [rest_client.is_ns_server_running] -->is_ns_server_running? 2021-05-05 19:18:55 | INFO | MainProcess | MainThread | [cluster_helper.wait_for_ns_servers_or_assert] ns_server @ 10.112.206.105:8091 is running 2021-05-05 19:18:55 | INFO | MainProcess | MainThread | [basetestcase.tearDown] ============== basetestcase cleanup was finished for test #1 suite_setUp ============== 2021-05-05 19:18:55 | INFO | MainProcess | MainThread | [basetestcase.tearDown] closing all ssh connections 2021-05-05 19:18:55 | INFO | MainProcess | MainThread | [basetestcase.tearDown] closing all memcached connections 2021-05-05 19:18:55 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.101 with username:root, attempt#1 of 5 2021-05-05 19:18:55 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.101 as root 2021-05-05 19:18:55 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:18:55 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:18:55 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.101 with username:root, attempt#1 of 5 2021-05-05 19:18:56 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.101 as root 2021-05-05 19:18:56 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:18:56 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:18:56 | INFO | MainProcess | MainThread | [remote_util.is_couchbase_installed] 10.112.206.101 **** The linux version file /opt/couchbase/ VERSION.txt exists 2021-05-05 19:18:56 | INFO | MainProcess | MainThread | [remote_util.stop_couchbase] Running systemd command on this server 2021-05-05 19:18:56 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] running command.raw on 10.112.206.101: systemctl stop couchbase-server.service 2021-05-05 19:18:57 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:18:57 | INFO | MainProcess | MainThread | [basetestcase.stop_server] Couchbase stopped 2021-05-05 19:18:57 | INFO | MainProcess | MainThread | [basetestcase.sleep] sleep for 5 secs. ... 2021-05-05 19:19:02 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] running command.raw on 10.112.206.101: rm -rf /opt/couchbase/var/lib/couchbase/data/* 2021-05-05 19:19:02 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:19:02 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] running command.raw on 10.112.206.101: rm -rf /opt/couchbase/var/lib/couchbase/config/* 2021-05-05 19:19:02 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:19:02 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.101 with username:root, attempt#1 of 5 2021-05-05 19:19:03 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.101 as root 2021-05-05 19:19:03 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:19:03 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:19:03 | INFO | MainProcess | MainThread | [remote_util.is_couchbase_installed] 10.112.206.101 **** The linux version file /opt/couchbase/ VERSION.txt exists 2021-05-05 19:19:03 | INFO | MainProcess | MainThread | [remote_util.is_process_running] Checking for process beam.smp on linux 2021-05-05 19:19:03 | INFO | MainProcess | MainThread | [remote_util.start_couchbase] Starting couchbase server 2021-05-05 19:19:03 | INFO | MainProcess | MainThread | [remote_util.start_couchbase] Running systemd command on this server 2021-05-05 19:19:03 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] running command.raw on 10.112.206.101: systemctl start couchbase-server.service 2021-05-05 19:19:03 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:19:03 | INFO | MainProcess | MainThread | [remote_util.sleep] 10.112.206.101:sleep for 5 secs. waiting for couchbase server to come up ... 2021-05-05 19:19:08 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] running command.raw on 10.112.206.101: systemctl status couchbase-server.service | grep ExecStop=/opt/couchbase/bin/couchbase-server 2021-05-05 19:19:08 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:19:08 | INFO | MainProcess | MainThread | [remote_util.start_couchbase] Couchbase server status: [' Process: 28663 ExecStop=/opt/couchbase/bin/couchbase-server -k (code=exited, status=0/SUCCESS)'] 2021-05-05 19:19:08 | INFO | MainProcess | MainThread | [remote_util.is_process_running] Checking for process beam.smp on linux 2021-05-05 19:19:08 | INFO | MainProcess | MainThread | [remote_util.is_process_running] process beam.smp is running on 10.112.206.101: with pid 28744 2021-05-05 19:19:08 | INFO | MainProcess | MainThread | [basetestcase.start_server] Couchbase started 2021-05-05 19:19:08 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.102 with username:root, attempt#1 of 5 2021-05-05 19:19:08 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.102 as root 2021-05-05 19:19:08 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:19:09 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:19:09 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.102 with username:root, attempt#1 of 5 2021-05-05 19:19:09 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.102 as root 2021-05-05 19:19:09 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:19:09 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:19:09 | INFO | MainProcess | MainThread | [remote_util.is_couchbase_installed] 10.112.206.102 **** The linux version file /opt/couchbase/ VERSION.txt exists 2021-05-05 19:19:09 | INFO | MainProcess | MainThread | [remote_util.stop_couchbase] Running systemd command on this server 2021-05-05 19:19:09 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] running command.raw on 10.112.206.102: systemctl stop couchbase-server.service 2021-05-05 19:19:10 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:19:10 | INFO | MainProcess | MainThread | [basetestcase.stop_server] Couchbase stopped 2021-05-05 19:19:11 | INFO | MainProcess | MainThread | [basetestcase.sleep] sleep for 5 secs. ... 2021-05-05 19:19:16 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] running command.raw on 10.112.206.102: rm -rf /opt/couchbase/var/lib/couchbase/data/* 2021-05-05 19:19:16 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:19:16 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] running command.raw on 10.112.206.102: rm -rf /opt/couchbase/var/lib/couchbase/config/* 2021-05-05 19:19:16 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:19:16 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.102 with username:root, attempt#1 of 5 2021-05-05 19:19:16 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.102 as root 2021-05-05 19:19:16 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:19:16 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:19:16 | INFO | MainProcess | MainThread | [remote_util.is_couchbase_installed] 10.112.206.102 **** The linux version file /opt/couchbase/ VERSION.txt exists 2021-05-05 19:19:16 | INFO | MainProcess | MainThread | [remote_util.is_process_running] Checking for process beam.smp on linux 2021-05-05 19:19:16 | INFO | MainProcess | MainThread | [remote_util.start_couchbase] Starting couchbase server 2021-05-05 19:19:16 | INFO | MainProcess | MainThread | [remote_util.start_couchbase] Running systemd command on this server 2021-05-05 19:19:16 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] running command.raw on 10.112.206.102: systemctl start couchbase-server.service 2021-05-05 19:19:16 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:19:16 | INFO | MainProcess | MainThread | [remote_util.sleep] 10.112.206.102:sleep for 5 secs. waiting for couchbase server to come up ... 2021-05-05 19:19:21 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] running command.raw on 10.112.206.102: systemctl status couchbase-server.service | grep ExecStop=/opt/couchbase/bin/couchbase-server 2021-05-05 19:19:21 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:19:21 | INFO | MainProcess | MainThread | [remote_util.start_couchbase] Couchbase server status: [' Process: 9538 ExecStop=/opt/couchbase/bin/couchbase-server -k (code=exited, status=0/SUCCESS)'] 2021-05-05 19:19:21 | INFO | MainProcess | MainThread | [remote_util.is_process_running] Checking for process beam.smp on linux 2021-05-05 19:19:21 | INFO | MainProcess | MainThread | [remote_util.is_process_running] process beam.smp is running on 10.112.206.102: with pid 9619 2021-05-05 19:19:21 | INFO | MainProcess | MainThread | [basetestcase.start_server] Couchbase started 2021-05-05 19:19:21 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.103 with username:root, attempt#1 of 5 2021-05-05 19:19:21 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.103 as root 2021-05-05 19:19:21 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:19:22 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:19:22 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.103 with username:root, attempt#1 of 5 2021-05-05 19:19:22 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.103 as root 2021-05-05 19:19:22 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:19:22 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:19:22 | INFO | MainProcess | MainThread | [remote_util.is_couchbase_installed] 10.112.206.103 **** The linux version file /opt/couchbase/ VERSION.txt exists 2021-05-05 19:19:22 | INFO | MainProcess | MainThread | [remote_util.stop_couchbase] Running systemd command on this server 2021-05-05 19:19:22 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] running command.raw on 10.112.206.103: systemctl stop couchbase-server.service 2021-05-05 19:19:24 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:19:24 | INFO | MainProcess | MainThread | [basetestcase.stop_server] Couchbase stopped 2021-05-05 19:19:24 | INFO | MainProcess | MainThread | [basetestcase.sleep] sleep for 5 secs. ... 2021-05-05 19:19:29 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] running command.raw on 10.112.206.103: rm -rf /opt/couchbase/var/lib/couchbase/data/* 2021-05-05 19:19:29 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:19:29 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] running command.raw on 10.112.206.103: rm -rf /opt/couchbase/var/lib/couchbase/config/* 2021-05-05 19:19:29 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:19:29 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.103 with username:root, attempt#1 of 5 2021-05-05 19:19:29 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.103 as root 2021-05-05 19:19:29 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:19:29 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:19:29 | INFO | MainProcess | MainThread | [remote_util.is_couchbase_installed] 10.112.206.103 **** The linux version file /opt/couchbase/ VERSION.txt exists 2021-05-05 19:19:29 | INFO | MainProcess | MainThread | [remote_util.is_process_running] Checking for process beam.smp on linux 2021-05-05 19:19:29 | INFO | MainProcess | MainThread | [remote_util.start_couchbase] Starting couchbase server 2021-05-05 19:19:29 | INFO | MainProcess | MainThread | [remote_util.start_couchbase] Running systemd command on this server 2021-05-05 19:19:29 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] running command.raw on 10.112.206.103: systemctl start couchbase-server.service 2021-05-05 19:19:29 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:19:29 | INFO | MainProcess | MainThread | [remote_util.sleep] 10.112.206.103:sleep for 5 secs. waiting for couchbase server to come up ... 2021-05-05 19:19:34 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] running command.raw on 10.112.206.103: systemctl status couchbase-server.service | grep ExecStop=/opt/couchbase/bin/couchbase-server 2021-05-05 19:19:34 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:19:34 | INFO | MainProcess | MainThread | [remote_util.start_couchbase] Couchbase server status: [' Process: 7220 ExecStop=/opt/couchbase/bin/couchbase-server -k (code=exited, status=0/SUCCESS)'] 2021-05-05 19:19:34 | INFO | MainProcess | MainThread | [remote_util.is_process_running] Checking for process beam.smp on linux 2021-05-05 19:19:34 | INFO | MainProcess | MainThread | [remote_util.is_process_running] process beam.smp is running on 10.112.206.103: with pid 7301 2021-05-05 19:19:34 | INFO | MainProcess | MainThread | [basetestcase.start_server] Couchbase started 2021-05-05 19:19:34 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.104 with username:root, attempt#1 of 5 2021-05-05 19:19:34 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.104 as root 2021-05-05 19:19:35 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:19:35 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:19:35 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.104 with username:root, attempt#1 of 5 2021-05-05 19:19:35 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.104 as root 2021-05-05 19:19:35 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:19:35 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:19:35 | INFO | MainProcess | MainThread | [remote_util.is_couchbase_installed] 10.112.206.104 **** The linux version file /opt/couchbase/ VERSION.txt exists 2021-05-05 19:19:35 | INFO | MainProcess | MainThread | [remote_util.stop_couchbase] Running systemd command on this server 2021-05-05 19:19:35 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] running command.raw on 10.112.206.104: systemctl stop couchbase-server.service 2021-05-05 19:19:37 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:19:37 | INFO | MainProcess | MainThread | [basetestcase.stop_server] Couchbase stopped 2021-05-05 19:19:37 | INFO | MainProcess | MainThread | [basetestcase.sleep] sleep for 5 secs. ... 2021-05-05 19:19:42 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] running command.raw on 10.112.206.104: rm -rf /opt/couchbase/var/lib/couchbase/data/* 2021-05-05 19:19:42 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:19:42 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] running command.raw on 10.112.206.104: rm -rf /opt/couchbase/var/lib/couchbase/config/* 2021-05-05 19:19:42 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:19:42 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.104 with username:root, attempt#1 of 5 2021-05-05 19:19:42 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.104 as root 2021-05-05 19:19:42 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:19:42 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:19:42 | INFO | MainProcess | MainThread | [remote_util.is_couchbase_installed] 10.112.206.104 **** The linux version file /opt/couchbase/ VERSION.txt exists 2021-05-05 19:19:42 | INFO | MainProcess | MainThread | [remote_util.is_process_running] Checking for process beam.smp on linux 2021-05-05 19:19:42 | INFO | MainProcess | MainThread | [remote_util.start_couchbase] Starting couchbase server 2021-05-05 19:19:42 | INFO | MainProcess | MainThread | [remote_util.start_couchbase] Running systemd command on this server 2021-05-05 19:19:42 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] running command.raw on 10.112.206.104: systemctl start couchbase-server.service 2021-05-05 19:19:42 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:19:42 | INFO | MainProcess | MainThread | [remote_util.sleep] 10.112.206.104:sleep for 5 secs. waiting for couchbase server to come up ... 2021-05-05 19:19:47 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] running command.raw on 10.112.206.104: systemctl status couchbase-server.service | grep ExecStop=/opt/couchbase/bin/couchbase-server 2021-05-05 19:19:47 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:19:47 | INFO | MainProcess | MainThread | [remote_util.start_couchbase] Couchbase server status: [' Process: 8899 ExecStop=/opt/couchbase/bin/couchbase-server -k (code=exited, status=0/SUCCESS)'] 2021-05-05 19:19:47 | INFO | MainProcess | MainThread | [remote_util.is_process_running] Checking for process beam.smp on linux 2021-05-05 19:19:48 | INFO | MainProcess | MainThread | [remote_util.is_process_running] process beam.smp is running on 10.112.206.104: with pid 8980 2021-05-05 19:19:48 | INFO | MainProcess | MainThread | [basetestcase.start_server] Couchbase started 2021-05-05 19:19:48 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.105 with username:root, attempt#1 of 5 2021-05-05 19:19:48 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.105 as root 2021-05-05 19:19:48 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:19:48 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:19:48 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.105 with username:root, attempt#1 of 5 2021-05-05 19:19:48 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.105 as root 2021-05-05 19:19:48 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:19:48 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:19:48 | INFO | MainProcess | MainThread | [remote_util.is_couchbase_installed] 10.112.206.105 **** The linux version file /opt/couchbase/ VERSION.txt exists 2021-05-05 19:19:49 | INFO | MainProcess | MainThread | [remote_util.stop_couchbase] Running systemd command on this server 2021-05-05 19:19:49 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] running command.raw on 10.112.206.105: systemctl stop couchbase-server.service 2021-05-05 19:19:50 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:19:50 | INFO | MainProcess | MainThread | [basetestcase.stop_server] Couchbase stopped 2021-05-05 19:19:50 | INFO | MainProcess | MainThread | [basetestcase.sleep] sleep for 5 secs. ... 2021-05-05 19:19:55 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] running command.raw on 10.112.206.105: rm -rf /opt/couchbase/var/lib/couchbase/data/* 2021-05-05 19:19:55 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:19:55 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] running command.raw on 10.112.206.105: rm -rf /opt/couchbase/var/lib/couchbase/config/* 2021-05-05 19:19:55 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:19:55 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.105 with username:root, attempt#1 of 5 2021-05-05 19:19:55 | INFO | MainProcess | MainThread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.105 as root 2021-05-05 19:19:55 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:19:56 | INFO | MainProcess | MainThread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:19:56 | INFO | MainProcess | MainThread | [remote_util.is_couchbase_installed] 10.112.206.105 **** The linux version file /opt/couchbase/ VERSION.txt exists 2021-05-05 19:19:56 | INFO | MainProcess | MainThread | [remote_util.is_process_running] Checking for process beam.smp on linux 2021-05-05 19:19:56 | INFO | MainProcess | MainThread | [remote_util.start_couchbase] Starting couchbase server 2021-05-05 19:19:56 | INFO | MainProcess | MainThread | [remote_util.start_couchbase] Running systemd command on this server 2021-05-05 19:19:56 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] running command.raw on 10.112.206.105: systemctl start couchbase-server.service 2021-05-05 19:19:56 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:19:56 | INFO | MainProcess | MainThread | [remote_util.sleep] 10.112.206.105:sleep for 5 secs. waiting for couchbase server to come up ... 2021-05-05 19:20:01 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] running command.raw on 10.112.206.105: systemctl status couchbase-server.service | grep ExecStop=/opt/couchbase/bin/couchbase-server 2021-05-05 19:20:01 | INFO | MainProcess | MainThread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:20:01 | INFO | MainProcess | MainThread | [remote_util.start_couchbase] Couchbase server status: [' Process: 7501 ExecStop=/opt/couchbase/bin/couchbase-server -k (code=exited, status=0/SUCCESS)'] 2021-05-05 19:20:01 | INFO | MainProcess | MainThread | [remote_util.is_process_running] Checking for process beam.smp on linux 2021-05-05 19:20:01 | INFO | MainProcess | MainThread | [remote_util.is_process_running] process beam.smp is running on 10.112.206.105: with pid 7582 2021-05-05 19:20:01 | INFO | MainProcess | MainThread | [basetestcase.start_server] Couchbase started 2021-05-05 19:20:01 | INFO | MainProcess | MainThread | [basetestcase.sleep] sleep for 10 secs. ... Cluster instance shutdown with force 2021-05-05 19:20:11 | INFO | MainProcess | MainThread | [eventing_base.tearDown] ============== EventingBaseTest tearDown has completed ============== ok ---------------------------------------------------------------------- Ran 1 test in 542.934s OK -->result: test_eventing_rebalance_with_multiple_kv_nodes (eventing.eventing_rebalance_collection.EventingRebalanceCollection) ... 2021-05-05 19:20:11 | INFO | MainProcess | test_thread | [eventing_base.setUp] ============== EventingBaseTest setup has started ============== 2021-05-05 19:20:11 | INFO | MainProcess | test_thread | [eventing_base.setUp] Starting Test: test_eventing_rebalance_with_multiple_kv_nodes 2021-05-05 19:20:11 | INFO | MainProcess | test_thread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.101 with username:root, attempt#1 of 5 2021-05-05 19:20:11 | INFO | MainProcess | test_thread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.101 as root 2021-05-05 19:20:11 | INFO | MainProcess | test_thread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:20:11 | INFO | MainProcess | test_thread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:20:11 | INFO | MainProcess | test_thread | [rest_client.is_ns_server_running] -->is_ns_server_running? 2021-05-05 19:20:11 | INFO | MainProcess | test_thread | [rest_client.get_nodes_version] Node version in cluster 7.0.0-5102-enterprise 2021-05-05 19:20:11 | ERROR | MainProcess | test_thread | [rest_client._http_request] GET http://10.112.206.101:8091/pools/default body: headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==', 'Accept': '*/*'} error: 404 reason: unknown b'"unknown pool"' auth: Administrator:password 2021-05-05 19:20:11 | INFO | MainProcess | test_thread | [rest_client.get_nodes_versions] Node versions in cluster [] 2021-05-05 19:20:11 | INFO | MainProcess | test_thread | [basetestcase.setUp] ============== basetestcase setup was started for test #1 test_eventing_rebalance_with_multiple_kv_nodes============== 2021-05-05 19:20:11 | ERROR | MainProcess | test_thread | [rest_client._http_request] GET http://10.112.206.101:8091/pools/default/ body: headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==', 'Accept': '*/*'} error: 404 reason: unknown b'"unknown pool"' auth: Administrator:password 2021-05-05 19:20:11 | INFO | MainProcess | test_thread | [bucket_helper.delete_all_buckets_or_assert] Could not find any buckets for node 10.112.206.101, nothing to delete 2021-05-05 19:20:11 | INFO | MainProcess | test_thread | [bucket_helper.delete_all_buckets_or_assert] Could not find any buckets for node 10.112.206.102, nothing to delete 2021-05-05 19:20:11 | INFO | MainProcess | test_thread | [bucket_helper.delete_all_buckets_or_assert] Could not find any buckets for node 10.112.206.103, nothing to delete 2021-05-05 19:20:11 | INFO | MainProcess | test_thread | [bucket_helper.delete_all_buckets_or_assert] Could not find any buckets for node 10.112.206.104, nothing to delete 2021-05-05 19:20:11 | INFO | MainProcess | test_thread | [bucket_helper.delete_all_buckets_or_assert] Could not find any buckets for node 10.112.206.105, nothing to delete 2021-05-05 19:20:11 | INFO | MainProcess | test_thread | [rest_client.is_ns_server_running] -->is_ns_server_running? 2021-05-05 19:20:11 | INFO | MainProcess | test_thread | [cluster_helper.wait_for_ns_servers_or_assert] waiting for ns_server @ 10.112.206.101:8091 2021-05-05 19:20:11 | INFO | MainProcess | test_thread | [rest_client.is_ns_server_running] -->is_ns_server_running? 2021-05-05 19:20:11 | INFO | MainProcess | test_thread | [cluster_helper.wait_for_ns_servers_or_assert] ns_server @ 10.112.206.101:8091 is running 2021-05-05 19:20:11 | INFO | MainProcess | test_thread | [cluster_helper.wait_for_ns_servers_or_assert] waiting for ns_server @ 10.112.206.102:8091 2021-05-05 19:20:11 | INFO | MainProcess | test_thread | [rest_client.is_ns_server_running] -->is_ns_server_running? 2021-05-05 19:20:11 | INFO | MainProcess | test_thread | [cluster_helper.wait_for_ns_servers_or_assert] ns_server @ 10.112.206.102:8091 is running 2021-05-05 19:20:11 | INFO | MainProcess | test_thread | [cluster_helper.wait_for_ns_servers_or_assert] waiting for ns_server @ 10.112.206.103:8091 2021-05-05 19:20:11 | INFO | MainProcess | test_thread | [rest_client.is_ns_server_running] -->is_ns_server_running? 2021-05-05 19:20:11 | INFO | MainProcess | test_thread | [cluster_helper.wait_for_ns_servers_or_assert] ns_server @ 10.112.206.103:8091 is running 2021-05-05 19:20:11 | INFO | MainProcess | test_thread | [cluster_helper.wait_for_ns_servers_or_assert] waiting for ns_server @ 10.112.206.104:8091 2021-05-05 19:20:11 | INFO | MainProcess | test_thread | [rest_client.is_ns_server_running] -->is_ns_server_running? 2021-05-05 19:20:11 | INFO | MainProcess | test_thread | [cluster_helper.wait_for_ns_servers_or_assert] ns_server @ 10.112.206.104:8091 is running 2021-05-05 19:20:11 | INFO | MainProcess | test_thread | [cluster_helper.wait_for_ns_servers_or_assert] waiting for ns_server @ 10.112.206.105:8091 2021-05-05 19:20:11 | INFO | MainProcess | test_thread | [rest_client.is_ns_server_running] -->is_ns_server_running? 2021-05-05 19:20:11 | INFO | MainProcess | test_thread | [cluster_helper.wait_for_ns_servers_or_assert] ns_server @ 10.112.206.105:8091 is running 2021-05-05 19:20:11 | INFO | MainProcess | test_thread | [basetestcase.setUp] Building docker image with java sdk client 2021-05-05 19:20:15 | ERROR | MainProcess | test_thread | [rest_client._http_request] GET http://10.112.206.101:8091/pools/default body: headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==', 'Accept': '*/*'} error: 404 reason: unknown b'"unknown pool"' auth: Administrator:password 2021-05-05 19:20:15 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] cannot find service node eventing in cluster 2021-05-05 19:20:15 | INFO | MainProcess | test_thread | [eventing_base.tearDown] ============== EventingBaseTest tearDown has started ============== 2021-05-05 19:20:15 | ERROR | MainProcess | test_thread | [rest_client._http_request] GET http://10.112.206.101:8091/pools/default body: headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==', 'Accept': '*/*'} error: 404 reason: unknown b'"unknown pool"' auth: Administrator:password 2021-05-05 19:20:15 | ERROR | MainProcess | test_thread | [rest_client._http_request] GET http://10.112.206.101:8091/pools/default body: headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==', 'Accept': '*/*'} error: 404 reason: unknown b'"unknown pool"' auth: Administrator:password 2021-05-05 19:20:15 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] cannot find service node eventing in cluster 2021-05-05 19:20:16 | ERROR | MainProcess | test_thread | [rest_client._http_request] GET http://10.112.206.101:8091/pools/default body: headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==', 'Accept': '*/*'} error: 404 reason: unknown b'"unknown pool"' auth: Administrator:password 2021-05-05 19:20:16 | INFO | MainProcess | test_thread | [basetestcase.print_cluster_stats] ------- Cluster statistics ------- 2021-05-05 19:20:16 | INFO | MainProcess | test_thread | [basetestcase.print_cluster_stats] --- End of cluster statistics --- 2021-05-05 19:20:16 | INFO | MainProcess | test_thread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.101 with username:root, attempt#1 of 5 2021-05-05 19:20:16 | INFO | MainProcess | test_thread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.101 as root 2021-05-05 19:20:16 | INFO | MainProcess | test_thread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:20:16 | INFO | MainProcess | test_thread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:20:16 | INFO | MainProcess | test_thread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.102 with username:root, attempt#1 of 5 2021-05-05 19:20:16 | INFO | MainProcess | test_thread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.102 as root 2021-05-05 19:20:16 | INFO | MainProcess | test_thread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:20:16 | INFO | MainProcess | test_thread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:20:16 | INFO | MainProcess | test_thread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.103 with username:root, attempt#1 of 5 2021-05-05 19:20:17 | INFO | MainProcess | test_thread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.103 as root 2021-05-05 19:20:17 | INFO | MainProcess | test_thread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:20:17 | INFO | MainProcess | test_thread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:20:17 | INFO | MainProcess | test_thread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.104 with username:root, attempt#1 of 5 2021-05-05 19:20:17 | INFO | MainProcess | test_thread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.104 as root 2021-05-05 19:20:17 | INFO | MainProcess | test_thread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:20:17 | INFO | MainProcess | test_thread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:20:17 | INFO | MainProcess | test_thread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.105 with username:root, attempt#1 of 5 2021-05-05 19:20:17 | INFO | MainProcess | test_thread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.105 as root 2021-05-05 19:20:17 | INFO | MainProcess | test_thread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:20:18 | INFO | MainProcess | test_thread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:20:33 | INFO | MainProcess | test_thread | [basetestcase.tearDown] ============== basetestcase cleanup was started for test #1 test_eventing_rebalance_with_multiple_kv_nodes ============== 2021-05-05 19:20:33 | ERROR | MainProcess | test_thread | [rest_client._http_request] GET http://10.112.206.101:8091/pools/default/ body: headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==', 'Accept': '*/*'} error: 404 reason: unknown b'"unknown pool"' auth: Administrator:password 2021-05-05 19:20:33 | INFO | MainProcess | test_thread | [bucket_helper.delete_all_buckets_or_assert] Could not find any buckets for node 10.112.206.101, nothing to delete 2021-05-05 19:20:33 | INFO | MainProcess | test_thread | [bucket_helper.delete_all_buckets_or_assert] Could not find any buckets for node 10.112.206.102, nothing to delete 2021-05-05 19:20:33 | INFO | MainProcess | test_thread | [bucket_helper.delete_all_buckets_or_assert] Could not find any buckets for node 10.112.206.103, nothing to delete 2021-05-05 19:20:33 | INFO | MainProcess | test_thread | [bucket_helper.delete_all_buckets_or_assert] Could not find any buckets for node 10.112.206.104, nothing to delete 2021-05-05 19:20:33 | INFO | MainProcess | test_thread | [bucket_helper.delete_all_buckets_or_assert] Could not find any buckets for node 10.112.206.105, nothing to delete 2021-05-05 19:20:33 | INFO | MainProcess | test_thread | [rest_client.is_ns_server_running] -->is_ns_server_running? 2021-05-05 19:20:33 | INFO | MainProcess | test_thread | [cluster_helper.wait_for_ns_servers_or_assert] waiting for ns_server @ 10.112.206.101:8091 2021-05-05 19:20:33 | INFO | MainProcess | test_thread | [rest_client.is_ns_server_running] -->is_ns_server_running? 2021-05-05 19:20:33 | INFO | MainProcess | test_thread | [cluster_helper.wait_for_ns_servers_or_assert] ns_server @ 10.112.206.101:8091 is running 2021-05-05 19:20:33 | INFO | MainProcess | test_thread | [cluster_helper.wait_for_ns_servers_or_assert] waiting for ns_server @ 10.112.206.102:8091 2021-05-05 19:20:33 | INFO | MainProcess | test_thread | [rest_client.is_ns_server_running] -->is_ns_server_running? 2021-05-05 19:20:33 | INFO | MainProcess | test_thread | [cluster_helper.wait_for_ns_servers_or_assert] ns_server @ 10.112.206.102:8091 is running 2021-05-05 19:20:33 | INFO | MainProcess | test_thread | [cluster_helper.wait_for_ns_servers_or_assert] waiting for ns_server @ 10.112.206.103:8091 2021-05-05 19:20:33 | INFO | MainProcess | test_thread | [rest_client.is_ns_server_running] -->is_ns_server_running? 2021-05-05 19:20:33 | INFO | MainProcess | test_thread | [cluster_helper.wait_for_ns_servers_or_assert] ns_server @ 10.112.206.103:8091 is running 2021-05-05 19:20:33 | INFO | MainProcess | test_thread | [cluster_helper.wait_for_ns_servers_or_assert] waiting for ns_server @ 10.112.206.104:8091 2021-05-05 19:20:33 | INFO | MainProcess | test_thread | [rest_client.is_ns_server_running] -->is_ns_server_running? 2021-05-05 19:20:33 | INFO | MainProcess | test_thread | [cluster_helper.wait_for_ns_servers_or_assert] ns_server @ 10.112.206.104:8091 is running 2021-05-05 19:20:33 | INFO | MainProcess | test_thread | [cluster_helper.wait_for_ns_servers_or_assert] waiting for ns_server @ 10.112.206.105:8091 2021-05-05 19:20:33 | INFO | MainProcess | test_thread | [rest_client.is_ns_server_running] -->is_ns_server_running? 2021-05-05 19:20:33 | INFO | MainProcess | test_thread | [cluster_helper.wait_for_ns_servers_or_assert] ns_server @ 10.112.206.105:8091 is running 2021-05-05 19:20:33 | INFO | MainProcess | test_thread | [basetestcase.tearDown] ============== basetestcase cleanup was finished for test #1 test_eventing_rebalance_with_multiple_kv_nodes ============== 2021-05-05 19:20:33 | INFO | MainProcess | test_thread | [basetestcase.tearDown] closing all ssh connections 2021-05-05 19:20:33 | INFO | MainProcess | test_thread | [basetestcase.tearDown] closing all memcached connections 2021-05-05 19:20:33 | INFO | MainProcess | test_thread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.101 with username:root, attempt#1 of 5 2021-05-05 19:20:33 | INFO | MainProcess | test_thread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.101 as root 2021-05-05 19:20:33 | INFO | MainProcess | test_thread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:20:33 | INFO | MainProcess | test_thread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:20:33 | INFO | MainProcess | test_thread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.101 with username:root, attempt#1 of 5 2021-05-05 19:20:33 | INFO | MainProcess | test_thread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.101 as root 2021-05-05 19:20:33 | INFO | MainProcess | test_thread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:20:34 | INFO | MainProcess | test_thread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:20:34 | INFO | MainProcess | test_thread | [remote_util.is_couchbase_installed] 10.112.206.101 **** The linux version file /opt/couchbase/ VERSION.txt exists 2021-05-05 19:20:34 | INFO | MainProcess | test_thread | [remote_util.stop_couchbase] Running systemd command on this server 2021-05-05 19:20:34 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] running command.raw on 10.112.206.101: systemctl stop couchbase-server.service 2021-05-05 19:20:35 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:20:35 | INFO | MainProcess | test_thread | [basetestcase.stop_server] Couchbase stopped 2021-05-05 19:20:35 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 5 secs. ... 2021-05-05 19:20:40 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] running command.raw on 10.112.206.101: rm -rf /opt/couchbase/var/lib/couchbase/data/* 2021-05-05 19:20:40 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:20:40 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] running command.raw on 10.112.206.101: rm -rf /opt/couchbase/var/lib/couchbase/config/* 2021-05-05 19:20:40 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:20:40 | INFO | MainProcess | test_thread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.101 with username:root, attempt#1 of 5 2021-05-05 19:20:40 | INFO | MainProcess | test_thread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.101 as root 2021-05-05 19:20:40 | INFO | MainProcess | test_thread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:20:41 | INFO | MainProcess | test_thread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:20:41 | INFO | MainProcess | test_thread | [remote_util.is_couchbase_installed] 10.112.206.101 **** The linux version file /opt/couchbase/ VERSION.txt exists 2021-05-05 19:20:41 | INFO | MainProcess | test_thread | [remote_util.is_process_running] Checking for process beam.smp on linux 2021-05-05 19:20:41 | INFO | MainProcess | test_thread | [remote_util.start_couchbase] Starting couchbase server 2021-05-05 19:20:41 | INFO | MainProcess | test_thread | [remote_util.start_couchbase] Running systemd command on this server 2021-05-05 19:20:41 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] running command.raw on 10.112.206.101: systemctl start couchbase-server.service 2021-05-05 19:20:41 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:20:41 | INFO | MainProcess | test_thread | [remote_util.sleep] 10.112.206.101:sleep for 5 secs. waiting for couchbase server to come up ... 2021-05-05 19:20:46 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] running command.raw on 10.112.206.101: systemctl status couchbase-server.service | grep ExecStop=/opt/couchbase/bin/couchbase-server 2021-05-05 19:20:46 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:20:46 | INFO | MainProcess | test_thread | [remote_util.start_couchbase] Couchbase server status: [' Process: 31098 ExecStop=/opt/couchbase/bin/couchbase-server -k (code=exited, status=0/SUCCESS)'] 2021-05-05 19:20:46 | INFO | MainProcess | test_thread | [remote_util.is_process_running] Checking for process beam.smp on linux 2021-05-05 19:20:46 | INFO | MainProcess | test_thread | [remote_util.is_process_running] process beam.smp is running on 10.112.206.101: with pid 31179 2021-05-05 19:20:46 | INFO | MainProcess | test_thread | [basetestcase.start_server] Couchbase started 2021-05-05 19:20:46 | INFO | MainProcess | test_thread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.102 with username:root, attempt#1 of 5 2021-05-05 19:20:46 | INFO | MainProcess | test_thread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.102 as root 2021-05-05 19:20:46 | INFO | MainProcess | test_thread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:20:46 | INFO | MainProcess | test_thread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:20:46 | INFO | MainProcess | test_thread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.102 with username:root, attempt#1 of 5 2021-05-05 19:20:46 | INFO | MainProcess | test_thread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.102 as root 2021-05-05 19:20:46 | INFO | MainProcess | test_thread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:20:47 | INFO | MainProcess | test_thread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:20:47 | INFO | MainProcess | test_thread | [remote_util.is_couchbase_installed] 10.112.206.102 **** The linux version file /opt/couchbase/ VERSION.txt exists 2021-05-05 19:20:47 | INFO | MainProcess | test_thread | [remote_util.stop_couchbase] Running systemd command on this server 2021-05-05 19:20:47 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] running command.raw on 10.112.206.102: systemctl stop couchbase-server.service 2021-05-05 19:20:48 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:20:48 | INFO | MainProcess | test_thread | [basetestcase.stop_server] Couchbase stopped 2021-05-05 19:20:48 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 5 secs. ... 2021-05-05 19:20:53 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] running command.raw on 10.112.206.102: rm -rf /opt/couchbase/var/lib/couchbase/data/* 2021-05-05 19:20:53 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:20:53 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] running command.raw on 10.112.206.102: rm -rf /opt/couchbase/var/lib/couchbase/config/* 2021-05-05 19:20:53 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:20:53 | INFO | MainProcess | test_thread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.102 with username:root, attempt#1 of 5 2021-05-05 19:20:53 | INFO | MainProcess | test_thread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.102 as root 2021-05-05 19:20:54 | INFO | MainProcess | test_thread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:20:54 | INFO | MainProcess | test_thread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:20:54 | INFO | MainProcess | test_thread | [remote_util.is_couchbase_installed] 10.112.206.102 **** The linux version file /opt/couchbase/ VERSION.txt exists 2021-05-05 19:20:54 | INFO | MainProcess | test_thread | [remote_util.is_process_running] Checking for process beam.smp on linux 2021-05-05 19:20:54 | INFO | MainProcess | test_thread | [remote_util.start_couchbase] Starting couchbase server 2021-05-05 19:20:54 | INFO | MainProcess | test_thread | [remote_util.start_couchbase] Running systemd command on this server 2021-05-05 19:20:54 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] running command.raw on 10.112.206.102: systemctl start couchbase-server.service 2021-05-05 19:20:54 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:20:54 | INFO | MainProcess | test_thread | [remote_util.sleep] 10.112.206.102:sleep for 5 secs. waiting for couchbase server to come up ... 2021-05-05 19:20:59 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] running command.raw on 10.112.206.102: systemctl status couchbase-server.service | grep ExecStop=/opt/couchbase/bin/couchbase-server 2021-05-05 19:20:59 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:20:59 | INFO | MainProcess | test_thread | [remote_util.start_couchbase] Couchbase server status: [' Process: 11231 ExecStop=/opt/couchbase/bin/couchbase-server -k (code=exited, status=0/SUCCESS)'] 2021-05-05 19:20:59 | INFO | MainProcess | test_thread | [remote_util.is_process_running] Checking for process beam.smp on linux 2021-05-05 19:20:59 | INFO | MainProcess | test_thread | [remote_util.is_process_running] process beam.smp is running on 10.112.206.102: with pid 11312 2021-05-05 19:20:59 | INFO | MainProcess | test_thread | [basetestcase.start_server] Couchbase started 2021-05-05 19:20:59 | INFO | MainProcess | test_thread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.103 with username:root, attempt#1 of 5 2021-05-05 19:20:59 | INFO | MainProcess | test_thread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.103 as root 2021-05-05 19:20:59 | INFO | MainProcess | test_thread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:20:59 | INFO | MainProcess | test_thread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:20:59 | INFO | MainProcess | test_thread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.103 with username:root, attempt#1 of 5 2021-05-05 19:21:00 | INFO | MainProcess | test_thread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.103 as root 2021-05-05 19:21:00 | INFO | MainProcess | test_thread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:21:00 | INFO | MainProcess | test_thread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:21:00 | INFO | MainProcess | test_thread | [remote_util.is_couchbase_installed] 10.112.206.103 **** The linux version file /opt/couchbase/ VERSION.txt exists 2021-05-05 19:21:00 | INFO | MainProcess | test_thread | [remote_util.stop_couchbase] Running systemd command on this server 2021-05-05 19:21:00 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] running command.raw on 10.112.206.103: systemctl stop couchbase-server.service 2021-05-05 19:21:02 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:21:02 | INFO | MainProcess | test_thread | [basetestcase.stop_server] Couchbase stopped 2021-05-05 19:21:02 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 5 secs. ... 2021-05-05 19:21:07 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] running command.raw on 10.112.206.103: rm -rf /opt/couchbase/var/lib/couchbase/data/* 2021-05-05 19:21:07 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:21:07 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] running command.raw on 10.112.206.103: rm -rf /opt/couchbase/var/lib/couchbase/config/* 2021-05-05 19:21:07 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:21:07 | INFO | MainProcess | test_thread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.103 with username:root, attempt#1 of 5 2021-05-05 19:21:07 | INFO | MainProcess | test_thread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.103 as root 2021-05-05 19:21:07 | INFO | MainProcess | test_thread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:21:07 | INFO | MainProcess | test_thread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:21:07 | INFO | MainProcess | test_thread | [remote_util.is_couchbase_installed] 10.112.206.103 **** The linux version file /opt/couchbase/ VERSION.txt exists 2021-05-05 19:21:07 | INFO | MainProcess | test_thread | [remote_util.is_process_running] Checking for process beam.smp on linux 2021-05-05 19:21:07 | INFO | MainProcess | test_thread | [remote_util.start_couchbase] Starting couchbase server 2021-05-05 19:21:07 | INFO | MainProcess | test_thread | [remote_util.start_couchbase] Running systemd command on this server 2021-05-05 19:21:07 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] running command.raw on 10.112.206.103: systemctl start couchbase-server.service 2021-05-05 19:21:07 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:21:07 | INFO | MainProcess | test_thread | [remote_util.sleep] 10.112.206.103:sleep for 5 secs. waiting for couchbase server to come up ... 2021-05-05 19:21:12 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] running command.raw on 10.112.206.103: systemctl status couchbase-server.service | grep ExecStop=/opt/couchbase/bin/couchbase-server 2021-05-05 19:21:12 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:21:12 | INFO | MainProcess | test_thread | [remote_util.start_couchbase] Couchbase server status: [' Process: 8913 ExecStop=/opt/couchbase/bin/couchbase-server -k (code=exited, status=0/SUCCESS)'] 2021-05-05 19:21:12 | INFO | MainProcess | test_thread | [remote_util.is_process_running] Checking for process beam.smp on linux 2021-05-05 19:21:12 | INFO | MainProcess | test_thread | [remote_util.is_process_running] process beam.smp is running on 10.112.206.103: with pid 8994 2021-05-05 19:21:12 | INFO | MainProcess | test_thread | [basetestcase.start_server] Couchbase started 2021-05-05 19:21:12 | INFO | MainProcess | test_thread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.104 with username:root, attempt#1 of 5 2021-05-05 19:21:12 | INFO | MainProcess | test_thread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.104 as root 2021-05-05 19:21:13 | INFO | MainProcess | test_thread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:21:13 | INFO | MainProcess | test_thread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:21:13 | INFO | MainProcess | test_thread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.104 with username:root, attempt#1 of 5 2021-05-05 19:21:13 | INFO | MainProcess | test_thread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.104 as root 2021-05-05 19:21:13 | INFO | MainProcess | test_thread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:21:13 | INFO | MainProcess | test_thread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:21:13 | INFO | MainProcess | test_thread | [remote_util.is_couchbase_installed] 10.112.206.104 **** The linux version file /opt/couchbase/ VERSION.txt exists 2021-05-05 19:21:13 | INFO | MainProcess | test_thread | [remote_util.stop_couchbase] Running systemd command on this server 2021-05-05 19:21:13 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] running command.raw on 10.112.206.104: systemctl stop couchbase-server.service 2021-05-05 19:21:15 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:21:15 | INFO | MainProcess | test_thread | [basetestcase.stop_server] Couchbase stopped 2021-05-05 19:21:15 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 5 secs. ... 2021-05-05 19:21:20 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] running command.raw on 10.112.206.104: rm -rf /opt/couchbase/var/lib/couchbase/data/* 2021-05-05 19:21:20 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:21:20 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] running command.raw on 10.112.206.104: rm -rf /opt/couchbase/var/lib/couchbase/config/* 2021-05-05 19:21:20 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:21:20 | INFO | MainProcess | test_thread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.104 with username:root, attempt#1 of 5 2021-05-05 19:21:20 | INFO | MainProcess | test_thread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.104 as root 2021-05-05 19:21:20 | INFO | MainProcess | test_thread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:21:20 | INFO | MainProcess | test_thread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:21:20 | INFO | MainProcess | test_thread | [remote_util.is_couchbase_installed] 10.112.206.104 **** The linux version file /opt/couchbase/ VERSION.txt exists 2021-05-05 19:21:20 | INFO | MainProcess | test_thread | [remote_util.is_process_running] Checking for process beam.smp on linux 2021-05-05 19:21:20 | INFO | MainProcess | test_thread | [remote_util.start_couchbase] Starting couchbase server 2021-05-05 19:21:20 | INFO | MainProcess | test_thread | [remote_util.start_couchbase] Running systemd command on this server 2021-05-05 19:21:20 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] running command.raw on 10.112.206.104: systemctl start couchbase-server.service 2021-05-05 19:21:20 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:21:20 | INFO | MainProcess | test_thread | [remote_util.sleep] 10.112.206.104:sleep for 5 secs. waiting for couchbase server to come up ... 2021-05-05 19:21:25 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] running command.raw on 10.112.206.104: systemctl status couchbase-server.service | grep ExecStop=/opt/couchbase/bin/couchbase-server 2021-05-05 19:21:26 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:21:26 | INFO | MainProcess | test_thread | [remote_util.start_couchbase] Couchbase server status: [' Process: 10669 ExecStop=/opt/couchbase/bin/couchbase-server -k (code=exited, status=0/SUCCESS)'] 2021-05-05 19:21:26 | INFO | MainProcess | test_thread | [remote_util.is_process_running] Checking for process beam.smp on linux 2021-05-05 19:21:26 | INFO | MainProcess | test_thread | [remote_util.is_process_running] process beam.smp is running on 10.112.206.104: with pid 10750 2021-05-05 19:21:26 | INFO | MainProcess | test_thread | [basetestcase.start_server] Couchbase started 2021-05-05 19:21:26 | INFO | MainProcess | test_thread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.105 with username:root, attempt#1 of 5 2021-05-05 19:21:26 | INFO | MainProcess | test_thread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.105 as root 2021-05-05 19:21:26 | INFO | MainProcess | test_thread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:21:26 | INFO | MainProcess | test_thread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:21:26 | INFO | MainProcess | test_thread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.105 with username:root, attempt#1 of 5 2021-05-05 19:21:26 | INFO | MainProcess | test_thread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.105 as root 2021-05-05 19:21:26 | INFO | MainProcess | test_thread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:21:26 | INFO | MainProcess | test_thread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:21:26 | INFO | MainProcess | test_thread | [remote_util.is_couchbase_installed] 10.112.206.105 **** The linux version file /opt/couchbase/ VERSION.txt exists 2021-05-05 19:21:26 | INFO | MainProcess | test_thread | [remote_util.stop_couchbase] Running systemd command on this server 2021-05-05 19:21:26 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] running command.raw on 10.112.206.105: systemctl stop couchbase-server.service 2021-05-05 19:21:28 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:21:28 | INFO | MainProcess | test_thread | [basetestcase.stop_server] Couchbase stopped 2021-05-05 19:21:28 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 5 secs. ... 2021-05-05 19:21:33 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] running command.raw on 10.112.206.105: rm -rf /opt/couchbase/var/lib/couchbase/data/* 2021-05-05 19:21:33 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:21:33 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] running command.raw on 10.112.206.105: rm -rf /opt/couchbase/var/lib/couchbase/config/* 2021-05-05 19:21:33 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:21:33 | INFO | MainProcess | test_thread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.105 with username:root, attempt#1 of 5 2021-05-05 19:21:33 | INFO | MainProcess | test_thread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.105 as root 2021-05-05 19:21:33 | INFO | MainProcess | test_thread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:21:34 | INFO | MainProcess | test_thread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:21:34 | INFO | MainProcess | test_thread | [remote_util.is_couchbase_installed] 10.112.206.105 **** The linux version file /opt/couchbase/ VERSION.txt exists 2021-05-05 19:21:34 | INFO | MainProcess | test_thread | [remote_util.is_process_running] Checking for process beam.smp on linux 2021-05-05 19:21:34 | INFO | MainProcess | test_thread | [remote_util.start_couchbase] Starting couchbase server 2021-05-05 19:21:34 | INFO | MainProcess | test_thread | [remote_util.start_couchbase] Running systemd command on this server 2021-05-05 19:21:34 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] running command.raw on 10.112.206.105: systemctl start couchbase-server.service 2021-05-05 19:21:34 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:21:34 | INFO | MainProcess | test_thread | [remote_util.sleep] 10.112.206.105:sleep for 5 secs. waiting for couchbase server to come up ... 2021-05-05 19:21:39 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] running command.raw on 10.112.206.105: systemctl status couchbase-server.service | grep ExecStop=/opt/couchbase/bin/couchbase-server 2021-05-05 19:21:39 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:21:39 | INFO | MainProcess | test_thread | [remote_util.start_couchbase] Couchbase server status: [' Process: 9174 ExecStop=/opt/couchbase/bin/couchbase-server -k (code=exited, status=0/SUCCESS)'] 2021-05-05 19:21:39 | INFO | MainProcess | test_thread | [remote_util.is_process_running] Checking for process beam.smp on linux 2021-05-05 19:21:39 | INFO | MainProcess | test_thread | [remote_util.is_process_running] process beam.smp is running on 10.112.206.105: with pid 9255 2021-05-05 19:21:39 | INFO | MainProcess | test_thread | [basetestcase.start_server] Couchbase started 2021-05-05 19:21:39 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 10 secs. ... Cluster instance shutdown with force 2021-05-05 19:21:49 | INFO | MainProcess | test_thread | [eventing_base.tearDown] ============== EventingBaseTest tearDown has completed ============== 2021-05-05 19:21:49 | INFO | MainProcess | test_thread | [basetestcase.setUp] initializing cluster 2021-05-05 19:21:49 | INFO | MainProcess | test_thread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.101 with username:root, attempt#1 of 5 2021-05-05 19:21:49 | INFO | MainProcess | test_thread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.101 as root 2021-05-05 19:21:49 | INFO | MainProcess | test_thread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:21:49 | INFO | MainProcess | test_thread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:21:49 | INFO | MainProcess | test_thread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.101 with username:root, attempt#1 of 5 2021-05-05 19:21:49 | INFO | MainProcess | test_thread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.101 as root 2021-05-05 19:21:50 | INFO | MainProcess | test_thread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:21:50 | INFO | MainProcess | test_thread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:21:50 | INFO | MainProcess | test_thread | [remote_util.is_couchbase_installed] 10.112.206.101 **** The linux version file /opt/couchbase/ VERSION.txt exists 2021-05-05 19:21:50 | INFO | MainProcess | test_thread | [remote_util.stop_couchbase] Running systemd command on this server 2021-05-05 19:21:50 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] running command.raw on 10.112.206.101: systemctl stop couchbase-server.service 2021-05-05 19:21:51 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:21:51 | INFO | MainProcess | test_thread | [basetestcase.stop_server] Couchbase stopped 2021-05-05 19:21:51 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 5 secs. ... 2021-05-05 19:21:56 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] running command.raw on 10.112.206.101: rm -rf /opt/couchbase/var/lib/couchbase/data/* 2021-05-05 19:21:56 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:21:56 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] running command.raw on 10.112.206.101: rm -rf /opt/couchbase/var/lib/couchbase/config/* 2021-05-05 19:21:56 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:21:56 | INFO | MainProcess | test_thread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.101 with username:root, attempt#1 of 5 2021-05-05 19:21:56 | INFO | MainProcess | test_thread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.101 as root 2021-05-05 19:21:57 | INFO | MainProcess | test_thread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:21:57 | INFO | MainProcess | test_thread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:21:57 | INFO | MainProcess | test_thread | [remote_util.is_couchbase_installed] 10.112.206.101 **** The linux version file /opt/couchbase/ VERSION.txt exists 2021-05-05 19:21:57 | INFO | MainProcess | test_thread | [remote_util.is_process_running] Checking for process beam.smp on linux 2021-05-05 19:21:57 | INFO | MainProcess | test_thread | [remote_util.start_couchbase] Starting couchbase server 2021-05-05 19:21:57 | INFO | MainProcess | test_thread | [remote_util.start_couchbase] Running systemd command on this server 2021-05-05 19:21:57 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] running command.raw on 10.112.206.101: systemctl start couchbase-server.service 2021-05-05 19:21:57 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:21:57 | INFO | MainProcess | test_thread | [remote_util.sleep] 10.112.206.101:sleep for 5 secs. waiting for couchbase server to come up ... 2021-05-05 19:22:02 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] running command.raw on 10.112.206.101: systemctl status couchbase-server.service | grep ExecStop=/opt/couchbase/bin/couchbase-server 2021-05-05 19:22:02 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:22:02 | INFO | MainProcess | test_thread | [remote_util.start_couchbase] Couchbase server status: [' Process: 31480 ExecStop=/opt/couchbase/bin/couchbase-server -k (code=exited, status=0/SUCCESS)'] 2021-05-05 19:22:02 | INFO | MainProcess | test_thread | [remote_util.is_process_running] Checking for process beam.smp on linux 2021-05-05 19:22:02 | INFO | MainProcess | test_thread | [remote_util.is_process_running] process beam.smp is running on 10.112.206.101: with pid 31561 2021-05-05 19:22:02 | INFO | MainProcess | test_thread | [basetestcase.start_server] Couchbase started 2021-05-05 19:22:02 | INFO | MainProcess | test_thread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.102 with username:root, attempt#1 of 5 2021-05-05 19:22:02 | INFO | MainProcess | test_thread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.102 as root 2021-05-05 19:22:02 | INFO | MainProcess | test_thread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:22:02 | INFO | MainProcess | test_thread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:22:02 | INFO | MainProcess | test_thread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.102 with username:root, attempt#1 of 5 2021-05-05 19:22:03 | INFO | MainProcess | test_thread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.102 as root 2021-05-05 19:22:03 | INFO | MainProcess | test_thread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:22:03 | INFO | MainProcess | test_thread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:22:03 | INFO | MainProcess | test_thread | [remote_util.is_couchbase_installed] 10.112.206.102 **** The linux version file /opt/couchbase/ VERSION.txt exists 2021-05-05 19:22:03 | INFO | MainProcess | test_thread | [remote_util.stop_couchbase] Running systemd command on this server 2021-05-05 19:22:03 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] running command.raw on 10.112.206.102: systemctl stop couchbase-server.service 2021-05-05 19:22:04 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:22:04 | INFO | MainProcess | test_thread | [basetestcase.stop_server] Couchbase stopped 2021-05-05 19:22:04 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 5 secs. ... 2021-05-05 19:22:09 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] running command.raw on 10.112.206.102: rm -rf /opt/couchbase/var/lib/couchbase/data/* 2021-05-05 19:22:09 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:22:09 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] running command.raw on 10.112.206.102: rm -rf /opt/couchbase/var/lib/couchbase/config/* 2021-05-05 19:22:10 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:22:10 | INFO | MainProcess | test_thread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.102 with username:root, attempt#1 of 5 2021-05-05 19:22:10 | INFO | MainProcess | test_thread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.102 as root 2021-05-05 19:22:10 | INFO | MainProcess | test_thread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:22:10 | INFO | MainProcess | test_thread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:22:10 | INFO | MainProcess | test_thread | [remote_util.is_couchbase_installed] 10.112.206.102 **** The linux version file /opt/couchbase/ VERSION.txt exists 2021-05-05 19:22:10 | INFO | MainProcess | test_thread | [remote_util.is_process_running] Checking for process beam.smp on linux 2021-05-05 19:22:10 | INFO | MainProcess | test_thread | [remote_util.start_couchbase] Starting couchbase server 2021-05-05 19:22:10 | INFO | MainProcess | test_thread | [remote_util.start_couchbase] Running systemd command on this server 2021-05-05 19:22:10 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] running command.raw on 10.112.206.102: systemctl start couchbase-server.service 2021-05-05 19:22:10 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:22:10 | INFO | MainProcess | test_thread | [remote_util.sleep] 10.112.206.102:sleep for 5 secs. waiting for couchbase server to come up ... 2021-05-05 19:22:15 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] running command.raw on 10.112.206.102: systemctl status couchbase-server.service | grep ExecStop=/opt/couchbase/bin/couchbase-server 2021-05-05 19:22:15 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:22:15 | INFO | MainProcess | test_thread | [remote_util.start_couchbase] Couchbase server status: [' Process: 11611 ExecStop=/opt/couchbase/bin/couchbase-server -k (code=exited, status=0/SUCCESS)'] 2021-05-05 19:22:15 | INFO | MainProcess | test_thread | [remote_util.is_process_running] Checking for process beam.smp on linux 2021-05-05 19:22:15 | INFO | MainProcess | test_thread | [remote_util.is_process_running] process beam.smp is running on 10.112.206.102: with pid 11692 2021-05-05 19:22:15 | INFO | MainProcess | test_thread | [basetestcase.start_server] Couchbase started 2021-05-05 19:22:15 | INFO | MainProcess | test_thread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.103 with username:root, attempt#1 of 5 2021-05-05 19:22:15 | INFO | MainProcess | test_thread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.103 as root 2021-05-05 19:22:15 | INFO | MainProcess | test_thread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:22:16 | INFO | MainProcess | test_thread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:22:16 | INFO | MainProcess | test_thread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.103 with username:root, attempt#1 of 5 2021-05-05 19:22:16 | INFO | MainProcess | test_thread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.103 as root 2021-05-05 19:22:16 | INFO | MainProcess | test_thread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:22:16 | INFO | MainProcess | test_thread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:22:16 | INFO | MainProcess | test_thread | [remote_util.is_couchbase_installed] 10.112.206.103 **** The linux version file /opt/couchbase/ VERSION.txt exists 2021-05-05 19:22:16 | INFO | MainProcess | test_thread | [remote_util.stop_couchbase] Running systemd command on this server 2021-05-05 19:22:16 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] running command.raw on 10.112.206.103: systemctl stop couchbase-server.service 2021-05-05 19:22:18 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:22:18 | INFO | MainProcess | test_thread | [basetestcase.stop_server] Couchbase stopped 2021-05-05 19:22:18 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 5 secs. ... 2021-05-05 19:22:23 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] running command.raw on 10.112.206.103: rm -rf /opt/couchbase/var/lib/couchbase/data/* 2021-05-05 19:22:23 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:22:23 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] running command.raw on 10.112.206.103: rm -rf /opt/couchbase/var/lib/couchbase/config/* 2021-05-05 19:22:23 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:22:23 | INFO | MainProcess | test_thread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.103 with username:root, attempt#1 of 5 2021-05-05 19:22:23 | INFO | MainProcess | test_thread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.103 as root 2021-05-05 19:22:23 | INFO | MainProcess | test_thread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:22:23 | INFO | MainProcess | test_thread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:22:23 | INFO | MainProcess | test_thread | [remote_util.is_couchbase_installed] 10.112.206.103 **** The linux version file /opt/couchbase/ VERSION.txt exists 2021-05-05 19:22:23 | INFO | MainProcess | test_thread | [remote_util.is_process_running] Checking for process beam.smp on linux 2021-05-05 19:22:23 | INFO | MainProcess | test_thread | [remote_util.start_couchbase] Starting couchbase server 2021-05-05 19:22:23 | INFO | MainProcess | test_thread | [remote_util.start_couchbase] Running systemd command on this server 2021-05-05 19:22:23 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] running command.raw on 10.112.206.103: systemctl start couchbase-server.service 2021-05-05 19:22:23 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:22:23 | INFO | MainProcess | test_thread | [remote_util.sleep] 10.112.206.103:sleep for 5 secs. waiting for couchbase server to come up ... 2021-05-05 19:22:28 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] running command.raw on 10.112.206.103: systemctl status couchbase-server.service | grep ExecStop=/opt/couchbase/bin/couchbase-server 2021-05-05 19:22:28 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:22:28 | INFO | MainProcess | test_thread | [remote_util.start_couchbase] Couchbase server status: [' Process: 9293 ExecStop=/opt/couchbase/bin/couchbase-server -k (code=exited, status=0/SUCCESS)'] 2021-05-05 19:22:28 | INFO | MainProcess | test_thread | [remote_util.is_process_running] Checking for process beam.smp on linux 2021-05-05 19:22:28 | INFO | MainProcess | test_thread | [remote_util.is_process_running] process beam.smp is running on 10.112.206.103: with pid 9374 2021-05-05 19:22:28 | INFO | MainProcess | test_thread | [basetestcase.start_server] Couchbase started 2021-05-05 19:22:28 | INFO | MainProcess | test_thread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.104 with username:root, attempt#1 of 5 2021-05-05 19:22:29 | INFO | MainProcess | test_thread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.104 as root 2021-05-05 19:22:29 | INFO | MainProcess | test_thread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:22:29 | INFO | MainProcess | test_thread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:22:29 | INFO | MainProcess | test_thread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.104 with username:root, attempt#1 of 5 2021-05-05 19:22:29 | INFO | MainProcess | test_thread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.104 as root 2021-05-05 19:22:29 | INFO | MainProcess | test_thread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:22:29 | INFO | MainProcess | test_thread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:22:29 | INFO | MainProcess | test_thread | [remote_util.is_couchbase_installed] 10.112.206.104 **** The linux version file /opt/couchbase/ VERSION.txt exists 2021-05-05 19:22:29 | INFO | MainProcess | test_thread | [remote_util.stop_couchbase] Running systemd command on this server 2021-05-05 19:22:29 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] running command.raw on 10.112.206.104: systemctl stop couchbase-server.service 2021-05-05 19:22:31 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:22:31 | INFO | MainProcess | test_thread | [basetestcase.stop_server] Couchbase stopped 2021-05-05 19:22:31 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 5 secs. ... 2021-05-05 19:22:36 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] running command.raw on 10.112.206.104: rm -rf /opt/couchbase/var/lib/couchbase/data/* 2021-05-05 19:22:36 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:22:36 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] running command.raw on 10.112.206.104: rm -rf /opt/couchbase/var/lib/couchbase/config/* 2021-05-05 19:22:36 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:22:36 | INFO | MainProcess | test_thread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.104 with username:root, attempt#1 of 5 2021-05-05 19:22:36 | INFO | MainProcess | test_thread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.104 as root 2021-05-05 19:22:36 | INFO | MainProcess | test_thread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:22:36 | INFO | MainProcess | test_thread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:22:36 | INFO | MainProcess | test_thread | [remote_util.is_couchbase_installed] 10.112.206.104 **** The linux version file /opt/couchbase/ VERSION.txt exists 2021-05-05 19:22:36 | INFO | MainProcess | test_thread | [remote_util.is_process_running] Checking for process beam.smp on linux 2021-05-05 19:22:36 | INFO | MainProcess | test_thread | [remote_util.start_couchbase] Starting couchbase server 2021-05-05 19:22:36 | INFO | MainProcess | test_thread | [remote_util.start_couchbase] Running systemd command on this server 2021-05-05 19:22:36 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] running command.raw on 10.112.206.104: systemctl start couchbase-server.service 2021-05-05 19:22:36 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:22:36 | INFO | MainProcess | test_thread | [remote_util.sleep] 10.112.206.104:sleep for 5 secs. waiting for couchbase server to come up ... 2021-05-05 19:22:41 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] running command.raw on 10.112.206.104: systemctl status couchbase-server.service | grep ExecStop=/opt/couchbase/bin/couchbase-server 2021-05-05 19:22:42 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:22:42 | INFO | MainProcess | test_thread | [remote_util.start_couchbase] Couchbase server status: [' Process: 11049 ExecStop=/opt/couchbase/bin/couchbase-server -k (code=exited, status=0/SUCCESS)'] 2021-05-05 19:22:42 | INFO | MainProcess | test_thread | [remote_util.is_process_running] Checking for process beam.smp on linux 2021-05-05 19:22:42 | INFO | MainProcess | test_thread | [remote_util.is_process_running] process beam.smp is running on 10.112.206.104: with pid 11131 2021-05-05 19:22:42 | INFO | MainProcess | test_thread | [basetestcase.start_server] Couchbase started 2021-05-05 19:22:42 | INFO | MainProcess | test_thread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.105 with username:root, attempt#1 of 5 2021-05-05 19:22:42 | INFO | MainProcess | test_thread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.105 as root 2021-05-05 19:22:42 | INFO | MainProcess | test_thread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:22:42 | INFO | MainProcess | test_thread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:22:42 | INFO | MainProcess | test_thread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.105 with username:root, attempt#1 of 5 2021-05-05 19:22:42 | INFO | MainProcess | test_thread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.105 as root 2021-05-05 19:22:42 | INFO | MainProcess | test_thread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:22:42 | INFO | MainProcess | test_thread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:22:42 | INFO | MainProcess | test_thread | [remote_util.is_couchbase_installed] 10.112.206.105 **** The linux version file /opt/couchbase/ VERSION.txt exists 2021-05-05 19:22:43 | INFO | MainProcess | test_thread | [remote_util.stop_couchbase] Running systemd command on this server 2021-05-05 19:22:43 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] running command.raw on 10.112.206.105: systemctl stop couchbase-server.service 2021-05-05 19:22:44 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:22:44 | INFO | MainProcess | test_thread | [basetestcase.stop_server] Couchbase stopped 2021-05-05 19:22:44 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 5 secs. ... 2021-05-05 19:22:49 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] running command.raw on 10.112.206.105: rm -rf /opt/couchbase/var/lib/couchbase/data/* 2021-05-05 19:22:49 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:22:49 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] running command.raw on 10.112.206.105: rm -rf /opt/couchbase/var/lib/couchbase/config/* 2021-05-05 19:22:49 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:22:49 | INFO | MainProcess | test_thread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.105 with username:root, attempt#1 of 5 2021-05-05 19:22:49 | INFO | MainProcess | test_thread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.105 as root 2021-05-05 19:22:49 | INFO | MainProcess | test_thread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:22:50 | INFO | MainProcess | test_thread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:22:50 | INFO | MainProcess | test_thread | [remote_util.is_couchbase_installed] 10.112.206.105 **** The linux version file /opt/couchbase/ VERSION.txt exists 2021-05-05 19:22:50 | INFO | MainProcess | test_thread | [remote_util.is_process_running] Checking for process beam.smp on linux 2021-05-05 19:22:50 | INFO | MainProcess | test_thread | [remote_util.start_couchbase] Starting couchbase server 2021-05-05 19:22:50 | INFO | MainProcess | test_thread | [remote_util.start_couchbase] Running systemd command on this server 2021-05-05 19:22:50 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] running command.raw on 10.112.206.105: systemctl start couchbase-server.service 2021-05-05 19:22:50 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:22:50 | INFO | MainProcess | test_thread | [remote_util.sleep] 10.112.206.105:sleep for 5 secs. waiting for couchbase server to come up ... 2021-05-05 19:22:55 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] running command.raw on 10.112.206.105: systemctl status couchbase-server.service | grep ExecStop=/opt/couchbase/bin/couchbase-server 2021-05-05 19:22:55 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:22:55 | INFO | MainProcess | test_thread | [remote_util.start_couchbase] Couchbase server status: [' Process: 9554 ExecStop=/opt/couchbase/bin/couchbase-server -k (code=exited, status=0/SUCCESS)'] 2021-05-05 19:22:55 | INFO | MainProcess | test_thread | [remote_util.is_process_running] Checking for process beam.smp on linux 2021-05-05 19:22:55 | INFO | MainProcess | test_thread | [remote_util.is_process_running] process beam.smp is running on 10.112.206.105: with pid 9635 2021-05-05 19:22:55 | INFO | MainProcess | test_thread | [basetestcase.start_server] Couchbase started 2021-05-05 19:22:55 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 10 secs. ... 2021-05-05 19:23:05 | INFO | MainProcess | Cluster_Thread | [task.execute] server: ip:10.112.206.101 port:8091 ssh_username:root, nodes/self 2021-05-05 19:23:05 | INFO | MainProcess | Cluster_Thread | [task.execute] {'uptime': '64', 'memoryTotal': 1930829824, 'memoryFree': 1527664640, 'mcdMemoryReserved': 1473, 'mcdMemoryAllocated': 1473, 'status': 'healthy', 'hostname': '10.112.206.101:8091', 'clusterCompatibility': 458752, 'clusterMembership': 'active', 'recoveryType': 'none', 'version': '7.0.0-5102-enterprise', 'os': 'x86_64-unknown-linux-gnu', 'ports': [], 'availableStorage': [], 'storage': [], 'memoryQuota': 256, 'moxi': 11211, 'memcached': 11210, 'id': 'ns_1@10.112.206.101', 'ip': '10.112.206.101', 'rest_username': '', 'rest_password': '', 'port': '8091', 'services': ['kv'], 'storageTotalRam': 1841, 'curr_items': 0} 2021-05-05 19:23:05 | ERROR | MainProcess | Cluster_Thread | [rest_client._http_request] GET http://10.112.206.101:8091/pools/default body: headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==', 'Accept': '*/*'} error: 404 reason: unknown b'"unknown pool"' auth: Administrator:password 2021-05-05 19:23:05 | INFO | MainProcess | Cluster_Thread | [rest_client.init_cluster_memoryQuota] pools/default params : memoryQuota=986 2021-05-05 19:23:05 | INFO | MainProcess | Cluster_Thread | [rest_client.init_node_services] --> init_node_services(Administrator,password,10.112.206.101,8091,['kv']) 2021-05-05 19:23:05 | INFO | MainProcess | Cluster_Thread | [rest_client.init_node_services] /node/controller/setupServices params on 10.112.206.101: 8091:hostname=10.112.206.101&user=Administrator&password=password&services=kv 2021-05-05 19:23:05 | INFO | MainProcess | Cluster_Thread | [rest_client.init_cluster] --> in init_cluster...Administrator,password,8091 2021-05-05 19:23:05 | INFO | MainProcess | Cluster_Thread | [rest_client.init_cluster] settings/web params on 10.112.206.101:8091:port=8091&username=Administrator&password=password 2021-05-05 19:23:05 | INFO | MainProcess | Cluster_Thread | [rest_client.init_cluster] --> status:True 2021-05-05 19:23:05 | INFO | MainProcess | Cluster_Thread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.101 with username:root, attempt#1 of 5 2021-05-05 19:23:05 | INFO | MainProcess | Cluster_Thread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.101 as root 2021-05-05 19:23:06 | INFO | MainProcess | Cluster_Thread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:23:06 | INFO | MainProcess | Cluster_Thread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:23:06 | INFO | MainProcess | Cluster_Thread | [remote_util.execute_command_raw] running command.raw on 10.112.206.101: curl --silent --show-error http://Administrator:password@localhost:8091/diag/eval -X POST -d 'ns_config:set(allow_nonlocal_eval, true).' 2021-05-05 19:23:06 | INFO | MainProcess | Cluster_Thread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:23:06 | INFO | MainProcess | Cluster_Thread | [remote_util.enable_diag_eval_on_non_local_hosts] ['ok'] 2021-05-05 19:23:06 | INFO | MainProcess | Cluster_Thread | [rest_client.diag_eval] /diag/eval status on 10.112.206.101:8091: True content: [7,0] command: cluster_compat_mode:get_compat_version(). 2021-05-05 19:23:06 | INFO | MainProcess | Cluster_Thread | [rest_client.diag_eval] /diag/eval status on 10.112.206.101:8091: True content: [7,0] command: cluster_compat_mode:get_compat_version(). 2021-05-05 19:23:06 | INFO | MainProcess | Cluster_Thread | [rest_client.set_indexer_storage_mode] settings/indexes params : storageMode=plasma 2021-05-05 19:23:06 | INFO | MainProcess | Cluster_Thread | [task.execute] server: ip:10.112.206.102 port:8091 ssh_username:root, nodes/self 2021-05-05 19:23:06 | INFO | MainProcess | Cluster_Thread | [task.execute] {'uptime': '53', 'memoryTotal': 1930829824, 'memoryFree': 1598443520, 'mcdMemoryReserved': 1473, 'mcdMemoryAllocated': 1473, 'status': 'healthy', 'hostname': '10.112.206.102:8091', 'clusterCompatibility': 458752, 'clusterMembership': 'active', 'recoveryType': 'none', 'version': '7.0.0-5102-enterprise', 'os': 'x86_64-unknown-linux-gnu', 'ports': [], 'availableStorage': [], 'storage': [], 'memoryQuota': 256, 'moxi': 11211, 'memcached': 11210, 'id': 'ns_1@10.112.206.102', 'ip': '10.112.206.102', 'rest_username': '', 'rest_password': '', 'port': '8091', 'services': ['kv'], 'storageTotalRam': 1841, 'curr_items': 0} 2021-05-05 19:23:06 | ERROR | MainProcess | Cluster_Thread | [rest_client._http_request] GET http://10.112.206.102:8091/pools/default body: headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==', 'Accept': '*/*'} error: 404 reason: unknown b'"unknown pool"' auth: Administrator:password 2021-05-05 19:23:06 | INFO | MainProcess | Cluster_Thread | [rest_client.init_cluster_memoryQuota] pools/default params : memoryQuota=986 2021-05-05 19:23:06 | INFO | MainProcess | Cluster_Thread | [rest_client.init_cluster] --> in init_cluster...Administrator,password,8091 2021-05-05 19:23:06 | INFO | MainProcess | Cluster_Thread | [rest_client.init_cluster] settings/web params on 10.112.206.102:8091:port=8091&username=Administrator&password=password 2021-05-05 19:23:06 | INFO | MainProcess | Cluster_Thread | [rest_client.init_cluster] --> status:True 2021-05-05 19:23:06 | INFO | MainProcess | Cluster_Thread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.102 with username:root, attempt#1 of 5 2021-05-05 19:23:06 | INFO | MainProcess | Cluster_Thread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.102 as root 2021-05-05 19:23:06 | INFO | MainProcess | Cluster_Thread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:23:07 | INFO | MainProcess | Cluster_Thread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:23:07 | INFO | MainProcess | Cluster_Thread | [remote_util.execute_command_raw] running command.raw on 10.112.206.102: curl --silent --show-error http://Administrator:password@localhost:8091/diag/eval -X POST -d 'ns_config:set(allow_nonlocal_eval, true).' 2021-05-05 19:23:07 | INFO | MainProcess | Cluster_Thread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:23:07 | INFO | MainProcess | Cluster_Thread | [remote_util.enable_diag_eval_on_non_local_hosts] ['ok'] 2021-05-05 19:23:07 | INFO | MainProcess | Cluster_Thread | [rest_client.diag_eval] /diag/eval status on 10.112.206.102:8091: True content: [7,0] command: cluster_compat_mode:get_compat_version(). 2021-05-05 19:23:07 | INFO | MainProcess | Cluster_Thread | [rest_client.diag_eval] /diag/eval status on 10.112.206.102:8091: True content: [7,0] command: cluster_compat_mode:get_compat_version(). 2021-05-05 19:23:07 | INFO | MainProcess | Cluster_Thread | [rest_client.set_indexer_storage_mode] settings/indexes params : storageMode=plasma 2021-05-05 19:23:07 | INFO | MainProcess | Cluster_Thread | [task.execute] server: ip:10.112.206.103 port:8091 ssh_username:root, nodes/self 2021-05-05 19:23:07 | INFO | MainProcess | Cluster_Thread | [task.execute] {'uptime': '39', 'memoryTotal': 1930829824, 'memoryFree': 1598717952, 'mcdMemoryReserved': 1473, 'mcdMemoryAllocated': 1473, 'status': 'healthy', 'hostname': '10.112.206.103:8091', 'clusterCompatibility': 458752, 'clusterMembership': 'active', 'recoveryType': 'none', 'version': '7.0.0-5102-enterprise', 'os': 'x86_64-unknown-linux-gnu', 'ports': [], 'availableStorage': [], 'storage': [], 'memoryQuota': 256, 'moxi': 11211, 'memcached': 11210, 'id': 'ns_1@10.112.206.103', 'ip': '10.112.206.103', 'rest_username': '', 'rest_password': '', 'port': '8091', 'services': ['kv'], 'storageTotalRam': 1841, 'curr_items': 0} 2021-05-05 19:23:07 | ERROR | MainProcess | Cluster_Thread | [rest_client._http_request] GET http://10.112.206.103:8091/pools/default body: headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==', 'Accept': '*/*'} error: 404 reason: unknown b'"unknown pool"' auth: Administrator:password 2021-05-05 19:23:07 | INFO | MainProcess | Cluster_Thread | [rest_client.init_cluster_memoryQuota] pools/default params : memoryQuota=986 2021-05-05 19:23:07 | INFO | MainProcess | Cluster_Thread | [rest_client.init_cluster] --> in init_cluster...Administrator,password,8091 2021-05-05 19:23:07 | INFO | MainProcess | Cluster_Thread | [rest_client.init_cluster] settings/web params on 10.112.206.103:8091:port=8091&username=Administrator&password=password 2021-05-05 19:23:07 | INFO | MainProcess | Cluster_Thread | [rest_client.init_cluster] --> status:True 2021-05-05 19:23:07 | INFO | MainProcess | Cluster_Thread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.103 with username:root, attempt#1 of 5 2021-05-05 19:23:07 | INFO | MainProcess | Cluster_Thread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.103 as root 2021-05-05 19:23:07 | INFO | MainProcess | Cluster_Thread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:23:08 | INFO | MainProcess | Cluster_Thread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:23:08 | INFO | MainProcess | Cluster_Thread | [remote_util.execute_command_raw] running command.raw on 10.112.206.103: curl --silent --show-error http://Administrator:password@localhost:8091/diag/eval -X POST -d 'ns_config:set(allow_nonlocal_eval, true).' 2021-05-05 19:23:08 | INFO | MainProcess | Cluster_Thread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:23:08 | INFO | MainProcess | Cluster_Thread | [remote_util.enable_diag_eval_on_non_local_hosts] ['ok'] 2021-05-05 19:23:08 | INFO | MainProcess | Cluster_Thread | [rest_client.diag_eval] /diag/eval status on 10.112.206.103:8091: True content: [7,0] command: cluster_compat_mode:get_compat_version(). 2021-05-05 19:23:08 | INFO | MainProcess | Cluster_Thread | [rest_client.diag_eval] /diag/eval status on 10.112.206.103:8091: True content: [7,0] command: cluster_compat_mode:get_compat_version(). 2021-05-05 19:23:08 | INFO | MainProcess | Cluster_Thread | [rest_client.set_indexer_storage_mode] settings/indexes params : storageMode=plasma 2021-05-05 19:23:08 | INFO | MainProcess | Cluster_Thread | [task.execute] server: ip:10.112.206.104 port:8091 ssh_username:root, nodes/self 2021-05-05 19:23:08 | INFO | MainProcess | Cluster_Thread | [task.execute] {'uptime': '28', 'memoryTotal': 1930829824, 'memoryFree': 1592360960, 'mcdMemoryReserved': 1473, 'mcdMemoryAllocated': 1473, 'status': 'healthy', 'hostname': '10.112.206.104:8091', 'clusterCompatibility': 458752, 'clusterMembership': 'active', 'recoveryType': 'none', 'version': '7.0.0-5102-enterprise', 'os': 'x86_64-unknown-linux-gnu', 'ports': [], 'availableStorage': [], 'storage': [], 'memoryQuota': 256, 'moxi': 11211, 'memcached': 11210, 'id': 'ns_1@10.112.206.104', 'ip': '10.112.206.104', 'rest_username': '', 'rest_password': '', 'port': '8091', 'services': ['kv'], 'storageTotalRam': 1841, 'curr_items': 0} 2021-05-05 19:23:08 | ERROR | MainProcess | Cluster_Thread | [rest_client._http_request] GET http://10.112.206.104:8091/pools/default body: headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==', 'Accept': '*/*'} error: 404 reason: unknown b'"unknown pool"' auth: Administrator:password 2021-05-05 19:23:08 | INFO | MainProcess | Cluster_Thread | [rest_client.init_cluster_memoryQuota] pools/default params : memoryQuota=986 2021-05-05 19:23:08 | INFO | MainProcess | Cluster_Thread | [rest_client.init_cluster] --> in init_cluster...Administrator,password,8091 2021-05-05 19:23:08 | INFO | MainProcess | Cluster_Thread | [rest_client.init_cluster] settings/web params on 10.112.206.104:8091:port=8091&username=Administrator&password=password 2021-05-05 19:23:08 | INFO | MainProcess | Cluster_Thread | [rest_client.init_cluster] --> status:True 2021-05-05 19:23:08 | INFO | MainProcess | Cluster_Thread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.104 with username:root, attempt#1 of 5 2021-05-05 19:23:08 | INFO | MainProcess | Cluster_Thread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.104 as root 2021-05-05 19:23:08 | INFO | MainProcess | Cluster_Thread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:23:09 | INFO | MainProcess | Cluster_Thread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:23:09 | INFO | MainProcess | Cluster_Thread | [remote_util.execute_command_raw] running command.raw on 10.112.206.104: curl --silent --show-error http://Administrator:password@localhost:8091/diag/eval -X POST -d 'ns_config:set(allow_nonlocal_eval, true).' 2021-05-05 19:23:09 | INFO | MainProcess | Cluster_Thread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:23:09 | INFO | MainProcess | Cluster_Thread | [remote_util.enable_diag_eval_on_non_local_hosts] ['ok'] 2021-05-05 19:23:09 | INFO | MainProcess | Cluster_Thread | [rest_client.diag_eval] /diag/eval status on 10.112.206.104:8091: True content: [7,0] command: cluster_compat_mode:get_compat_version(). 2021-05-05 19:23:09 | INFO | MainProcess | Cluster_Thread | [rest_client.diag_eval] /diag/eval status on 10.112.206.104:8091: True content: [7,0] command: cluster_compat_mode:get_compat_version(). 2021-05-05 19:23:09 | INFO | MainProcess | Cluster_Thread | [rest_client.set_indexer_storage_mode] settings/indexes params : storageMode=plasma 2021-05-05 19:23:09 | INFO | MainProcess | Cluster_Thread | [task.execute] server: ip:10.112.206.105 port:8091 ssh_username:root, nodes/self 2021-05-05 19:23:09 | INFO | MainProcess | Cluster_Thread | [task.execute] {'uptime': '13', 'memoryTotal': 1930829824, 'memoryFree': 1596346368, 'mcdMemoryReserved': 1473, 'mcdMemoryAllocated': 1473, 'status': 'healthy', 'hostname': '10.112.206.105:8091', 'clusterCompatibility': 458752, 'clusterMembership': 'active', 'recoveryType': 'none', 'version': '7.0.0-5102-enterprise', 'os': 'x86_64-unknown-linux-gnu', 'ports': [], 'availableStorage': [], 'storage': [], 'memoryQuota': 256, 'moxi': 11211, 'memcached': 11210, 'id': 'ns_1@10.112.206.105', 'ip': '10.112.206.105', 'rest_username': '', 'rest_password': '', 'port': '8091', 'services': ['kv'], 'storageTotalRam': 1841, 'curr_items': 0} 2021-05-05 19:23:09 | ERROR | MainProcess | Cluster_Thread | [rest_client._http_request] GET http://10.112.206.105:8091/pools/default body: headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==', 'Accept': '*/*'} error: 404 reason: unknown b'"unknown pool"' auth: Administrator:password 2021-05-05 19:23:09 | INFO | MainProcess | Cluster_Thread | [rest_client.init_cluster_memoryQuota] pools/default params : memoryQuota=986 2021-05-05 19:23:09 | INFO | MainProcess | Cluster_Thread | [rest_client.init_cluster] --> in init_cluster...Administrator,password,8091 2021-05-05 19:23:09 | INFO | MainProcess | Cluster_Thread | [rest_client.init_cluster] settings/web params on 10.112.206.105:8091:port=8091&username=Administrator&password=password 2021-05-05 19:23:09 | INFO | MainProcess | Cluster_Thread | [rest_client.init_cluster] --> status:True 2021-05-05 19:23:09 | INFO | MainProcess | Cluster_Thread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.105 with username:root, attempt#1 of 5 2021-05-05 19:23:09 | INFO | MainProcess | Cluster_Thread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.105 as root 2021-05-05 19:23:09 | INFO | MainProcess | Cluster_Thread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:23:10 | INFO | MainProcess | Cluster_Thread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:23:10 | INFO | MainProcess | Cluster_Thread | [remote_util.execute_command_raw] running command.raw on 10.112.206.105: curl --silent --show-error http://Administrator:password@localhost:8091/diag/eval -X POST -d 'ns_config:set(allow_nonlocal_eval, true).' 2021-05-05 19:23:10 | INFO | MainProcess | Cluster_Thread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:23:10 | INFO | MainProcess | Cluster_Thread | [remote_util.enable_diag_eval_on_non_local_hosts] ['ok'] 2021-05-05 19:23:10 | INFO | MainProcess | Cluster_Thread | [rest_client.diag_eval] /diag/eval status on 10.112.206.105:8091: True content: [7,0] command: cluster_compat_mode:get_compat_version(). 2021-05-05 19:23:10 | INFO | MainProcess | Cluster_Thread | [rest_client.diag_eval] /diag/eval status on 10.112.206.105:8091: True content: [7,0] command: cluster_compat_mode:get_compat_version(). 2021-05-05 19:23:10 | INFO | MainProcess | Cluster_Thread | [rest_client.set_indexer_storage_mode] settings/indexes params : storageMode=plasma 2021-05-05 19:23:10 | INFO | MainProcess | test_thread | [basetestcase.add_built_in_server_user] **** add built-in 'cbadminbucket' user to node 10.112.206.101 **** 2021-05-05 19:23:10 | ERROR | MainProcess | test_thread | [rest_client._http_request] DELETE http://10.112.206.101:8091/settings/rbac/users/local/cbadminbucket body: headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==', 'Accept': '*/*'} error: 404 reason: unknown b'"User was not found."' auth: Administrator:password 2021-05-05 19:23:10 | INFO | MainProcess | test_thread | [internal_user.delete_user] Exception while deleting user. Exception is -b'"User was not found."' 2021-05-05 19:23:10 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 5 secs. ... 2021-05-05 19:23:15 | INFO | MainProcess | test_thread | [basetestcase.add_built_in_server_user] **** add 'admin' role to 'cbadminbucket' user **** 2021-05-05 19:23:15 | INFO | MainProcess | test_thread | [basetestcase.setUp] done initializing cluster 2021-05-05 19:23:16 | INFO | MainProcess | Cluster_Thread | [task.add_nodes] adding node 10.112.206.102:8091 to cluster 2021-05-05 19:23:16 | INFO | MainProcess | Cluster_Thread | [rest_client.add_node] adding remote node @10.112.206.102:8091 to this cluster @10.112.206.101:8091 2021-05-05 19:23:26 | INFO | MainProcess | Cluster_Thread | [rest_client.monitorRebalance] rebalance progress took 10.01 seconds 2021-05-05 19:23:26 | INFO | MainProcess | Cluster_Thread | [rest_client.monitorRebalance] sleep for 10 seconds after rebalance... 2021-05-05 19:23:43 | INFO | MainProcess | Cluster_Thread | [task.add_nodes] adding node 10.112.206.103:8091 to cluster 2021-05-05 19:23:43 | INFO | MainProcess | Cluster_Thread | [rest_client.add_node] adding remote node @10.112.206.103:8091 to this cluster @10.112.206.101:8091 2021-05-05 19:23:53 | INFO | MainProcess | Cluster_Thread | [rest_client.monitorRebalance] rebalance progress took 10.01 seconds 2021-05-05 19:23:53 | INFO | MainProcess | Cluster_Thread | [rest_client.monitorRebalance] sleep for 10 seconds after rebalance... 2021-05-05 19:24:11 | INFO | MainProcess | Cluster_Thread | [task.add_nodes] adding node 10.112.206.104:8091 to cluster 2021-05-05 19:24:11 | INFO | MainProcess | Cluster_Thread | [rest_client.add_node] adding remote node @10.112.206.104:8091 to this cluster @10.112.206.101:8091 2021-05-05 19:24:21 | INFO | MainProcess | Cluster_Thread | [rest_client.monitorRebalance] rebalance progress took 10.01 seconds 2021-05-05 19:24:21 | INFO | MainProcess | Cluster_Thread | [rest_client.monitorRebalance] sleep for 10 seconds after rebalance... 2021-05-05 19:24:39 | INFO | MainProcess | Cluster_Thread | [task.add_nodes] adding node 10.112.206.105:8091 to cluster 2021-05-05 19:24:39 | INFO | MainProcess | Cluster_Thread | [rest_client.add_node] adding remote node @10.112.206.105:8091 to this cluster @10.112.206.101:8091 2021-05-05 19:24:49 | INFO | MainProcess | Cluster_Thread | [rest_client.monitorRebalance] rebalance progress took 10.01 seconds 2021-05-05 19:24:49 | INFO | MainProcess | Cluster_Thread | [rest_client.monitorRebalance] sleep for 10 seconds after rebalance... 2021-05-05 19:25:06 | INFO | MainProcess | Cluster_Thread | [rest_client.rebalance] rebalance params : {'knownNodes': 'ns_1@10.112.206.101,ns_1@10.112.206.102,ns_1@10.112.206.103,ns_1@10.112.206.104,ns_1@10.112.206.105', 'ejectedNodes': '', 'user': 'Administrator', 'password': 'password'} 2021-05-05 19:25:06 | INFO | MainProcess | Cluster_Thread | [rest_client.rebalance] rebalance operation started 2021-05-05 19:25:16 | INFO | MainProcess | Cluster_Thread | [task.check] Rebalance - status: none, progress: 100.00% 2021-05-05 19:25:16 | INFO | MainProcess | Cluster_Thread | [task.check] rebalancing was completed with progress: 100% in 10.01484489440918 sec 2021-05-05 19:25:16 | INFO | MainProcess | test_thread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.101 with username:root, attempt#1 of 5 2021-05-05 19:25:16 | INFO | MainProcess | test_thread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.101 as root 2021-05-05 19:25:16 | INFO | MainProcess | test_thread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:25:17 | INFO | MainProcess | test_thread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:25:17 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] running command.raw on 10.112.206.101: curl --silent --show-error http://Administrator:password@localhost:8091/diag/eval -X POST -d 'ns_config:set(allow_nonlocal_eval, true).' 2021-05-05 19:25:17 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 19:25:17 | INFO | MainProcess | test_thread | [remote_util.enable_diag_eval_on_non_local_hosts] ['ok'] 2021-05-05 19:25:17 | INFO | MainProcess | test_thread | [basetestcase.enable_diag_eval_on_non_local_hosts] Enabled diag/eval for non-local hosts from 10.112.206.101 2021-05-05 19:25:17 | INFO | MainProcess | test_thread | [basetestcase.setUp] ============== basetestcase setup was finished for test #1 test_eventing_rebalance_with_multiple_kv_nodes ============== 2021-05-05 19:25:17 | INFO | MainProcess | test_thread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.101 with username:root, attempt#1 of 5 2021-05-05 19:25:17 | INFO | MainProcess | test_thread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.101 as root 2021-05-05 19:25:17 | INFO | MainProcess | test_thread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:25:17 | INFO | MainProcess | test_thread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:25:17 | INFO | MainProcess | test_thread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.102 with username:root, attempt#1 of 5 2021-05-05 19:25:17 | INFO | MainProcess | test_thread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.102 as root 2021-05-05 19:25:17 | INFO | MainProcess | test_thread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:25:18 | INFO | MainProcess | test_thread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:25:18 | INFO | MainProcess | test_thread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.103 with username:root, attempt#1 of 5 2021-05-05 19:25:18 | INFO | MainProcess | test_thread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.103 as root 2021-05-05 19:25:18 | INFO | MainProcess | test_thread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:25:18 | INFO | MainProcess | test_thread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:25:18 | INFO | MainProcess | test_thread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.104 with username:root, attempt#1 of 5 2021-05-05 19:25:18 | INFO | MainProcess | test_thread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.104 as root 2021-05-05 19:25:18 | INFO | MainProcess | test_thread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:25:19 | INFO | MainProcess | test_thread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:25:19 | INFO | MainProcess | test_thread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.105 with username:root, attempt#1 of 5 2021-05-05 19:25:19 | INFO | MainProcess | test_thread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.105 as root 2021-05-05 19:25:19 | INFO | MainProcess | test_thread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:25:19 | INFO | MainProcess | test_thread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:25:35 | INFO | MainProcess | test_thread | [basetestcase.print_cluster_stats] ------- Cluster statistics ------- 2021-05-05 19:25:35 | INFO | MainProcess | test_thread | [basetestcase.print_cluster_stats] 10.112.206.101:8091 => {'services': ['kv'], 'cpu_utilization': 48.68421052631579, 'mem_free': 1511448576, 'mem_total': 1930829824, 'swap_mem_used': 0, 'swap_mem_total': 1107292160} 2021-05-05 19:25:35 | INFO | MainProcess | test_thread | [basetestcase.print_cluster_stats] 10.112.206.102:8091 => {'services': ['kv'], 'cpu_utilization': 12.6984126984127, 'mem_free': 1566130176, 'mem_total': 1930829824, 'swap_mem_used': 0, 'swap_mem_total': 1107292160} 2021-05-05 19:25:35 | INFO | MainProcess | test_thread | [basetestcase.print_cluster_stats] 10.112.206.103:8091 => {'services': ['kv'], 'cpu_utilization': 1.836734693877551, 'mem_free': 1563357184, 'mem_total': 1930829824, 'swap_mem_used': 0, 'swap_mem_total': 1107292160} 2021-05-05 19:25:35 | INFO | MainProcess | test_thread | [basetestcase.print_cluster_stats] 10.112.206.104:8091 => {'services': ['eventing'], 'cpu_utilization': 1.634320735444331, 'mem_free': 1555218432, 'mem_total': 1930829824, 'swap_mem_used': 0, 'swap_mem_total': 1107292160} 2021-05-05 19:25:35 | INFO | MainProcess | test_thread | [basetestcase.print_cluster_stats] 10.112.206.105:8091 => {'services': ['index', 'n1ql'], 'cpu_utilization': 2.346938775510204, 'mem_free': 1518342144, 'mem_total': 1930829824, 'swap_mem_used': 0, 'swap_mem_total': 1107292160} 2021-05-05 19:25:35 | INFO | MainProcess | test_thread | [basetestcase.print_cluster_stats] --- End of cluster statistics --- 2021-05-05 19:25:35 | INFO | MainProcess | test_thread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.101 with username:root, attempt#1 of 5 2021-05-05 19:25:35 | INFO | MainProcess | test_thread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.101 as root 2021-05-05 19:25:35 | INFO | MainProcess | test_thread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 19:25:35 | INFO | MainProcess | test_thread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 19:25:37 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:25:37 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of eventing nodes in cluster: [ip:10.112.206.104 port:8091 ssh_username:root] 2021-05-05 19:25:37 | INFO | MainProcess | test_thread | [rest_client.set_indexer_storage_mode] settings/indexes params : storageMode=plasma 2021-05-05 19:25:37 | INFO | MainProcess | test_thread | [eventing_base.setUp] Setting the min possible memory quota so that adding mode nodes to the cluster wouldn't be a problem. 2021-05-05 19:25:37 | INFO | MainProcess | test_thread | [rest_client.set_service_memoryQuota] pools/default params : memoryQuota=330 2021-05-05 19:25:37 | INFO | MainProcess | test_thread | [rest_client.set_service_memoryQuota] pools/default params : indexMemoryQuota=256 2021-05-05 19:25:37 | INFO | MainProcess | test_thread | [rest_client.set_service_memoryQuota] pools/default params : eventingMemoryQuota=512 2021-05-05 19:25:37 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:25:37 | INFO | MainProcess | test_thread | [eventing_base.setUp] ============== EventingBaseTest setup has completed ============== 2021-05-05 19:25:37 | INFO | MainProcess | test_thread | [rest_client.set_service_memoryQuota] pools/default params : memoryQuota=700 2021-05-05 19:25:37 | INFO | MainProcess | test_thread | [eventing_rebalance_collection.setUp] 200 2021-05-05 19:25:38 | INFO | MainProcess | Cluster_Thread | [rest_client.create_bucket] http://10.112.206.101:8091/pools/default/buckets with param: name=src_bucket&ramQuotaMB=200&replicaNumber=0&bucketType=membase&replicaIndex=1&threadsNumber=3&flushEnabled=1&evictionPolicy=valueOnly&compressionMode=passive&storageBackend=couchstore 2021-05-05 19:25:38 | INFO | MainProcess | Cluster_Thread | [rest_client.create_bucket] 0.03 seconds to create bucket src_bucket 2021-05-05 19:25:38 | INFO | MainProcess | Cluster_Thread | [bucket_helper.wait_for_memcached] waiting for memcached bucket : src_bucket in 10.112.206.101 to accept set ops 2021-05-05 19:25:39 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.101:11210 src_bucket 2021-05-05 19:25:39 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.102:11210 src_bucket 2021-05-05 19:25:39 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.103:11210 src_bucket 2021-05-05 19:25:39 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.101:11210 src_bucket 2021-05-05 19:25:39 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.102:11210 src_bucket 2021-05-05 19:25:39 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.103:11210 src_bucket 2021-05-05 19:25:40 | WARNING | MainProcess | Cluster_Thread | [task.check] vbucket map not ready after try 0 2021-05-05 19:25:40 | INFO | MainProcess | Cluster_Thread | [bucket_helper.wait_for_memcached] waiting for memcached bucket : src_bucket in 10.112.206.101 to accept set ops 2021-05-05 19:25:40 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.101:11210 src_bucket 2021-05-05 19:25:40 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.102:11210 src_bucket 2021-05-05 19:25:40 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.103:11210 src_bucket 2021-05-05 19:25:40 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.101:11210 src_bucket 2021-05-05 19:25:40 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.102:11210 src_bucket 2021-05-05 19:25:40 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.103:11210 src_bucket 2021-05-05 19:25:40 | WARNING | MainProcess | Cluster_Thread | [task.check] vbucket map not ready after try 1 2021-05-05 19:25:40 | INFO | MainProcess | Cluster_Thread | [bucket_helper.wait_for_memcached] waiting for memcached bucket : src_bucket in 10.112.206.101 to accept set ops 2021-05-05 19:25:40 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.101:11210 src_bucket 2021-05-05 19:25:41 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.102:11210 src_bucket 2021-05-05 19:25:41 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.103:11210 src_bucket 2021-05-05 19:25:41 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.101:11210 src_bucket 2021-05-05 19:25:41 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.102:11210 src_bucket 2021-05-05 19:25:41 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.103:11210 src_bucket 2021-05-05 19:25:41 | WARNING | MainProcess | Cluster_Thread | [task.check] vbucket map not ready after try 2 2021-05-05 19:25:41 | INFO | MainProcess | Cluster_Thread | [bucket_helper.wait_for_memcached] waiting for memcached bucket : src_bucket in 10.112.206.101 to accept set ops 2021-05-05 19:25:41 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.101:11210 src_bucket 2021-05-05 19:25:41 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.102:11210 src_bucket 2021-05-05 19:25:41 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.103:11210 src_bucket 2021-05-05 19:25:41 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.101:11210 src_bucket 2021-05-05 19:25:42 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.102:11210 src_bucket 2021-05-05 19:25:42 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.103:11210 src_bucket 2021-05-05 19:25:42 | WARNING | MainProcess | Cluster_Thread | [task.check] vbucket map not ready after try 3 2021-05-05 19:25:42 | INFO | MainProcess | Cluster_Thread | [bucket_helper.wait_for_memcached] waiting for memcached bucket : src_bucket in 10.112.206.101 to accept set ops 2021-05-05 19:25:42 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.101:11210 src_bucket 2021-05-05 19:25:42 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.102:11210 src_bucket 2021-05-05 19:25:42 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.103:11210 src_bucket 2021-05-05 19:25:42 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.101:11210 src_bucket 2021-05-05 19:25:42 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.102:11210 src_bucket 2021-05-05 19:25:42 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.103:11210 src_bucket 2021-05-05 19:25:43 | WARNING | MainProcess | Cluster_Thread | [task.check] vbucket map not ready after try 4 2021-05-05 19:25:43 | INFO | MainProcess | Cluster_Thread | [bucket_helper.wait_for_memcached] waiting for memcached bucket : src_bucket in 10.112.206.101 to accept set ops 2021-05-05 19:25:43 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.101:11210 src_bucket 2021-05-05 19:25:43 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.102:11210 src_bucket 2021-05-05 19:25:43 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.103:11210 src_bucket 2021-05-05 19:25:43 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.101:11210 src_bucket 2021-05-05 19:25:43 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.102:11210 src_bucket 2021-05-05 19:25:43 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.103:11210 src_bucket 2021-05-05 19:25:43 | WARNING | MainProcess | Cluster_Thread | [task.check] vbucket map not ready after try 5 2021-05-05 19:25:44 | INFO | MainProcess | Cluster_Thread | [rest_client.create_bucket] http://10.112.206.101:8091/pools/default/buckets with param: name=dst_bucket&ramQuotaMB=200&replicaNumber=0&bucketType=membase&replicaIndex=1&threadsNumber=3&flushEnabled=1&evictionPolicy=valueOnly&compressionMode=passive&storageBackend=couchstore 2021-05-05 19:25:44 | INFO | MainProcess | Cluster_Thread | [rest_client.create_bucket] 0.04 seconds to create bucket dst_bucket 2021-05-05 19:25:44 | INFO | MainProcess | Cluster_Thread | [bucket_helper.wait_for_memcached] waiting for memcached bucket : dst_bucket in 10.112.206.101 to accept set ops 2021-05-05 19:25:45 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.101:11210 dst_bucket 2021-05-05 19:25:45 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.102:11210 dst_bucket 2021-05-05 19:25:46 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.103:11210 dst_bucket 2021-05-05 19:25:46 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.101:11210 dst_bucket 2021-05-05 19:25:46 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.102:11210 dst_bucket 2021-05-05 19:25:46 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.103:11210 dst_bucket 2021-05-05 19:25:46 | WARNING | MainProcess | Cluster_Thread | [task.check] vbucket map not ready after try 0 2021-05-05 19:25:46 | INFO | MainProcess | Cluster_Thread | [bucket_helper.wait_for_memcached] waiting for memcached bucket : dst_bucket in 10.112.206.101 to accept set ops 2021-05-05 19:25:46 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.101:11210 dst_bucket 2021-05-05 19:25:47 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.102:11210 dst_bucket 2021-05-05 19:25:47 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.103:11210 dst_bucket 2021-05-05 19:25:47 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.101:11210 dst_bucket 2021-05-05 19:25:47 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.102:11210 dst_bucket 2021-05-05 19:25:47 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.103:11210 dst_bucket 2021-05-05 19:25:47 | WARNING | MainProcess | Cluster_Thread | [task.check] vbucket map not ready after try 1 2021-05-05 19:25:47 | INFO | MainProcess | Cluster_Thread | [bucket_helper.wait_for_memcached] waiting for memcached bucket : dst_bucket in 10.112.206.101 to accept set ops 2021-05-05 19:25:47 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.101:11210 dst_bucket 2021-05-05 19:25:47 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.102:11210 dst_bucket 2021-05-05 19:25:47 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.103:11210 dst_bucket 2021-05-05 19:25:48 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.101:11210 dst_bucket 2021-05-05 19:25:48 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.102:11210 dst_bucket 2021-05-05 19:25:48 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.103:11210 dst_bucket 2021-05-05 19:25:48 | WARNING | MainProcess | Cluster_Thread | [task.check] vbucket map not ready after try 2 2021-05-05 19:25:48 | INFO | MainProcess | Cluster_Thread | [bucket_helper.wait_for_memcached] waiting for memcached bucket : dst_bucket in 10.112.206.101 to accept set ops 2021-05-05 19:25:48 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.101:11210 dst_bucket 2021-05-05 19:25:48 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.102:11210 dst_bucket 2021-05-05 19:25:48 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.103:11210 dst_bucket 2021-05-05 19:25:48 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.101:11210 dst_bucket 2021-05-05 19:25:48 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.102:11210 dst_bucket 2021-05-05 19:25:49 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.103:11210 dst_bucket 2021-05-05 19:25:49 | WARNING | MainProcess | Cluster_Thread | [task.check] vbucket map not ready after try 3 2021-05-05 19:25:49 | INFO | MainProcess | Cluster_Thread | [bucket_helper.wait_for_memcached] waiting for memcached bucket : dst_bucket in 10.112.206.101 to accept set ops 2021-05-05 19:25:49 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.101:11210 dst_bucket 2021-05-05 19:25:49 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.102:11210 dst_bucket 2021-05-05 19:25:49 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.103:11210 dst_bucket 2021-05-05 19:25:49 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.101:11210 dst_bucket 2021-05-05 19:25:49 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.102:11210 dst_bucket 2021-05-05 19:25:49 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.103:11210 dst_bucket 2021-05-05 19:25:50 | WARNING | MainProcess | Cluster_Thread | [task.check] vbucket map not ready after try 4 2021-05-05 19:25:50 | INFO | MainProcess | Cluster_Thread | [bucket_helper.wait_for_memcached] waiting for memcached bucket : dst_bucket in 10.112.206.101 to accept set ops 2021-05-05 19:25:50 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.101:11210 dst_bucket 2021-05-05 19:25:50 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.102:11210 dst_bucket 2021-05-05 19:25:50 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.103:11210 dst_bucket 2021-05-05 19:25:50 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.101:11210 dst_bucket 2021-05-05 19:25:50 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.102:11210 dst_bucket 2021-05-05 19:25:50 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.103:11210 dst_bucket 2021-05-05 19:25:50 | WARNING | MainProcess | Cluster_Thread | [task.check] vbucket map not ready after try 5 2021-05-05 19:25:51 | INFO | MainProcess | Cluster_Thread | [rest_client.create_bucket] http://10.112.206.101:8091/pools/default/buckets with param: name=metadata&ramQuotaMB=200&replicaNumber=0&bucketType=membase&replicaIndex=1&threadsNumber=3&flushEnabled=1&evictionPolicy=valueOnly&compressionMode=passive&storageBackend=couchstore 2021-05-05 19:25:51 | INFO | MainProcess | Cluster_Thread | [rest_client.create_bucket] 0.04 seconds to create bucket metadata 2021-05-05 19:25:51 | INFO | MainProcess | Cluster_Thread | [bucket_helper.wait_for_memcached] waiting for memcached bucket : metadata in 10.112.206.101 to accept set ops 2021-05-05 19:25:52 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.101:11210 metadata 2021-05-05 19:25:52 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.102:11210 metadata 2021-05-05 19:25:53 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.103:11210 metadata 2021-05-05 19:25:53 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.101:11210 metadata 2021-05-05 19:25:53 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.102:11210 metadata 2021-05-05 19:25:53 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.103:11210 metadata 2021-05-05 19:25:53 | WARNING | MainProcess | Cluster_Thread | [task.check] vbucket map not ready after try 0 2021-05-05 19:25:53 | INFO | MainProcess | Cluster_Thread | [bucket_helper.wait_for_memcached] waiting for memcached bucket : metadata in 10.112.206.101 to accept set ops 2021-05-05 19:25:54 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.101:11210 metadata 2021-05-05 19:25:54 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.102:11210 metadata 2021-05-05 19:25:54 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.103:11210 metadata 2021-05-05 19:25:54 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.101:11210 metadata 2021-05-05 19:25:54 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.102:11210 metadata 2021-05-05 19:25:54 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.103:11210 metadata 2021-05-05 19:25:54 | WARNING | MainProcess | Cluster_Thread | [task.check] vbucket map not ready after try 1 2021-05-05 19:25:54 | INFO | MainProcess | Cluster_Thread | [bucket_helper.wait_for_memcached] waiting for memcached bucket : metadata in 10.112.206.101 to accept set ops 2021-05-05 19:25:54 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.101:11210 metadata 2021-05-05 19:25:54 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.102:11210 metadata 2021-05-05 19:25:55 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.103:11210 metadata 2021-05-05 19:25:55 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.101:11210 metadata 2021-05-05 19:25:55 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.102:11210 metadata 2021-05-05 19:25:55 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.103:11210 metadata 2021-05-05 19:25:55 | WARNING | MainProcess | Cluster_Thread | [task.check] vbucket map not ready after try 2 2021-05-05 19:25:55 | INFO | MainProcess | Cluster_Thread | [bucket_helper.wait_for_memcached] waiting for memcached bucket : metadata in 10.112.206.101 to accept set ops 2021-05-05 19:25:55 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.101:11210 metadata 2021-05-05 19:25:55 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.102:11210 metadata 2021-05-05 19:25:55 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.103:11210 metadata 2021-05-05 19:25:55 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.101:11210 metadata 2021-05-05 19:25:56 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.102:11210 metadata 2021-05-05 19:25:56 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.103:11210 metadata 2021-05-05 19:25:56 | WARNING | MainProcess | Cluster_Thread | [task.check] vbucket map not ready after try 3 2021-05-05 19:25:56 | INFO | MainProcess | Cluster_Thread | [bucket_helper.wait_for_memcached] waiting for memcached bucket : metadata in 10.112.206.101 to accept set ops 2021-05-05 19:25:56 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.101:11210 metadata 2021-05-05 19:25:56 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.102:11210 metadata 2021-05-05 19:25:56 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.103:11210 metadata 2021-05-05 19:25:56 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.101:11210 metadata 2021-05-05 19:25:56 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.102:11210 metadata 2021-05-05 19:25:56 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.103:11210 metadata 2021-05-05 19:25:57 | WARNING | MainProcess | Cluster_Thread | [task.check] vbucket map not ready after try 4 2021-05-05 19:25:57 | INFO | MainProcess | Cluster_Thread | [bucket_helper.wait_for_memcached] waiting for memcached bucket : metadata in 10.112.206.101 to accept set ops 2021-05-05 19:25:57 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.101:11210 metadata 2021-05-05 19:25:57 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.102:11210 metadata 2021-05-05 19:25:57 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.103:11210 metadata 2021-05-05 19:25:57 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.101:11210 metadata 2021-05-05 19:25:57 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.102:11210 metadata 2021-05-05 19:25:57 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 10.112.206.103:11210 metadata 2021-05-05 19:25:57 | WARNING | MainProcess | Cluster_Thread | [task.check] vbucket map not ready after try 5 2021-05-05 19:25:57 | INFO | MainProcess | test_thread | [rest_client.create_scope] http://10.112.206.104:8091/pools/default/buckets/dst_bucket/scopes with params: name=scope_0 2021-05-05 19:25:57 | INFO | MainProcess | test_thread | [rest_client.create_scope] Scope created dst_bucket->scope_0 {'uid': '1'} 2021-05-05 19:25:57 | INFO | MainProcess | test_thread | [rest_client.create_scope] http://10.112.206.104:8091/pools/default/buckets/dst_bucket/scopes with params: name=scope_1 2021-05-05 19:25:57 | INFO | MainProcess | test_thread | [rest_client.create_scope] Scope created dst_bucket->scope_1 {'uid': '2'} 2021-05-05 19:25:57 | INFO | MainProcess | test_thread | [rest_client.create_scope] http://10.112.206.104:8091/pools/default/buckets/dst_bucket/scopes with params: name=scope_2 2021-05-05 19:25:57 | INFO | MainProcess | test_thread | [rest_client.create_scope] Scope created dst_bucket->scope_2 {'uid': '3'} 2021-05-05 19:25:58 | INFO | MainProcess | test_thread | [rest_client.create_scope] http://10.112.206.104:8091/pools/default/buckets/dst_bucket/scopes with params: name=scope_3 2021-05-05 19:25:58 | INFO | MainProcess | test_thread | [rest_client.create_scope] Scope created dst_bucket->scope_3 {'uid': '4'} 2021-05-05 19:25:58 | INFO | MainProcess | test_thread | [rest_client.create_scope] http://10.112.206.104:8091/pools/default/buckets/dst_bucket/scopes with params: name=scope_4 2021-05-05 19:25:58 | INFO | MainProcess | test_thread | [rest_client.create_scope] Scope created dst_bucket->scope_4 {'uid': '5'} 2021-05-05 19:25:58 | INFO | MainProcess | test_thread | [rest_client.create_scope] http://10.112.206.104:8091/pools/default/buckets/src_bucket/scopes with params: name=scope_0 2021-05-05 19:25:58 | INFO | MainProcess | test_thread | [rest_client.create_scope] Scope created src_bucket->scope_0 {'uid': '1'} 2021-05-05 19:25:58 | INFO | MainProcess | test_thread | [rest_client.create_scope] http://10.112.206.104:8091/pools/default/buckets/src_bucket/scopes with params: name=scope_1 2021-05-05 19:25:58 | INFO | MainProcess | test_thread | [rest_client.create_scope] Scope created src_bucket->scope_1 {'uid': '2'} 2021-05-05 19:25:58 | INFO | MainProcess | test_thread | [rest_client.create_scope] http://10.112.206.104:8091/pools/default/buckets/src_bucket/scopes with params: name=scope_2 2021-05-05 19:25:58 | INFO | MainProcess | test_thread | [rest_client.create_scope] Scope created src_bucket->scope_2 {'uid': '3'} 2021-05-05 19:25:58 | INFO | MainProcess | test_thread | [rest_client.create_scope] http://10.112.206.104:8091/pools/default/buckets/src_bucket/scopes with params: name=scope_3 2021-05-05 19:25:58 | INFO | MainProcess | test_thread | [rest_client.create_scope] Scope created src_bucket->scope_3 {'uid': '4'} 2021-05-05 19:25:58 | INFO | MainProcess | test_thread | [rest_client.create_scope] http://10.112.206.104:8091/pools/default/buckets/src_bucket/scopes with params: name=scope_4 2021-05-05 19:25:58 | INFO | MainProcess | test_thread | [rest_client.create_scope] Scope created src_bucket->scope_4 {'uid': '5'} 2021-05-05 19:25:58 | INFO | MainProcess | test_thread | [rest_client._create_single_collection] http://10.112.206.104:8091/pools/default/buckets/dst_bucket/scopes/scope_1/collections with params: name=coll_0 2021-05-05 19:25:58 | INFO | MainProcess | test_thread | [rest_client.create_collection] Collection created dst_bucket->scope_1->coll_0 manifest:{'uid': '6'} 2021-05-05 19:25:58 | INFO | MainProcess | test_thread | [rest_client._create_single_collection] http://10.112.206.104:8091/pools/default/buckets/dst_bucket/scopes/scope_1/collections with params: name=coll_1 2021-05-05 19:25:58 | INFO | MainProcess | test_thread | [rest_client.create_collection] Collection created dst_bucket->scope_1->coll_1 manifest:{'uid': '7'} 2021-05-05 19:25:58 | INFO | MainProcess | test_thread | [rest_client._create_single_collection] http://10.112.206.104:8091/pools/default/buckets/dst_bucket/scopes/scope_1/collections with params: name=coll_2 2021-05-05 19:25:58 | INFO | MainProcess | test_thread | [rest_client.create_collection] Collection created dst_bucket->scope_1->coll_2 manifest:{'uid': '8'} 2021-05-05 19:25:58 | INFO | MainProcess | test_thread | [rest_client._create_single_collection] http://10.112.206.104:8091/pools/default/buckets/dst_bucket/scopes/scope_1/collections with params: name=coll_3 2021-05-05 19:25:58 | INFO | MainProcess | test_thread | [rest_client.create_collection] Collection created dst_bucket->scope_1->coll_3 manifest:{'uid': '9'} 2021-05-05 19:25:58 | INFO | MainProcess | test_thread | [rest_client._create_single_collection] http://10.112.206.104:8091/pools/default/buckets/dst_bucket/scopes/scope_1/collections with params: name=coll_4 2021-05-05 19:25:58 | INFO | MainProcess | test_thread | [rest_client.create_collection] Collection created dst_bucket->scope_1->coll_4 manifest:{'uid': 'a'} 2021-05-05 19:25:58 | INFO | MainProcess | test_thread | [rest_client._create_single_collection] http://10.112.206.104:8091/pools/default/buckets/src_bucket/scopes/scope_1/collections with params: name=coll_0 2021-05-05 19:25:58 | INFO | MainProcess | test_thread | [rest_client.create_collection] Collection created src_bucket->scope_1->coll_0 manifest:{'uid': '6'} 2021-05-05 19:25:58 | INFO | MainProcess | test_thread | [rest_client._create_single_collection] http://10.112.206.104:8091/pools/default/buckets/src_bucket/scopes/scope_1/collections with params: name=coll_1 2021-05-05 19:25:58 | INFO | MainProcess | test_thread | [rest_client.create_collection] Collection created src_bucket->scope_1->coll_1 manifest:{'uid': '7'} 2021-05-05 19:25:58 | INFO | MainProcess | test_thread | [rest_client._create_single_collection] http://10.112.206.104:8091/pools/default/buckets/src_bucket/scopes/scope_1/collections with params: name=coll_2 2021-05-05 19:25:58 | INFO | MainProcess | test_thread | [rest_client.create_collection] Collection created src_bucket->scope_1->coll_2 manifest:{'uid': '8'} 2021-05-05 19:25:58 | INFO | MainProcess | test_thread | [rest_client._create_single_collection] http://10.112.206.104:8091/pools/default/buckets/src_bucket/scopes/scope_1/collections with params: name=coll_3 2021-05-05 19:25:58 | INFO | MainProcess | test_thread | [rest_client.create_collection] Collection created src_bucket->scope_1->coll_3 manifest:{'uid': '9'} 2021-05-05 19:25:58 | INFO | MainProcess | test_thread | [rest_client._create_single_collection] http://10.112.206.104:8091/pools/default/buckets/src_bucket/scopes/scope_1/collections with params: name=coll_4 2021-05-05 19:25:58 | INFO | MainProcess | test_thread | [rest_client.create_collection] Collection created src_bucket->scope_1->coll_4 manifest:{'uid': 'a'} 2021-05-05 19:25:59 | INFO | MainProcess | test_thread | [eventing_base.create_function_with_collection] saving function bucket_op 2021-05-05 19:26:01 | INFO | MainProcess | test_thread | [eventing_base.create_function_with_collection] saving function timers 2021-05-05 19:26:02 | INFO | MainProcess | test_thread | [eventing_base.create_function_with_collection] saving function sbm 2021-05-05 19:26:04 | INFO | MainProcess | test_thread | [eventing_base.create_function_with_collection] saving function curl 2021-05-05 19:26:05 | INFO | MainProcess | test_thread | [eventing_base.create_function_with_collection] saving function n1ql 2021-05-05 19:26:06 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 5 secs. Waiting for bucket_op to deployed... ... 2021-05-05 19:26:11 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 5 secs. Waiting for bucket_op to deployed... ... 2021-05-05 19:26:17 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 5 secs. Waiting for bucket_op to deployed... ... 2021-05-05 19:26:22 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 5 secs. Waiting for bucket_op to deployed... ... 2021-05-05 19:26:28 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 5 secs. Waiting for timers to deployed... ... 2021-05-05 19:26:33 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 5 secs. Waiting for timers to deployed... ... 2021-05-05 19:26:38 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 5 secs. Waiting for timers to deployed... ... 2021-05-05 19:26:43 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 5 secs. Waiting for timers to deployed... ... 2021-05-05 19:26:49 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 5 secs. Waiting for sbm to deployed... ... 2021-05-05 19:26:54 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 5 secs. Waiting for sbm to deployed... ... 2021-05-05 19:26:59 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 5 secs. Waiting for sbm to deployed... ... 2021-05-05 19:27:04 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 5 secs. Waiting for sbm to deployed... ... 2021-05-05 19:27:11 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 5 secs. Waiting for curl to deployed... ... 2021-05-05 19:27:16 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 5 secs. Waiting for curl to deployed... ... 2021-05-05 19:27:21 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 5 secs. Waiting for curl to deployed... ... 2021-05-05 19:27:26 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 5 secs. Waiting for curl to deployed... ... 2021-05-05 19:27:33 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 5 secs. Waiting for n1ql to deployed... ... 2021-05-05 19:27:38 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 5 secs. Waiting for n1ql to deployed... ... 2021-05-05 19:27:43 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 5 secs. Waiting for n1ql to deployed... ... 2021-05-05 19:27:48 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 5 secs. Waiting for n1ql to deployed... ... 2021-05-05 19:27:53 | INFO | MainProcess | Cluster_Thread | [task.execute_for_collection] java -jar java_sdk_client/collections/target/javaclient/javaclient.jar -i 10.112.206.101 -u Administrator -p password -b src_bucket -s _default -c _default -n 40320 -pc 100 -pu 0 -pd 0 -l uniform -dsn 1 -dpx doc_ -dt Person -de 0 -ds 500 -ac False -st 0 -en 40319 -o False -sd False 2021-05-05 19:27:56 | INFO | MainProcess | test_thread | [rest_client.rebalance_reached] rebalance reached >100% in 3.00666880607605 seconds 2021-05-05 19:28:37 | INFO | MainProcess | Cluster_Thread | [task.execute_for_collection] java -jar java_sdk_client/collections/target/javaclient/javaclient.jar -i 10.112.206.101 -u Administrator -p password -b src_bucket -s scope_1 -c coll_1 -n 40320 -pc 100 -pu 0 -pd 0 -l uniform -dsn 1 -dpx doc_ -dt Person -de 0 -ds 500 -ac False -st 0 -en 40319 -o False -sd False 2021-05-05 19:29:20 | INFO | MainProcess | Cluster_Thread | [rest_client.rebalance] rebalance params : {'knownNodes': 'ns_1@10.112.206.101,ns_1@10.112.206.102,ns_1@10.112.206.103,ns_1@10.112.206.104,ns_1@10.112.206.105', 'ejectedNodes': '', 'user': 'Administrator', 'password': 'password'} 2021-05-05 19:29:20 | INFO | MainProcess | Cluster_Thread | [rest_client.rebalance] rebalance operation started 2021-05-05 19:29:30 | INFO | MainProcess | Cluster_Thread | [rest_client._rebalance_status_and_progress] rebalance percentage : 70.00 % 2021-05-05 19:29:30 | INFO | MainProcess | Cluster_Thread | [task.check] Rebalance - status: running, progress: 70.00% 2021-05-05 19:29:50 | INFO | MainProcess | Cluster_Thread | [task.check] Rebalance - status: none, progress: 100.00% 2021-05-05 19:29:50 | INFO | MainProcess | Cluster_Thread | [task.check] rebalancing was completed with progress: 100% in 30.028761863708496 sec 2021-05-05 19:29:50 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of eventing nodes in cluster: [ip:10.112.206.104 port:8091 ssh_username:root] 2021-05-05 19:29:50 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:29:50 | INFO | MainProcess | test_thread | [tuq_helper.run_cbq_query] RUN QUERY create primary index on dst_bucket.scope_1.coll_0 2021-05-05 19:29:50 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=create+primary+index+on+dst_bucket.scope_1.coll_0 2021-05-05 19:29:54 | INFO | MainProcess | test_thread | [tuq_helper.run_cbq_query] TOTAL ELAPSED TIME: 3.181259774s 2021-05-05 19:29:54 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:29:54 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_0 2021-05-05 19:29:54 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_0 to complete bucket operations... Current : 30429 Expected : 40320 ... 2021-05-05 19:30:24 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:30:24 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_0 2021-05-05 19:30:24 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:30:24 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_0 2021-05-05 19:30:24 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_0 to complete bucket operations... Current : 30429 Expected : 40320 ... 2021-05-05 19:30:54 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:30:54 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_0 2021-05-05 19:30:54 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:30:54 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_0 2021-05-05 19:30:54 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_0 to complete bucket operations... Current : 30429 Expected : 40320 ... 2021-05-05 19:31:24 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:31:24 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_0 2021-05-05 19:31:24 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:31:24 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_0 2021-05-05 19:31:24 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_0 to complete bucket operations... Current : 30429 Expected : 40320 ... 2021-05-05 19:31:54 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:31:54 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_0 2021-05-05 19:31:54 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:31:54 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_0 2021-05-05 19:31:54 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_0 to complete bucket operations... Current : 30429 Expected : 40320 ... 2021-05-05 19:32:24 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:32:24 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_0 2021-05-05 19:32:24 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:32:24 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_0 2021-05-05 19:32:24 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_0 to complete bucket operations... Current : 30429 Expected : 40320 ... 2021-05-05 19:32:54 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:32:54 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_0 2021-05-05 19:32:54 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:32:54 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_0 2021-05-05 19:32:54 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_0 to complete bucket operations... Current : 30429 Expected : 40320 ... 2021-05-05 19:33:24 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:33:24 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_0 2021-05-05 19:33:24 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:33:24 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_0 2021-05-05 19:33:24 | INFO | MainProcess | test_thread | [eventing_base.verify_doc_count_collections] Final docs count... Current : 40320 Expected : 40320 2021-05-05 19:33:24 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of eventing nodes in cluster: [ip:10.112.206.104 port:8091 ssh_username:root] 2021-05-05 19:33:24 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:33:24 | INFO | MainProcess | test_thread | [tuq_helper.run_cbq_query] RUN QUERY create primary index on dst_bucket.scope_1.coll_1 2021-05-05 19:33:24 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=create+primary+index+on+dst_bucket.scope_1.coll_1 2021-05-05 19:33:28 | INFO | MainProcess | test_thread | [tuq_helper.run_cbq_query] TOTAL ELAPSED TIME: 3.538210578s 2021-05-05 19:33:28 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:33:28 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_1 2021-05-05 19:33:28 | INFO | MainProcess | test_thread | [eventing_base.verify_doc_count_collections] Final docs count... Current : 40320 Expected : 40320 2021-05-05 19:33:28 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of eventing nodes in cluster: [ip:10.112.206.104 port:8091 ssh_username:root] 2021-05-05 19:33:28 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:33:28 | INFO | MainProcess | test_thread | [tuq_helper.run_cbq_query] RUN QUERY create primary index on dst_bucket.scope_1.coll_3 2021-05-05 19:33:28 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=create+primary+index+on+dst_bucket.scope_1.coll_3 2021-05-05 19:33:31 | INFO | MainProcess | test_thread | [tuq_helper.run_cbq_query] TOTAL ELAPSED TIME: 3.086427645s 2021-05-05 19:33:31 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:33:31 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 19:33:31 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 2080 Expected : 40320 ... 2021-05-05 19:34:01 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:34:01 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 19:34:01 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:34:01 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 19:34:01 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 2272 Expected : 40320 ... 2021-05-05 19:34:31 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:34:31 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 19:34:31 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:34:31 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 19:34:31 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 2466 Expected : 40320 ... 2021-05-05 19:35:01 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:35:01 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 19:35:01 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:35:01 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 19:35:01 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 2660 Expected : 40320 ... 2021-05-05 19:35:31 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:35:31 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 19:35:31 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:35:31 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 19:35:31 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 2856 Expected : 40320 ... 2021-05-05 19:36:01 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:36:01 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 19:36:01 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:36:01 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 19:36:01 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 3044 Expected : 40320 ... 2021-05-05 19:36:31 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:36:31 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 19:36:31 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:36:31 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 19:36:31 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 3234 Expected : 40320 ... 2021-05-05 19:37:01 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:37:01 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 19:37:01 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:37:01 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 19:37:01 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 3420 Expected : 40320 ... 2021-05-05 19:37:31 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:37:31 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 19:37:31 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:37:31 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 19:37:31 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 3606 Expected : 40320 ... 2021-05-05 19:38:01 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:38:01 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 19:38:01 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:38:01 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 19:38:01 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 3788 Expected : 40320 ... 2021-05-05 19:38:31 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:38:31 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 19:38:31 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:38:31 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 19:38:31 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 3974 Expected : 40320 ... 2021-05-05 19:39:01 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:39:01 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 19:39:01 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:39:01 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 19:39:01 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 4160 Expected : 40320 ... 2021-05-05 19:39:31 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:39:31 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 19:39:31 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:39:31 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 19:39:31 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 4350 Expected : 40320 ... 2021-05-05 19:40:01 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:40:01 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 19:40:01 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:40:01 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 19:40:01 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 4538 Expected : 40320 ... 2021-05-05 19:40:31 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:40:31 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 19:40:31 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:40:31 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 19:40:31 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 4726 Expected : 40320 ... 2021-05-05 19:41:01 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:41:01 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 19:41:01 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:41:01 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 19:41:01 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 4914 Expected : 40320 ... 2021-05-05 19:41:31 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:41:31 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 19:41:31 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:41:31 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 19:41:31 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 5100 Expected : 40320 ... 2021-05-05 19:42:01 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:42:01 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 19:42:01 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:42:01 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 19:42:01 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 5286 Expected : 40320 ... 2021-05-05 19:42:31 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:42:31 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 19:42:32 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:42:32 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 19:42:32 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 5472 Expected : 40320 ... 2021-05-05 19:43:02 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:43:02 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 19:43:02 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:43:02 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 19:43:02 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 5660 Expected : 40320 ... 2021-05-05 19:43:32 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:43:32 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 19:43:32 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:43:32 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 19:43:32 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 5848 Expected : 40320 ... 2021-05-05 19:44:02 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:44:02 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 19:44:02 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:44:02 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 19:44:02 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 6034 Expected : 40320 ... 2021-05-05 19:44:32 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:44:32 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 19:44:32 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:44:32 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 19:44:32 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 6220 Expected : 40320 ... 2021-05-05 19:45:02 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:45:02 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 19:45:02 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:45:02 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 19:45:02 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 6402 Expected : 40320 ... 2021-05-05 19:45:32 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:45:32 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 19:45:32 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:45:32 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 19:45:32 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 6588 Expected : 40320 ... 2021-05-05 19:46:02 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:46:02 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 19:46:02 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:46:02 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 19:46:02 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 6776 Expected : 40320 ... 2021-05-05 19:46:32 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:46:32 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 19:46:32 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:46:32 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 19:46:32 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 6964 Expected : 40320 ... 2021-05-05 19:47:02 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:47:02 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 19:47:02 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:47:02 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 19:47:02 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 7152 Expected : 40320 ... 2021-05-05 19:47:32 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:47:32 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 19:47:32 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:47:32 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 19:47:32 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 7334 Expected : 40320 ... 2021-05-05 19:48:02 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:48:02 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 19:48:02 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:48:02 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 19:48:02 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 7530 Expected : 40320 ... 2021-05-05 19:48:32 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:48:32 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 19:48:32 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:48:32 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 19:48:32 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 7728 Expected : 40320 ... 2021-05-05 19:49:02 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:49:02 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 19:49:02 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:49:02 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 19:49:02 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 7927 Expected : 40320 ... 2021-05-05 19:49:32 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:49:32 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 19:49:32 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:49:32 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 19:49:32 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 8121 Expected : 40320 ... 2021-05-05 19:50:02 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:50:02 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 19:50:02 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:50:02 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 19:50:02 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 8313 Expected : 40320 ... 2021-05-05 19:50:32 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:50:32 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 19:50:32 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:50:32 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 19:50:32 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 8503 Expected : 40320 ... 2021-05-05 19:51:02 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:51:02 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 19:51:02 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:51:02 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 19:51:02 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 8695 Expected : 40320 ... 2021-05-05 19:51:32 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:51:32 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 19:51:32 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:51:32 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 19:51:32 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 8887 Expected : 40320 ... 2021-05-05 19:52:02 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:52:02 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 19:52:02 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:52:02 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 19:52:02 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 9079 Expected : 40320 ... 2021-05-05 19:52:32 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:52:32 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 19:52:32 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:52:32 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 19:52:32 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 9271 Expected : 40320 ... 2021-05-05 19:53:02 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:53:02 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 19:53:02 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:53:02 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 19:53:02 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 9463 Expected : 40320 ... 2021-05-05 19:53:32 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:53:32 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 19:53:32 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:53:32 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 19:53:32 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 9657 Expected : 40320 ... 2021-05-05 19:54:02 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:54:02 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 19:54:03 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:54:03 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 19:54:03 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 9849 Expected : 40320 ... 2021-05-05 19:54:33 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:54:33 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 19:54:33 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:54:33 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 19:54:33 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 10043 Expected : 40320 ... 2021-05-05 19:55:03 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:55:03 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 19:55:03 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:55:03 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 19:55:03 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 10167 Expected : 40320 ... 2021-05-05 19:55:33 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:55:33 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 19:55:33 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:55:33 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 19:55:33 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 10167 Expected : 40320 ... 2021-05-05 19:56:03 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:56:03 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 19:56:03 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:56:03 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 19:56:03 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 10167 Expected : 40320 ... 2021-05-05 19:56:33 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:56:33 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 19:56:33 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:56:33 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 19:56:33 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 10276 Expected : 40320 ... 2021-05-05 19:57:03 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:57:03 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 19:57:03 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:57:03 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 19:57:03 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 10494 Expected : 40320 ... 2021-05-05 19:57:33 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:57:33 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 19:57:33 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:57:33 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 19:57:33 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 10706 Expected : 40320 ... 2021-05-05 19:58:03 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:58:03 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 19:58:03 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:58:03 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 19:58:03 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 10894 Expected : 40320 ... 2021-05-05 19:58:33 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:58:33 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 19:58:33 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:58:33 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 19:58:33 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 11082 Expected : 40320 ... 2021-05-05 19:59:03 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:59:03 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 19:59:03 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:59:03 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 19:59:03 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 11256 Expected : 40320 ... 2021-05-05 19:59:33 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:59:33 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 19:59:33 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 19:59:33 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 19:59:33 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 11456 Expected : 40320 ... 2021-05-05 20:00:03 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:00:03 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:00:03 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:00:03 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:00:03 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 11656 Expected : 40320 ... 2021-05-05 20:00:33 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:00:33 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:00:33 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:00:33 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:00:33 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 11852 Expected : 40320 ... 2021-05-05 20:01:03 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:01:03 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:01:03 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:01:03 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:01:03 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 12044 Expected : 40320 ... 2021-05-05 20:01:33 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:01:33 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:01:33 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:01:33 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:01:33 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 12244 Expected : 40320 ... 2021-05-05 20:02:03 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:02:03 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:02:03 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:02:03 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:02:03 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 12440 Expected : 40320 ... 2021-05-05 20:02:33 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:02:33 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:02:33 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:02:33 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:02:33 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 12636 Expected : 40320 ... 2021-05-05 20:03:03 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:03:03 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:03:03 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:03:03 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:03:03 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 12830 Expected : 40320 ... 2021-05-05 20:03:33 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:03:33 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:03:33 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:03:33 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:03:33 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 13026 Expected : 40320 ... 2021-05-05 20:04:03 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:04:03 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:04:03 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:04:03 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:04:03 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 13224 Expected : 40320 ... 2021-05-05 20:04:33 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:04:33 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:04:33 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:04:33 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:04:33 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 13422 Expected : 40320 ... 2021-05-05 20:05:03 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:05:03 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:05:03 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:05:03 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:05:03 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 13618 Expected : 40320 ... 2021-05-05 20:05:33 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:05:33 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:05:33 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:05:33 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:05:33 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 13818 Expected : 40320 ... 2021-05-05 20:06:03 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:06:03 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:06:03 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:06:03 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:06:03 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 14016 Expected : 40320 ... 2021-05-05 20:06:34 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:06:34 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:06:34 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:06:34 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:06:34 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 14216 Expected : 40320 ... 2021-05-05 20:07:04 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:07:04 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:07:04 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:07:04 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:07:04 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 14416 Expected : 40320 ... 2021-05-05 20:07:34 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:07:34 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:07:34 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:07:34 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:07:34 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 14612 Expected : 40320 ... 2021-05-05 20:08:04 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:08:04 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:08:04 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:08:04 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:08:04 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 14810 Expected : 40320 ... 2021-05-05 20:08:34 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:08:34 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:08:34 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:08:34 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:08:34 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 15010 Expected : 40320 ... 2021-05-05 20:09:04 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:09:04 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:09:04 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:09:04 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:09:04 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 15210 Expected : 40320 ... 2021-05-05 20:09:34 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:09:34 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:09:34 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:09:34 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:09:34 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 15408 Expected : 40320 ... 2021-05-05 20:10:04 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:10:04 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:10:04 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:10:04 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:10:04 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 15608 Expected : 40320 ... 2021-05-05 20:10:34 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:10:34 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:10:34 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:10:34 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:10:34 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 15806 Expected : 40320 ... 2021-05-05 20:11:04 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:11:04 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:11:04 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:11:04 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:11:04 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 16004 Expected : 40320 ... 2021-05-05 20:11:34 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:11:34 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:11:34 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:11:34 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:11:34 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 16202 Expected : 40320 ... 2021-05-05 20:12:04 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:12:04 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:12:04 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:12:04 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:12:04 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 16398 Expected : 40320 ... 2021-05-05 20:12:34 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:12:34 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:12:34 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:12:34 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:12:34 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 16586 Expected : 40320 ... 2021-05-05 20:13:04 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:13:04 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:13:04 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:13:04 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:13:04 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 16778 Expected : 40320 ... 2021-05-05 20:13:34 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:13:34 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:13:34 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:13:34 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:13:34 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 16964 Expected : 40320 ... 2021-05-05 20:14:04 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:14:04 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:14:04 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:14:04 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:14:04 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 17142 Expected : 40320 ... 2021-05-05 20:14:34 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:14:34 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:14:34 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:14:34 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:14:34 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 17310 Expected : 40320 ... 2021-05-05 20:15:04 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:15:04 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:15:04 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:15:04 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:15:04 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 17496 Expected : 40320 ... 2021-05-05 20:15:34 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:15:34 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:15:34 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:15:34 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:15:34 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 17694 Expected : 40320 ... 2021-05-05 20:16:04 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:16:04 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:16:04 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:16:04 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:16:04 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 17890 Expected : 40320 ... 2021-05-05 20:16:34 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:16:34 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:16:34 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:16:34 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:16:34 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 18066 Expected : 40320 ... 2021-05-05 20:17:04 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:17:04 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:17:04 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:17:04 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:17:04 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 18252 Expected : 40320 ... 2021-05-05 20:17:34 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:17:34 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:17:34 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:17:34 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:17:34 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 18442 Expected : 40320 ... 2021-05-05 20:18:04 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:18:04 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:18:04 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:18:04 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:18:04 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 18618 Expected : 40320 ... 2021-05-05 20:18:34 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:18:34 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:18:35 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:18:35 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:18:35 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 18798 Expected : 40320 ... 2021-05-05 20:19:05 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:19:05 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:19:05 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:19:05 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:19:05 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 18988 Expected : 40320 ... 2021-05-05 20:19:35 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:19:35 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:19:35 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:19:35 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:19:35 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 19182 Expected : 40320 ... 2021-05-05 20:20:25 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:20:25 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:20:25 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:20:25 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:20:25 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 19210 Expected : 40320 ... 2021-05-05 20:20:55 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:20:55 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:20:55 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:20:55 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:20:55 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 19411 Expected : 40320 ... 2021-05-05 20:21:25 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:21:25 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:21:25 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:21:25 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:21:25 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 19611 Expected : 40320 ... 2021-05-05 20:21:55 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:21:55 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:21:55 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:21:55 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:21:55 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 19813 Expected : 40320 ... 2021-05-05 20:22:25 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:22:25 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:22:25 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:22:25 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:22:25 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 20017 Expected : 40320 ... 2021-05-05 20:22:55 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:22:55 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:22:55 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:22:55 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:22:55 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 20217 Expected : 40320 ... 2021-05-05 20:23:25 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:23:25 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:23:25 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:23:25 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:23:25 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 20415 Expected : 40320 ... 2021-05-05 20:23:55 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:23:55 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:23:55 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:23:55 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:23:55 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 20611 Expected : 40320 ... 2021-05-05 20:24:25 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:24:25 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:24:25 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:24:25 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:24:25 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 20809 Expected : 40320 ... 2021-05-05 20:24:55 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:24:55 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:24:55 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:24:55 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:24:55 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 21007 Expected : 40320 ... 2021-05-05 20:25:25 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:25:25 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:25:25 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:25:25 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:25:25 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 21219 Expected : 40320 ... 2021-05-05 20:25:55 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:25:55 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:25:55 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:25:55 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:25:55 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 21421 Expected : 40320 ... 2021-05-05 20:26:26 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:26:26 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:26:26 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:26:26 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:26:26 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 21625 Expected : 40320 ... 2021-05-05 20:26:56 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:26:56 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:26:56 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:26:56 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:26:56 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 21831 Expected : 40320 ... 2021-05-05 20:27:26 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:27:26 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:27:26 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:27:26 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:27:26 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 22031 Expected : 40320 ... 2021-05-05 20:27:56 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:27:56 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:27:56 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:27:56 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:27:56 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 22233 Expected : 40320 ... 2021-05-05 20:28:26 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:28:26 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:28:26 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:28:26 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:28:26 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 22435 Expected : 40320 ... 2021-05-05 20:28:56 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:28:56 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:28:56 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:28:56 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:28:56 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 22624 Expected : 40320 ... 2021-05-05 20:29:26 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:29:26 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:29:26 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:29:26 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:29:26 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 22826 Expected : 40320 ... 2021-05-05 20:29:56 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:29:56 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:29:56 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:29:56 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:29:56 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 23030 Expected : 40320 ... 2021-05-05 20:30:26 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:30:26 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:30:26 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:30:26 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:30:26 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 23208 Expected : 40320 ... 2021-05-05 20:30:56 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:30:56 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:30:56 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:30:56 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:30:56 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 23404 Expected : 40320 ... 2021-05-05 20:31:26 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:31:26 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:31:26 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:31:26 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:31:26 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 23611 Expected : 40320 ... 2021-05-05 20:31:56 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:31:56 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:31:56 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:31:56 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:31:56 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 23812 Expected : 40320 ... 2021-05-05 20:32:26 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:32:26 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:32:26 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:32:26 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:32:26 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 24016 Expected : 40320 ... 2021-05-05 20:32:56 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:32:56 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:32:56 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:32:56 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:32:56 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 24216 Expected : 40320 ... 2021-05-05 20:33:26 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:33:26 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:33:26 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:33:26 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:33:26 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 24420 Expected : 40320 ... 2021-05-05 20:33:56 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:33:56 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:33:56 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:33:56 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:33:56 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 24620 Expected : 40320 ... 2021-05-05 20:34:26 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:34:26 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:34:26 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:34:26 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:34:26 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 24820 Expected : 40320 ... 2021-05-05 20:34:56 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:34:56 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:34:56 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:34:56 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:34:56 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 25024 Expected : 40320 ... 2021-05-05 20:35:26 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:35:26 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:35:26 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:35:26 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:35:26 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 25226 Expected : 40320 ... 2021-05-05 20:35:56 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:35:56 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:35:56 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:35:56 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:35:56 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 25422 Expected : 40320 ... 2021-05-05 20:36:26 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:36:26 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:36:26 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:36:26 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:36:26 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 25630 Expected : 40320 ... 2021-05-05 20:36:56 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:36:56 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:36:56 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:36:56 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:36:57 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 25832 Expected : 40320 ... 2021-05-05 20:37:27 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:37:27 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:37:27 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:37:27 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:37:27 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 26043 Expected : 40320 ... 2021-05-05 20:37:57 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:37:57 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:37:57 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:37:57 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:37:57 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 26253 Expected : 40320 ... 2021-05-05 20:38:27 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:38:27 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:38:27 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:38:27 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:38:27 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 26459 Expected : 40320 ... 2021-05-05 20:38:57 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:38:57 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:38:57 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:38:57 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:38:57 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 26660 Expected : 40320 ... 2021-05-05 20:39:27 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:39:27 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:39:27 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:39:27 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:39:27 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 26875 Expected : 40320 ... 2021-05-05 20:39:57 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:39:57 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:39:57 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:39:57 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:39:57 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 27081 Expected : 40320 ... 2021-05-05 20:40:27 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:40:27 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:40:27 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:40:27 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:40:27 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 27275 Expected : 40320 ... 2021-05-05 20:40:57 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:40:57 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:40:57 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:40:57 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:40:57 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 27473 Expected : 40320 ... 2021-05-05 20:41:27 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:41:27 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:41:27 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:41:27 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:41:27 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 27659 Expected : 40320 ... 2021-05-05 20:41:57 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:41:57 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:41:57 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:41:57 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:41:57 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 27861 Expected : 40320 ... 2021-05-05 20:42:27 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:42:27 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:42:27 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:42:27 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:42:27 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 28063 Expected : 40320 ... 2021-05-05 20:42:57 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:42:57 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:42:57 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:42:57 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:42:57 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 28267 Expected : 40320 ... 2021-05-05 20:43:27 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:43:27 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:43:27 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:43:27 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:43:27 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 28467 Expected : 40320 ... 2021-05-05 20:43:57 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:43:57 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:43:57 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:43:57 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:43:57 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 28682 Expected : 40320 ... 2021-05-05 20:44:27 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:44:27 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:44:27 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:44:27 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:44:27 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 28901 Expected : 40320 ... 2021-05-05 20:44:57 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:44:57 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:44:57 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:44:57 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:44:57 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 29113 Expected : 40320 ... 2021-05-05 20:45:27 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:45:27 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:45:27 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:45:27 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:45:27 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 29311 Expected : 40320 ... 2021-05-05 20:45:57 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:45:57 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:45:57 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:45:57 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:45:57 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 29511 Expected : 40320 ... 2021-05-05 20:46:27 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:46:27 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:46:27 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:46:27 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:46:27 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 29707 Expected : 40320 ... 2021-05-05 20:46:57 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:46:57 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:46:57 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:46:57 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:46:57 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 29911 Expected : 40320 ... 2021-05-05 20:47:27 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:47:27 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:47:27 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:47:28 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:47:28 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 30113 Expected : 40320 ... 2021-05-05 20:47:58 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:47:58 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:47:58 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:47:58 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:47:58 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 30311 Expected : 40320 ... 2021-05-05 20:48:28 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:48:28 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:48:28 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:48:28 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:48:28 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 30511 Expected : 40320 ... 2021-05-05 20:48:58 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:48:58 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:48:58 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:48:58 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:48:58 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 30711 Expected : 40320 ... 2021-05-05 20:49:28 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:49:28 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:49:28 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:49:28 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:49:28 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 30907 Expected : 40320 ... 2021-05-05 20:49:58 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:49:58 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:49:58 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:49:58 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:49:58 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 31113 Expected : 40320 ... 2021-05-05 20:50:28 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:50:28 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:50:28 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:50:28 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:50:28 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 31313 Expected : 40320 ... 2021-05-05 20:50:58 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:50:58 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:50:58 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:50:58 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:50:58 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 31511 Expected : 40320 ... 2021-05-05 20:51:28 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:51:28 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:51:28 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:51:28 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:51:28 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 31709 Expected : 40320 ... 2021-05-05 20:51:58 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:51:58 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:51:58 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:51:58 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:51:58 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 31909 Expected : 40320 ... 2021-05-05 20:52:28 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:52:28 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:52:28 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:52:28 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:52:28 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 32107 Expected : 40320 ... 2021-05-05 20:52:58 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:52:58 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:52:58 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:52:58 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:52:58 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 32309 Expected : 40320 ... 2021-05-05 20:53:28 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:53:28 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:53:28 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:53:28 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:53:28 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 32509 Expected : 40320 ... 2021-05-05 20:53:58 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:53:58 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:53:58 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:53:58 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:53:58 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 32711 Expected : 40320 ... 2021-05-05 20:54:28 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:54:28 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:54:28 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:54:28 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:54:28 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 32911 Expected : 40320 ... 2021-05-05 20:54:58 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:54:58 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:54:58 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:54:58 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:54:58 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 33119 Expected : 40320 ... 2021-05-05 20:55:28 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:55:28 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:55:28 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:55:28 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:55:28 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 33321 Expected : 40320 ... 2021-05-05 20:55:58 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:55:58 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:55:58 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:55:58 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:55:58 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 33523 Expected : 40320 ... 2021-05-05 20:56:28 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:56:28 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:56:28 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:56:28 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:56:28 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 33721 Expected : 40320 ... 2021-05-05 20:56:58 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:56:58 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:56:58 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:56:58 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:56:58 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 33923 Expected : 40320 ... 2021-05-05 20:57:28 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:57:28 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:57:28 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:57:28 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:57:28 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 34121 Expected : 40320 ... 2021-05-05 20:57:58 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:57:58 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:57:58 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:57:58 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:57:58 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 34321 Expected : 40320 ... 2021-05-05 20:58:28 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:58:28 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:58:28 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:58:28 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:58:28 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 34525 Expected : 40320 ... 2021-05-05 20:58:58 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:58:58 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:58:58 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:58:58 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:58:58 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 34727 Expected : 40320 ... 2021-05-05 20:59:29 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:59:29 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:59:29 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:59:29 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:59:29 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 34925 Expected : 40320 ... 2021-05-05 20:59:59 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:59:59 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:59:59 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 20:59:59 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 20:59:59 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 35125 Expected : 40320 ... 2021-05-05 21:00:29 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 21:00:29 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 21:00:29 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 21:00:29 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 21:00:29 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 35323 Expected : 40320 ... 2021-05-05 21:00:59 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 21:00:59 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 21:00:59 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 21:00:59 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 21:00:59 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 35521 Expected : 40320 ... 2021-05-05 21:01:29 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 21:01:29 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 21:01:29 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 21:01:29 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 21:01:29 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 35723 Expected : 40320 ... 2021-05-05 21:01:59 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 21:01:59 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 21:01:59 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 21:01:59 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 21:01:59 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 35923 Expected : 40320 ... 2021-05-05 21:02:29 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 21:02:29 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 21:02:29 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 21:02:29 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 21:02:29 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 36125 Expected : 40320 ... 2021-05-05 21:02:59 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 21:02:59 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 21:02:59 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 21:02:59 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 21:02:59 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 36323 Expected : 40320 ... 2021-05-05 21:03:29 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 21:03:29 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 21:03:29 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 21:03:29 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 21:03:29 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 36533 Expected : 40320 ... 2021-05-05 21:03:59 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 21:03:59 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 21:03:59 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 21:03:59 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 21:03:59 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 36737 Expected : 40320 ... 2021-05-05 21:04:29 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 21:04:29 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 21:04:29 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 21:04:29 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 21:04:29 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 36939 Expected : 40320 ... 2021-05-05 21:04:59 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 21:04:59 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 21:04:59 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 21:04:59 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 21:04:59 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 37143 Expected : 40320 ... 2021-05-05 21:05:29 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 21:05:29 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 21:05:29 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 21:05:29 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 21:05:29 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 37345 Expected : 40320 ... 2021-05-05 21:05:59 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 21:05:59 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 21:05:59 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 21:05:59 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 21:05:59 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 37541 Expected : 40320 ... 2021-05-05 21:06:29 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 21:06:29 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 21:06:29 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 21:06:29 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 21:06:29 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 37743 Expected : 40320 ... 2021-05-05 21:06:59 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 21:06:59 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 21:06:59 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 21:06:59 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 21:06:59 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 37945 Expected : 40320 ... 2021-05-05 21:07:29 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 21:07:29 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 21:07:29 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 21:07:29 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 21:07:29 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 38032 Expected : 40320 ... 2021-05-05 21:07:59 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 21:07:59 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 21:07:59 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 21:07:59 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 21:07:59 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 38032 Expected : 40320 ... 2021-05-05 21:08:29 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 21:08:29 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 21:08:29 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 21:08:29 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 21:08:29 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 38032 Expected : 40320 ... 2021-05-05 21:08:59 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 21:08:59 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 21:08:59 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 21:08:59 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 21:08:59 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 38032 Expected : 40320 ... 2021-05-05 21:09:29 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 21:09:29 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 21:09:29 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 21:09:29 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 21:09:29 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 38032 Expected : 40320 ... 2021-05-05 21:09:59 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 21:09:59 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 21:09:59 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 21:09:59 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 21:09:59 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 38032 Expected : 40320 ... 2021-05-05 21:10:29 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 21:10:29 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 21:10:29 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 21:10:29 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 21:10:29 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 38032 Expected : 40320 ... 2021-05-05 21:10:59 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 21:11:00 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 21:11:00 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 21:11:00 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 21:11:00 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 38032 Expected : 40320 ... 2021-05-05 21:11:30 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 21:11:30 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 21:11:30 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 21:11:30 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 21:11:30 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 38032 Expected : 40320 ... 2021-05-05 21:12:00 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 21:12:00 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 21:12:00 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 21:12:00 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 21:12:00 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 38032 Expected : 40320 ... 2021-05-05 21:12:30 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 21:12:30 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 21:12:30 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 21:12:30 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 21:12:30 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 38032 Expected : 40320 ... 2021-05-05 21:13:00 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 21:13:00 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 21:13:00 | INFO | MainProcess | test_thread | [eventing_base.bucket_compaction] Compacting bucket : dst_bucket 2021-05-05 21:13:00 | INFO | MainProcess | test_thread | [rest_client.compact_bucket] bucket compaction successful 2021-05-05 21:13:00 | INFO | MainProcess | test_thread | [eventing_base.bucket_compaction] Compacting bucket : metadata 2021-05-05 21:13:00 | INFO | MainProcess | test_thread | [rest_client.compact_bucket] bucket compaction successful 2021-05-05 21:13:00 | INFO | MainProcess | test_thread | [eventing_base.bucket_compaction] Compacting bucket : src_bucket 2021-05-05 21:13:00 | INFO | MainProcess | test_thread | [rest_client.compact_bucket] bucket compaction successful 2021-05-05 21:13:00 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 21:13:00 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 21:13:00 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 38032 Expected : 40320 ... 2021-05-05 21:13:30 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 21:13:30 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 21:13:30 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 21:13:30 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 21:13:30 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 38032 Expected : 40320 ... 2021-05-05 21:14:00 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 21:14:00 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 21:14:00 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 21:14:00 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 21:14:00 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 38032 Expected : 40320 ... 2021-05-05 21:14:30 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 21:14:30 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 21:14:30 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 21:14:30 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 21:14:30 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 38032 Expected : 40320 ... 2021-05-05 21:15:00 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 21:15:00 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 21:15:00 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 21:15:00 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 21:15:00 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 38032 Expected : 40320 ... 2021-05-05 21:15:30 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 21:15:30 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 21:15:30 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 21:15:30 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 21:15:30 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 38032 Expected : 40320 ... 2021-05-05 21:16:00 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 21:16:00 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 21:16:00 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 21:16:00 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 21:16:00 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 38032 Expected : 40320 ... 2021-05-05 21:16:30 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 21:16:30 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 21:16:30 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 21:16:30 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 21:16:30 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 38032 Expected : 40320 ... 2021-05-05 21:17:00 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 21:17:00 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 21:17:00 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 21:17:00 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 21:17:00 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 30 secs. Waiting for handler code dst_bucket.scope_1.coll_3 to complete bucket operations... Current : 38032 Expected : 40320 ... 2021-05-05 21:17:30 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 21:17:30 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 21:17:30 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:10.112.206.105 port:8091 ssh_username:root] 2021-05-05 21:17:30 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket.scope_1.coll_3 2021-05-05 21:17:30 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of eventing nodes in cluster: [ip:10.112.206.104 port:8091 ssh_username:root] 2021-05-05 21:17:30 | INFO | MainProcess | test_thread | [eventing_base.print_eventing_stats_from_all_eventing_nodes] Stats for Node 10.112.206.104 is [ { "dcp_feed_boundary": "everything", "event_processing_stats": { "adhoc_timer_response_received": 100, "agg_messages_sent_to_worker": 74174, "agg_queue_memory": 0, "agg_queue_memory_cap": 53477376, "agg_queue_size": 0, "agg_queue_size_cap": 100000, "agg_timer_feedback_queue_cap": 500, "dcp_mutation": 120960, "dcp_mutation_sent_to_worker": 40320, "dcp_snapshot": 116668, "dcp_stream_req_counter": 1024, "dcp_streamreq": 1024, "dcp_system_event": 10240, "execution_stats": 6564, "failure_stats": 6564, "latency_stats": 6564, "lcb_exception_stats": 6564, "log_level": 1, "num_processed_events": 40320, "processed_events_size": 36095576, "thr_count": 1, "thr_map": 1, "v8_init": 1, "v8_load": 1 }, "events_remaining": { "dcp_backlog": 0 }, "execution_stats": { "agg_queue_memory": 0, "agg_queue_size": 0, "curl": { "delete": 0, "get": 40320, "head": 0, "post": 0, "put": 0 }, "curl_success_count": 38032, "dcp_delete_msg_counter": 0, "dcp_delete_parse_failure": 0, "dcp_mutation_msg_counter": 40320, "dcp_mutation_parse_failure": 0, "enqueued_dcp_delete_msg_counter": 0, "enqueued_dcp_mutation_msg_counter": 40320, "enqueued_timer_msg_counter": 0, "feedback_queue_size": 0, "filtered_dcp_delete_counter": 0, "filtered_dcp_mutation_counter": 0, "lcb_retry_failure": 0, "messages_parsed": 74170, "no_op_counter": 0, "num_processed_events": 40320, "on_delete_failure": 0, "on_delete_success": 0, "on_update_failure": 0, "on_update_success": 40320, "processed_events_size": 36095576, "timer_callback_failure": 0, "timer_callback_success": 0, "timer_cancel_counter": 0, "timer_create_counter": 0, "timer_create_failure": 0, "timer_msg_counter": 0, "timer_responses_sent": 0, "timestamp": { "13006": "2021-05-05T15:46:46Z" }, "uv_msg_parse_failure": 0, "uv_try_write_failure_counter": 0 }, "failure_stats": { "app_worker_setting_events_lost": 0, "bkt_ops_cas_mismatch_count": 0, "bucket_op_exception_count": 0, "checkpoint_failure_count": 0, "curl_failure_count": 2286, "curl_max_resp_size_exceeded": 0, "curl_non_200_response": 0, "curl_timeout_count": 2, "dcp_events_lost": 0, "debugger_events_lost": 0, "delete_events_lost": 0, "mutation_events_lost": 0, "n1ql_op_exception_count": 0, "timeout_count": 0, "timer_callback_missing_counter": 0, "timer_context_size_exceeded_counter": 0, "timer_events_lost": 0, "timestamp": { "13006": "2021-05-05T15:46:46Z" }, "v8worker_events_lost": 0 }, "function_id": 1020397118, "function_name": "curl", "gocb_creds_request_counter": 78, "internal_vb_distribution_stats": { "worker_curl_0": "[0-1023]" }, "latency_percentile_stats": { "100": 9999900, "50": 301100, "80": 317200, "90": 341900, "95": 370700, "99": 414200 }, "lcb_creds_request_counter": 80738, "lcb_exception_stats": {}, "metastore_stats": {}, "planner_stats": [ { "host_name": "10.112.206.104:8096", "start_vb": 0, "vb_count": 1024 } ], "vb_distribution_stats_from_metadata": { "10.112.206.104:8096": { "worker_curl_0": "[0-1023]" } }, "worker_pids": { "worker_curl_0": 13006 } }, { "dcp_feed_boundary": "everything", "event_processing_stats": { "adhoc_timer_response_received": 4, "agg_messages_sent_to_worker": 74068, "agg_queue_memory": 0, "agg_queue_memory_cap": 106954752, "agg_queue_size": 0, "agg_queue_size_cap": 100000, "agg_timer_feedback_queue_cap": 500, "dcp_mutation": 120960, "dcp_mutation_sent_to_worker": 40320, "dcp_snapshot": 118376, "dcp_stream_req_counter": 1024, "dcp_streamreq": 1024, "dcp_system_event": 10240, "execution_stats": 6543, "failure_stats": 6543, "latency_stats": 6543, "lcb_exception_stats": 6543, "log_level": 1, "num_processed_events": 40320, "processed_events_size": 36095576, "thr_count": 1, "thr_map": 1, "v8_init": 1, "v8_load": 1 }, "events_remaining": { "dcp_backlog": 0 }, "execution_stats": { "agg_queue_memory": 0, "agg_queue_size": 0, "curl": { "delete": 0, "get": 0, "head": 0, "post": 0, "put": 0 }, "curl_success_count": 0, "dcp_delete_msg_counter": 0, "dcp_delete_parse_failure": 0, "dcp_mutation_msg_counter": 40320, "dcp_mutation_parse_failure": 0, "enqueued_dcp_delete_msg_counter": 0, "enqueued_dcp_mutation_msg_counter": 40320, "enqueued_timer_msg_counter": 0, "feedback_queue_size": 0, "filtered_dcp_delete_counter": 0, "filtered_dcp_mutation_counter": 0, "lcb_retry_failure": 0, "messages_parsed": 74064, "no_op_counter": 0, "num_processed_events": 40320, "on_delete_failure": 0, "on_delete_success": 0, "on_update_failure": 0, "on_update_success": 40320, "processed_events_size": 36095576, "timer_callback_failure": 0, "timer_callback_success": 0, "timer_cancel_counter": 0, "timer_create_counter": 0, "timer_create_failure": 0, "timer_msg_counter": 0, "timer_responses_sent": 0, "timestamp": { "13027": "2021-05-05T15:46:45Z" }, "uv_msg_parse_failure": 0, "uv_try_write_failure_counter": 0 }, "failure_stats": { "app_worker_setting_events_lost": 0, "bkt_ops_cas_mismatch_count": 0, "bucket_op_exception_count": 0, "checkpoint_failure_count": 0, "curl_failure_count": 0, "curl_max_resp_size_exceeded": 0, "curl_non_200_response": 0, "curl_timeout_count": 0, "dcp_events_lost": 0, "debugger_events_lost": 0, "delete_events_lost": 0, "mutation_events_lost": 0, "n1ql_op_exception_count": 0, "timeout_count": 0, "timer_callback_missing_counter": 0, "timer_context_size_exceeded_counter": 0, "timer_events_lost": 0, "timestamp": { "13027": "2021-05-05T15:46:45Z" }, "v8worker_events_lost": 0 }, "function_id": 3865328468, "function_name": "n1ql", "gocb_creds_request_counter": 78, "internal_vb_distribution_stats": { "worker_n1ql_0": "[0-1023]" }, "latency_percentile_stats": { "100": 3232200, "50": 5100, "80": 8100, "90": 12900, "95": 27900, "99": 60000 }, "lcb_creds_request_counter": 80738, "lcb_exception_stats": {}, "metastore_stats": {}, "planner_stats": [ { "host_name": "10.112.206.104:8096", "start_vb": 0, "vb_count": 1024 } ], "vb_distribution_stats_from_metadata": { "10.112.206.104:8096": { "worker_n1ql_0": "[0-1023]" } }, "worker_pids": { "worker_n1ql_0": 13027 } }, { "dcp_feed_boundary": "everything", "event_processing_stats": { "adhoc_timer_response_received": 1, "agg_messages_sent_to_worker": 42008, "agg_queue_memory": 0, "agg_queue_memory_cap": 106954752, "agg_queue_size": 0, "agg_queue_size_cap": 100000, "agg_timer_feedback_queue_cap": 500, "dcp_mutation": 90531, "dcp_mutation_sent_to_worker": 9891, "dcp_snapshot": 1024, "dcp_stream_req_counter": 1024, "dcp_streamreq": 1024, "execution_stats": 6217, "failure_stats": 6217, "latency_stats": 6217, "lcb_exception_stats": 6217, "log_level": 1, "num_processed_events": 9891, "processed_events_size": 8853896, "thr_count": 1, "thr_map": 1, "v8_init": 1, "v8_load": 1, "worker_spawn_counter": 1 }, "events_remaining": { "dcp_backlog": 0 }, "execution_stats": { "agg_queue_memory": 0, "agg_queue_size": 0, "curl": { "delete": 0, "get": 0, "head": 0, "post": 0, "put": 0 }, "curl_success_count": 0, "dcp_delete_msg_counter": 0, "dcp_delete_parse_failure": 0, "dcp_mutation_msg_counter": 9891, "dcp_mutation_parse_failure": 0, "enqueued_dcp_delete_msg_counter": 0, "enqueued_dcp_mutation_msg_counter": 9891, "enqueued_timer_msg_counter": 0, "feedback_queue_size": 0, "filtered_dcp_delete_counter": 0, "filtered_dcp_mutation_counter": 0, "lcb_retry_failure": 0, "messages_parsed": 42004, "no_op_counter": 0, "num_processed_events": 9891, "on_delete_failure": 0, "on_delete_success": 0, "on_update_failure": 0, "on_update_success": 9891, "processed_events_size": 8853896, "timer_callback_failure": 0, "timer_callback_success": 0, "timer_cancel_counter": 0, "timer_create_counter": 0, "timer_create_failure": 0, "timer_msg_counter": 0, "timer_responses_sent": 0, "timestamp": { "21233": "2021-05-05T15:46:46Z" }, "uv_msg_parse_failure": 0, "uv_try_write_failure_counter": 0 }, "failure_stats": { "app_worker_setting_events_lost": 0, "bkt_ops_cas_mismatch_count": 0, "bucket_op_exception_count": 0, "checkpoint_failure_count": 0, "curl_failure_count": 0, "curl_max_resp_size_exceeded": 0, "curl_non_200_response": 0, "curl_timeout_count": 0, "dcp_events_lost": 0, "debugger_events_lost": 0, "delete_events_lost": 0, "mutation_events_lost": 0, "n1ql_op_exception_count": 0, "timeout_count": 0, "timer_callback_missing_counter": 0, "timer_context_size_exceeded_counter": 0, "timer_events_lost": 0, "timestamp": { "21233": "2021-05-05T15:46:46Z" }, "v8worker_events_lost": 0 }, "function_id": 2633971858, "function_name": "bucket_op", "gocb_creds_request_counter": 78, "internal_vb_distribution_stats": { "worker_bucket_op_0": "[0-1023]" }, "latency_percentile_stats": { "100": 1904300, "50": 700, "80": 2000, "90": 6800, "95": 11700, "99": 15900 }, "lcb_creds_request_counter": 80738, "lcb_exception_stats": {}, "metastore_stats": {}, "planner_stats": [ { "host_name": "10.112.206.104:8096", "start_vb": 0, "vb_count": 1024 } ], "vb_distribution_stats_from_metadata": { "10.112.206.104:8096": { "worker_bucket_op_0": "[0-1023]" } }, "worker_pids": { "worker_bucket_op_0": 21233 } }, { "dcp_feed_boundary": "everything", "event_processing_stats": { "adhoc_timer_response_received": 3, "agg_messages_sent_to_worker": 74396, "agg_queue_memory": 0, "agg_queue_memory_cap": 53477376, "agg_queue_size": 0, "agg_queue_size_cap": 100000, "agg_timer_feedback_queue_cap": 500, "dcp_mutation": 120960, "dcp_mutation_sent_to_worker": 40320, "dcp_snapshot": 117569, "dcp_stream_req_counter": 1024, "dcp_streamreq": 1024, "dcp_system_event": 10240, "execution_stats": 6608, "failure_stats": 6608, "latency_stats": 6608, "lcb_exception_stats": 6608, "log_level": 1, "num_processed_events": 40320, "processed_events_size": 36095576, "thr_count": 1, "thr_map": 1, "timer_events": 40320, "timer_responses_received": 40320, "v8_init": 1, "v8_load": 1 }, "events_remaining": { "dcp_backlog": 0 }, "execution_stats": { "agg_queue_memory": 0, "agg_queue_size": 0, "curl": { "delete": 0, "get": 0, "head": 0, "post": 0, "put": 0 }, "curl_success_count": 0, "dcp_delete_msg_counter": 0, "dcp_delete_parse_failure": 0, "dcp_mutation_msg_counter": 40320, "dcp_mutation_parse_failure": 0, "enqueued_dcp_delete_msg_counter": 0, "enqueued_dcp_mutation_msg_counter": 40320, "enqueued_timer_msg_counter": 0, "feedback_queue_size": 0, "filtered_dcp_delete_counter": 0, "filtered_dcp_mutation_counter": 0, "lcb_retry_failure": 0, "messages_parsed": 74392, "no_op_counter": 0, "num_processed_events": 40320, "on_delete_failure": 0, "on_delete_success": 0, "on_update_failure": 0, "on_update_success": 40320, "processed_events_size": 36095576, "timer_callback_failure": 0, "timer_callback_success": 40320, "timer_cancel_counter": 0, "timer_create_counter": 40320, "timer_create_failure": 0, "timer_msg_counter": 40320, "timer_responses_sent": 0, "timestamp": { "12968": "2021-05-05T15:46:46Z" }, "uv_msg_parse_failure": 0, "uv_try_write_failure_counter": 0 }, "failure_stats": { "app_worker_setting_events_lost": 0, "bkt_ops_cas_mismatch_count": 0, "bucket_op_exception_count": 0, "checkpoint_failure_count": 0, "curl_failure_count": 0, "curl_max_resp_size_exceeded": 0, "curl_non_200_response": 0, "curl_timeout_count": 0, "dcp_events_lost": 0, "debugger_events_lost": 0, "delete_events_lost": 0, "mutation_events_lost": 0, "n1ql_op_exception_count": 0, "timeout_count": 0, "timer_callback_missing_counter": 0, "timer_context_size_exceeded_counter": 0, "timer_events_lost": 0, "timestamp": { "12968": "2021-05-05T15:46:46Z" }, "v8worker_events_lost": 0 }, "function_id": 1710735491, "function_name": "timers", "gocb_creds_request_counter": 78, "internal_vb_distribution_stats": { "worker_timers_0": "[0-1023]" }, "latency_percentile_stats": { "100": 76200, "50": 1800, "80": 2600, "90": 3600, "95": 5900, "99": 17200 }, "lcb_creds_request_counter": 80738, "lcb_exception_stats": {}, "metastore_stats": { "metastore_delete_err": 0, "metastore_deletes": 0, "metastore_not_found": 0, "metastore_scan": 0, "metastore_scan_due": 0, "metastore_scan_err": 0, "metastore_set": 0, "metastore_set_err": 0 }, "planner_stats": [ { "host_name": "10.112.206.104:8096", "start_vb": 0, "vb_count": 1024 } ], "vb_distribution_stats_from_metadata": { "10.112.206.104:8096": { "worker_timers_0": "[0-1023]" } }, "worker_pids": { "worker_timers_0": 12968 } }, { "dcp_feed_boundary": "everything", "event_processing_stats": { "adhoc_timer_response_received": 2, "agg_messages_sent_to_worker": 74285, "agg_queue_memory": 0, "agg_queue_memory_cap": 53477376, "agg_queue_size": 0, "agg_queue_size_cap": 100000, "agg_timer_feedback_queue_cap": 500, "dcp_mutation": 120960, "dcp_mutation_sent_to_worker": 40320, "dcp_mutation_suppressed_counter": 40320, "dcp_snapshot": 118012, "dcp_stream_req_counter": 1024, "dcp_streamreq": 1024, "dcp_system_event": 10240, "execution_stats": 6586, "failure_stats": 6586, "latency_stats": 6586, "lcb_exception_stats": 6586, "log_level": 1, "num_processed_events": 40320, "processed_events_size": 36097612, "thr_count": 1, "thr_map": 1, "v8_init": 1, "v8_load": 1 }, "events_remaining": { "dcp_backlog": 0 }, "execution_stats": { "agg_queue_memory": 0, "agg_queue_size": 0, "curl": { "delete": 0, "get": 0, "head": 0, "post": 0, "put": 0 }, "curl_success_count": 0, "dcp_delete_msg_counter": 0, "dcp_delete_parse_failure": 0, "dcp_mutation_msg_counter": 40320, "dcp_mutation_parse_failure": 0, "enqueued_dcp_delete_msg_counter": 0, "enqueued_dcp_mutation_msg_counter": 40320, "enqueued_timer_msg_counter": 0, "feedback_queue_size": 0, "filtered_dcp_delete_counter": 0, "filtered_dcp_mutation_counter": 0, "lcb_retry_failure": 0, "messages_parsed": 74281, "no_op_counter": 0, "num_processed_events": 40320, "on_delete_failure": 0, "on_delete_success": 0, "on_update_failure": 0, "on_update_success": 40320, "processed_events_size": 36097612, "timer_callback_failure": 0, "timer_callback_success": 0, "timer_cancel_counter": 0, "timer_create_counter": 0, "timer_create_failure": 0, "timer_msg_counter": 0, "timer_responses_sent": 0, "timestamp": { "12988": "2021-05-05T15:46:45Z" }, "uv_msg_parse_failure": 0, "uv_try_write_failure_counter": 0 }, "failure_stats": { "app_worker_setting_events_lost": 0, "bkt_ops_cas_mismatch_count": 0, "bucket_op_exception_count": 0, "checkpoint_failure_count": 0, "curl_failure_count": 0, "curl_max_resp_size_exceeded": 0, "curl_non_200_response": 0, "curl_timeout_count": 0, "dcp_events_lost": 0, "debugger_events_lost": 0, "delete_events_lost": 0, "mutation_events_lost": 0, "n1ql_op_exception_count": 0, "timeout_count": 0, "timer_callback_missing_counter": 0, "timer_context_size_exceeded_counter": 0, "timer_events_lost": 0, "timestamp": { "12988": "2021-05-05T15:46:45Z" }, "v8worker_events_lost": 0 }, "function_id": 1994838142, "function_name": "sbm", "gocb_creds_request_counter": 78, "internal_vb_distribution_stats": { "worker_sbm_0": "[0-1023]" }, "latency_percentile_stats": { "100": 158000, "50": 1400, "80": 2800, "90": 4000, "95": 7400, "99": 13300 }, "lcb_creds_request_counter": 80738, "lcb_exception_stats": {}, "metastore_stats": {}, "planner_stats": [ { "host_name": "10.112.206.104:8096", "start_vb": 0, "vb_count": 1024 } ], "vb_distribution_stats_from_metadata": { "10.112.206.104:8096": { "worker_sbm_0": "[0-1023]" } }, "worker_pids": { "worker_sbm_0": 12988 } } ] 2021-05-05 21:17:35 | INFO | MainProcess | test_thread | [eventing_base.verify_doc_count_collections] Stats for Node 10.112.206.104 is [ { "dcp_feed_boundary": "everything", "event_processing_stats": { "adhoc_timer_response_received": 1, "agg_messages_sent_to_worker": 42008, "agg_queue_memory": 0, "agg_queue_memory_cap": 106954752, "agg_queue_size": 0, "agg_queue_size_cap": 100000, "agg_timer_feedback_queue_cap": 500, "dcp_mutation": 90531, "dcp_mutation_sent_to_worker": 9891, "dcp_snapshot": 1024, "dcp_stream_req_counter": 1024, "dcp_streamreq": 1024, "execution_stats": 6217, "failure_stats": 6217, "latency_stats": 6217, "lcb_exception_stats": 6217, "log_level": 1, "num_processed_events": 9891, "processed_events_size": 8853896, "thr_count": 1, "thr_map": 1, "v8_init": 1, "v8_load": 1, "worker_spawn_counter": 1 }, "events_remaining": { "dcp_backlog": 0 }, "execution_stats": { "agg_queue_memory": 0, "agg_queue_size": 0, "curl": { "delete": 0, "get": 0, "head": 0, "post": 0, "put": 0 }, "curl_success_count": 0, "dcp_delete_msg_counter": 0, "dcp_delete_parse_failure": 0, "dcp_mutation_msg_counter": 9891, "dcp_mutation_parse_failure": 0, "enqueued_dcp_delete_msg_counter": 0, "enqueued_dcp_mutation_msg_counter": 9891, "enqueued_timer_msg_counter": 0, "feedback_queue_size": 0, "filtered_dcp_delete_counter": 0, "filtered_dcp_mutation_counter": 0, "lcb_retry_failure": 0, "messages_parsed": 42004, "no_op_counter": 0, "num_processed_events": 9891, "on_delete_failure": 0, "on_delete_success": 0, "on_update_failure": 0, "on_update_success": 9891, "processed_events_size": 8853896, "timer_callback_failure": 0, "timer_callback_success": 0, "timer_cancel_counter": 0, "timer_create_counter": 0, "timer_create_failure": 0, "timer_msg_counter": 0, "timer_responses_sent": 0, "timestamp": { "21233": "2021-05-05T15:46:46Z" }, "uv_msg_parse_failure": 0, "uv_try_write_failure_counter": 0 }, "failure_stats": { "app_worker_setting_events_lost": 0, "bkt_ops_cas_mismatch_count": 0, "bucket_op_exception_count": 0, "checkpoint_failure_count": 0, "curl_failure_count": 0, "curl_max_resp_size_exceeded": 0, "curl_non_200_response": 0, "curl_timeout_count": 0, "dcp_events_lost": 0, "debugger_events_lost": 0, "delete_events_lost": 0, "mutation_events_lost": 0, "n1ql_op_exception_count": 0, "timeout_count": 0, "timer_callback_missing_counter": 0, "timer_context_size_exceeded_counter": 0, "timer_events_lost": 0, "timestamp": { "21233": "2021-05-05T15:46:46Z" }, "v8worker_events_lost": 0 }, "function_id": 2633971858, "function_name": "bucket_op", "gocb_creds_request_counter": 78, "internal_vb_distribution_stats": { "worker_bucket_op_0": "[0-1023]" }, "latency_percentile_stats": { "100": 1904300, "50": 700, "80": 2000, "90": 6800, "95": 11700, "99": 15900 }, "lcb_creds_request_counter": 80738, "lcb_exception_stats": {}, "metastore_stats": {}, "planner_stats": [ { "host_name": "10.112.206.104:8096", "start_vb": 0, "vb_count": 1024 } ], "vb_distribution_stats_from_metadata": { "10.112.206.104:8096": { "worker_bucket_op_0": "[0-1023]" } }, "worker_pids": { "worker_bucket_op_0": 21233 } }, { "dcp_feed_boundary": "everything", "event_processing_stats": { "adhoc_timer_response_received": 3, "agg_messages_sent_to_worker": 74396, "agg_queue_memory": 0, "agg_queue_memory_cap": 53477376, "agg_queue_size": 0, "agg_queue_size_cap": 100000, "agg_timer_feedback_queue_cap": 500, "dcp_mutation": 120960, "dcp_mutation_sent_to_worker": 40320, "dcp_snapshot": 117569, "dcp_stream_req_counter": 1024, "dcp_streamreq": 1024, "dcp_system_event": 10240, "execution_stats": 6608, "failure_stats": 6608, "latency_stats": 6608, "lcb_exception_stats": 6608, "log_level": 1, "num_processed_events": 40320, "processed_events_size": 36095576, "thr_count": 1, "thr_map": 1, "timer_events": 40320, "timer_responses_received": 40320, "v8_init": 1, "v8_load": 1 }, "events_remaining": { "dcp_backlog": 0 }, "execution_stats": { "agg_queue_memory": 0, "agg_queue_size": 0, "curl": { "delete": 0, "get": 0, "head": 0, "post": 0, "put": 0 }, "curl_success_count": 0, "dcp_delete_msg_counter": 0, "dcp_delete_parse_failure": 0, "dcp_mutation_msg_counter": 40320, "dcp_mutation_parse_failure": 0, "enqueued_dcp_delete_msg_counter": 0, "enqueued_dcp_mutation_msg_counter": 40320, "enqueued_timer_msg_counter": 0, "feedback_queue_size": 0, "filtered_dcp_delete_counter": 0, "filtered_dcp_mutation_counter": 0, "lcb_retry_failure": 0, "messages_parsed": 74392, "no_op_counter": 0, "num_processed_events": 40320, "on_delete_failure": 0, "on_delete_success": 0, "on_update_failure": 0, "on_update_success": 40320, "processed_events_size": 36095576, "timer_callback_failure": 0, "timer_callback_success": 40320, "timer_cancel_counter": 0, "timer_create_counter": 40320, "timer_create_failure": 0, "timer_msg_counter": 40320, "timer_responses_sent": 0, "timestamp": { "12968": "2021-05-05T15:46:46Z" }, "uv_msg_parse_failure": 0, "uv_try_write_failure_counter": 0 }, "failure_stats": { "app_worker_setting_events_lost": 0, "bkt_ops_cas_mismatch_count": 0, "bucket_op_exception_count": 0, "checkpoint_failure_count": 0, "curl_failure_count": 0, "curl_max_resp_size_exceeded": 0, "curl_non_200_response": 0, "curl_timeout_count": 0, "dcp_events_lost": 0, "debugger_events_lost": 0, "delete_events_lost": 0, "mutation_events_lost": 0, "n1ql_op_exception_count": 0, "timeout_count": 0, "timer_callback_missing_counter": 0, "timer_context_size_exceeded_counter": 0, "timer_events_lost": 0, "timestamp": { "12968": "2021-05-05T15:46:46Z" }, "v8worker_events_lost": 0 }, "function_id": 1710735491, "function_name": "timers", "gocb_creds_request_counter": 78, "internal_vb_distribution_stats": { "worker_timers_0": "[0-1023]" }, "latency_percentile_stats": { "100": 76200, "50": 1800, "80": 2600, "90": 3600, "95": 5900, "99": 17200 }, "lcb_creds_request_counter": 80738, "lcb_exception_stats": {}, "metastore_stats": { "metastore_delete_err": 0, "metastore_deletes": 0, "metastore_not_found": 0, "metastore_scan": 0, "metastore_scan_due": 0, "metastore_scan_err": 0, "metastore_set": 0, "metastore_set_err": 0 }, "planner_stats": [ { "host_name": "10.112.206.104:8096", "start_vb": 0, "vb_count": 1024 } ], "vb_distribution_stats_from_metadata": { "10.112.206.104:8096": { "worker_timers_0": "[0-1023]" } }, "worker_pids": { "worker_timers_0": 12968 } }, { "dcp_feed_boundary": "everything", "event_processing_stats": { "adhoc_timer_response_received": 2, "agg_messages_sent_to_worker": 74285, "agg_queue_memory": 0, "agg_queue_memory_cap": 53477376, "agg_queue_size": 0, "agg_queue_size_cap": 100000, "agg_timer_feedback_queue_cap": 500, "dcp_mutation": 120960, "dcp_mutation_sent_to_worker": 40320, "dcp_mutation_suppressed_counter": 40320, "dcp_snapshot": 118012, "dcp_stream_req_counter": 1024, "dcp_streamreq": 1024, "dcp_system_event": 10240, "execution_stats": 6586, "failure_stats": 6586, "latency_stats": 6586, "lcb_exception_stats": 6586, "log_level": 1, "num_processed_events": 40320, "processed_events_size": 36097612, "thr_count": 1, "thr_map": 1, "v8_init": 1, "v8_load": 1 }, "events_remaining": { "dcp_backlog": 0 }, "execution_stats": { "agg_queue_memory": 0, "agg_queue_size": 0, "curl": { "delete": 0, "get": 0, "head": 0, "post": 0, "put": 0 }, "curl_success_count": 0, "dcp_delete_msg_counter": 0, "dcp_delete_parse_failure": 0, "dcp_mutation_msg_counter": 40320, "dcp_mutation_parse_failure": 0, "enqueued_dcp_delete_msg_counter": 0, "enqueued_dcp_mutation_msg_counter": 40320, "enqueued_timer_msg_counter": 0, "feedback_queue_size": 0, "filtered_dcp_delete_counter": 0, "filtered_dcp_mutation_counter": 0, "lcb_retry_failure": 0, "messages_parsed": 74281, "no_op_counter": 0, "num_processed_events": 40320, "on_delete_failure": 0, "on_delete_success": 0, "on_update_failure": 0, "on_update_success": 40320, "processed_events_size": 36097612, "timer_callback_failure": 0, "timer_callback_success": 0, "timer_cancel_counter": 0, "timer_create_counter": 0, "timer_create_failure": 0, "timer_msg_counter": 0, "timer_responses_sent": 0, "timestamp": { "12988": "2021-05-05T15:46:45Z" }, "uv_msg_parse_failure": 0, "uv_try_write_failure_counter": 0 }, "failure_stats": { "app_worker_setting_events_lost": 0, "bkt_ops_cas_mismatch_count": 0, "bucket_op_exception_count": 0, "checkpoint_failure_count": 0, "curl_failure_count": 0, "curl_max_resp_size_exceeded": 0, "curl_non_200_response": 0, "curl_timeout_count": 0, "dcp_events_lost": 0, "debugger_events_lost": 0, "delete_events_lost": 0, "mutation_events_lost": 0, "n1ql_op_exception_count": 0, "timeout_count": 0, "timer_callback_missing_counter": 0, "timer_context_size_exceeded_counter": 0, "timer_events_lost": 0, "timestamp": { "12988": "2021-05-05T15:46:45Z" }, "v8worker_events_lost": 0 }, "function_id": 1994838142, "function_name": "sbm", "gocb_creds_request_counter": 78, "internal_vb_distribution_stats": { "worker_sbm_0": "[0-1023]" }, "latency_percentile_stats": { "100": 158000, "50": 1400, "80": 2800, "90": 4000, "95": 7400, "99": 13300 }, "lcb_creds_request_counter": 80738, "lcb_exception_stats": {}, "metastore_stats": {}, "planner_stats": [ { "host_name": "10.112.206.104:8096", "start_vb": 0, "vb_count": 1024 } ], "vb_distribution_stats_from_metadata": { "10.112.206.104:8096": { "worker_sbm_0": "[0-1023]" } }, "worker_pids": { "worker_sbm_0": 12988 } }, { "dcp_feed_boundary": "everything", "event_processing_stats": { "adhoc_timer_response_received": 100, "agg_messages_sent_to_worker": 74174, "agg_queue_memory": 0, "agg_queue_memory_cap": 53477376, "agg_queue_size": 0, "agg_queue_size_cap": 100000, "agg_timer_feedback_queue_cap": 500, "dcp_mutation": 120960, "dcp_mutation_sent_to_worker": 40320, "dcp_snapshot": 116668, "dcp_stream_req_counter": 1024, "dcp_streamreq": 1024, "dcp_system_event": 10240, "execution_stats": 6564, "failure_stats": 6564, "latency_stats": 6564, "lcb_exception_stats": 6564, "log_level": 1, "num_processed_events": 40320, "processed_events_size": 36095576, "thr_count": 1, "thr_map": 1, "v8_init": 1, "v8_load": 1 }, "events_remaining": { "dcp_backlog": 0 }, "execution_stats": { "agg_queue_memory": 0, "agg_queue_size": 0, "curl": { "delete": 0, "get": 40320, "head": 0, "post": 0, "put": 0 }, "curl_success_count": 38032, "dcp_delete_msg_counter": 0, "dcp_delete_parse_failure": 0, "dcp_mutation_msg_counter": 40320, "dcp_mutation_parse_failure": 0, "enqueued_dcp_delete_msg_counter": 0, "enqueued_dcp_mutation_msg_counter": 40320, "enqueued_timer_msg_counter": 0, "feedback_queue_size": 0, "filtered_dcp_delete_counter": 0, "filtered_dcp_mutation_counter": 0, "lcb_retry_failure": 0, "messages_parsed": 74170, "no_op_counter": 0, "num_processed_events": 40320, "on_delete_failure": 0, "on_delete_success": 0, "on_update_failure": 0, "on_update_success": 40320, "processed_events_size": 36095576, "timer_callback_failure": 0, "timer_callback_success": 0, "timer_cancel_counter": 0, "timer_create_counter": 0, "timer_create_failure": 0, "timer_msg_counter": 0, "timer_responses_sent": 0, "timestamp": { "13006": "2021-05-05T15:46:46Z" }, "uv_msg_parse_failure": 0, "uv_try_write_failure_counter": 0 }, "failure_stats": { "app_worker_setting_events_lost": 0, "bkt_ops_cas_mismatch_count": 0, "bucket_op_exception_count": 0, "checkpoint_failure_count": 0, "curl_failure_count": 2286, "curl_max_resp_size_exceeded": 0, "curl_non_200_response": 0, "curl_timeout_count": 2, "dcp_events_lost": 0, "debugger_events_lost": 0, "delete_events_lost": 0, "mutation_events_lost": 0, "n1ql_op_exception_count": 0, "timeout_count": 0, "timer_callback_missing_counter": 0, "timer_context_size_exceeded_counter": 0, "timer_events_lost": 0, "timestamp": { "13006": "2021-05-05T15:46:46Z" }, "v8worker_events_lost": 0 }, "function_id": 1020397118, "function_name": "curl", "gocb_creds_request_counter": 78, "internal_vb_distribution_stats": { "worker_curl_0": "[0-1023]" }, "latency_percentile_stats": { "100": 9999900, "50": 301100, "80": 317200, "90": 341900, "95": 370700, "99": 414200 }, "lcb_creds_request_counter": 80738, "lcb_exception_stats": {}, "metastore_stats": {}, "planner_stats": [ { "host_name": "10.112.206.104:8096", "start_vb": 0, "vb_count": 1024 } ], "vb_distribution_stats_from_metadata": { "10.112.206.104:8096": { "worker_curl_0": "[0-1023]" } }, "worker_pids": { "worker_curl_0": 13006 } }, { "dcp_feed_boundary": "everything", "event_processing_stats": { "adhoc_timer_response_received": 4, "agg_messages_sent_to_worker": 74068, "agg_queue_memory": 0, "agg_queue_memory_cap": 106954752, "agg_queue_size": 0, "agg_queue_size_cap": 100000, "agg_timer_feedback_queue_cap": 500, "dcp_mutation": 120960, "dcp_mutation_sent_to_worker": 40320, "dcp_snapshot": 118376, "dcp_stream_req_counter": 1024, "dcp_streamreq": 1024, "dcp_system_event": 10240, "execution_stats": 6543, "failure_stats": 6543, "latency_stats": 6543, "lcb_exception_stats": 6543, "log_level": 1, "num_processed_events": 40320, "processed_events_size": 36095576, "thr_count": 1, "thr_map": 1, "v8_init": 1, "v8_load": 1 }, "events_remaining": { "dcp_backlog": 0 }, "execution_stats": { "agg_queue_memory": 0, "agg_queue_size": 0, "curl": { "delete": 0, "get": 0, "head": 0, "post": 0, "put": 0 }, "curl_success_count": 0, "dcp_delete_msg_counter": 0, "dcp_delete_parse_failure": 0, "dcp_mutation_msg_counter": 40320, "dcp_mutation_parse_failure": 0, "enqueued_dcp_delete_msg_counter": 0, "enqueued_dcp_mutation_msg_counter": 40320, "enqueued_timer_msg_counter": 0, "feedback_queue_size": 0, "filtered_dcp_delete_counter": 0, "filtered_dcp_mutation_counter": 0, "lcb_retry_failure": 0, "messages_parsed": 74064, "no_op_counter": 0, "num_processed_events": 40320, "on_delete_failure": 0, "on_delete_success": 0, "on_update_failure": 0, "on_update_success": 40320, "processed_events_size": 36095576, "timer_callback_failure": 0, "timer_callback_success": 0, "timer_cancel_counter": 0, "timer_create_counter": 0, "timer_create_failure": 0, "timer_msg_counter": 0, "timer_responses_sent": 0, "timestamp": { "13027": "2021-05-05T15:46:45Z" }, "uv_msg_parse_failure": 0, "uv_try_write_failure_counter": 0 }, "failure_stats": { "app_worker_setting_events_lost": 0, "bkt_ops_cas_mismatch_count": 0, "bucket_op_exception_count": 0, "checkpoint_failure_count": 0, "curl_failure_count": 0, "curl_max_resp_size_exceeded": 0, "curl_non_200_response": 0, "curl_timeout_count": 0, "dcp_events_lost": 0, "debugger_events_lost": 0, "delete_events_lost": 0, "mutation_events_lost": 0, "n1ql_op_exception_count": 0, "timeout_count": 0, "timer_callback_missing_counter": 0, "timer_context_size_exceeded_counter": 0, "timer_events_lost": 0, "timestamp": { "13027": "2021-05-05T15:46:45Z" }, "v8worker_events_lost": 0 }, "function_id": 3865328468, "function_name": "n1ql", "gocb_creds_request_counter": 78, "internal_vb_distribution_stats": { "worker_n1ql_0": "[0-1023]" }, "latency_percentile_stats": { "100": 3232200, "50": 5100, "80": 8100, "90": 12900, "95": 27900, "99": 60000 }, "lcb_creds_request_counter": 80738, "lcb_exception_stats": {}, "metastore_stats": {}, "planner_stats": [ { "host_name": "10.112.206.104:8096", "start_vb": 0, "vb_count": 1024 } ], "vb_distribution_stats_from_metadata": { "10.112.206.104:8096": { "worker_n1ql_0": "[0-1023]" } }, "worker_pids": { "worker_n1ql_0": 13027 } } ] 2021-05-05 21:17:35 | INFO | MainProcess | test_thread | [eventing_base.verify_doc_count_collections] Execution stats update_success: 9891 update_failure: 0 delete_success: 0 delete_failure: 0 2021-05-05 21:17:35 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of eventing nodes in cluster: [ip:10.112.206.104 port:8091 ssh_username:root] 2021-05-05 21:17:35 | INFO | MainProcess | test_thread | [eventing_base.print_eventing_stats_from_all_eventing_nodes] Stats for Node 10.112.206.104 is [ { "dcp_feed_boundary": "everything", "event_processing_stats": { "adhoc_timer_response_received": 4, "agg_messages_sent_to_worker": 74093, "agg_queue_memory": 0, "agg_queue_memory_cap": 106954752, "agg_queue_size": 0, "agg_queue_size_cap": 100000, "agg_timer_feedback_queue_cap": 500, "dcp_mutation": 120960, "dcp_mutation_sent_to_worker": 40320, "dcp_snapshot": 118376, "dcp_stream_req_counter": 1024, "dcp_streamreq": 1024, "dcp_system_event": 10240, "execution_stats": 6548, "failure_stats": 6548, "latency_stats": 6548, "lcb_exception_stats": 6548, "log_level": 1, "num_processed_events": 40320, "processed_events_size": 36095576, "thr_count": 1, "thr_map": 1, "v8_init": 1, "v8_load": 1 }, "events_remaining": { "dcp_backlog": 0 }, "execution_stats": { "agg_queue_memory": 0, "agg_queue_size": 0, "curl": { "delete": 0, "get": 0, "head": 0, "post": 0, "put": 0 }, "curl_success_count": 0, "dcp_delete_msg_counter": 0, "dcp_delete_parse_failure": 0, "dcp_mutation_msg_counter": 40320, "dcp_mutation_parse_failure": 0, "enqueued_dcp_delete_msg_counter": 0, "enqueued_dcp_mutation_msg_counter": 40320, "enqueued_timer_msg_counter": 0, "feedback_queue_size": 0, "filtered_dcp_delete_counter": 0, "filtered_dcp_mutation_counter": 0, "lcb_retry_failure": 0, "messages_parsed": 74089, "no_op_counter": 0, "num_processed_events": 40320, "on_delete_failure": 0, "on_delete_success": 0, "on_update_failure": 0, "on_update_success": 40320, "processed_events_size": 36095576, "timer_callback_failure": 0, "timer_callback_success": 0, "timer_cancel_counter": 0, "timer_create_counter": 0, "timer_create_failure": 0, "timer_msg_counter": 0, "timer_responses_sent": 0, "timestamp": { "13027": "2021-05-05T15:46:50Z" }, "uv_msg_parse_failure": 0, "uv_try_write_failure_counter": 0 }, "failure_stats": { "app_worker_setting_events_lost": 0, "bkt_ops_cas_mismatch_count": 0, "bucket_op_exception_count": 0, "checkpoint_failure_count": 0, "curl_failure_count": 0, "curl_max_resp_size_exceeded": 0, "curl_non_200_response": 0, "curl_timeout_count": 0, "dcp_events_lost": 0, "debugger_events_lost": 0, "delete_events_lost": 0, "mutation_events_lost": 0, "n1ql_op_exception_count": 0, "timeout_count": 0, "timer_callback_missing_counter": 0, "timer_context_size_exceeded_counter": 0, "timer_events_lost": 0, "timestamp": { "13027": "2021-05-05T15:46:50Z" }, "v8worker_events_lost": 0 }, "function_id": 3865328468, "function_name": "n1ql", "gocb_creds_request_counter": 78, "internal_vb_distribution_stats": { "worker_n1ql_0": "[0-1023]" }, "latency_percentile_stats": { "100": 3232200, "50": 5100, "80": 8100, "90": 12900, "95": 27900, "99": 60000 }, "lcb_creds_request_counter": 80738, "lcb_exception_stats": {}, "metastore_stats": {}, "planner_stats": [ { "host_name": "10.112.206.104:8096", "start_vb": 0, "vb_count": 1024 } ], "vb_distribution_stats_from_metadata": { "10.112.206.104:8096": { "worker_n1ql_0": "[0-1023]" } }, "worker_pids": { "worker_n1ql_0": 13027 } }, { "dcp_feed_boundary": "everything", "event_processing_stats": { "adhoc_timer_response_received": 1, "agg_messages_sent_to_worker": 42033, "agg_queue_memory": 0, "agg_queue_memory_cap": 106954752, "agg_queue_size": 0, "agg_queue_size_cap": 100000, "agg_timer_feedback_queue_cap": 500, "dcp_mutation": 90531, "dcp_mutation_sent_to_worker": 9891, "dcp_snapshot": 1024, "dcp_stream_req_counter": 1024, "dcp_streamreq": 1024, "execution_stats": 6222, "failure_stats": 6222, "latency_stats": 6222, "lcb_exception_stats": 6222, "log_level": 1, "num_processed_events": 9891, "processed_events_size": 8853896, "thr_count": 1, "thr_map": 1, "v8_init": 1, "v8_load": 1, "worker_spawn_counter": 1 }, "events_remaining": { "dcp_backlog": 0 }, "execution_stats": { "agg_queue_memory": 0, "agg_queue_size": 0, "curl": { "delete": 0, "get": 0, "head": 0, "post": 0, "put": 0 }, "curl_success_count": 0, "dcp_delete_msg_counter": 0, "dcp_delete_parse_failure": 0, "dcp_mutation_msg_counter": 9891, "dcp_mutation_parse_failure": 0, "enqueued_dcp_delete_msg_counter": 0, "enqueued_dcp_mutation_msg_counter": 9891, "enqueued_timer_msg_counter": 0, "feedback_queue_size": 0, "filtered_dcp_delete_counter": 0, "filtered_dcp_mutation_counter": 0, "lcb_retry_failure": 0, "messages_parsed": 42029, "no_op_counter": 0, "num_processed_events": 9891, "on_delete_failure": 0, "on_delete_success": 0, "on_update_failure": 0, "on_update_success": 9891, "processed_events_size": 8853896, "timer_callback_failure": 0, "timer_callback_success": 0, "timer_cancel_counter": 0, "timer_create_counter": 0, "timer_create_failure": 0, "timer_msg_counter": 0, "timer_responses_sent": 0, "timestamp": { "21233": "2021-05-05T15:46:51Z" }, "uv_msg_parse_failure": 0, "uv_try_write_failure_counter": 0 }, "failure_stats": { "app_worker_setting_events_lost": 0, "bkt_ops_cas_mismatch_count": 0, "bucket_op_exception_count": 0, "checkpoint_failure_count": 0, "curl_failure_count": 0, "curl_max_resp_size_exceeded": 0, "curl_non_200_response": 0, "curl_timeout_count": 0, "dcp_events_lost": 0, "debugger_events_lost": 0, "delete_events_lost": 0, "mutation_events_lost": 0, "n1ql_op_exception_count": 0, "timeout_count": 0, "timer_callback_missing_counter": 0, "timer_context_size_exceeded_counter": 0, "timer_events_lost": 0, "timestamp": { "21233": "2021-05-05T15:46:51Z" }, "v8worker_events_lost": 0 }, "function_id": 2633971858, "function_name": "bucket_op", "gocb_creds_request_counter": 78, "internal_vb_distribution_stats": { "worker_bucket_op_0": "[0-1023]" }, "latency_percentile_stats": { "100": 1904300, "50": 700, "80": 2000, "90": 6800, "95": 11700, "99": 15900 }, "lcb_creds_request_counter": 80738, "lcb_exception_stats": {}, "metastore_stats": {}, "planner_stats": [ { "host_name": "10.112.206.104:8096", "start_vb": 0, "vb_count": 1024 } ], "vb_distribution_stats_from_metadata": { "10.112.206.104:8096": { "worker_bucket_op_0": "[0-1023]" } }, "worker_pids": { "worker_bucket_op_0": 21233 } }, { "dcp_feed_boundary": "everything", "event_processing_stats": { "adhoc_timer_response_received": 3, "agg_messages_sent_to_worker": 74421, "agg_queue_memory": 0, "agg_queue_memory_cap": 53477376, "agg_queue_size": 0, "agg_queue_size_cap": 100000, "agg_timer_feedback_queue_cap": 500, "dcp_mutation": 120960, "dcp_mutation_sent_to_worker": 40320, "dcp_snapshot": 117569, "dcp_stream_req_counter": 1024, "dcp_streamreq": 1024, "dcp_system_event": 10240, "execution_stats": 6613, "failure_stats": 6613, "latency_stats": 6613, "lcb_exception_stats": 6613, "log_level": 1, "num_processed_events": 40320, "processed_events_size": 36095576, "thr_count": 1, "thr_map": 1, "timer_events": 40320, "timer_responses_received": 40320, "v8_init": 1, "v8_load": 1 }, "events_remaining": { "dcp_backlog": 0 }, "execution_stats": { "agg_queue_memory": 0, "agg_queue_size": 0, "curl": { "delete": 0, "get": 0, "head": 0, "post": 0, "put": 0 }, "curl_success_count": 0, "dcp_delete_msg_counter": 0, "dcp_delete_parse_failure": 0, "dcp_mutation_msg_counter": 40320, "dcp_mutation_parse_failure": 0, "enqueued_dcp_delete_msg_counter": 0, "enqueued_dcp_mutation_msg_counter": 40320, "enqueued_timer_msg_counter": 0, "feedback_queue_size": 0, "filtered_dcp_delete_counter": 0, "filtered_dcp_mutation_counter": 0, "lcb_retry_failure": 0, "messages_parsed": 74417, "no_op_counter": 0, "num_processed_events": 40320, "on_delete_failure": 0, "on_delete_success": 0, "on_update_failure": 0, "on_update_success": 40320, "processed_events_size": 36095576, "timer_callback_failure": 0, "timer_callback_success": 40320, "timer_cancel_counter": 0, "timer_create_counter": 40320, "timer_create_failure": 0, "timer_msg_counter": 40320, "timer_responses_sent": 0, "timestamp": { "12968": "2021-05-05T15:46:51Z" }, "uv_msg_parse_failure": 0, "uv_try_write_failure_counter": 0 }, "failure_stats": { "app_worker_setting_events_lost": 0, "bkt_ops_cas_mismatch_count": 0, "bucket_op_exception_count": 0, "checkpoint_failure_count": 0, "curl_failure_count": 0, "curl_max_resp_size_exceeded": 0, "curl_non_200_response": 0, "curl_timeout_count": 0, "dcp_events_lost": 0, "debugger_events_lost": 0, "delete_events_lost": 0, "mutation_events_lost": 0, "n1ql_op_exception_count": 0, "timeout_count": 0, "timer_callback_missing_counter": 0, "timer_context_size_exceeded_counter": 0, "timer_events_lost": 0, "timestamp": { "12968": "2021-05-05T15:46:51Z" }, "v8worker_events_lost": 0 }, "function_id": 1710735491, "function_name": "timers", "gocb_creds_request_counter": 78, "internal_vb_distribution_stats": { "worker_timers_0": "[0-1023]" }, "latency_percentile_stats": { "100": 76200, "50": 1800, "80": 2600, "90": 3600, "95": 5900, "99": 17200 }, "lcb_creds_request_counter": 80738, "lcb_exception_stats": {}, "metastore_stats": { "metastore_delete_err": 0, "metastore_deletes": 0, "metastore_not_found": 0, "metastore_scan": 0, "metastore_scan_due": 0, "metastore_scan_err": 0, "metastore_set": 0, "metastore_set_err": 0 }, "planner_stats": [ { "host_name": "10.112.206.104:8096", "start_vb": 0, "vb_count": 1024 } ], "vb_distribution_stats_from_metadata": { "10.112.206.104:8096": { "worker_timers_0": "[0-1023]" } }, "worker_pids": { "worker_timers_0": 12968 } }, { "dcp_feed_boundary": "everything", "event_processing_stats": { "adhoc_timer_response_received": 2, "agg_messages_sent_to_worker": 74310, "agg_queue_memory": 0, "agg_queue_memory_cap": 53477376, "agg_queue_size": 0, "agg_queue_size_cap": 100000, "agg_timer_feedback_queue_cap": 500, "dcp_mutation": 120960, "dcp_mutation_sent_to_worker": 40320, "dcp_mutation_suppressed_counter": 40320, "dcp_snapshot": 118012, "dcp_stream_req_counter": 1024, "dcp_streamreq": 1024, "dcp_system_event": 10240, "execution_stats": 6591, "failure_stats": 6591, "latency_stats": 6591, "lcb_exception_stats": 6591, "log_level": 1, "num_processed_events": 40320, "processed_events_size": 36097612, "thr_count": 1, "thr_map": 1, "v8_init": 1, "v8_load": 1 }, "events_remaining": { "dcp_backlog": 0 }, "execution_stats": { "agg_queue_memory": 0, "agg_queue_size": 0, "curl": { "delete": 0, "get": 0, "head": 0, "post": 0, "put": 0 }, "curl_success_count": 0, "dcp_delete_msg_counter": 0, "dcp_delete_parse_failure": 0, "dcp_mutation_msg_counter": 40320, "dcp_mutation_parse_failure": 0, "enqueued_dcp_delete_msg_counter": 0, "enqueued_dcp_mutation_msg_counter": 40320, "enqueued_timer_msg_counter": 0, "feedback_queue_size": 0, "filtered_dcp_delete_counter": 0, "filtered_dcp_mutation_counter": 0, "lcb_retry_failure": 0, "messages_parsed": 74306, "no_op_counter": 0, "num_processed_events": 40320, "on_delete_failure": 0, "on_delete_success": 0, "on_update_failure": 0, "on_update_success": 40320, "processed_events_size": 36097612, "timer_callback_failure": 0, "timer_callback_success": 0, "timer_cancel_counter": 0, "timer_create_counter": 0, "timer_create_failure": 0, "timer_msg_counter": 0, "timer_responses_sent": 0, "timestamp": { "12988": "2021-05-05T15:46:50Z" }, "uv_msg_parse_failure": 0, "uv_try_write_failure_counter": 0 }, "failure_stats": { "app_worker_setting_events_lost": 0, "bkt_ops_cas_mismatch_count": 0, "bucket_op_exception_count": 0, "checkpoint_failure_count": 0, "curl_failure_count": 0, "curl_max_resp_size_exceeded": 0, "curl_non_200_response": 0, "curl_timeout_count": 0, "dcp_events_lost": 0, "debugger_events_lost": 0, "delete_events_lost": 0, "mutation_events_lost": 0, "n1ql_op_exception_count": 0, "timeout_count": 0, "timer_callback_missing_counter": 0, "timer_context_size_exceeded_counter": 0, "timer_events_lost": 0, "timestamp": { "12988": "2021-05-05T15:46:50Z" }, "v8worker_events_lost": 0 }, "function_id": 1994838142, "function_name": "sbm", "gocb_creds_request_counter": 78, "internal_vb_distribution_stats": { "worker_sbm_0": "[0-1023]" }, "latency_percentile_stats": { "100": 158000, "50": 1400, "80": 2800, "90": 4000, "95": 7400, "99": 13300 }, "lcb_creds_request_counter": 80738, "lcb_exception_stats": {}, "metastore_stats": {}, "planner_stats": [ { "host_name": "10.112.206.104:8096", "start_vb": 0, "vb_count": 1024 } ], "vb_distribution_stats_from_metadata": { "10.112.206.104:8096": { "worker_sbm_0": "[0-1023]" } }, "worker_pids": { "worker_sbm_0": 12988 } }, { "dcp_feed_boundary": "everything", "event_processing_stats": { "adhoc_timer_response_received": 100, "agg_messages_sent_to_worker": 74199, "agg_queue_memory": 0, "agg_queue_memory_cap": 53477376, "agg_queue_size": 0, "agg_queue_size_cap": 100000, "agg_timer_feedback_queue_cap": 500, "dcp_mutation": 120960, "dcp_mutation_sent_to_worker": 40320, "dcp_snapshot": 116668, "dcp_stream_req_counter": 1024, "dcp_streamreq": 1024, "dcp_system_event": 10240, "execution_stats": 6569, "failure_stats": 6569, "latency_stats": 6569, "lcb_exception_stats": 6569, "log_level": 1, "num_processed_events": 40320, "processed_events_size": 36095576, "thr_count": 1, "thr_map": 1, "v8_init": 1, "v8_load": 1 }, "events_remaining": { "dcp_backlog": 0 }, "execution_stats": { "agg_queue_memory": 0, "agg_queue_size": 0, "curl": { "delete": 0, "get": 40320, "head": 0, "post": 0, "put": 0 }, "curl_success_count": 38032, "dcp_delete_msg_counter": 0, "dcp_delete_parse_failure": 0, "dcp_mutation_msg_counter": 40320, "dcp_mutation_parse_failure": 0, "enqueued_dcp_delete_msg_counter": 0, "enqueued_dcp_mutation_msg_counter": 40320, "enqueued_timer_msg_counter": 0, "feedback_queue_size": 0, "filtered_dcp_delete_counter": 0, "filtered_dcp_mutation_counter": 0, "lcb_retry_failure": 0, "messages_parsed": 74195, "no_op_counter": 0, "num_processed_events": 40320, "on_delete_failure": 0, "on_delete_success": 0, "on_update_failure": 0, "on_update_success": 40320, "processed_events_size": 36095576, "timer_callback_failure": 0, "timer_callback_success": 0, "timer_cancel_counter": 0, "timer_create_counter": 0, "timer_create_failure": 0, "timer_msg_counter": 0, "timer_responses_sent": 0, "timestamp": { "13006": "2021-05-05T15:46:51Z" }, "uv_msg_parse_failure": 0, "uv_try_write_failure_counter": 0 }, "failure_stats": { "app_worker_setting_events_lost": 0, "bkt_ops_cas_mismatch_count": 0, "bucket_op_exception_count": 0, "checkpoint_failure_count": 0, "curl_failure_count": 2286, "curl_max_resp_size_exceeded": 0, "curl_non_200_response": 0, "curl_timeout_count": 2, "dcp_events_lost": 0, "debugger_events_lost": 0, "delete_events_lost": 0, "mutation_events_lost": 0, "n1ql_op_exception_count": 0, "timeout_count": 0, "timer_callback_missing_counter": 0, "timer_context_size_exceeded_counter": 0, "timer_events_lost": 0, "timestamp": { "13006": "2021-05-05T15:46:51Z" }, "v8worker_events_lost": 0 }, "function_id": 1020397118, "function_name": "curl", "gocb_creds_request_counter": 78, "internal_vb_distribution_stats": { "worker_curl_0": "[0-1023]" }, "latency_percentile_stats": { "100": 9999900, "50": 301100, "80": 317200, "90": 341900, "95": 370700, "99": 414200 }, "lcb_creds_request_counter": 80738, "lcb_exception_stats": {}, "metastore_stats": {}, "planner_stats": [ { "host_name": "10.112.206.104:8096", "start_vb": 0, "vb_count": 1024 } ], "vb_distribution_stats_from_metadata": { "10.112.206.104:8096": { "worker_curl_0": "[0-1023]" } }, "worker_pids": { "worker_curl_0": 13006 } } ] 2021-05-05 21:17:35 | INFO | MainProcess | test_thread | [eventing_base.tearDown] ============== EventingBaseTest tearDown has started ============== 2021-05-05 21:17:35 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of eventing nodes in cluster: [ip:10.112.206.104 port:8091 ssh_username:root] 2021-05-05 21:17:35 | INFO | MainProcess | test_thread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.104 with username:root, attempt#1 of 5 2021-05-05 21:17:35 | INFO | MainProcess | test_thread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.104 as root 2021-05-05 21:17:36 | INFO | MainProcess | test_thread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 21:17:36 | INFO | MainProcess | test_thread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 21:17:36 | INFO | MainProcess | test_thread | [rest_client.diag_eval] /diag/eval status on 10.112.206.104:8091: True content: "/opt/couchbase/var/lib/couchbase/logs" command: filename:absname(element(2, application:get_env(ns_server,error_logger_mf_dir))). 2021-05-05 21:17:36 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] running command.raw on 10.112.206.104: zgrep "panic" "/opt/couchbase/var/lib/couchbase/logs"/eventing.log* | wc -l 2021-05-05 21:17:36 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 21:17:36 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] running command.raw on 10.112.206.104: ls "/opt/couchbase/var/lib/couchbase/logs"/../crash/| wc -l 2021-05-05 21:17:36 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] command executed successfully with root 2021-05-05 21:17:36 | INFO | MainProcess | test_thread | [eventing_base.check_eventing_logs_for_panic] ===== CORE DUMPS SEEN ON EVENTING NODES, SERVER 10.112.206.104 : 2 crashes seen ===== 2021-05-05 21:17:36 | INFO | MainProcess | test_thread | [basetestcase.print_cluster_stats] ------- Cluster statistics ------- 2021-05-05 21:17:36 | INFO | MainProcess | test_thread | [basetestcase.print_cluster_stats] 10.112.206.101:8091 => {'services': ['kv'], 'cpu_utilization': 16.31455399061033, 'mem_free': 1215897600, 'mem_total': 1930829824, 'swap_mem_used': 0, 'swap_mem_total': 1107292160} 2021-05-05 21:17:36 | INFO | MainProcess | test_thread | [basetestcase.print_cluster_stats] 10.112.206.102:8091 => {'services': ['kv'], 'cpu_utilization': 13.35578002244669, 'mem_free': 1281114112, 'mem_total': 1930829824, 'swap_mem_used': 0, 'swap_mem_total': 1107292160} 2021-05-05 21:17:36 | INFO | MainProcess | test_thread | [basetestcase.print_cluster_stats] 10.112.206.103:8091 => {'services': ['kv'], 'cpu_utilization': 6.351039260969977, 'mem_free': 1285537792, 'mem_total': 1930829824, 'swap_mem_used': 0, 'swap_mem_total': 1107292160} 2021-05-05 21:17:36 | INFO | MainProcess | test_thread | [basetestcase.print_cluster_stats] 10.112.206.104:8091 => {'services': ['eventing'], 'cpu_utilization': 5.826271186440678, 'mem_free': 1095053312, 'mem_total': 1930829824, 'swap_mem_used': 0, 'swap_mem_total': 1107292160} 2021-05-05 21:17:36 | INFO | MainProcess | test_thread | [basetestcase.print_cluster_stats] 10.112.206.105:8091 => {'services': ['index', 'n1ql'], 'cpu_utilization': 2.600216684723727, 'mem_free': 1111498752, 'mem_total': 1930829824, 'swap_mem_used': 0, 'swap_mem_total': 1107292160} 2021-05-05 21:17:36 | INFO | MainProcess | test_thread | [basetestcase.print_cluster_stats] --- End of cluster statistics --- 2021-05-05 21:17:36 | INFO | MainProcess | test_thread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.101 with username:root, attempt#1 of 5 2021-05-05 21:17:37 | INFO | MainProcess | test_thread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.101 as root 2021-05-05 21:17:37 | INFO | MainProcess | test_thread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 21:17:37 | INFO | MainProcess | test_thread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 21:17:37 | INFO | MainProcess | test_thread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.102 with username:root, attempt#1 of 5 2021-05-05 21:17:37 | INFO | MainProcess | test_thread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.102 as root 2021-05-05 21:17:37 | INFO | MainProcess | test_thread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 21:17:37 | INFO | MainProcess | test_thread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 21:17:37 | INFO | MainProcess | test_thread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.103 with username:root, attempt#1 of 5 2021-05-05 21:17:38 | INFO | MainProcess | test_thread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.103 as root 2021-05-05 21:17:38 | INFO | MainProcess | test_thread | [remote_util.extract_remote_info] os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-05-05 21:17:38 | INFO | MainProcess | test_thread | [remote_util.extract_remote_info] extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-05-05 21:17:38 | INFO | MainProcess | test_thread | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.104 with username:root, attempt#1 of 5 2021-05-05 15:47:38 | INFO | MainProcess | test_thread | [remote_util.ssh_connect_with_retries] SSH Connected to 10.112.206.104 as root ERROR ERROR ====================================================================== ERROR: test_eventing_rebalance_with_multiple_kv_nodes (eventing.eventing_rebalance_collection.EventingRebalanceCollection) ---------------------------------------------------------------------- Traceback (most recent call last): File "pytests/eventing/eventing_rebalance_collection.py", line 349, in test_eventing_rebalance_with_multiple_kv_nodes self.verify_all_handler(self.docs_per_day * self.num_docs) File "pytests/eventing/eventing_rebalance_collection.py", line 81, in verify_all_handler self.verify_doc_count_collections("dst_bucket.scope_1.coll_3", number_of_docs) File "./pytests/eventing/eventing_base.py", line 1045, in verify_doc_count_collections raise Exception("missing data in destination bucket. Current : {0} " Exception: missing data in destination bucket. Current : 38032 Expected : 40320 dcp_backlog : 0 TIMERS_IN_PAST : 0 lcb_exceptions : {'10.112.206.104': {}} ====================================================================== ERROR: test_eventing_rebalance_with_multiple_kv_nodes (eventing.eventing_rebalance_collection.EventingRebalanceCollection) ---------------------------------------------------------------------- Traceback (most recent call last): File "pytests/eventing/eventing_rebalance_collection.py", line 57, in tearDown super(EventingRebalanceCollection, self).tearDown() File "./pytests/eventing/eventing_base.py", line 147, in tearDown super(EventingBaseTest, self).tearDown() File "./pytests/query_tests_helper.py", line 62, in tearDown super(QueryHelperTests, self).tearDown() File "pytests/basetestcase.py", line 527, in tearDown _tasks = self.cluster.async_log_scan(self.servers, self.log_scan_file_prefix+"_AFTER") File "lib/couchbase_helper/cluster.py", line 1411, in async_log_scan File "lib/tasks/task.py", line 6493, in __init__ self.log_scanner = LogScanner(server=self.server, exclude_keywords=exclude_keywords, File "./lib/log_scanner.py", line 56, in __init__ File "lib/remote/remote_util.py", line 267, in __init__ File "lib/remote/remote_util.py", line 3552, in extract_remote_info File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/paramiko/sftp_client.py", line 801, in get OSError: [Errno 24] Too many open files: 'etc-os-release-c7dd668d-9583-4f14-9087-4b83c3e1a29e' ---------------------------------------------------------------------- Ran 1 test in 7027.093s FAILED (errors=2) 2021-05-05 15:47:38 | INFO | MainProcess | Thread-370 | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.101 with username:root, attempt#1 of 5 2021-05-05 15:47:38 | INFO | MainProcess | Thread-371 | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.102 with username:root, attempt#1 of 5 2021-05-05 15:47:38 | ERROR | MainProcess | Thread-370 | [remote_util.ssh_connect_with_retries] Can't establish SSH session (unknown reason) to node 10.112.206.101 : [Errno 24] Too many open files with credentials root:couchbase 2021-05-05 15:47:38 | INFO | MainProcess | Thread-372 | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.103 with username:root, attempt#1 of 5 2021-05-05 15:47:38 | INFO | MainProcess | Thread-373 | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.104 with username:root, attempt#1 of 5 2021-05-05 15:47:38 | ERROR | MainProcess | Thread-371 | [remote_util.ssh_connect_with_retries] Can't establish SSH session (unknown reason) to node 10.112.206.102 : [Errno 24] Too many open files with credentials root:couchbase 2021-05-05 15:47:38 | INFO | MainProcess | Thread-374 | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.105 with username:root, attempt#1 of 5 2021-05-05 15:47:38 | ERROR | MainProcess | Thread-372 | [remote_util.ssh_connect_with_retries] Can't establish SSH session (unknown reason) to node 10.112.206.103 : [Errno 24] Too many open files with credentials root:couchbase 2021-05-05 15:47:38 | ERROR | MainProcess | Thread-373 | [remote_util.ssh_connect_with_retries] Can't establish SSH session (unknown reason) to node 10.112.206.104 : [Errno 24] Too many open files with credentials root:couchbase 2021-05-05 15:47:38 | INFO | MainProcess | Thread-370 | [remote_util.ssh_connect_with_retries] Retrying with exponential back off delay for 10 secs. 2021-05-05 15:47:38 | INFO | MainProcess | Thread-371 | [remote_util.ssh_connect_with_retries] Retrying with exponential back off delay for 10 secs. 2021-05-05 15:47:38 | ERROR | MainProcess | Thread-374 | [remote_util.ssh_connect_with_retries] Can't establish SSH session (unknown reason) to node 10.112.206.105 : [Errno 24] Too many open files with credentials root:couchbase 2021-05-05 15:47:38 | INFO | MainProcess | Thread-372 | [remote_util.ssh_connect_with_retries] Retrying with exponential back off delay for 10 secs. 2021-05-05 15:47:38 | INFO | MainProcess | Thread-373 | [remote_util.ssh_connect_with_retries] Retrying with exponential back off delay for 10 secs. 2021-05-05 15:47:38 | INFO | MainProcess | Thread-374 | [remote_util.ssh_connect_with_retries] Retrying with exponential back off delay for 10 secs. 2021-05-05 21:17:48 | INFO | MainProcess | Thread-370 | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.101 with username:root, attempt#2 of 5 2021-05-05 21:17:48 | INFO | MainProcess | Thread-374 | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.105 with username:root, attempt#2 of 5 2021-05-05 21:17:48 | INFO | MainProcess | Thread-372 | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.103 with username:root, attempt#2 of 5 2021-05-05 21:17:48 | INFO | MainProcess | Thread-371 | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.102 with username:root, attempt#2 of 5 2021-05-05 21:17:48 | INFO | MainProcess | Thread-373 | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.104 with username:root, attempt#2 of 5 2021-05-05 15:47:48 | ERROR | MainProcess | Thread-372 | [remote_util.ssh_connect_with_retries] Can't establish SSH session (unknown reason) to node 10.112.206.103 : [Errno 24] Too many open files with credentials root:couchbase 2021-05-05 15:47:48 | INFO | MainProcess | Thread-372 | [remote_util.ssh_connect_with_retries] Retrying with exponential back off delay for 20 secs. 2021-05-05 15:47:48 | ERROR | MainProcess | Thread-371 | [remote_util.ssh_connect_with_retries] Can't establish SSH session (unknown reason) to node 10.112.206.102 : [Errno 24] Too many open files with credentials root:couchbase 2021-05-05 15:47:48 | INFO | MainProcess | Thread-371 | [remote_util.ssh_connect_with_retries] Retrying with exponential back off delay for 20 secs. 2021-05-05 15:47:48 | ERROR | MainProcess | Thread-373 | [remote_util.ssh_connect_with_retries] Can't establish SSH session (unknown reason) to node 10.112.206.104 : [Errno 24] Too many open files with credentials root:couchbase 2021-05-05 15:47:48 | INFO | MainProcess | Thread-373 | [remote_util.ssh_connect_with_retries] Retrying with exponential back off delay for 20 secs. 2021-05-05 15:47:48 | ERROR | MainProcess | Thread-370 | [remote_util.ssh_connect_with_retries] Can't establish SSH session (unknown reason) to node 10.112.206.101 : [Errno 24] Too many open files with credentials root:couchbase 2021-05-05 15:47:48 | INFO | MainProcess | Thread-370 | [remote_util.ssh_connect_with_retries] Retrying with exponential back off delay for 20 secs. 2021-05-05 15:47:48 | ERROR | MainProcess | Thread-374 | [remote_util.ssh_connect_with_retries] Can't establish SSH session (unknown reason) to node 10.112.206.105 : [Errno 24] Too many open files with credentials root:couchbase 2021-05-05 15:47:48 | INFO | MainProcess | Thread-374 | [remote_util.ssh_connect_with_retries] Retrying with exponential back off delay for 20 secs. 2021-05-05 21:18:08 | INFO | MainProcess | Thread-372 | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.103 with username:root, attempt#3 of 5 2021-05-05 21:18:08 | INFO | MainProcess | Thread-371 | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.102 with username:root, attempt#3 of 5 2021-05-05 15:48:08 | ERROR | MainProcess | Thread-372 | [remote_util.ssh_connect_with_retries] Can't establish SSH session (unknown reason) to node 10.112.206.103 : [Errno 24] Too many open files with credentials root:couchbase 2021-05-05 15:48:08 | INFO | MainProcess | Thread-372 | [remote_util.ssh_connect_with_retries] Retrying with exponential back off delay for 40 secs. 2021-05-05 15:48:08 | INFO | MainProcess | Thread-373 | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.104 with username:root, attempt#3 of 5 2021-05-05 15:48:08 | INFO | MainProcess | Thread-370 | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.101 with username:root, attempt#3 of 5 2021-05-05 15:48:08 | ERROR | MainProcess | Thread-373 | [remote_util.ssh_connect_with_retries] Can't establish SSH session (unknown reason) to node 10.112.206.104 : [Errno 24] Too many open files with credentials root:couchbase 2021-05-05 15:48:08 | INFO | MainProcess | Thread-373 | [remote_util.ssh_connect_with_retries] Retrying with exponential back off delay for 40 secs. 2021-05-05 15:48:08 | ERROR | MainProcess | Thread-370 | [remote_util.ssh_connect_with_retries] Can't establish SSH session (unknown reason) to node 10.112.206.101 : [Errno 24] Too many open files with credentials root:couchbase 2021-05-05 15:48:08 | INFO | MainProcess | Thread-370 | [remote_util.ssh_connect_with_retries] Retrying with exponential back off delay for 40 secs. 2021-05-05 15:48:08 | ERROR | MainProcess | Thread-371 | [remote_util.ssh_connect_with_retries] Can't establish SSH session (unknown reason) to node 10.112.206.102 : [Errno 24] Too many open files with credentials root:couchbase 2021-05-05 15:48:08 | INFO | MainProcess | Thread-371 | [remote_util.ssh_connect_with_retries] Retrying with exponential back off delay for 40 secs. 2021-05-05 15:48:08 | INFO | MainProcess | Thread-374 | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.105 with username:root, attempt#3 of 5 2021-05-05 15:48:08 | ERROR | MainProcess | Thread-374 | [remote_util.ssh_connect_with_retries] Can't establish SSH session (unknown reason) to node 10.112.206.105 : [Errno 24] Too many open files with credentials root:couchbase 2021-05-05 15:48:08 | INFO | MainProcess | Thread-374 | [remote_util.ssh_connect_with_retries] Retrying with exponential back off delay for 40 secs. 2021-05-05 15:48:48 | INFO | MainProcess | Thread-372 | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.103 with username:root, attempt#4 of 5 2021-05-05 15:48:48 | ERROR | MainProcess | Thread-372 | [remote_util.ssh_connect_with_retries] Can't establish SSH session (unknown reason) to node 10.112.206.103 : [Errno 24] Too many open files with credentials root:couchbase 2021-05-05 15:48:48 | INFO | MainProcess | Thread-372 | [remote_util.ssh_connect_with_retries] Retrying with exponential back off delay for 80 secs. 2021-05-05 15:48:48 | INFO | MainProcess | Thread-373 | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.104 with username:root, attempt#4 of 5 2021-05-05 15:48:48 | INFO | MainProcess | Thread-370 | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.101 with username:root, attempt#4 of 5 2021-05-05 15:48:48 | ERROR | MainProcess | Thread-373 | [remote_util.ssh_connect_with_retries] Can't establish SSH session (unknown reason) to node 10.112.206.104 : [Errno 24] Too many open files with credentials root:couchbase 2021-05-05 15:48:48 | INFO | MainProcess | Thread-373 | [remote_util.ssh_connect_with_retries] Retrying with exponential back off delay for 80 secs. 2021-05-05 15:48:48 | ERROR | MainProcess | Thread-370 | [remote_util.ssh_connect_with_retries] Can't establish SSH session (unknown reason) to node 10.112.206.101 : [Errno 24] Too many open files with credentials root:couchbase 2021-05-05 15:48:48 | INFO | MainProcess | Thread-370 | [remote_util.ssh_connect_with_retries] Retrying with exponential back off delay for 80 secs. 2021-05-05 15:48:48 | INFO | MainProcess | Thread-371 | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.102 with username:root, attempt#4 of 5 2021-05-05 15:48:48 | ERROR | MainProcess | Thread-371 | [remote_util.ssh_connect_with_retries] Can't establish SSH session (unknown reason) to node 10.112.206.102 : [Errno 24] Too many open files with credentials root:couchbase 2021-05-05 15:48:48 | INFO | MainProcess | Thread-371 | [remote_util.ssh_connect_with_retries] Retrying with exponential back off delay for 80 secs. 2021-05-05 15:48:48 | INFO | MainProcess | Thread-374 | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.105 with username:root, attempt#4 of 5 2021-05-05 15:48:48 | ERROR | MainProcess | Thread-374 | [remote_util.ssh_connect_with_retries] Can't establish SSH session (unknown reason) to node 10.112.206.105 : [Errno 24] Too many open files with credentials root:couchbase 2021-05-05 15:48:48 | INFO | MainProcess | Thread-374 | [remote_util.ssh_connect_with_retries] Retrying with exponential back off delay for 80 secs. 2021-05-05 21:20:08 | INFO | MainProcess | Thread-372 | [remote_util.ssh_connect_with_retries] SSH Connecting to 10.112.206.103 with username:root, attempt#5 of 5