2021-04-23 18:07:49,898 | test | INFO | MainProcess | MainThread | [basetestcase:log_setup_status:542] ========= CBASDatasetsAndCollections setup started for test #52 test_KV_collection_deletion_does_not_effect_dataset_on_other_collections ========= 2021-04-23 18:07:50,351 | test | INFO | MainProcess | MainThread | [basetestcase:log_setup_status:542] ========= BaseTestCase setup started for test #52 test_KV_collection_deletion_does_not_effect_dataset_on_other_collections ========= 2021-04-23 18:07:50,720 | test | INFO | MainProcess | MainThread | [basetestcase:setUp:318] Initializing cluster 2021-04-23 18:07:54,167 | test | ERROR | MainProcess | MainThread | [rest_client:_http_request:752] Socket error while connecting to http://172.23.123.102:8091/nodes/self error [Errno 111] Connection refused 2021-04-23 18:07:56,186 | test | ERROR | MainProcess | MainThread | [rest_client:_http_request:752] Socket error while connecting to http://172.23.123.102:8091/nodes/self error [Errno 111] Connection refused 2021-04-23 18:07:58,201 | test | ERROR | MainProcess | MainThread | [rest_client:_http_request:752] Socket error while connecting to http://172.23.123.102:8091/nodes/self error [Errno 111] Connection refused 2021-04-23 18:08:00,220 | test | ERROR | MainProcess | MainThread | [rest_client:_http_request:752] Socket error while connecting to http://172.23.123.102:8091/nodes/self error [Errno 111] Connection refused 2021-04-23 18:08:02,237 | test | ERROR | MainProcess | MainThread | [rest_client:_http_request:752] Socket error while connecting to http://172.23.123.102:8091/nodes/self error [Errno 111] Connection refused 2021-04-23 18:08:04,253 | test | ERROR | MainProcess | MainThread | [rest_client:_http_request:752] Socket error while connecting to http://172.23.123.102:8091/nodes/self error [Errno 111] Connection refused 2021-04-23 18:08:10,732 | test | ERROR | MainProcess | MainThread | [rest_client:_http_request:752] Socket error while connecting to http://172.23.123.101:8091/nodes/self error [Errno 111] Connection refused 2021-04-23 18:08:12,746 | test | ERROR | MainProcess | MainThread | [rest_client:_http_request:752] Socket error while connecting to http://172.23.123.101:8091/nodes/self error [Errno 111] Connection refused 2021-04-23 18:08:14,763 | test | ERROR | MainProcess | MainThread | [rest_client:_http_request:752] Socket error while connecting to http://172.23.123.101:8091/nodes/self error [Errno 111] Connection refused 2021-04-23 18:08:16,779 | test | ERROR | MainProcess | MainThread | [rest_client:_http_request:752] Socket error while connecting to http://172.23.123.101:8091/nodes/self error [Errno 111] Connection refused 2021-04-23 18:08:18,796 | test | ERROR | MainProcess | MainThread | [rest_client:_http_request:752] Socket error while connecting to http://172.23.123.101:8091/nodes/self error [Errno 111] Connection refused 2021-04-23 18:08:20,815 | test | ERROR | MainProcess | MainThread | [rest_client:_http_request:752] Socket error while connecting to http://172.23.123.101:8091/nodes/self error [Errno 111] Connection refused 2021-04-23 18:08:26,142 | test | ERROR | MainProcess | MainThread | [rest_client:_http_request:752] Socket error while connecting to http://172.23.123.105:8091/nodes/self error [Errno 111] Connection refused 2021-04-23 18:08:28,158 | test | ERROR | MainProcess | MainThread | [rest_client:_http_request:752] Socket error while connecting to http://172.23.123.105:8091/nodes/self error [Errno 111] Connection refused 2021-04-23 18:08:30,177 | test | ERROR | MainProcess | MainThread | [rest_client:_http_request:752] Socket error while connecting to http://172.23.123.105:8091/nodes/self error [Errno 111] Connection refused 2021-04-23 18:08:32,194 | test | ERROR | MainProcess | MainThread | [rest_client:_http_request:752] Socket error while connecting to http://172.23.123.105:8091/nodes/self error [Errno 111] Connection refused 2021-04-23 18:08:34,211 | test | ERROR | MainProcess | MainThread | [rest_client:_http_request:752] Socket error while connecting to http://172.23.123.105:8091/nodes/self error [Errno 111] Connection refused 2021-04-23 18:08:36,227 | test | ERROR | MainProcess | MainThread | [rest_client:_http_request:752] Socket error while connecting to http://172.23.123.105:8091/nodes/self error [Errno 111] Connection refused 2021-04-23 18:08:42,003 | test | ERROR | MainProcess | MainThread | [rest_client:_http_request:752] Socket error while connecting to http://172.23.123.100:8091/nodes/self error [Errno 111] Connection refused 2021-04-23 18:08:44,019 | test | ERROR | MainProcess | MainThread | [rest_client:_http_request:752] Socket error while connecting to http://172.23.123.100:8091/nodes/self error [Errno 111] Connection refused 2021-04-23 18:08:46,035 | test | ERROR | MainProcess | MainThread | [rest_client:_http_request:752] Socket error while connecting to http://172.23.123.100:8091/nodes/self error [Errno 111] Connection refused 2021-04-23 18:08:48,052 | test | ERROR | MainProcess | MainThread | [rest_client:_http_request:752] Socket error while connecting to http://172.23.123.100:8091/nodes/self error [Errno 111] Connection refused 2021-04-23 18:09:02,844 | test | WARNING | MainProcess | MainThread | [basetestcase:_initialize_nodes:593] RAM quota was defined less than 100 MB: 2021-04-23 18:09:14,835 | test | INFO | MainProcess | MainThread | [basetestcase:setUp:340] Couchbase Cluster: C1, Nodes: 172.23.123.102, 172.23.123.101, 172.23.123.105, 172.23.123.100 initialized 2021-04-23 18:09:14,857 | test | INFO | MainProcess | MainThread | [basetestcase:log_setup_status:542] ========= BaseTestCase setup finished for test #52 test_KV_collection_deletion_does_not_effect_dataset_on_other_collections ========= 2021-04-23 18:09:14,963 | test | INFO | MainProcess | MainThread | [cbas_base:setUp:34] Starting Test: test_KV_collection_deletion_does_not_effect_dataset_on_other_collections 2021-04-23 18:09:17,178 | test | INFO | MainProcess | MainThread | [cbas_base:setUp:148] Setting the min possible memory quota so that adding more nodes to the cluster wouldn't be a problem. 2021-04-23 18:09:17,196 | test | INFO | MainProcess | MainThread | [cbas_base:setUp:159] Setting 1024 memory quota for CBAS 2021-04-23 18:09:42,036 | test | INFO | MainProcess | MainThread | [rest_client:monitorRebalance:1446] Rebalance done. Taken 17.1719999313 seconds to complete 2021-04-23 18:09:42,046 | test | INFO | MainProcess | MainThread | [common_lib:sleep:22] Sleep 5 seconds. Reason: Wait after rebalance complete 2021-04-23 18:09:47,757 | test | INFO | MainProcess | MainThread | [cbas_base:cbas_logger:351] **********No buckets to disconnect********** 2021-04-23 18:09:47,838 | test | INFO | MainProcess | MainThread | [cbas_base:cbas_logger:351] **********No datasets to drop********** 2021-04-23 18:09:47,881 | test | INFO | MainProcess | MainThread | [cbas_base:cbas_logger:351] **********No buckets to drop********** 2021-04-23 18:09:47,881 | test | INFO | MainProcess | MainThread | [cbas_base:cleanup_cbas:399] Drop Dataverse other than Default and Metadata 2021-04-23 18:09:47,934 | test | INFO | MainProcess | MainThread | [cbas_base:cbas_logger:351] **********No dataverse to drop********** 2021-04-23 18:09:58,440 | test | INFO | MainProcess | pool-54-thread-5 | [table_view:display:72] Rebalance Overview +----------------+----------+-----------------------+---------------+--------------+ | Nodes | Services | Version | CPU | Status | +----------------+----------+-----------------------+---------------+--------------+ | 172.23.123.102 | kv, n1ql | 7.0.0-5017-enterprise | 0.60286360211 | Cluster node | | 172.23.123.105 | cbas | 7.0.0-5017-enterprise | 26.9317895797 | Cluster node | | 172.23.123.101 | ['kv'] | | | <--- IN --- | | 172.23.123.100 | ['cbas'] | | | <--- IN --- | +----------------+----------+-----------------------+---------------+--------------+ 2021-04-23 18:09:58,460 | test | INFO | MainProcess | pool-54-thread-5 | [task:check:355] Rebalance - status: running, progress: 0.0 2021-04-23 18:10:03,513 | test | INFO | MainProcess | pool-54-thread-5 | [task:check:355] Rebalance - status: running, progress: 37.5 2021-04-23 18:10:13,595 | test | INFO | MainProcess | pool-54-thread-5 | [task:check:355] Rebalance - status: none, progress: 100 2021-04-23 18:10:13,619 | test | INFO | MainProcess | pool-54-thread-5 | [task:check:414] Rebalance completed with progress: 100% in 15.1799998283 sec 2021-04-23 18:10:13,783 | test | INFO | MainProcess | MainThread | [bucket_ready_functions:create_buckets_using_json_data:1799] Creating required buckets from template 2021-04-23 18:10:14,002 | test | INFO | MainProcess | MainThread | [bucket_ready_functions:wait_for_collection_creation_to_complete:2260] Waiting for all collections to be created 2021-04-23 18:10:14,108 | test | INFO | MainProcess | MainThread | [table_view:display:72] Bucket statistics +---------+-----------+----------+------------+-----+-------+-----------+----------+-----------+ | Bucket | Type | Replicas | Durability | TTL | Items | RAM Quota | RAM Used | Disk Used | +---------+-----------+----------+------------+-----+-------+-----------+----------+-----------+ | default | couchbase | 1 | none | 0 | 0 | 536870912 | 0 | 0 | +---------+-----------+----------+------------+-----+-------+-----------+----------+-----------+ 2021-04-23 18:10:14,109 | test | INFO | MainProcess | MainThread | [cbas_base:collectionSetUp:699] Creating required SDK clients for client_pool 2021-04-23 18:10:20,891 | test | INFO | MainProcess | MainThread | [table_view:display:72] Cluster statistics +----------------+----------+-----------------+------------+------------+------------------------+-------------------+-----------------------+ | Node | Services | CPU_utilization | Mem_total | Mem_free | Swap_mem_used | Active / Replica | Version | +----------------+----------+-----------------+------------+------------+------------------------+-------------------+-----------------------+ | 172.23.123.101 | kv | 0.628614533568 | 4201672704 | 3613728768 | 82837504 / 3758092288 | 0 / 0 | 7.0.0-5017-enterprise | | 172.23.123.102 | kv, n1ql | 1.30718954248 | 4201672704 | 3351687168 | 0 / 3758092288 | 0 / 0 | 7.0.0-5017-enterprise | | 172.23.123.105 | cbas | 0.952619704187 | 4201672704 | 2982195200 | 180092928 / 3758092288 | 0 / 0 | 7.0.0-5017-enterprise | | 172.23.123.100 | cbas | 1.50715900528 | 4201672704 | 3195031552 | 0 / 3758092288 | 0 / 0 | 7.0.0-5017-enterprise | +----------------+----------+-----------------+------------+------------+------------------------+-------------------+-----------------------+ 2021-04-23 18:10:20,892 | test | INFO | MainProcess | MainThread | [bucket_ready_functions:perform_tasks_from_spec:4651] Performing scope/collection specific operations 2021-04-23 18:10:20,898 | test | INFO | MainProcess | MainThread | [bucket_ready_functions:perform_tasks_from_spec:4741] Done Performing scope/collection specific operations 2021-04-23 18:10:24,509 | test | INFO | MainProcess | MainThread | [bucket_ready_functions:validate_docs_per_collections_all_buckets:4405] Validating collection stats and item counts 2021-04-23 18:10:29,073 | test | ERROR | MainProcess | MainThread | [bucket_ready_functions:validate_manifest_uid:4287] Bucket UID mismatch. Expected: 0, Actual: 1 2021-04-23 18:10:29,073 | test | WARNING | MainProcess | MainThread | [bucket_ready_functions:validate_docs_per_collections_all_buckets:4422] Bucket manifest UID mismatch! 2021-04-23 18:10:30,470 | test | INFO | MainProcess | MainThread | [cbas_base:setUp:337] === CBAS_BASE setup was finished for test #52 test_KV_collection_deletion_does_not_effect_dataset_on_other_collections === 2021-04-23 18:10:30,470 | test | INFO | MainProcess | MainThread | [basetestcase:log_setup_status:542] ========= CBASDatasetsAndCollections setUp Finished for test #52 test_KV_collection_deletion_does_not_effect_dataset_on_other_collections ========= 2021-04-23 18:10:30,471 | test | INFO | MainProcess | MainThread | [cbas_collection_functional_test:test_KV_collection_deletion_does_not_effect_dataset_on_other_collections:1634] Test started 2021-04-23 18:10:30,471 | test | INFO | MainProcess | MainThread | [cbas_utils_v2:create_datasets_on_all_collections:2134] Creating Datasets on all KV collections 2021-04-23 18:10:44,622 | test | INFO | MainProcess | MainThread | [cbas_utils_v2:verify_index_used:2829] INDEX-SEARCH is found in EXPLAIN hence indexed data will be scanned to serve EXPLAIN SELECT VALUE v FROM Default.Uqoo v WHERE age > 2 2021-04-23 18:10:46,579 | test | INFO | MainProcess | MainThread | [cbas_utils_v2:verify_index_used:2829] INDEX-SEARCH is found in EXPLAIN hence indexed data will be scanned to serve EXPLAIN SELECT VALUE v FROM Default.Jn90fxoReYIq0JjSbDVF4HQ v WHERE age > 2 2021-04-23 18:10:47,453 | test | INFO | MainProcess | MainThread | [cbas_utils_v2:verify_index_used:2829] INDEX-SEARCH is found in EXPLAIN hence indexed data will be scanned to serve EXPLAIN SELECT VALUE v FROM Default.WPlbJWOAIBM7Tw0dcr9HvH v WHERE age > 2 2021-04-23 18:10:49,696 | test | INFO | MainProcess | MainThread | [cbas_utils_v2:verify_index_used:2829] INDEX-SEARCH is found in EXPLAIN hence indexed data will be scanned to serve EXPLAIN SELECT VALUE v FROM Default.i7JTpnRmmklyIZdn4H v WHERE age > 2 2021-04-23 18:10:53,765 | test | INFO | MainProcess | MainThread | [cbas_utils_v2:verify_index_used:2829] INDEX-SEARCH is found in EXPLAIN hence indexed data will be scanned to serve EXPLAIN SELECT VALUE v FROM Default.UQpiCELkPCsOM2UP3xK0cG v WHERE age > 2 2021-04-23 18:10:55,602 | test | INFO | MainProcess | MainThread | [cbas_utils_v2:verify_index_used:2829] INDEX-SEARCH is found in EXPLAIN hence indexed data will be scanned to serve EXPLAIN SELECT VALUE v FROM Default.k44dQe6pWs v WHERE age > 2 2021-04-23 18:10:57,960 | test | INFO | MainProcess | MainThread | [cbas_utils_v2:verify_index_used:2829] INDEX-SEARCH is found in EXPLAIN hence indexed data will be scanned to serve EXPLAIN SELECT VALUE v FROM Default.mZ03dlcw9iu5KuZp6yc v WHERE age > 2 2021-04-23 18:10:59,953 | test | INFO | MainProcess | MainThread | [cbas_utils_v2:verify_index_used:2829] INDEX-SEARCH is found in EXPLAIN hence indexed data will be scanned to serve EXPLAIN SELECT VALUE v FROM Default.QsyrVrOSVg7 v WHERE age > 2 2021-04-23 18:11:01,644 | test | INFO | MainProcess | MainThread | [cbas_utils_v2:verify_index_used:2829] INDEX-SEARCH is found in EXPLAIN hence indexed data will be scanned to serve EXPLAIN SELECT VALUE v FROM default._default._default v WHERE age > 2 2021-04-23 18:11:12,038 | test | INFO | MainProcess | MainThread | [cbas_utils_v2:verify_index_used:2829] INDEX-SEARCH is found in EXPLAIN hence indexed data will be scanned to serve EXPLAIN SELECT VALUE v FROM Default.Uqoo v WHERE age > 2 2021-04-23 18:11:22,240 | test | INFO | MainProcess | MainThread | [cbas_utils_v2:verify_index_used:2829] INDEX-SEARCH is found in EXPLAIN hence indexed data will be scanned to serve EXPLAIN SELECT VALUE v FROM Default.Jn90fxoReYIq0JjSbDVF4HQ v WHERE age > 2 2021-04-23 18:11:22,367 | test | INFO | MainProcess | MainThread | [cbas_utils_v2:verify_index_used:2829] INDEX-SEARCH is found in EXPLAIN hence indexed data will be scanned to serve EXPLAIN SELECT VALUE v FROM Default.WPlbJWOAIBM7Tw0dcr9HvH v WHERE age > 2 2021-04-23 18:11:22,483 | test | INFO | MainProcess | MainThread | [cbas_utils_v2:verify_index_used:2829] INDEX-SEARCH is found in EXPLAIN hence indexed data will be scanned to serve EXPLAIN SELECT VALUE v FROM Default.i7JTpnRmmklyIZdn4H v WHERE age > 2 2021-04-23 18:11:22,602 | test | INFO | MainProcess | MainThread | [cbas_utils_v2:verify_index_used:2829] INDEX-SEARCH is found in EXPLAIN hence indexed data will be scanned to serve EXPLAIN SELECT VALUE v FROM Default.UQpiCELkPCsOM2UP3xK0cG v WHERE age > 2 2021-04-23 18:11:22,713 | test | INFO | MainProcess | MainThread | [cbas_utils_v2:verify_index_used:2829] INDEX-SEARCH is found in EXPLAIN hence indexed data will be scanned to serve EXPLAIN SELECT VALUE v FROM Default.k44dQe6pWs v WHERE age > 2 2021-04-23 18:11:22,812 | test | INFO | MainProcess | MainThread | [cbas_utils_v2:verify_index_used:2829] INDEX-SEARCH is found in EXPLAIN hence indexed data will be scanned to serve EXPLAIN SELECT VALUE v FROM Default.mZ03dlcw9iu5KuZp6yc v WHERE age > 2 2021-04-23 18:11:23,023 | test | INFO | MainProcess | MainThread | [cbas_utils_v2:verify_index_used:2829] INDEX-SEARCH is found in EXPLAIN hence indexed data will be scanned to serve EXPLAIN SELECT VALUE v FROM default._default._default v WHERE age > 2 2021-04-23 18:11:23,023 | test | INFO | MainProcess | MainThread | [cbas_collection_functional_test:test_KV_collection_deletion_does_not_effect_dataset_on_other_collections:1690] Test finished 2021-04-23 18:11:23,025 | test | INFO | MainProcess | MainThread | [basetestcase:log_setup_status:542] ========= CBASDatasetsAndCollections tearDown Started for test #52 test_KV_collection_deletion_does_not_effect_dataset_on_other_collections ========= 2021-04-23 18:11:23,841 | test | CRITICAL | MainProcess | MainThread | [basetestcase:check_coredump_exist:727] 172.23.123.102: 1 core dump seen 2021-04-23 18:11:23,878 | test | INFO | MainProcess | MainThread | [basetestcase:get_gdb:670] running gdb --batch /opt/couchbase/bin/ memcached -c /opt/couchbase/var/lib/couchbase/crash/1cb75420-d6c5-41d7-7c37da8f-3071784e.core -ex "bt full" -ex quit 2021-04-23 18:11:23,960 | test | CRITICAL | MainProcess | MainThread | [basetestcase:check_coredump_exist:728] 172.23.123.102: Stack Trace of first crash - 1cb75420-d6c5-41d7-7c37da8f-3071784e.dmp Core was generated by `/opt/couchbase/bin/memcached -C /opt/couchbase/var/lib/couchbase/config/memcach'. #0 0x000000000049d1b4 in ?? () #0 0x000000000049d1b4 in ?? () No symbol table info available. #1 0x0000000000000040 in ?? () No symbol table info available. #2 0x00007faa651a04c0 in ?? () No symbol table info available. #3 0x0000000000000040 in ?? () No symbol table info available. #4 0x0000000000000040 in ?? () No symbol table info available. #5 0x0000000000000040 in ?? () No symbol table info available. #6 0x00007faac67e9a80 in ?? () No symbol table info available. #7 0x00007faabc0c3000 in ?? () No symbol table info available. #8 0x00007faabc0a2018 in ?? () No symbol table info available. #9 0x00007faa65c4b260 in ?? () No symbol table info available. #10 0x00007faa651a03c0 in ?? () No symbol table info available. #11 0x00007faac67e9950 in ?? () No symbol table info available. #12 0x000000000000000d in ?? () No symbol table info available. #13 0x00007faa651a0398 in ?? () No symbol table info available. #14 0x00007faa651a03c0 in ?? () No symbol table info available. #15 0x00007faac67e9928 in ?? () No symbol table info available. #16 0x000000000074be3c in ?? () No symbol table info available. #17 0x00007faabc0bc400 in ?? () No symbol table info available. #18 0x00007faa651a0480 in ?? () No symbol table info available. #19 0x00007faac67e9ac0 in ?? () No symbol table info available. #20 0x00007faac67e9930 in ?? () No symbol table info available. #21 0x00007faabc0b2fc8 in ?? () No symbol table info available. #22 0x000000000000000d in ?? () No symbol table info available. #23 0x00007faa664f6ed8 in ?? () No symbol table info available. #24 0x0000000000000001 in ?? () No symbol table info available. #25 0x00007faa664f6e40 in ?? () No symbol table info available. #26 0x00000000006c5bfb in ?? () No symbol table info available. #27 0x00007faa651a0480 in ?? () No symbol table info available. #28 0x00007faa651a04c0 in ?? () No symbol table info available. #29 0x00007faa651a04c0 in ?? () No symbol table info available. #30 0x03ff7faac67e9bb0 in ?? () No symbol table info available. #31 0x00007faaa46acf00 in ?? () No symbol table info available. #32 0x00007faabc0c2c00 in ?? () No symbol table info available. #33 0x00007faac67e99b0 in ?? () No symbol table info available. #34 0x00007faac67e9a80 in ?? () No symbol table info available. #35 0x00007faabc0c3000 in ?? () No symbol table info available. #36 0x00007faabc0a2018 in ?? () No symbol table info available. #37 0x00007faa65c4b260 in ?? () No symbol table info available. #38 0x00000000006aa03b in ?? () No symbol table info available. #39 0x00007faa65b61a00 in ?? () No symbol table info available. #40 0x00007faa885ae180 in ?? () No symbol table info available. #41 0x00007faabc0c2800 in ?? () No symbol table info available. #42 0x00007faabc0c3000 in ?? () No symbol table info available. #43 0x00007faabc0c3000 in ?? () No symbol table info available. #44 0x0000000000744098 in ?? () No symbol table info available. #45 0x00007faac67e9c48 in ?? () No symbol table info available. #46 0x00007faa651a03c0 in ?? () No symbol table info available. #47 0x00007faa651a0340 in ?? () No symbol table info available. #48 0x00007faabc0a2000 in ?? () No symbol table info available. #49 0x00007faa651a03c0 in ?? () No symbol table info available. #50 0x00007faac67e9bb0 in ?? () No symbol table info available. #51 0x00007faac67e9ac0 in ?? () No symbol table info available. #52 0x00000000007448f0 in ?? () No symbol table info available. #53 0x00007faac67e9c70 in ?? () No symbol table info available. #54 0x00007faac67e9af0 in ?? () No symbol table info available. #55 0x0000000100000008 in ?? () No symbol table info available. #56 0x00007faac67e9a80 in ?? () No symbol table info available. #57 0x00007faaa45c12b8 in ?? () No symbol table info available. #58 0x0000000100000008 in ?? () No symbol table info available. #59 0x0000000100000008 in ?? () No symbol table info available. #60 0x0000000100000008 in ?? () No symbol table info available. #61 0x0000000000000004 in ?? () No symbol table info available. #62 0x00007faad20c8ee2 in ?? () No symbol table info available. #63 0x00007faac67e9df0 in ?? () No symbol table info available. #64 0x000000000000000f in ?? () No symbol table info available. #65 0x0000000000f03fb8 in ?? () No symbol table info available. #66 0x0000000100000008 in ?? () No symbol table info available. #67 0x00007faa00000000 in ?? () No symbol table info available. #68 0x0000000000000000 in ?? () No symbol table info available. 2021-04-23 18:11:23,960 | test | INFO | MainProcess | MainThread | [basetestcase:get_full_thread_dump:679] running gdb -p `(pidof memcached)` -ex "thread apply all bt" -ex detach -ex quit 2021-04-23 18:11:38,503 | test | INFO | MainProcess | MainThread | [cluster_ready_functions:trigger_cb_collect_on_cluster:1085] Running cbcollect on node ns_1@172.23.123.100,ns_1@172.23.123.101,ns_1@172.23.123.102,ns_1@172.23.123.105 2021-04-23 18:11:48,539 | test | INFO | MainProcess | MainThread | [cluster_ready_functions:trigger_cb_collect_on_cluster:1088] ns_1@172.23.123.100,ns_1@172.23.123.101,ns_1@172.23.123.102,ns_1@172.23.123.105 - cbcollect status: True 2021-04-23 18:11:48,539 | test | INFO | MainProcess | MainThread | [cluster_ready_functions:wait_for_cb_collect_to_complete:1093] Polling active_tasks to check cbcollect status 2021-04-23 18:15:29,266 | test | INFO | MainProcess | MainThread | [cluster_ready_functions:copy_cb_collect_logs:1120] ns_1@172.23.123.100: Copying cbcollect ZIP file to Client 2021-04-23 18:15:30,934 | test | INFO | MainProcess | MainThread | [cluster_ready_functions:copy_cb_collect_logs:1120] ns_1@172.23.123.101: Copying cbcollect ZIP file to Client 2021-04-23 18:15:32,062 | test | INFO | MainProcess | MainThread | [cluster_ready_functions:copy_cb_collect_logs:1120] ns_1@172.23.123.102: Copying cbcollect ZIP file to Client 2021-04-23 18:15:33,089 | test | INFO | MainProcess | MainThread | [cluster_ready_functions:copy_cb_collect_logs:1120] ns_1@172.23.123.105: Copying cbcollect ZIP file to Client 2021-04-23 18:15:47,359 | test | INFO | MainProcess | MainThread | [rest_client:monitorRebalance:1446] Rebalance done. Taken 11.1289999485 seconds to complete 2021-04-23 18:15:47,372 | test | INFO | MainProcess | MainThread | [common_lib:sleep:22] Sleep 5 seconds. Reason: Wait after rebalance complete