Started by user Balakumaran G [EnvInject] - Loading node environment variables. Building remotely on slv-sc2402-32g-12c (volume_test_collections) in workspace /data/workspace/temp_rebalance_even_win2 [WS-CLEANUP] Deleting project workspace... Cloning the remote Git repository Cloning repository https://github.com/couchbaselabs/TAF/ > /usr/bin/git init /data/workspace/temp_rebalance_even_win2 # timeout=10 Fetching upstream changes from https://github.com/couchbaselabs/TAF/ > /usr/bin/git --version # timeout=10 > /usr/bin/git fetch --tags --progress https://github.com/couchbaselabs/TAF/ +refs/heads/*:refs/remotes/origin/* > /usr/bin/git config remote.origin.url https://github.com/couchbaselabs/TAF/ # timeout=10 > /usr/bin/git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10 > /usr/bin/git config remote.origin.url https://github.com/couchbaselabs/TAF/ # timeout=10 Fetching upstream changes from https://github.com/couchbaselabs/TAF/ > /usr/bin/git fetch --tags --progress https://github.com/couchbaselabs/TAF/ +refs/heads/*:refs/remotes/origin/* > /usr/bin/git rev-parse refs/remotes/origin/master^{commit} # timeout=10 > /usr/bin/git rev-parse refs/remotes/origin/origin/master^{commit} # timeout=10 Checking out Revision 2ed6005ebd7c801945999bdc8462961a031622d5 (refs/remotes/origin/master) > /usr/bin/git config core.sparsecheckout # timeout=10 > /usr/bin/git checkout -f 2ed6005ebd7c801945999bdc8462961a031622d5 > /usr/bin/git rev-list 2ed6005ebd7c801945999bdc8462961a031622d5 # timeout=10 [temp_rebalance_even_win2] $ /bin/sh -xe /tmp/jenkins3637468833464877691.sh + echo '[global] username:Administrator password:Membase123 port:8091 index_port:9102 n1ql_port:8093 [membase] rest_username:Administrator rest_password:password [servers] 1:_1 2:_2 3:_3 4:_4 #5:_5 #6:_6 [_1] ip:172.23.136.125 services:kv [_2] ip:172.23.136.124 services:kv [_3] ip:172.23.136.109 services:kv [_6] ip:172.23.136.110 services:kv [_4] ip:172.23.136.101 services:kv [_5] ip:172.23.136.105 services:kv' [temp_rebalance_even_win2] $ /bin/sh -xe /tmp/jenkins6321023991550947286.sh + git checkout master Switched to a new branch 'master' Branch master set up to track remote branch master from origin. + git pull origin master From https://github.com/couchbaselabs/TAF * branch master -> FETCH_HEAD Already up-to-date. + git clone https://github.com/sumedhpb/guides.git Cloning into 'guides'... + jython_path=/opt/jython/bin/jython + guides/gradlew check -s Starting a Gradle Daemon, 14 busy Daemons could not be reused, use --status for details > Configure project : Using Transaction_client :: 1.1.0 > Task :compileJava Note: /data/workspace/temp_rebalance_even_win2/src/main/java/com/couchbase/test/transactions/SimpleTransaction.java uses or overrides a deprecated API. Note: Recompile with -Xlint:deprecation for details. > Task :processResources > Task :classes > Task :compileTestJava NO-SOURCE > Task :processTestResources NO-SOURCE > Task :testClasses UP-TO-DATE > Task :test NO-SOURCE > Task :check UP-TO-DATE Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0. Use '--warning-mode all' to show the individual deprecation warnings. See https://docs.gradle.org/6.2.2/userguide/command_line_interface.html#sec:command_line_warnings BUILD SUCCESSFUL in 5s 2 actionable tasks: 2 executed + guides/gradlew -v ------------------------------------------------------------ Gradle 6.2.2 ------------------------------------------------------------ Build time: 2020-03-04 08:49:31 UTC Revision: 7d0bf6dcb46c143bcc3b7a0fa40a8e5ca28e5856 Kotlin: 1.3.61 Groovy: 2.5.8 Ant: Apache Ant(TM) version 1.10.7 compiled on September 1 2019 JVM: 1.8.0_242 (Oracle Corporation 25.242-b08) OS: Linux 3.10.0-1062.18.1.el7.x86_64 amd64 + guides/gradlew --refresh-dependencies testrunner -P jython=/opt/jython/bin/jython -P 'args=-i /tmp/win10-bucket-ops-11111222223334444.ini -p rerun=False,get-cbcollect-info=True,crash_warning=True -t bucket_collections.collections_rebalance.CollectionsRebalance.test_data_load_collections_with_graceful_failover_rebalance_out,nodes_init=4,nodes_failover=2,override_spec_params=durability;replicas,durability=MAJORITY,replicas=2,bucket_spec=multi_bucket.buckets_for_rebalance_tests,data_load_stage=during,skip_validations=False -m rest' > Configure project : Using Transaction_client :: 1.1.0 Running: /opt/jython/bin/jython -J-cp /data/workspace/temp_rebalance_even_win2/build/classes/java/main:/data/workspace/temp_rebalance_even_win2/build/resources/main:/root/.gradle/caches/modules-2/files-2.1/com.jcraft/jsch/0.1.54/da3584329a263616e277e15462b387addd1b208d/jsch-0.1.54.jar:/root/.gradle/caches/modules-2/files-2.1/com.couchbase.client/couchbase-transactions/1.1.0/a295271b66684b05dd5345d7b7a6232e03054ef9/couchbase-transactions-1.1.0.jar:/root/.gradle/caches/modules-2/files-2.1/com.couchbase.client/java-client/3.0.7/892363ab817451b9bb8fa91861c0d66c480056eb/java-client-3.0.7.jar:/root/.gradle/caches/modules-2/files-2.1/com.couchbase.client/core-io/2.0.8/d21bea503f8be1ce9d68143e4aaf2f203a48d2b7/core-io-2.0.8.jar:/root/.gradle/caches/modules-2/files-2.1/io.projectreactor/reactor-core/3.3.8.RELEASE/e6b2e8dc1f6548ae216b2ecdee0c0a32176317a/reactor-core-3.3.8.RELEASE.jar:/root/.gradle/caches/modules-2/files-2.1/org.reactivestreams/reactive-streams/1.0.3/d9fb7a7926ffa635b3dcaa5049fb2bfa25b3e7d0/reactive-streams-1.0.3.jar scripts/ssh.py -i /tmp/win10-bucket-ops-11111222223334444.ini -p rerun=False,get-cbcollect-info=True,crash_warning=True -t bucket_collections.collections_rebalance.CollectionsRebalance.test_data_load_collections_with_graceful_failover_rebalance_out,nodes_init=4,nodes_failover=2,override_spec_params=durability;replicas,durability=MAJORITY,replicas=2,bucket_spec=multi_bucket.buckets_for_rebalance_tests,data_load_stage=during,skip_validations=False -m rest Running: /opt/jython/bin/jython -J-cp /data/workspace/temp_rebalance_even_win2/build/classes/java/main:/data/workspace/temp_rebalance_even_win2/build/resources/main:/root/.gradle/caches/modules-2/files-2.1/com.jcraft/jsch/0.1.54/da3584329a263616e277e15462b387addd1b208d/jsch-0.1.54.jar:/root/.gradle/caches/modules-2/files-2.1/com.couchbase.client/couchbase-transactions/1.1.0/a295271b66684b05dd5345d7b7a6232e03054ef9/couchbase-transactions-1.1.0.jar:/root/.gradle/caches/modules-2/files-2.1/com.couchbase.client/java-client/3.0.7/892363ab817451b9bb8fa91861c0d66c480056eb/java-client-3.0.7.jar:/root/.gradle/caches/modules-2/files-2.1/com.couchbase.client/core-io/2.0.8/d21bea503f8be1ce9d68143e4aaf2f203a48d2b7/core-io-2.0.8.jar:/root/.gradle/caches/modules-2/files-2.1/io.projectreactor/reactor-core/3.3.8.RELEASE/e6b2e8dc1f6548ae216b2ecdee0c0a32176317a/reactor-core-3.3.8.RELEASE.jar:/root/.gradle/caches/modules-2/files-2.1/org.reactivestreams/reactive-streams/1.0.3/d9fb7a7926ffa635b3dcaa5049fb2bfa25b3e7d0/reactive-streams-1.0.3.jar scripts/eagles_all_around.py -i /tmp/win10-bucket-ops-11111222223334444.ini -p rerun=False,get-cbcollect-info=True,crash_warning=True -t bucket_collections.collections_rebalance.CollectionsRebalance.test_data_load_collections_with_graceful_failover_rebalance_out,nodes_init=4,nodes_failover=2,override_spec_params=durability;replicas,durability=MAJORITY,replicas=2,bucket_spec=multi_bucket.buckets_for_rebalance_tests,data_load_stage=during,skip_validations=False -m rest Running: /opt/jython/bin/jython -J-cp /data/workspace/temp_rebalance_even_win2/build/classes/java/main:/data/workspace/temp_rebalance_even_win2/build/resources/main:/root/.gradle/caches/modules-2/files-2.1/com.jcraft/jsch/0.1.54/da3584329a263616e277e15462b387addd1b208d/jsch-0.1.54.jar:/root/.gradle/caches/modules-2/files-2.1/com.couchbase.client/couchbase-transactions/1.1.0/a295271b66684b05dd5345d7b7a6232e03054ef9/couchbase-transactions-1.1.0.jar:/root/.gradle/caches/modules-2/files-2.1/com.couchbase.client/java-client/3.0.7/892363ab817451b9bb8fa91861c0d66c480056eb/java-client-3.0.7.jar:/root/.gradle/caches/modules-2/files-2.1/com.couchbase.client/core-io/2.0.8/d21bea503f8be1ce9d68143e4aaf2f203a48d2b7/core-io-2.0.8.jar:/root/.gradle/caches/modules-2/files-2.1/io.projectreactor/reactor-core/3.3.8.RELEASE/e6b2e8dc1f6548ae216b2ecdee0c0a32176317a/reactor-core-3.3.8.RELEASE.jar:/root/.gradle/caches/modules-2/files-2.1/org.reactivestreams/reactive-streams/1.0.3/d9fb7a7926ffa635b3dcaa5049fb2bfa25b3e7d0/reactive-streams-1.0.3.jar:build/classes/java/main scripts/install.py -i /tmp/win10-bucket-ops-11111222223334444.ini -p rerun=False,get-cbcollect-info=True,crash_warning=True -t bucket_collections.collections_rebalance.CollectionsRebalance.test_data_load_collections_with_graceful_failover_rebalance_out,nodes_init=4,nodes_failover=2,override_spec_params=durability;replicas,durability=MAJORITY,replicas=2,bucket_spec=multi_bucket.buckets_for_rebalance_tests,data_load_stage=during,skip_validations=False -m rest Running: /opt/jython/bin/jython -J-cp /data/workspace/temp_rebalance_even_win2/build/classes/java/main:/data/workspace/temp_rebalance_even_win2/build/resources/main:/root/.gradle/caches/modules-2/files-2.1/com.jcraft/jsch/0.1.54/da3584329a263616e277e15462b387addd1b208d/jsch-0.1.54.jar:/root/.gradle/caches/modules-2/files-2.1/com.couchbase.client/couchbase-transactions/1.1.0/a295271b66684b05dd5345d7b7a6232e03054ef9/couchbase-transactions-1.1.0.jar:/root/.gradle/caches/modules-2/files-2.1/com.couchbase.client/java-client/3.0.7/892363ab817451b9bb8fa91861c0d66c480056eb/java-client-3.0.7.jar:/root/.gradle/caches/modules-2/files-2.1/com.couchbase.client/core-io/2.0.8/d21bea503f8be1ce9d68143e4aaf2f203a48d2b7/core-io-2.0.8.jar:/root/.gradle/caches/modules-2/files-2.1/io.projectreactor/reactor-core/3.3.8.RELEASE/e6b2e8dc1f6548ae216b2ecdee0c0a32176317a/reactor-core-3.3.8.RELEASE.jar:/root/.gradle/caches/modules-2/files-2.1/org.reactivestreams/reactive-streams/1.0.3/d9fb7a7926ffa635b3dcaa5049fb2bfa25b3e7d0/reactive-streams-1.0.3.jar:build/classes/java/main:src/main/resources testrunner.py -i /tmp/win10-bucket-ops-11111222223334444.ini -p rerun=False,get-cbcollect-info=True,crash_warning=True -t bucket_collections.collections_rebalance.CollectionsRebalance.test_data_load_collections_with_graceful_failover_rebalance_out,nodes_init=4,nodes_failover=2,override_spec_params=durability;replicas,durability=MAJORITY,replicas=2,bucket_spec=multi_bucket.buckets_for_rebalance_tests,data_load_stage=during,skip_validations=False -m rest Running: /opt/jython/bin/jython -J-cp /data/workspace/temp_rebalance_even_win2/build/classes/java/main:/data/workspace/temp_rebalance_even_win2/build/resources/main:/root/.gradle/caches/modules-2/files-2.1/com.jcraft/jsch/0.1.54/da3584329a263616e277e15462b387addd1b208d/jsch-0.1.54.jar:/root/.gradle/caches/modules-2/files-2.1/com.couchbase.client/couchbase-transactions/1.1.0/a295271b66684b05dd5345d7b7a6232e03054ef9/couchbase-transactions-1.1.0.jar:/root/.gradle/caches/modules-2/files-2.1/com.couchbase.client/java-client/3.0.7/892363ab817451b9bb8fa91861c0d66c480056eb/java-client-3.0.7.jar:/root/.gradle/caches/modules-2/files-2.1/com.couchbase.client/core-io/2.0.8/d21bea503f8be1ce9d68143e4aaf2f203a48d2b7/core-io-2.0.8.jar:/root/.gradle/caches/modules-2/files-2.1/io.projectreactor/reactor-core/3.3.8.RELEASE/e6b2e8dc1f6548ae216b2ecdee0c0a32176317a/reactor-core-3.3.8.RELEASE.jar:/root/.gradle/caches/modules-2/files-2.1/org.reactivestreams/reactive-streams/1.0.3/d9fb7a7926ffa635b3dcaa5049fb2bfa25b3e7d0/reactive-streams-1.0.3.jar:build/classes/java/main:src/main/resources scripts/rerun_jobs.py -i /tmp/win10-bucket-ops-11111222223334444.ini -p rerun=False,get-cbcollect-info=True,crash_warning=True -t bucket_collections.collections_rebalance.CollectionsRebalance.test_data_load_collections_with_graceful_failover_rebalance_out,nodes_init=4,nodes_failover=2,override_spec_params=durability;replicas,durability=MAJORITY,replicas=2,bucket_spec=multi_bucket.buckets_for_rebalance_tests,data_load_stage=during,skip_validations=False -m rest > Task :compileJava UP-TO-DATE > Task :testrunner Global Test input params: {'cluster_name': 'win10-bucket-ops-11111222223334444', 'crash_warning': 'True', 'get-cbcollect-info': 'True', 'ini': '/tmp/win10-bucket-ops-11111222223334444.ini', 'num_nodes': 4, 'rerun': 'False'} Logs will be stored at /data/workspace/temp_rebalance_even_win2/logs/testrunner-21-Mar-18_03-43-40/test_1 guides/gradlew --refresh-dependencies testrunner -P jython=/opt/jython/bin/jython -P 'args=-i /tmp/win10-bucket-ops-11111222223334444.ini rerun=False,get-cbcollect-info=True,crash_warning=True -t bucket_collections.collections_rebalance.CollectionsRebalance.test_data_load_collections_with_graceful_failover_rebalance_out,nodes_init=4,nodes_failover=2,override_spec_params=durability;replicas,durability=MAJORITY,replicas=2,bucket_spec=multi_bucket.buckets_for_rebalance_tests,data_load_stage=during,skip_validations=False' Test Input params: {'data_load_stage': 'during', 'cluster_name': 'win10-bucket-ops-11111222223334444', 'nodes_failover': '2', 'ini': '/tmp/win10-bucket-ops-11111222223334444.ini', 'get-cbcollect-info': 'True', 'replicas': '2', 'durability': 'MAJORITY', 'crash_warning': 'True', 'skip_validations': 'False', 'rerun': 'False', 'num_nodes': 4, 'logs_folder': '/data/workspace/temp_rebalance_even_win2/logs/testrunner-21-Mar-18_03-43-40/test_1', 'nodes_init': '4', 'bucket_spec': 'multi_bucket.buckets_for_rebalance_tests', 'case_number': 1, 'override_spec_params': 'durability;replicas'} test_data_load_collections_with_graceful_failover_rebalance_out (bucket_collections.collections_rebalance.CollectionsRebalance) ... 2021-03-18 03:43:52,365 | test | INFO | MainThread | [basetestcase:log_setup_status:540] ========= CollectionsRebalance setup started for test #1 test_data_load_collections_with_graceful_failover_rebalance_out ========= 2021-03-18 03:43:54,217 | test | INFO | MainThread | [basetestcase:log_setup_status:540] ========= BaseTestCase setup started for test #1 test_data_load_collections_with_graceful_failover_rebalance_out ========= 2021-03-18 03:44:27,417 | test | INFO | MainThread | [rest_client:monitorRebalance:1461] Rebalance done. Taken 11.5320000648 seconds to complete 2021-03-18 03:44:27,434 | test | INFO | MainThread | [common_lib:sleep:22] Sleep 5 seconds. Reason: Wait after rebalance complete 2021-03-18 03:44:32,778 | test | INFO | MainThread | [basetestcase:setUp:318] Initializing cluster 2021-03-18 03:47:50,112 | test | WARNING | MainThread | [basetestcase:_initialize_nodes:587] RAM quota was defined less than 100 MB: 2021-03-18 03:48:15,509 | test | INFO | MainThread | [basetestcase:setUp:340] Couchbase Cluster: C1, Nodes: 172.23.136.125, 172.23.136.124, 172.23.136.109, 172.23.136.101 initialized 2021-03-18 03:48:15,523 | test | INFO | MainThread | [basetestcase:log_setup_status:540] ========= BaseTestCase setup finished for test #1 test_data_load_collections_with_graceful_failover_rebalance_out ========= 2021-03-18 03:48:15,543 | test | INFO | MainThread | [basetestcase:log_setup_status:540] ========= CollectionBase setup started for test #1 test_data_load_collections_with_graceful_failover_rebalance_out ========= 2021-03-18 03:48:29,476 | test | INFO | pool-1-thread-5 | [table_view:display:72] Rebalance Overview +----------------+----------+-----------------------+---------------+--------------+ | Nodes | Services | Version | CPU | Status | +----------------+----------+-----------------------+---------------+--------------+ | 172.23.136.125 | kv | 7.0.0-4723-enterprise | 11.7996066798 | Cluster node | | 172.23.136.124 | None | | | <--- IN --- | | 172.23.136.109 | None | | | <--- IN --- | | 172.23.136.101 | None | | | <--- IN --- | +----------------+----------+-----------------------+---------------+--------------+ 2021-03-18 03:48:29,499 | test | INFO | pool-1-thread-5 | [task:check:333] Rebalance - status: running, progress: 0.0 2021-03-18 03:48:39,545 | test | INFO | pool-1-thread-5 | [task:check:333] Rebalance - status: none, progress: 100 2021-03-18 03:48:39,559 | test | INFO | pool-1-thread-5 | [task:check:392] Rebalance completed with progress: 100% in 10.0820000172 sec 2021-03-18 03:48:39,782 | test | INFO | MainThread | [bucket_ready_functions:create_buckets_using_json_data:1792] Creating required buckets from template 2021-03-18 03:48:46,691 | test | INFO | MainThread | [bucket_ready_functions:wait_for_collection_creation_to_complete:2254] Waiting for all collections to be created 2021-03-18 03:48:46,838 | test | INFO | MainThread | [table_view:display:72] Bucket statistics +---------+-----------+----------+------------+-----+-------+-----------+----------+-----------+ | Bucket | Type | Replicas | Durability | TTL | Items | RAM Quota | RAM Used | Disk Used | +---------+-----------+----------+------------+-----+-------+-----------+----------+-----------+ | bucket1 | couchbase | 2 | none | 0 | 0 | 419430400 | 0 | 0 | | bucket2 | ephemeral | 2 | none | 0 | 0 | 419430400 | 0 | 136 | | default | couchbase | 2 | none | 0 | 0 | 0 | 0 | 0 | +---------+-----------+----------+------------+-----+-------+-----------+----------+-----------+ 2021-03-18 03:48:46,841 | test | INFO | MainThread | [collections_base:collection_setup:144] Creating required SDK clients for client_pool 2021-03-18 03:49:06,515 | test | INFO | MainThread | [bucket_ready_functions:perform_tasks_from_spec:4647] Performing scope/collection specific operations 2021-03-18 03:49:06,530 | test | INFO | MainThread | [bucket_ready_functions:perform_tasks_from_spec:4737] Done Performing scope/collection specific operations 2021-03-18 03:49:55,790 | test | INFO | pool-1-thread-10 | [table_view:display:72] Ops trend for bucket 'default' +----------+-----------------------------------------------------------------------------------------------------------------+-----------+ | Min | Trend | Max | +----------+-----------------------------------------------------------------------------------------------------------------+-----------+ | 0.000 | ***..............................................................................****...........*...........**X | 11166.559 | | 9743.015 | ......................................................................................****..**X | 10026.383 | +----------+-----------------------------------------------------------------------------------------------------------------+-----------+ 2021-03-18 03:49:57,269 | test | INFO | pool-1-thread-11 | [table_view:display:72] Ops trend for bucket 'bucket1' +---------+-------------------------------------------------------------------------------------------------------------+----------+ | Min | Trend | Max | +---------+-------------------------------------------------------------------------------------------------------------+----------+ | 0.000 | ***..........................................................................*..........................**X | 1497.798 | | 0.000 | ****..................................................................***X | 998.755 | | 624.689 | ........................................X | 624.689 | | 499.476 | ................................***X | 499.476 | +---------+-------------------------------------------------------------------------------------------------------------+----------+ 2021-03-18 03:49:58,707 | test | INFO | pool-1-thread-12 | [table_view:display:72] Ops trend for bucket 'bucket2' +---------+--------------------------------------------------------------------------------------------------------------+---------+ | Min | Trend | Max | +---------+--------------------------------------------------------------------------------------------------------------+---------+ | 0.000 | ****........................................................******X | 499.750 | | 499.378 | .......................................................****............****................................X | 890.192 | +---------+--------------------------------------------------------------------------------------------------------------+---------+ 2021-03-18 03:49:58,772 | test | INFO | MainThread | [table_view:display:72] Cluster statistics +----------------+----------+-----------------+------------+------------+--------------------------+-------------------+-----------------------+ | Node | Services | CPU_utilization | Mem_total | Mem_free | Swap_mem_used | Active / Replica | Version | +----------------+----------+-----------------+------------+------------+--------------------------+-------------------+-----------------------+ | 172.23.136.124 | kv | 96.8515991613 | 6437847040 | 4013346816 | 3093405696 / 8265605120 | 112029 / 225066 | 7.0.0-4723-enterprise | | 172.23.136.101 | kv | 100 | 6437847040 | 3987247104 | 2851028992 / 7511588864 | 140012 / 275444 | 7.0.0-4723-enterprise | | 172.23.136.109 | kv | 99.9734161862 | 6437847040 | 3454373888 | 3291090944 / 7511588864 | 139888 / 280896 | 7.0.0-4723-enterprise | | 172.23.136.125 | kv | 97.0970256955 | 6437847040 | 3760988160 | 3305480192 / 10006401024 | 112354 / 224737 | 7.0.0-4723-enterprise | +----------------+----------+-----------------+------------+------------+--------------------------+-------------------+-----------------------+ 2021-03-18 03:50:23,553 | test | INFO | MainThread | [bucket_ready_functions:validate_docs_per_collections_all_buckets:4404] Validating collection stats and item counts 2021-03-18 03:52:10,414 | test | INFO | MainThread | [table_view:display:72] Bucket statistics +---------+-----------+----------+------------+-----+--------+------------+-----------+-----------+ | Bucket | Type | Replicas | Durability | TTL | Items | RAM Quota | RAM Used | Disk Used | +---------+-----------+----------+------------+-----+--------+------------+-----------+-----------+ | bucket1 | couchbase | 2 | none | 0 | 30000 | 419430400 | 134294704 | 119502759 | | bucket2 | ephemeral | 2 | none | 0 | 30000 | 419430400 | 111509024 | 136 | | default | couchbase | 2 | none | 0 | 500000 | 6291456000 | 571468688 | 424616750 | +---------+-----------+----------+------------+-----+--------+------------+-----------+-----------+ 2021-03-18 03:52:10,415 | test | INFO | MainThread | [basetestcase:log_setup_status:540] ========= CollectionBase setup complete for test #1 test_data_load_collections_with_graceful_failover_rebalance_out ========= 2021-03-18 03:52:26,095 | test | INFO | MainThread | [collections_rebalance:load_collections_with_rebalance:822] Doing collection data load during graceful_failover_rebalance_out 2021-03-18 03:52:26,095 | test | INFO | MainThread | [collections_rebalance:rebalance_operation:303] Starting rebalance operation of type : graceful_failover_rebalance_out 2021-03-18 03:52:26,098 | test | INFO | MainThread | [collections_rebalance:rebalance_operation:500] failing over nodes [ip:172.23.136.109 port:8091 ssh_username:Administrator, ip:172.23.136.101 port:8091 ssh_username:Administrator] 2021-03-18 03:54:59,507 | test | ERROR | pool-1-thread-14 | [rest_client:_http_request:752] Socket error while connecting to http://172.23.136.125:8091/pools/default/tasks error timed out ERROR 2021-03-18 03:55:01,549 | test | INFO | MainThread | [collections_base:tearDown:91] Bucket: bucket2, Active Resident ratio(DGM): 100% 2021-03-18 03:55:01,551 | test | INFO | MainThread | [collections_base:tearDown:94] Bucket: bucket2, Replica Resident ratio(DGM): 100% Build was aborted Aborted by Balakumaran G Archiving artifacts Recording test results ERROR: Step ‘Publish JUnit test result report’ failed: None of the test reports contained any result [description-setter] Description set: [BFA] Scanning build for known causes... [BFA] No failure causes found [BFA] Done. 0s Notifying upstream projects of job completion Finished: ABORTED