Started by user Balakumaran G [EnvInject] - Loading node environment variables. Building remotely on slv-s82202 (aws) in workspace /data/workspace/temp_rebalance_even [WS-CLEANUP] Deleting project workspace... [WS-CLEANUP] Done Cloning the remote Git repository Cloning repository git://github.com/bkumaran/TAF/ > /usr/bin/git init /data/workspace/temp_rebalance_even # timeout=10 Fetching upstream changes from git://github.com/bkumaran/TAF/ > /usr/bin/git --version # timeout=10 > /usr/bin/git fetch --tags --progress git://github.com/bkumaran/TAF/ +refs/heads/*:refs/remotes/origin/* > /usr/bin/git config remote.origin.url git://github.com/bkumaran/TAF/ # timeout=10 > /usr/bin/git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10 > /usr/bin/git config remote.origin.url git://github.com/bkumaran/TAF/ # timeout=10 Fetching upstream changes from git://github.com/bkumaran/TAF/ > /usr/bin/git fetch --tags --progress git://github.com/bkumaran/TAF/ +refs/heads/*:refs/remotes/origin/* > /usr/bin/git rev-parse refs/remotes/origin/master^{commit} # timeout=10 > /usr/bin/git rev-parse refs/remotes/origin/origin/master^{commit} # timeout=10 Checking out Revision d6f45f044d6ad45f54300043f27fd3f6f5f55956 (refs/remotes/origin/master) > /usr/bin/git config core.sparsecheckout # timeout=10 > /usr/bin/git checkout -f d6f45f044d6ad45f54300043f27fd3f6f5f55956 > /usr/bin/git rev-list d6f45f044d6ad45f54300043f27fd3f6f5f55956 # timeout=10 [temp_rebalance_even] $ /bin/sh -xe /tmp/jenkins9071784382280544409.sh + echo '[global] username:root password:couchbase port:8091 n1ql_port:8093 index_port:9102 #data_path:/data/dta #index_path:/data/idx #cbas_path:['\''/data/d1'\'','\''/data/d2'\'','\''/data/d3'\'','\''/data/d4'\'','\''/data/d5'\'','\''/data/d6'\''] [membase] rest_username:Administrator rest_password:password [cluster1] 1:_1 2:_2 [cluster2] 1:_3 2:_4 [servers] 1:_1 2:_2 3:_3 4:_4 5:_5 6:_6 [_1] ip:172.23.98.196 port:8091 services:kv,index,n1ql [_2] ip:172.23.98.195 port:8091 services:kv,n1ql [_3] ip:172.23.121.10 port:8091 services:kv [_4] ip:172.23.104.186 port:8091 services:cbas [_5] ip:172.23.120.201 [_6] ip:172.23.120.206' + echo '[global] username:root password:couchbase port:8091 n1ql_port:8093 index_port:9102 [membase] rest_username:Administrator rest_password:password [servers] 1:_1 2:_2 3:_3 4:_4 5:_5 6:_6 [_1] ip:172.23.98.196 [_2] ip:172.23.98.195 [_3] ip:172.23.121.10 [_4] ip:172.23.104.186 [_5] ip:172.23.120.201 [_6] ip:172.23.120.206' + git checkout master Switched to a new branch 'master' Branch master set up to track remote branch master from origin. + git pull origin master From git://github.com/bkumaran/TAF * branch master -> FETCH_HEAD Already up-to-date. + git clone https://github.com/sumedhpb/guides.git Cloning into 'guides'... + jython_path=/opt/jython/bin/jython + guides/gradlew check -s Starting a Gradle Daemon, 6 busy Daemons could not be reused, use --status for details > Task :compileJava Note: /data/workspace/temp_rebalance_even/src/main/java/com/couchbase/test/transactions/SimpleTransaction.java uses or overrides a deprecated API. Note: Recompile with -Xlint:deprecation for details. > Task :processResources > Task :classes > Task :compileTestJava NO-SOURCE > Task :processTestResources NO-SOURCE > Task :testClasses UP-TO-DATE > Task :test NO-SOURCE > Task :check UP-TO-DATE Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0. Use '--warning-mode all' to show the individual deprecation warnings. See https://docs.gradle.org/6.2.2/userguide/command_line_interface.html#sec:command_line_warnings BUILD SUCCESSFUL in 6s 2 actionable tasks: 2 executed + guides/gradlew -v ------------------------------------------------------------ Gradle 6.2.2 ------------------------------------------------------------ Build time: 2020-03-04 08:49:31 UTC Revision: 7d0bf6dcb46c143bcc3b7a0fa40a8e5ca28e5856 Kotlin: 1.3.61 Groovy: 2.5.8 Ant: Apache Ant(TM) version 1.10.7 compiled on September 1 2019 JVM: 1.8.0_252 (Oracle Corporation 25.252-b09) OS: Linux 3.10.0-957.21.3.el7.x86_64 amd64 + guides/gradlew install -P jython=/opt/jython/bin/jython -P 'args=-i /tmp/win10-bucket-ops.ini -p version=7.0.0-2477,product=cb,parallel=,init_nodes=True,url=' > Configure project : Running: /opt/jython/bin/jython -J-cp /data/workspace/temp_rebalance_even/build/classes/java/main:/data/workspace/temp_rebalance_even/build/resources/main:/root/.gradle/caches/modules-2/files-2.1/com.couchbase.client/couchbase-transactions/1.0.1-SNAPSHOT/c6ff42a1c4529a1d3645ef0a70df43588ac99b15/couchbase-transactions-1.0.1-SNAPSHOT.jar:/root/.gradle/caches/modules-2/files-2.1/com.jcraft/jsch/0.1.54/da3584329a263616e277e15462b387addd1b208d/jsch-0.1.54.jar:/root/.gradle/caches/modules-2/files-2.1/org.ini4j/ini4j/0.5.1/8e737d82ecac9ba6100a9bba71e92a381b75efdc/ini4j-0.5.1.jar:/root/.gradle/caches/modules-2/files-2.1/org.apache.logging.log4j/log4j-slf4j-impl/2.11.1/4b41b53a3a2d299ce381a69d165381ca19f62912/log4j-slf4j-impl-2.11.1.jar:/root/.gradle/caches/modules-2/files-2.1/com.google.guava/guava/22.0/3564ef3803de51fb0530a8377ec6100b33b0d073/guava-22.0.jar:/root/.gradle/caches/modules-2/files-2.1/commons-cli/commons-cli/1.4/c51c00206bb913cd8612b24abd9fa98ae89719b1/commons-cli-1.4.jar:/root/.gradle/caches/modules-2/files-2.1/net.sourceforge.argparse4j/argparse4j/0.8.1/2c8241f84acf6c924bd75be0dbd68e8d74fbcd70/argparse4j-0.8.1.jar:/root/.gradle/caches/modules-2/files-2.1/com.couchbase.client/java-client/3.0.5/50e852b87fb272cdfcaf77097bef8f253a7a863c/java-client-3.0.5.jar:/root/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-api/1.7.25/da76ca59f6a57ee3102f8f9bd9cee742973efa8a/slf4j-api-1.7.25.jar:/root/.gradle/caches/modules-2/files-2.1/org.apache.logging.log4j/log4j-core/2.11.1/592a48674c926b01a9a747c7831bcd82a9e6d6e4/log4j-core-2.11.1.jar:/root/.gradle/caches/modules-2/files-2.1/org.apache.logging.log4j/log4j-api/2.11.1/268f0fe4df3eefe052b57c87ec48517d64fb2a10/log4j-api-2.11.1.jar:/root/.gradle/caches/modules-2/files-2.1/com.google.code.findbugs/jsr305/1.3.9/40719ea6961c0cb6afaeb6a921eaa1f6afd4cfdf/jsr305-1.3.9.jar:/root/.gradle/caches/modules-2/files-2.1/com.google.errorprone/error_prone_annotations/2.0.18/5f65affce1684999e2f4024983835efc3504012e/error_prone_annotations-2.0.18.jar:/root/.gradle/caches/modules-2/files-2.1/com.google.j2objc/j2objc-annotations/1.1/ed28ded51a8b1c6b112568def5f4b455e6809019/j2objc-annotations-1.1.jar:/root/.gradle/caches/modules-2/files-2.1/org.codehaus.mojo/animal-sniffer-annotations/1.14/775b7e22fb10026eed3f86e8dc556dfafe35f2d5/animal-sniffer-annotations-1.14.jar:/root/.gradle/caches/modules-2/files-2.1/com.couchbase.client/core-io/2.0.6/786e661fb89e48d3bf45f84b5ba94a97152ba1ec/core-io-2.0.6.jar:/root/.gradle/caches/modules-2/files-2.1/io.projectreactor/reactor-core/3.3.5.RELEASE/3fe17d2b3c413c38f728049fd334ba9f69b416e1/reactor-core-3.3.5.RELEASE.jar:/root/.gradle/caches/modules-2/files-2.1/org.reactivestreams/reactive-streams/1.0.3/d9fb7a7926ffa635b3dcaa5049fb2bfa25b3e7d0/reactive-streams-1.0.3.jar scripts/ssh.py -i /tmp/win10-bucket-ops.ini -p version=7.0.0-2477,product=cb,parallel=,init_nodes=True,url= Running: /opt/jython/bin/jython -J-cp /data/workspace/temp_rebalance_even/build/classes/java/main:/data/workspace/temp_rebalance_even/build/resources/main:/root/.gradle/caches/modules-2/files-2.1/com.couchbase.client/couchbase-transactions/1.0.1-SNAPSHOT/c6ff42a1c4529a1d3645ef0a70df43588ac99b15/couchbase-transactions-1.0.1-SNAPSHOT.jar:/root/.gradle/caches/modules-2/files-2.1/com.jcraft/jsch/0.1.54/da3584329a263616e277e15462b387addd1b208d/jsch-0.1.54.jar:/root/.gradle/caches/modules-2/files-2.1/org.ini4j/ini4j/0.5.1/8e737d82ecac9ba6100a9bba71e92a381b75efdc/ini4j-0.5.1.jar:/root/.gradle/caches/modules-2/files-2.1/org.apache.logging.log4j/log4j-slf4j-impl/2.11.1/4b41b53a3a2d299ce381a69d165381ca19f62912/log4j-slf4j-impl-2.11.1.jar:/root/.gradle/caches/modules-2/files-2.1/com.google.guava/guava/22.0/3564ef3803de51fb0530a8377ec6100b33b0d073/guava-22.0.jar:/root/.gradle/caches/modules-2/files-2.1/commons-cli/commons-cli/1.4/c51c00206bb913cd8612b24abd9fa98ae89719b1/commons-cli-1.4.jar:/root/.gradle/caches/modules-2/files-2.1/net.sourceforge.argparse4j/argparse4j/0.8.1/2c8241f84acf6c924bd75be0dbd68e8d74fbcd70/argparse4j-0.8.1.jar:/root/.gradle/caches/modules-2/files-2.1/com.couchbase.client/java-client/3.0.5/50e852b87fb272cdfcaf77097bef8f253a7a863c/java-client-3.0.5.jar:/root/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-api/1.7.25/da76ca59f6a57ee3102f8f9bd9cee742973efa8a/slf4j-api-1.7.25.jar:/root/.gradle/caches/modules-2/files-2.1/org.apache.logging.log4j/log4j-core/2.11.1/592a48674c926b01a9a747c7831bcd82a9e6d6e4/log4j-core-2.11.1.jar:/root/.gradle/caches/modules-2/files-2.1/org.apache.logging.log4j/log4j-api/2.11.1/268f0fe4df3eefe052b57c87ec48517d64fb2a10/log4j-api-2.11.1.jar:/root/.gradle/caches/modules-2/files-2.1/com.google.code.findbugs/jsr305/1.3.9/40719ea6961c0cb6afaeb6a921eaa1f6afd4cfdf/jsr305-1.3.9.jar:/root/.gradle/caches/modules-2/files-2.1/com.google.errorprone/error_prone_annotations/2.0.18/5f65affce1684999e2f4024983835efc3504012e/error_prone_annotations-2.0.18.jar:/root/.gradle/caches/modules-2/files-2.1/com.google.j2objc/j2objc-annotations/1.1/ed28ded51a8b1c6b112568def5f4b455e6809019/j2objc-annotations-1.1.jar:/root/.gradle/caches/modules-2/files-2.1/org.codehaus.mojo/animal-sniffer-annotations/1.14/775b7e22fb10026eed3f86e8dc556dfafe35f2d5/animal-sniffer-annotations-1.14.jar:/root/.gradle/caches/modules-2/files-2.1/com.couchbase.client/core-io/2.0.6/786e661fb89e48d3bf45f84b5ba94a97152ba1ec/core-io-2.0.6.jar:/root/.gradle/caches/modules-2/files-2.1/io.projectreactor/reactor-core/3.3.5.RELEASE/3fe17d2b3c413c38f728049fd334ba9f69b416e1/reactor-core-3.3.5.RELEASE.jar:/root/.gradle/caches/modules-2/files-2.1/org.reactivestreams/reactive-streams/1.0.3/d9fb7a7926ffa635b3dcaa5049fb2bfa25b3e7d0/reactive-streams-1.0.3.jar:build/classes/java/main scripts/install.py -i /tmp/win10-bucket-ops.ini -p version=7.0.0-2477,product=cb,parallel=,init_nodes=True,url= Running: /opt/jython/bin/jython -J-cp /data/workspace/temp_rebalance_even/build/classes/java/main:/data/workspace/temp_rebalance_even/build/resources/main:/root/.gradle/caches/modules-2/files-2.1/com.couchbase.client/couchbase-transactions/1.0.1-SNAPSHOT/c6ff42a1c4529a1d3645ef0a70df43588ac99b15/couchbase-transactions-1.0.1-SNAPSHOT.jar:/root/.gradle/caches/modules-2/files-2.1/com.jcraft/jsch/0.1.54/da3584329a263616e277e15462b387addd1b208d/jsch-0.1.54.jar:/root/.gradle/caches/modules-2/files-2.1/org.ini4j/ini4j/0.5.1/8e737d82ecac9ba6100a9bba71e92a381b75efdc/ini4j-0.5.1.jar:/root/.gradle/caches/modules-2/files-2.1/org.apache.logging.log4j/log4j-slf4j-impl/2.11.1/4b41b53a3a2d299ce381a69d165381ca19f62912/log4j-slf4j-impl-2.11.1.jar:/root/.gradle/caches/modules-2/files-2.1/com.google.guava/guava/22.0/3564ef3803de51fb0530a8377ec6100b33b0d073/guava-22.0.jar:/root/.gradle/caches/modules-2/files-2.1/commons-cli/commons-cli/1.4/c51c00206bb913cd8612b24abd9fa98ae89719b1/commons-cli-1.4.jar:/root/.gradle/caches/modules-2/files-2.1/net.sourceforge.argparse4j/argparse4j/0.8.1/2c8241f84acf6c924bd75be0dbd68e8d74fbcd70/argparse4j-0.8.1.jar:/root/.gradle/caches/modules-2/files-2.1/com.couchbase.client/java-client/3.0.5/50e852b87fb272cdfcaf77097bef8f253a7a863c/java-client-3.0.5.jar:/root/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-api/1.7.25/da76ca59f6a57ee3102f8f9bd9cee742973efa8a/slf4j-api-1.7.25.jar:/root/.gradle/caches/modules-2/files-2.1/org.apache.logging.log4j/log4j-core/2.11.1/592a48674c926b01a9a747c7831bcd82a9e6d6e4/log4j-core-2.11.1.jar:/root/.gradle/caches/modules-2/files-2.1/org.apache.logging.log4j/log4j-api/2.11.1/268f0fe4df3eefe052b57c87ec48517d64fb2a10/log4j-api-2.11.1.jar:/root/.gradle/caches/modules-2/files-2.1/com.google.code.findbugs/jsr305/1.3.9/40719ea6961c0cb6afaeb6a921eaa1f6afd4cfdf/jsr305-1.3.9.jar:/root/.gradle/caches/modules-2/files-2.1/com.google.errorprone/error_prone_annotations/2.0.18/5f65affce1684999e2f4024983835efc3504012e/error_prone_annotations-2.0.18.jar:/root/.gradle/caches/modules-2/files-2.1/com.google.j2objc/j2objc-annotations/1.1/ed28ded51a8b1c6b112568def5f4b455e6809019/j2objc-annotations-1.1.jar:/root/.gradle/caches/modules-2/files-2.1/org.codehaus.mojo/animal-sniffer-annotations/1.14/775b7e22fb10026eed3f86e8dc556dfafe35f2d5/animal-sniffer-annotations-1.14.jar:/root/.gradle/caches/modules-2/files-2.1/com.couchbase.client/core-io/2.0.6/786e661fb89e48d3bf45f84b5ba94a97152ba1ec/core-io-2.0.6.jar:/root/.gradle/caches/modules-2/files-2.1/io.projectreactor/reactor-core/3.3.5.RELEASE/3fe17d2b3c413c38f728049fd334ba9f69b416e1/reactor-core-3.3.5.RELEASE.jar:/root/.gradle/caches/modules-2/files-2.1/org.reactivestreams/reactive-streams/1.0.3/d9fb7a7926ffa635b3dcaa5049fb2bfa25b3e7d0/reactive-streams-1.0.3.jar:build/classes/java/main:src/main/resources testrunner.py -i /tmp/win10-bucket-ops.ini -p version=7.0.0-2477,product=cb,parallel=,init_nodes=True,url= Running: /opt/jython/bin/jython -J-cp /data/workspace/temp_rebalance_even/build/classes/java/main:/data/workspace/temp_rebalance_even/build/resources/main:/root/.gradle/caches/modules-2/files-2.1/com.couchbase.client/couchbase-transactions/1.0.1-SNAPSHOT/c6ff42a1c4529a1d3645ef0a70df43588ac99b15/couchbase-transactions-1.0.1-SNAPSHOT.jar:/root/.gradle/caches/modules-2/files-2.1/com.jcraft/jsch/0.1.54/da3584329a263616e277e15462b387addd1b208d/jsch-0.1.54.jar:/root/.gradle/caches/modules-2/files-2.1/org.ini4j/ini4j/0.5.1/8e737d82ecac9ba6100a9bba71e92a381b75efdc/ini4j-0.5.1.jar:/root/.gradle/caches/modules-2/files-2.1/org.apache.logging.log4j/log4j-slf4j-impl/2.11.1/4b41b53a3a2d299ce381a69d165381ca19f62912/log4j-slf4j-impl-2.11.1.jar:/root/.gradle/caches/modules-2/files-2.1/com.google.guava/guava/22.0/3564ef3803de51fb0530a8377ec6100b33b0d073/guava-22.0.jar:/root/.gradle/caches/modules-2/files-2.1/commons-cli/commons-cli/1.4/c51c00206bb913cd8612b24abd9fa98ae89719b1/commons-cli-1.4.jar:/root/.gradle/caches/modules-2/files-2.1/net.sourceforge.argparse4j/argparse4j/0.8.1/2c8241f84acf6c924bd75be0dbd68e8d74fbcd70/argparse4j-0.8.1.jar:/root/.gradle/caches/modules-2/files-2.1/com.couchbase.client/java-client/3.0.5/50e852b87fb272cdfcaf77097bef8f253a7a863c/java-client-3.0.5.jar:/root/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-api/1.7.25/da76ca59f6a57ee3102f8f9bd9cee742973efa8a/slf4j-api-1.7.25.jar:/root/.gradle/caches/modules-2/files-2.1/org.apache.logging.log4j/log4j-core/2.11.1/592a48674c926b01a9a747c7831bcd82a9e6d6e4/log4j-core-2.11.1.jar:/root/.gradle/caches/modules-2/files-2.1/org.apache.logging.log4j/log4j-api/2.11.1/268f0fe4df3eefe052b57c87ec48517d64fb2a10/log4j-api-2.11.1.jar:/root/.gradle/caches/modules-2/files-2.1/com.google.code.findbugs/jsr305/1.3.9/40719ea6961c0cb6afaeb6a921eaa1f6afd4cfdf/jsr305-1.3.9.jar:/root/.gradle/caches/modules-2/files-2.1/com.google.errorprone/error_prone_annotations/2.0.18/5f65affce1684999e2f4024983835efc3504012e/error_prone_annotations-2.0.18.jar:/root/.gradle/caches/modules-2/files-2.1/com.google.j2objc/j2objc-annotations/1.1/ed28ded51a8b1c6b112568def5f4b455e6809019/j2objc-annotations-1.1.jar:/root/.gradle/caches/modules-2/files-2.1/org.codehaus.mojo/animal-sniffer-annotations/1.14/775b7e22fb10026eed3f86e8dc556dfafe35f2d5/animal-sniffer-annotations-1.14.jar:/root/.gradle/caches/modules-2/files-2.1/com.couchbase.client/core-io/2.0.6/786e661fb89e48d3bf45f84b5ba94a97152ba1ec/core-io-2.0.6.jar:/root/.gradle/caches/modules-2/files-2.1/io.projectreactor/reactor-core/3.3.5.RELEASE/3fe17d2b3c413c38f728049fd334ba9f69b416e1/reactor-core-3.3.5.RELEASE.jar:/root/.gradle/caches/modules-2/files-2.1/org.reactivestreams/reactive-streams/1.0.3/d9fb7a7926ffa635b3dcaa5049fb2bfa25b3e7d0/reactive-streams-1.0.3.jar:build/classes/java/main:src/main/resources scripts/rerun_jobs.py -i /tmp/win10-bucket-ops.ini -p version=7.0.0-2477,product=cb,parallel=,init_nodes=True,url= > Task :install 2020-06-28 04:06:04,490 - root - WARNING - URL: is invalid, will use version to locate build 2020-06-28 04:06:15,315 - root - INFO - Downloading build binary to 172.23.98.196:/tmp/couchbase-server-enterprise-7.0.0-2477-centos7.x86_64.rpm.. 2020-06-28 04:06:15,444 - root - INFO - Downloading build binary to 172.23.98.195:/tmp/couchbase-server-enterprise-7.0.0-2477-centos7.x86_64.rpm.. 2020-06-28 04:06:15,549 - root - INFO - Downloading build binary to 172.23.121.10:/tmp/couchbase-server-enterprise-7.0.0-2477-centos7.x86_64.rpm.. 2020-06-28 04:06:15,621 - root - INFO - Downloading build binary to 172.23.104.186:/tmp/couchbase-server-enterprise-7.0.0-2477-centos7.x86_64.rpm.. 2020-06-28 04:06:15,688 - root - INFO - Downloading build binary to 172.23.120.201:/tmp/couchbase-server-enterprise-7.0.0-2477-centos7.x86_64.rpm.. 2020-06-28 04:06:15,742 - root - INFO - Downloading build binary to 172.23.120.206:/tmp/couchbase-server-enterprise-7.0.0-2477-centos7.x86_64.rpm.. 2020-06-28 04:06:48,447 - root - INFO - Done with uninstall on 172.23.120.206. 2020-06-28 04:06:49,127 - root - INFO - Done with uninstall on 172.23.98.196. 2020-06-28 04:06:49,674 - root - INFO - Done with uninstall on 172.23.104.186. 2020-06-28 04:06:53,471 - root - INFO - Done with uninstall on 172.23.120.201. 2020-06-28 04:06:54,608 - root - INFO - Done with uninstall on 172.23.98.195. 2020-06-28 04:09:41,506 - root - INFO - Done with uninstall on 172.23.121.10. 2020-06-28 04:09:50,910 - root - INFO - Done with install on 172.23.120.206. 2020-06-28 04:09:57,654 - root - INFO - Done with install on 172.23.120.201. 2020-06-28 04:10:17,052 - root - INFO - Done with install on 172.23.104.186. 2020-06-28 04:10:52,611 - root - INFO - Setting KV memory quota as 1886 MB on 172.23.120.206 2020-06-28 04:10:52,756 - root - INFO - Done with init on 172.23.120.206. 2020-06-28 04:10:52,770 - root - INFO - Done with cleanup on 172.23.120.206. 2020-06-28 04:10:58,926 - root - INFO - Setting KV memory quota as 1886 MB on 172.23.120.201 2020-06-28 04:10:59,048 - root - INFO - Done with init on 172.23.120.201. 2020-06-28 04:10:59,072 - root - INFO - Done with cleanup on 172.23.120.201. 2020-06-28 04:11:18,671 - root - INFO - Setting KV memory quota as 3106 MB on 172.23.104.186 2020-06-28 04:11:18,907 - root - INFO - Done with init on 172.23.104.186. 2020-06-28 04:11:18,937 - root - INFO - Done with cleanup on 172.23.104.186. 2020-06-28 04:11:27,002 - root - INFO - Done with install on 172.23.98.196. 2020-06-28 04:11:29,900 - root - INFO - Done with install on 172.23.98.195. 2020-06-28 04:12:28,969 - root - INFO - Setting KV memory quota as 1894 MB on 172.23.98.196 2020-06-28 04:12:29,279 - root - INFO - Done with init on 172.23.98.196. 2020-06-28 04:12:29,335 - root - INFO - Done with cleanup on 172.23.98.196. 2020-06-28 04:12:31,835 - root - INFO - Setting KV memory quota as 1894 MB on 172.23.98.195 2020-06-28 04:12:32,154 - root - INFO - Done with init on 172.23.98.195. 2020-06-28 04:12:32,200 - root - INFO - Done with cleanup on 172.23.98.195. 2020-06-28 04:13:02,164 - root - INFO - Done with install on 172.23.121.10. 2020-06-28 04:14:03,780 - root - INFO - Setting KV memory quota as 2156 MB on 172.23.121.10 2020-06-28 04:14:03,947 - root - INFO - Done with init on 172.23.121.10. 2020-06-28 04:14:03,982 - root - INFO - Done with cleanup on 172.23.121.10. 2020-06-28 04:14:15,789 - root - INFO - ---------------------------------------------------------------------------------------------------- 2020-06-28 04:14:15,822 - root - INFO - node:172.23.98.196:8091 version:7.0.0-2477-enterprise aFamily:inet services:[u'kv'] 2020-06-28 04:14:15,851 - root - INFO - node:172.23.98.195:8091 version:7.0.0-2477-enterprise aFamily:inet services:[u'kv'] 2020-06-28 04:14:15,875 - root - INFO - node:172.23.121.10:8091 version:7.0.0-2477-enterprise aFamily:inet services:[u'kv'] 2020-06-28 04:14:15,903 - root - INFO - node:172.23.104.186:8091 version:7.0.0-2477-enterprise aFamily:inet services:[u'kv'] 2020-06-28 04:14:15,924 - root - INFO - node:172.23.120.201:8091 version:7.0.0-2477-enterprise aFamily:inet services:[u'kv'] 2020-06-28 04:14:15,944 - root - INFO - node:172.23.120.206:8091 version:7.0.0-2477-enterprise aFamily:inet services:[u'kv'] 2020-06-28 04:14:15,946 - root - INFO - ---------------------------------------------------------------------------------------------------- 2020-06-28 04:14:15,946 - root - INFO - ---------------------------------------------------------------------------------------------------- 2020-06-28 04:14:15,947 - root - INFO - INSTALL COMPLETED ON: 172.23.98.196 2020-06-28 04:14:15,947 - root - INFO - INSTALL COMPLETED ON: 172.23.98.195 2020-06-28 04:14:15,947 - root - INFO - INSTALL COMPLETED ON: 172.23.121.10 2020-06-28 04:14:15,947 - root - INFO - INSTALL COMPLETED ON: 172.23.104.186 2020-06-28 04:14:15,948 - root - INFO - INSTALL COMPLETED ON: 172.23.120.201 2020-06-28 04:14:15,948 - root - INFO - INSTALL COMPLETED ON: 172.23.120.206 2020-06-28 04:14:15,948 - root - INFO - ---------------------------------------------------------------------------------------------------- 2020-06-28 04:14:15,953 - root - INFO - TOTAL INSTALL TIME = 491.0 seconds Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0. Use '--warning-mode all' to show the individual deprecation warnings. See https://docs.gradle.org/6.2.2/userguide/command_line_interface.html#sec:command_line_warnings BUILD SUCCESSFUL in 8m 18s 1 actionable task: 1 executed + guides/gradlew --refresh-dependencies testrunner -P jython=/opt/jython/bin/jython -P 'args=-i /tmp/win10-bucket-ops.ini -p rerun=False,quota_percent=90,crash_warning=True -c conf/collections/collections_failover.conf -m rest' > Configure project : Running: /opt/jython/bin/jython -J-cp /data/workspace/temp_rebalance_even/build/classes/java/main:/data/workspace/temp_rebalance_even/build/resources/main:/root/.gradle/caches/modules-2/files-2.1/com.couchbase.client/couchbase-transactions/1.0.1-SNAPSHOT/c6ff42a1c4529a1d3645ef0a70df43588ac99b15/couchbase-transactions-1.0.1-SNAPSHOT.jar:/root/.gradle/caches/modules-2/files-2.1/com.jcraft/jsch/0.1.54/da3584329a263616e277e15462b387addd1b208d/jsch-0.1.54.jar:/root/.gradle/caches/modules-2/files-2.1/org.ini4j/ini4j/0.5.1/8e737d82ecac9ba6100a9bba71e92a381b75efdc/ini4j-0.5.1.jar:/root/.gradle/caches/modules-2/files-2.1/org.apache.logging.log4j/log4j-slf4j-impl/2.11.1/4b41b53a3a2d299ce381a69d165381ca19f62912/log4j-slf4j-impl-2.11.1.jar:/root/.gradle/caches/modules-2/files-2.1/com.google.guava/guava/22.0/3564ef3803de51fb0530a8377ec6100b33b0d073/guava-22.0.jar:/root/.gradle/caches/modules-2/files-2.1/commons-cli/commons-cli/1.4/c51c00206bb913cd8612b24abd9fa98ae89719b1/commons-cli-1.4.jar:/root/.gradle/caches/modules-2/files-2.1/net.sourceforge.argparse4j/argparse4j/0.8.1/2c8241f84acf6c924bd75be0dbd68e8d74fbcd70/argparse4j-0.8.1.jar:/root/.gradle/caches/modules-2/files-2.1/com.couchbase.client/java-client/3.0.5/50e852b87fb272cdfcaf77097bef8f253a7a863c/java-client-3.0.5.jar:/root/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-api/1.7.25/da76ca59f6a57ee3102f8f9bd9cee742973efa8a/slf4j-api-1.7.25.jar:/root/.gradle/caches/modules-2/files-2.1/org.apache.logging.log4j/log4j-core/2.11.1/592a48674c926b01a9a747c7831bcd82a9e6d6e4/log4j-core-2.11.1.jar:/root/.gradle/caches/modules-2/files-2.1/org.apache.logging.log4j/log4j-api/2.11.1/268f0fe4df3eefe052b57c87ec48517d64fb2a10/log4j-api-2.11.1.jar:/root/.gradle/caches/modules-2/files-2.1/com.google.code.findbugs/jsr305/1.3.9/40719ea6961c0cb6afaeb6a921eaa1f6afd4cfdf/jsr305-1.3.9.jar:/root/.gradle/caches/modules-2/files-2.1/com.google.errorprone/error_prone_annotations/2.0.18/5f65affce1684999e2f4024983835efc3504012e/error_prone_annotations-2.0.18.jar:/root/.gradle/caches/modules-2/files-2.1/com.google.j2objc/j2objc-annotations/1.1/ed28ded51a8b1c6b112568def5f4b455e6809019/j2objc-annotations-1.1.jar:/root/.gradle/caches/modules-2/files-2.1/org.codehaus.mojo/animal-sniffer-annotations/1.14/775b7e22fb10026eed3f86e8dc556dfafe35f2d5/animal-sniffer-annotations-1.14.jar:/root/.gradle/caches/modules-2/files-2.1/com.couchbase.client/core-io/2.0.6/786e661fb89e48d3bf45f84b5ba94a97152ba1ec/core-io-2.0.6.jar:/root/.gradle/caches/modules-2/files-2.1/io.projectreactor/reactor-core/3.3.5.RELEASE/3fe17d2b3c413c38f728049fd334ba9f69b416e1/reactor-core-3.3.5.RELEASE.jar:/root/.gradle/caches/modules-2/files-2.1/org.reactivestreams/reactive-streams/1.0.3/d9fb7a7926ffa635b3dcaa5049fb2bfa25b3e7d0/reactive-streams-1.0.3.jar scripts/ssh.py -i /tmp/win10-bucket-ops.ini -p rerun=False,quota_percent=90,crash_warning=True -c conf/collections/collections_failover.conf -m rest Running: /opt/jython/bin/jython -J-cp /data/workspace/temp_rebalance_even/build/classes/java/main:/data/workspace/temp_rebalance_even/build/resources/main:/root/.gradle/caches/modules-2/files-2.1/com.couchbase.client/couchbase-transactions/1.0.1-SNAPSHOT/c6ff42a1c4529a1d3645ef0a70df43588ac99b15/couchbase-transactions-1.0.1-SNAPSHOT.jar:/root/.gradle/caches/modules-2/files-2.1/com.jcraft/jsch/0.1.54/da3584329a263616e277e15462b387addd1b208d/jsch-0.1.54.jar:/root/.gradle/caches/modules-2/files-2.1/org.ini4j/ini4j/0.5.1/8e737d82ecac9ba6100a9bba71e92a381b75efdc/ini4j-0.5.1.jar:/root/.gradle/caches/modules-2/files-2.1/org.apache.logging.log4j/log4j-slf4j-impl/2.11.1/4b41b53a3a2d299ce381a69d165381ca19f62912/log4j-slf4j-impl-2.11.1.jar:/root/.gradle/caches/modules-2/files-2.1/com.google.guava/guava/22.0/3564ef3803de51fb0530a8377ec6100b33b0d073/guava-22.0.jar:/root/.gradle/caches/modules-2/files-2.1/commons-cli/commons-cli/1.4/c51c00206bb913cd8612b24abd9fa98ae89719b1/commons-cli-1.4.jar:/root/.gradle/caches/modules-2/files-2.1/net.sourceforge.argparse4j/argparse4j/0.8.1/2c8241f84acf6c924bd75be0dbd68e8d74fbcd70/argparse4j-0.8.1.jar:/root/.gradle/caches/modules-2/files-2.1/com.couchbase.client/java-client/3.0.5/50e852b87fb272cdfcaf77097bef8f253a7a863c/java-client-3.0.5.jar:/root/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-api/1.7.25/da76ca59f6a57ee3102f8f9bd9cee742973efa8a/slf4j-api-1.7.25.jar:/root/.gradle/caches/modules-2/files-2.1/org.apache.logging.log4j/log4j-core/2.11.1/592a48674c926b01a9a747c7831bcd82a9e6d6e4/log4j-core-2.11.1.jar:/root/.gradle/caches/modules-2/files-2.1/org.apache.logging.log4j/log4j-api/2.11.1/268f0fe4df3eefe052b57c87ec48517d64fb2a10/log4j-api-2.11.1.jar:/root/.gradle/caches/modules-2/files-2.1/com.google.code.findbugs/jsr305/1.3.9/40719ea6961c0cb6afaeb6a921eaa1f6afd4cfdf/jsr305-1.3.9.jar:/root/.gradle/caches/modules-2/files-2.1/com.google.errorprone/error_prone_annotations/2.0.18/5f65affce1684999e2f4024983835efc3504012e/error_prone_annotations-2.0.18.jar:/root/.gradle/caches/modules-2/files-2.1/com.google.j2objc/j2objc-annotations/1.1/ed28ded51a8b1c6b112568def5f4b455e6809019/j2objc-annotations-1.1.jar:/root/.gradle/caches/modules-2/files-2.1/org.codehaus.mojo/animal-sniffer-annotations/1.14/775b7e22fb10026eed3f86e8dc556dfafe35f2d5/animal-sniffer-annotations-1.14.jar:/root/.gradle/caches/modules-2/files-2.1/com.couchbase.client/core-io/2.0.6/786e661fb89e48d3bf45f84b5ba94a97152ba1ec/core-io-2.0.6.jar:/root/.gradle/caches/modules-2/files-2.1/io.projectreactor/reactor-core/3.3.5.RELEASE/3fe17d2b3c413c38f728049fd334ba9f69b416e1/reactor-core-3.3.5.RELEASE.jar:/root/.gradle/caches/modules-2/files-2.1/org.reactivestreams/reactive-streams/1.0.3/d9fb7a7926ffa635b3dcaa5049fb2bfa25b3e7d0/reactive-streams-1.0.3.jar:build/classes/java/main scripts/install.py -i /tmp/win10-bucket-ops.ini -p rerun=False,quota_percent=90,crash_warning=True -c conf/collections/collections_failover.conf -m rest Running: /opt/jython/bin/jython -J-cp /data/workspace/temp_rebalance_even/build/classes/java/main:/data/workspace/temp_rebalance_even/build/resources/main:/root/.gradle/caches/modules-2/files-2.1/com.couchbase.client/couchbase-transactions/1.0.1-SNAPSHOT/c6ff42a1c4529a1d3645ef0a70df43588ac99b15/couchbase-transactions-1.0.1-SNAPSHOT.jar:/root/.gradle/caches/modules-2/files-2.1/com.jcraft/jsch/0.1.54/da3584329a263616e277e15462b387addd1b208d/jsch-0.1.54.jar:/root/.gradle/caches/modules-2/files-2.1/org.ini4j/ini4j/0.5.1/8e737d82ecac9ba6100a9bba71e92a381b75efdc/ini4j-0.5.1.jar:/root/.gradle/caches/modules-2/files-2.1/org.apache.logging.log4j/log4j-slf4j-impl/2.11.1/4b41b53a3a2d299ce381a69d165381ca19f62912/log4j-slf4j-impl-2.11.1.jar:/root/.gradle/caches/modules-2/files-2.1/com.google.guava/guava/22.0/3564ef3803de51fb0530a8377ec6100b33b0d073/guava-22.0.jar:/root/.gradle/caches/modules-2/files-2.1/commons-cli/commons-cli/1.4/c51c00206bb913cd8612b24abd9fa98ae89719b1/commons-cli-1.4.jar:/root/.gradle/caches/modules-2/files-2.1/net.sourceforge.argparse4j/argparse4j/0.8.1/2c8241f84acf6c924bd75be0dbd68e8d74fbcd70/argparse4j-0.8.1.jar:/root/.gradle/caches/modules-2/files-2.1/com.couchbase.client/java-client/3.0.5/50e852b87fb272cdfcaf77097bef8f253a7a863c/java-client-3.0.5.jar:/root/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-api/1.7.25/da76ca59f6a57ee3102f8f9bd9cee742973efa8a/slf4j-api-1.7.25.jar:/root/.gradle/caches/modules-2/files-2.1/org.apache.logging.log4j/log4j-core/2.11.1/592a48674c926b01a9a747c7831bcd82a9e6d6e4/log4j-core-2.11.1.jar:/root/.gradle/caches/modules-2/files-2.1/org.apache.logging.log4j/log4j-api/2.11.1/268f0fe4df3eefe052b57c87ec48517d64fb2a10/log4j-api-2.11.1.jar:/root/.gradle/caches/modules-2/files-2.1/com.google.code.findbugs/jsr305/1.3.9/40719ea6961c0cb6afaeb6a921eaa1f6afd4cfdf/jsr305-1.3.9.jar:/root/.gradle/caches/modules-2/files-2.1/com.google.errorprone/error_prone_annotations/2.0.18/5f65affce1684999e2f4024983835efc3504012e/error_prone_annotations-2.0.18.jar:/root/.gradle/caches/modules-2/files-2.1/com.google.j2objc/j2objc-annotations/1.1/ed28ded51a8b1c6b112568def5f4b455e6809019/j2objc-annotations-1.1.jar:/root/.gradle/caches/modules-2/files-2.1/org.codehaus.mojo/animal-sniffer-annotations/1.14/775b7e22fb10026eed3f86e8dc556dfafe35f2d5/animal-sniffer-annotations-1.14.jar:/root/.gradle/caches/modules-2/files-2.1/com.couchbase.client/core-io/2.0.6/786e661fb89e48d3bf45f84b5ba94a97152ba1ec/core-io-2.0.6.jar:/root/.gradle/caches/modules-2/files-2.1/io.projectreactor/reactor-core/3.3.5.RELEASE/3fe17d2b3c413c38f728049fd334ba9f69b416e1/reactor-core-3.3.5.RELEASE.jar:/root/.gradle/caches/modules-2/files-2.1/org.reactivestreams/reactive-streams/1.0.3/d9fb7a7926ffa635b3dcaa5049fb2bfa25b3e7d0/reactive-streams-1.0.3.jar:build/classes/java/main:src/main/resources testrunner.py -i /tmp/win10-bucket-ops.ini -p rerun=False,quota_percent=90,crash_warning=True -c conf/collections/collections_failover.conf -m rest Running: /opt/jython/bin/jython -J-cp /data/workspace/temp_rebalance_even/build/classes/java/main:/data/workspace/temp_rebalance_even/build/resources/main:/root/.gradle/caches/modules-2/files-2.1/com.couchbase.client/couchbase-transactions/1.0.1-SNAPSHOT/c6ff42a1c4529a1d3645ef0a70df43588ac99b15/couchbase-transactions-1.0.1-SNAPSHOT.jar:/root/.gradle/caches/modules-2/files-2.1/com.jcraft/jsch/0.1.54/da3584329a263616e277e15462b387addd1b208d/jsch-0.1.54.jar:/root/.gradle/caches/modules-2/files-2.1/org.ini4j/ini4j/0.5.1/8e737d82ecac9ba6100a9bba71e92a381b75efdc/ini4j-0.5.1.jar:/root/.gradle/caches/modules-2/files-2.1/org.apache.logging.log4j/log4j-slf4j-impl/2.11.1/4b41b53a3a2d299ce381a69d165381ca19f62912/log4j-slf4j-impl-2.11.1.jar:/root/.gradle/caches/modules-2/files-2.1/com.google.guava/guava/22.0/3564ef3803de51fb0530a8377ec6100b33b0d073/guava-22.0.jar:/root/.gradle/caches/modules-2/files-2.1/commons-cli/commons-cli/1.4/c51c00206bb913cd8612b24abd9fa98ae89719b1/commons-cli-1.4.jar:/root/.gradle/caches/modules-2/files-2.1/net.sourceforge.argparse4j/argparse4j/0.8.1/2c8241f84acf6c924bd75be0dbd68e8d74fbcd70/argparse4j-0.8.1.jar:/root/.gradle/caches/modules-2/files-2.1/com.couchbase.client/java-client/3.0.5/50e852b87fb272cdfcaf77097bef8f253a7a863c/java-client-3.0.5.jar:/root/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-api/1.7.25/da76ca59f6a57ee3102f8f9bd9cee742973efa8a/slf4j-api-1.7.25.jar:/root/.gradle/caches/modules-2/files-2.1/org.apache.logging.log4j/log4j-core/2.11.1/592a48674c926b01a9a747c7831bcd82a9e6d6e4/log4j-core-2.11.1.jar:/root/.gradle/caches/modules-2/files-2.1/org.apache.logging.log4j/log4j-api/2.11.1/268f0fe4df3eefe052b57c87ec48517d64fb2a10/log4j-api-2.11.1.jar:/root/.gradle/caches/modules-2/files-2.1/com.google.code.findbugs/jsr305/1.3.9/40719ea6961c0cb6afaeb6a921eaa1f6afd4cfdf/jsr305-1.3.9.jar:/root/.gradle/caches/modules-2/files-2.1/com.google.errorprone/error_prone_annotations/2.0.18/5f65affce1684999e2f4024983835efc3504012e/error_prone_annotations-2.0.18.jar:/root/.gradle/caches/modules-2/files-2.1/com.google.j2objc/j2objc-annotations/1.1/ed28ded51a8b1c6b112568def5f4b455e6809019/j2objc-annotations-1.1.jar:/root/.gradle/caches/modules-2/files-2.1/org.codehaus.mojo/animal-sniffer-annotations/1.14/775b7e22fb10026eed3f86e8dc556dfafe35f2d5/animal-sniffer-annotations-1.14.jar:/root/.gradle/caches/modules-2/files-2.1/com.couchbase.client/core-io/2.0.6/786e661fb89e48d3bf45f84b5ba94a97152ba1ec/core-io-2.0.6.jar:/root/.gradle/caches/modules-2/files-2.1/io.projectreactor/reactor-core/3.3.5.RELEASE/3fe17d2b3c413c38f728049fd334ba9f69b416e1/reactor-core-3.3.5.RELEASE.jar:/root/.gradle/caches/modules-2/files-2.1/org.reactivestreams/reactive-streams/1.0.3/d9fb7a7926ffa635b3dcaa5049fb2bfa25b3e7d0/reactive-streams-1.0.3.jar:build/classes/java/main:src/main/resources scripts/rerun_jobs.py -i /tmp/win10-bucket-ops.ini -p rerun=False,quota_percent=90,crash_warning=True -c conf/collections/collections_failover.conf -m rest > Task :compileJava UP-TO-DATE > Task :testrunner Filename: conf/collections/collections_failover.conf Prefix: bucket_collections.collections_rebalance.CollectionsRebalance Global Test input params: {'cluster_name': 'win10-bucket-ops', 'conf_file': 'conf/collections/collections_failover.conf', 'crash_warning': 'True', 'ini': '/tmp/win10-bucket-ops.ini', 'num_nodes': 6, 'quota_percent': '90', 'rerun': 'False', 'spec': 'collections_failover'} Logs will be stored at /data/workspace/temp_rebalance_even/logs/testrunner-20-Jun-28_04-14-25/test_1 ./testrunner -i /tmp/win10-bucket-ops.ini rerun=False,quota_percent=90,crash_warning=True -t bucket_collections.collections_rebalance.CollectionsRebalance.test_data_load_collections_with_graceful_failover_recovery,nodes_init=5,nodes_failover=2,recovery_type=delta,bucket_spec=multi_bucket.buckets_for_rebalance_tests,data_load_stage=before,GROUP=P0_failover_and_recovery Test Input params: {'data_load_stage': 'before', 'cluster_name': 'win10-bucket-ops', 'quota_percent': '90', 'nodes_failover': '2', 'ini': '/tmp/win10-bucket-ops.ini', 'recovery_type': 'delta', 'conf_file': 'conf/collections/collections_failover.conf', 'crash_warning': 'True', 'spec': 'collections_failover', 'rerun': 'False', 'num_nodes': 6, 'logs_folder': '/data/workspace/temp_rebalance_even/logs/testrunner-20-Jun-28_04-14-25/test_1', 'nodes_init': '5', 'GROUP': 'P0_failover_and_recovery', 'bucket_spec': 'multi_bucket.buckets_for_rebalance_tests', 'case_number': 1} WARNING: Failed to import concurrent.futures module test_data_load_collections_with_graceful_failover_recovery (bucket_collections.collections_rebalance.CollectionsRebalance) ... 2020-06-28 04:14:27,348 | test | INFO | MainThread | [basetestcase:log_setup_status:461] ========= BaseTestCase setup started for test #1 test_data_load_collections_with_graceful_failover_recovery ========= 2020-06-28 04:14:29,272 | test | INFO | MainThread | [basetestcase:log_setup_status:461] ========= BaseTestCase setup started for test #1 test_data_load_collections_with_graceful_failover_recovery ========= 2020-06-28 04:15:09,552 | test | INFO | MainThread | [basetestcase:setUp:305] Initializing cluster 2020-06-28 04:15:10,033 | test | WARNING | MainThread | [basetestcase:_initialize_nodes:504] RAM quota was defined less than 100 MB: 2020-06-28 04:15:29,114 | test | INFO | MainThread | [basetestcase:setUp:327] Couchbase Cluster: default, Nodes: 172.23.98.196, 172.23.98.195, 172.23.121.10, 172.23.104.186, 172.23.120.201, 172.23.120.206 initialized 2020-06-28 04:15:29,190 | test | INFO | MainThread | [basetestcase:log_setup_status:461] ========= BaseTestCase setup finished for test #1 test_data_load_collections_with_graceful_failover_recovery ========= 2020-06-28 04:15:29,227 | test | INFO | MainThread | [basetestcase:log_setup_status:461] ========= CollectionBase setup started for test #1 test_data_load_collections_with_graceful_failover_recovery ========= 2020-06-28 04:15:54,730 | test | INFO | pool-2-thread-7 | [table_view:display:72] Rebalance Overview +----------------+----------+--------------+ | Nodes | Services | Status | +----------------+----------+--------------+ | 172.23.98.196 | kv | Cluster node | | 172.23.98.195 | None | <--- IN --- | | 172.23.121.10 | None | <--- IN --- | | 172.23.104.186 | None | <--- IN --- | | 172.23.120.201 | None | <--- IN --- | +----------------+----------+--------------+ 2020-06-28 04:15:54,752 | test | INFO | pool-2-thread-7 | [task:check:317] Rebalance - status: running, progress: 0 2020-06-28 04:16:04,846 | test | INFO | pool-2-thread-7 | [task:check:317] Rebalance - status: none, progress: 100 2020-06-28 04:16:04,888 | test | INFO | pool-2-thread-7 | [task:check:374] Rebalance completed with progress: 100% in 10.1579999924 sec 2020-06-28 04:16:05,348 | test | INFO | MainThread | [bucket_ready_functions:create_buckets_using_json_data:1691] Creating required buckets from template 2020-06-28 04:16:06,026 | test | INFO | Thread-1 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection YR9LOY4luhJ creation 2020-06-28 04:16:06,194 | test | INFO | Thread-3 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection xzM7PvH creation 2020-06-28 04:16:06,326 | test | INFO | Thread-5 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection tQBQTSuta creation 2020-06-28 04:16:08,890 | test | INFO | Thread-5 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection gcX3AEfgwjUW_wO_zv3CBpUsa46Az creation 2020-06-28 04:16:09,069 | test | INFO | Thread-7 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection o0N3wOz_szpm85Oh_Xk36DNmEub5a creation 2020-06-28 04:16:09,276 | test | INFO | Thread-8 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection collection1 creation 2020-06-28 04:16:11,119 | test | INFO | Thread-5 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection nJ7wT1_ creation 2020-06-28 04:16:11,375 | test | INFO | Thread-7 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection collection1 creation 2020-06-28 04:16:11,713 | test | INFO | Thread-8 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection collection2 creation 2020-06-28 04:16:13,401 | test | INFO | Thread-5 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection NrmCu creation 2020-06-28 04:16:14,000 | test | INFO | Thread-16 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection collection1 creation 2020-06-28 04:16:14,578 | test | INFO | Thread-18 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection collection1 creation 2020-06-28 04:16:16,127 | test | INFO | Thread-20 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection m_i5%efCs_DGuFU0V creation 2020-06-28 04:16:16,410 | test | INFO | Thread-16 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection collection2 creation 2020-06-28 04:16:16,868 | test | INFO | Thread-18 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection collection2 creation 2020-06-28 04:16:18,427 | test | INFO | Thread-20 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection HT-6aVVHbjybRfdiquyqiEi6mKQJVb creation 2020-06-28 04:16:20,753 | test | INFO | Thread-20 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection mZtnKdEx_JLL50YIUl053yTy- creation 2020-06-28 04:16:23,058 | test | INFO | Thread-20 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection RYcQb5nYQWdilaPy1R5 creation 2020-06-28 04:16:25,377 | test | INFO | Thread-20 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection Z-360IYm%Q70%j0h8HX0fs7%7 creation 2020-06-28 04:16:28,312 | test | INFO | Thread-28 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection FXpVwZq creation 2020-06-28 04:16:30,664 | test | INFO | Thread-28 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection FoWKvOLDoch3sVormwSRzAw creation 2020-06-28 04:16:33,003 | test | INFO | Thread-28 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection Pfykv8VNT creation 2020-06-28 04:16:35,301 | test | INFO | Thread-28 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection ecNELfLu3d0_BJxda7z6g23K3ySl9K creation 2020-06-28 04:16:37,601 | test | INFO | Thread-28 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection Rv7 creation 2020-06-28 04:16:40,368 | test | INFO | Thread-34 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection hpuPiuBkkN creation 2020-06-28 04:16:42,687 | test | INFO | Thread-34 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection 0l7Q1YSaGCeugOILfgKn7j creation 2020-06-28 04:16:44,984 | test | INFO | Thread-34 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection I creation 2020-06-28 04:16:47,285 | test | INFO | Thread-34 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection OfnJ5oPbizdC2uLSBwL5fTmmC63a9x creation 2020-06-28 04:16:49,596 | test | INFO | Thread-34 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection mt82PXIMwI-oHMlTCSo creation 2020-06-28 04:16:52,344 | test | INFO | Thread-40 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection 4UIGhS0y creation 2020-06-28 04:16:54,654 | test | INFO | Thread-40 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection 5d6SO0qAolejqvy creation 2020-06-28 04:16:57,052 | test | INFO | Thread-40 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection uPN0NJJyGz creation 2020-06-28 04:16:59,371 | test | INFO | Thread-40 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection bkZoO_3wEXyhwC8AXYnze0PC3c% creation 2020-06-28 04:17:01,678 | test | INFO | Thread-40 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection Mtjv4I9Xnj5WMVnmJh7tIiYMPZ7uWU creation 2020-06-28 04:17:04,484 | test | INFO | Thread-46 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection farGAmk43eEk-M4V creation 2020-06-28 04:17:06,811 | test | INFO | Thread-46 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection fhd_8avL3HP5wbdFDFHzGB53 creation 2020-06-28 04:17:09,095 | test | INFO | Thread-46 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection eVlA6bsPc4x creation 2020-06-28 04:17:11,437 | test | INFO | Thread-46 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection PmTw86LrE95nDsVXg1s9 creation 2020-06-28 04:17:13,785 | test | INFO | Thread-46 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection dz-539QN8fmRMWgYB7 creation 2020-06-28 04:17:16,979 | test | INFO | Thread-52 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection lMBM6oPHRX%pLT creation 2020-06-28 04:17:19,288 | test | INFO | Thread-52 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection WZFxsYg%h creation 2020-06-28 04:17:21,595 | test | INFO | Thread-52 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection j4YOZohcgSVb%M7jUyk2FLA7 creation 2020-06-28 04:17:23,911 | test | INFO | Thread-52 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection ZAakQdePOjAcctePrCvREtwJ5j creation 2020-06-28 04:17:26,236 | test | INFO | Thread-52 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection tfMe40V creation 2020-06-28 04:17:29,163 | test | INFO | Thread-58 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection collections_2 creation 2020-06-28 04:17:31,546 | test | INFO | Thread-58 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection -KawVJSFG5C9Zfvh3kWko creation 2020-06-28 04:17:33,855 | test | INFO | Thread-58 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection K creation 2020-06-28 04:17:36,188 | test | INFO | Thread-58 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection collection_1 creation 2020-06-28 04:17:38,536 | test | INFO | Thread-58 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection FMBlKz5OZp8 creation 2020-06-28 04:17:41,467 | test | INFO | Thread-64 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection nR82t creation 2020-06-28 04:17:43,819 | test | INFO | Thread-64 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection Y6IyYYOIEC_2gWc_lzEFpXPMyAD8 creation 2020-06-28 04:17:46,147 | test | INFO | Thread-64 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection YPij creation 2020-06-28 04:17:48,512 | test | INFO | Thread-64 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection collection1 creation 2020-06-28 04:17:50,825 | test | INFO | Thread-64 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection collection2 creation 2020-06-28 04:17:53,752 | test | INFO | Thread-70 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection PQiY69pi2i2ZZCDioY-xNjv creation 2020-06-28 04:17:56,111 | test | INFO | Thread-70 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection cnXMRgTEwSnPmkpT creation 2020-06-28 04:17:58,434 | test | INFO | Thread-70 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection UAeRQZFpU creation 2020-06-28 04:18:00,765 | test | INFO | Thread-70 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection mVAOE2FbXeDQtMVbJC creation 2020-06-28 04:18:03,121 | test | INFO | Thread-70 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection 9F2kL31bDBh7hgpiWj1QrZhvcr787 creation 2020-06-28 04:18:05,124 | test | INFO | MainThread | [bucket_ready_functions:wait_for_collection_creation_to_complete:2152] Waiting for all collections to be created 2020-06-28 04:18:05,611 | test | INFO | MainThread | [table_view:display:72] Bucket statistics +---------+-----------+----------+------------+-----+-------+------------+-----------+------------+ | Bucket | Type | Replicas | Durability | TTL | Items | RAM Quota | RAM Used | Disk Used | +---------+-----------+----------+------------+-----+-------+------------+-----------+------------+ | bucket1 | membase | 3 | none | 0 | 0 | 4194304000 | 399493472 | 293114488 | | bucket2 | ephemeral | 3 | none | 0 | 0 | 524288000 | 69688944 | 170 | | default | membase | 3 | none | 0 | 0 | 7864320000 | 787294408 | 1141941625 | +---------+-----------+----------+------------+-----+-------+------------+-----------+------------+ 2020-06-28 04:18:05,612 | test | INFO | MainThread | [collections_base:collection_setup:112] Creating required SDK clients for client_pool 2020-06-28 04:18:29,739 | test | INFO | MainThread | [common_lib:sleep:22] Sleep 10 seconds. Reason: MB-38497 2020-06-28 04:18:39,740 | test | INFO | MainThread | [bucket_ready_functions:perform_tasks_from_spec:4458] Performing scope/collection specific operations 2020-06-28 04:18:47,548 | test | INFO | pool-2-thread-12 | [table_view:display:72] Ops trend for bucket 'bucket2' +-------+----------------------------------------------------------------------------------------------------------+----------+ | Min | Trend | Max | +-------+----------------------------------------------------------------------------------------------------------+----------+ | 0.000 | **.........................................*....*......................................................X | 3781.962 | +-------+----------------------------------------------------------------------------------------------------------+----------+ 2020-06-28 04:18:47,641 | test | INFO | pool-2-thread-13 | [table_view:display:72] Ops trend for bucket 'default' +-------+----------------------------------------------------------------------------------------------------------+-----------+ | Min | Trend | Max | +-------+----------------------------------------------------------------------------------------------------------+-----------+ | 0.000 | **..........................................................*...................*......................X | 24910.803 | +-------+----------------------------------------------------------------------------------------------------------+-----------+ 2020-06-28 04:18:48,694 | test | INFO | pool-2-thread-14 | [table_view:display:72] Ops trend for bucket 'bucket1' +----------+---------------------------------------------------------------------------------------------------------+----------+ | Min | Trend | Max | +----------+---------------------------------------------------------------------------------------------------------+----------+ | 0.000 | **....................................................................................................X | 6597.208 | | 1075.924 | ...............*.........X | 1684.836 | | 0.000 | X | 0.000 | +----------+---------------------------------------------------------------------------------------------------------+----------+ 2020-06-28 04:18:48,825 | test | INFO | MainThread | [table_view:display:72] Cluster statistics +----------------+----------+-----------------+------------+------------+------------------------+-------------------+-----------------------+ | Node | Services | CPU_utilization | Mem_total | Mem_free | Swap_mem_used | Active / Replica | Version | +----------------+----------+-----------------+------------+------------+------------------------+-------------------+-----------------------+ | 172.23.98.196 | kv | 94.7236180905 | 3706343424 | 2436849664 | 0 / 3909087232 | 23498 / 70726 | 7.0.0-2477-enterprise | | 172.23.98.195 | kv | 91.959798995 | 3706343424 | 2540072960 | 115474432 / 3909087232 | 23249 / 67410 | 7.0.0-2477-enterprise | | 172.23.104.186 | kv | 93.9393939394 | 6078930944 | 4887187456 | 34983936 / 3758092288 | 17283 / 50529 | 7.0.0-2477-enterprise | | 172.23.120.201 | kv | 69.3717277487 | 3691520000 | 2590892032 | 31756288 / 3758092288 | 16756 / 51039 | 7.0.0-2477-enterprise | | 172.23.121.10 | kv | 99.4949494949 | 4219432960 | 3182792704 | 34078720 / 3758092288 | 17155 / 39287 | 7.0.0-2477-enterprise | +----------------+----------+-----------------+------------+------------+------------------------+-------------------+-----------------------+ 2020-06-28 04:19:03,068 | test | INFO | MainThread | [bucket_ready_functions:validate_docs_per_collections_all_buckets:4264] Validating collection stats and item counts 2020-06-28 04:19:03,312 | test | INFO | MainThread | [data_helper:direct_client:267] creating direct client 172.23.98.196:11210 bucket2 2020-06-28 04:19:06,512 | test | INFO | MainThread | [data_helper:direct_client:267] creating direct client 172.23.98.195:11210 bucket2 2020-06-28 04:19:09,657 | test | INFO | MainThread | [data_helper:direct_client:267] creating direct client 172.23.121.10:11210 bucket2 2020-06-28 04:19:12,283 | test | INFO | MainThread | [data_helper:direct_client:267] creating direct client 172.23.104.186:11210 bucket2 2020-06-28 04:19:15,280 | test | INFO | MainThread | [data_helper:direct_client:267] creating direct client 172.23.120.201:11210 bucket2 2020-06-28 04:19:17,895 | test | INFO | MainThread | [data_helper:direct_client:267] creating direct client 172.23.98.196:11210 bucket1 2020-06-28 04:19:21,017 | test | INFO | MainThread | [data_helper:direct_client:267] creating direct client 172.23.98.195:11210 bucket1 2020-06-28 04:19:23,719 | test | INFO | MainThread | [data_helper:direct_client:267] creating direct client 172.23.121.10:11210 bucket1 2020-06-28 04:19:26,746 | test | INFO | MainThread | [data_helper:direct_client:267] creating direct client 172.23.104.186:11210 bucket1 2020-06-28 04:19:29,280 | test | INFO | MainThread | [data_helper:direct_client:267] creating direct client 172.23.120.201:11210 bucket1 2020-06-28 04:19:32,276 | test | INFO | MainThread | [data_helper:direct_client:267] creating direct client 172.23.98.196:11210 default 2020-06-28 04:19:34,983 | test | INFO | MainThread | [data_helper:direct_client:267] creating direct client 172.23.98.195:11210 default 2020-06-28 04:19:39,260 | test | INFO | MainThread | [data_helper:direct_client:267] creating direct client 172.23.121.10:11210 default 2020-06-28 04:19:41,815 | test | INFO | MainThread | [data_helper:direct_client:267] creating direct client 172.23.104.186:11210 default 2020-06-28 04:19:44,368 | test | INFO | MainThread | [data_helper:direct_client:267] creating direct client 172.23.120.201:11210 default 2020-06-28 04:20:44,404 | test | INFO | MainThread | [table_view:display:72] Bucket statistics +---------+-----------+----------+------------+-----+--------+------------+-----------+------------+ | Bucket | Type | Replicas | Durability | TTL | Items | RAM Quota | RAM Used | Disk Used | +---------+-----------+----------+------------+-----+--------+------------+-----------+------------+ | bucket1 | membase | 3 | none | 0 | 15000 | 4194304000 | 456136832 | 884368297 | | bucket2 | ephemeral | 3 | none | 0 | 15000 | 524288000 | 85604064 | 170 | | default | membase | 3 | none | 0 | 125000 | 7864320000 | 715204416 | 1983322601 | +---------+-----------+----------+------------+-----+--------+------------+-----------+------------+ 2020-06-28 04:20:44,436 | test | INFO | MainThread | [basetestcase:log_setup_status:461] ========= CollectionBase setup complete for test #1 test_data_load_collections_with_graceful_failover_recovery ========= 2020-06-28 04:20:53,005 | test | INFO | MainThread | [collections_rebalance:load_collections_with_rebalance:535] Doing collection data load before graceful_failover_recovery 2020-06-28 04:20:53,010 | test | INFO | MainThread | [bucket_ready_functions:perform_tasks_from_spec:4458] Performing scope/collection specific operations 2020-06-28 04:20:53,688 | test | INFO | MainThread | [collections_rebalance:rebalance_operation:112] Starting rebalance operation of type : graceful_failover_recovery 2020-06-28 04:20:59,700 | test | INFO | pool-2-thread-28 | [table_view:display:72] Ops trend for bucket 'bucket2' +-------+---------------------------------------------------------------------------------------------------------+----------+ | Min | Trend | Max | +-------+---------------------------------------------------------------------------------------------------------+----------+ | 0.000 | *....*................................................................................................X | 2828.543 | +-------+---------------------------------------------------------------------------------------------------------+----------+ 2020-06-28 04:21:01,423 | test | INFO | pool-2-thread-27 | [table_view:display:72] Ops trend for bucket 'default' +-----------+---------------------------------------------------------------------------------------------------------+-----------+ | Min | Trend | Max | +-----------+---------------------------------------------------------------------------------------------------------+-----------+ | 0.000 | *...*.................................................................................................X | 20466.799 | | 14360.040 | .....................................................................X | 14360.040 | +-----------+---------------------------------------------------------------------------------------------------------+-----------+ 2020-06-28 04:21:01,726 | test | INFO | pool-2-thread-29 | [table_view:display:72] Ops trend for bucket 'bucket1' +---------+---------------------------------------------------------------------------------------------------------+----------+ | Min | Trend | Max | +---------+---------------------------------------------------------------------------------------------------------+----------+ | 0.000 | *........*............................................................................................X | 2888.999 | | 468.843 | ...............X | 468.843 | +---------+---------------------------------------------------------------------------------------------------------+----------+ 2020-06-28 04:22:54,456 | test | INFO | MainThread | [collections_rebalance:wait_for_failover_or_assert:84] 1 nodes failed over as expected in 0.0800001621246 seconds 2020-06-28 04:24:54,996 | test | INFO | MainThread | [collections_rebalance:wait_for_failover_or_assert:84] 2 nodes failed over as expected in 0.0840001106262 seconds 2020-06-28 04:24:54,996 | test | INFO | MainThread | [collections_rebalance:data_load_after_failover:63] Starting a sync data load after failover 2020-06-28 04:24:54,996 | test | INFO | MainThread | [bucket_ready_functions:perform_tasks_from_spec:4458] Performing scope/collection specific operations 2020-06-28 04:25:01,038 | test | INFO | pool-2-thread-13 | [table_view:display:72] Ops trend for bucket 'bucket2' +----------+---------------------------------------------------------------------------------------------------------+----------+ | Min | Trend | Max | +----------+---------------------------------------------------------------------------------------------------------+----------+ | 0.000 | *...............................*.....................................................................X | 4039.960 | | 2196.607 | .....................................................X | 2196.607 | +----------+---------------------------------------------------------------------------------------------------------+----------+ 2020-06-28 04:25:01,210 | test | INFO | pool-2-thread-5 | [table_view:display:72] Ops trend for bucket 'default' +-------+----------------------------------------------------------------------------------------------------------+-----------+ | Min | Trend | Max | +-------+----------------------------------------------------------------------------------------------------------+-----------+ | 0.000 | *.............*......................................................................*.................X | 21121.756 | +-------+----------------------------------------------------------------------------------------------------------+-----------+ 2020-06-28 04:25:02,384 | test | INFO | pool-2-thread-10 | [table_view:display:72] Ops trend for bucket 'bucket1' +----------+----------------------------------------------------------------------------------------------------------+----------+ | Min | Trend | Max | +----------+----------------------------------------------------------------------------------------------------------+----------+ | 0.000 | **....................................................*................................................X | 3569.860 | | 1803.393 | .................................................X | 1803.393 | +----------+----------------------------------------------------------------------------------------------------------+----------+ 2020-06-28 04:25:02,484 | test | WARNING | MainThread | [rest_client:get_nodes:1671] 172.23.104.186 - Node not part of cluster inactiveFailed 2020-06-28 04:25:02,486 | test | WARNING | MainThread | [rest_client:get_nodes:1671] 172.23.120.201 - Node not part of cluster inactiveFailed 2020-06-28 04:25:02,546 | test | WARNING | MainThread | [rest_client:get_nodes:1671] 172.23.104.186 - Node not part of cluster inactiveFailed 2020-06-28 04:25:02,546 | test | WARNING | MainThread | [rest_client:get_nodes:1671] 172.23.120.201 - Node not part of cluster inactiveFailed 2020-06-28 04:25:24,855 | test | INFO | pool-2-thread-22 | [table_view:display:72] Rebalance Overview +----------------+----------+--------------+ | Nodes | Services | Status | +----------------+----------+--------------+ | 172.23.98.196 | kv | Cluster node | | 172.23.98.195 | kv | Cluster node | | 172.23.104.186 | kv | Cluster node | | 172.23.120.201 | kv | Cluster node | | 172.23.121.10 | kv | Cluster node | +----------------+----------+--------------+ 2020-06-28 04:25:24,878 | test | INFO | pool-2-thread-22 | [task:check:317] Rebalance - status: running, progress: 0 2020-06-28 04:25:30,009 | test | INFO | pool-2-thread-22 | [task:check:317] Rebalance - status: running, progress: 0 2020-06-28 04:25:35,085 | test | INFO | pool-2-thread-22 | [task:check:317] Rebalance - status: running, progress: 0 2020-06-28 04:25:40,153 | test | INFO | pool-2-thread-22 | [task:check:317] Rebalance - status: running, progress: 0 Build was aborted Aborted by Balakumaran G Archiving artifacts Recording test results ERROR: Step ‘Publish JUnit test result report’ failed: None of the test reports contained any result [description-setter] Description set: [BFA] Scanning build for known causes... [BFA] No failure causes found [BFA] Done. 0s Notifying upstream projects of job completion temp_durability is disabled. Triggering skipped Finished: ABORTED