Started by user vikas chaudhary [EnvInject] - Loading node environment variables. Building remotely on slv-s63704-32g-12c (P0 jython_slave host_management PY3P0) in workspace /data/workspace/dev_testbed_blr3 > /usr/bin/git rev-parse --is-inside-work-tree # timeout=10 Fetching changes from the remote Git repository > /usr/bin/git config remote.origin.url https://github.com/couchbase/testrunner/ # timeout=10 Fetching upstream changes from https://github.com/couchbase/testrunner/ > /usr/bin/git --version # timeout=10 > /usr/bin/git fetch --tags --progress https://github.com/couchbase/testrunner/ +refs/heads/*:refs/remotes/origin/* > /usr/bin/git rev-parse refs/remotes/origin/master^{commit} # timeout=10 > /usr/bin/git rev-parse refs/remotes/origin/origin/master^{commit} # timeout=10 Checking out Revision 794925e2c9d3135b08ed0473ec17662a077f3fa5 (refs/remotes/origin/master) > /usr/bin/git config core.sparsecheckout # timeout=10 > /usr/bin/git checkout -f 794925e2c9d3135b08ed0473ec17662a077f3fa5 > /usr/bin/git rev-list 794925e2c9d3135b08ed0473ec17662a077f3fa5 # timeout=10 [dev_testbed_blr3] $ /bin/sh -xe /tmp/jenkins7455741916493202211.sh + git checkout master Switched to branch 'master' M java_sdk_client + git reset --hard origin/master HEAD is now at 794925e remove single quote in bucket name. + git pull origin master From https://github.com/couchbase/testrunner * branch master -> FETCH_HEAD Already up-to-date. + '[' None '!=' None ']' + service docker start Redirecting to /bin/systemctl start docker.service + git submodule init + git submodule update --init --force --remote Submodule path 'java_sdk_client': checked out '8ae805783999ba5069f66511449402eda9d73931' + '[' -f /etc/redhat-release ']' + yum install -y maven Loaded plugins: fastestmirror Loading mirror speeds from cached hostfile * base: centos.mirror.ndchost.com * extras: mirrors.xtom.com * updates: mirror.sjc02.svwh.net Package maven-3.0.5-17.el7.noarch already installed and latest version Nothing to do + '[' -f /etc/lsb-release ']' + chmod 777 java_sdk_client/collections/configure.sh + mvn -f java_sdk_client/collections/pom.xml clean install OpenJDK 64-Bit Server VM warning: ignoring option MaxPermSize=512m; support was removed in 8.0 [INFO] Scanning for projects... [INFO] [INFO] ------------------------------------------------------------------------ [INFO] Building javaclient 1.0-SNAPSHOT [INFO] ------------------------------------------------------------------------ [INFO] [INFO] --- maven-clean-plugin:2.4.1:clean (default-clean) @ javaclient --- [INFO] Deleting /data/workspace/dev_testbed_blr3/java_sdk_client/collections/target [INFO] [INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ javaclient --- [WARNING] Using platform encoding (UTF-8 actually) to copy filtered resources, i.e. build is platform dependent! [INFO] Copying 1 resource [INFO] [INFO] --- maven-compiler-plugin:2.3.1:compile (default-compile) @ javaclient --- [WARNING] File encoding has not been set, using platform encoding UTF-8, i.e. build is platform dependent! [INFO] Compiling 19 source files to /data/workspace/dev_testbed_blr3/java_sdk_client/collections/target/classes [INFO] [INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ javaclient --- [WARNING] Using platform encoding (UTF-8 actually) to copy filtered resources, i.e. build is platform dependent! [INFO] skip non existing resourceDirectory /data/workspace/dev_testbed_blr3/java_sdk_client/collections/src/test/resources [INFO] [INFO] --- maven-compiler-plugin:2.3.1:testCompile (default-testCompile) @ javaclient --- [INFO] No sources to compile [INFO] [INFO] --- maven-surefire-plugin:2.10:test (default-test) @ javaclient --- [INFO] No tests to run. [INFO] Surefire report directory: /data/workspace/dev_testbed_blr3/java_sdk_client/collections/target/surefire-reports ------------------------------------------------------- T E S T S ------------------------------------------------------- Results : Tests run: 0, Failures: 0, Errors: 0, Skipped: 0 [INFO] [INFO] --- maven-dependency-plugin:2.1:copy-dependencies (copy-dependencies) @ javaclient --- [INFO] Copying core-io-2.1.3.jar to /data/workspace/dev_testbed_blr3/java_sdk_client/collections/target/javaclient/lib/core-io-2.1.3.jar [INFO] Copying java-client-3.1.3.jar to /data/workspace/dev_testbed_blr3/java_sdk_client/collections/target/javaclient/lib/java-client-3.1.3.jar [INFO] Copying jackson-annotations-2.10.3.jar to /data/workspace/dev_testbed_blr3/java_sdk_client/collections/target/javaclient/lib/jackson-annotations-2.10.3.jar [INFO] Copying jackson-core-2.10.3.jar to /data/workspace/dev_testbed_blr3/java_sdk_client/collections/target/javaclient/lib/jackson-core-2.10.3.jar [INFO] Copying jackson-databind-2.10.3.jar to /data/workspace/dev_testbed_blr3/java_sdk_client/collections/target/javaclient/lib/jackson-databind-2.10.3.jar [INFO] Copying javafaker-1.0.2.jar to /data/workspace/dev_testbed_blr3/java_sdk_client/collections/target/javaclient/lib/javafaker-1.0.2.jar [INFO] Copying generex-1.0.2.jar to /data/workspace/dev_testbed_blr3/java_sdk_client/collections/target/javaclient/lib/generex-1.0.2.jar [INFO] Copying json-simple-1.1.1.jar to /data/workspace/dev_testbed_blr3/java_sdk_client/collections/target/javaclient/lib/json-simple-1.1.1.jar [INFO] Copying commons-codec-1.11.jar to /data/workspace/dev_testbed_blr3/java_sdk_client/collections/target/javaclient/lib/commons-codec-1.11.jar [INFO] Copying commons-logging-1.2.jar to /data/workspace/dev_testbed_blr3/java_sdk_client/collections/target/javaclient/lib/commons-logging-1.2.jar [INFO] Copying automaton-1.11-8.jar to /data/workspace/dev_testbed_blr3/java_sdk_client/collections/target/javaclient/lib/automaton-1.11-8.jar [INFO] Copying reactor-core-3.4.3.jar to /data/workspace/dev_testbed_blr3/java_sdk_client/collections/target/javaclient/lib/reactor-core-3.4.3.jar [INFO] Copying rxjava-2.2.0.jar to /data/workspace/dev_testbed_blr3/java_sdk_client/collections/target/javaclient/lib/rxjava-2.2.0.jar [INFO] Copying junit-4.12.jar to /data/workspace/dev_testbed_blr3/java_sdk_client/collections/target/javaclient/lib/junit-4.12.jar [INFO] Copying log4j-1.2.17.jar to /data/workspace/dev_testbed_blr3/java_sdk_client/collections/target/javaclient/lib/log4j-1.2.17.jar [INFO] Copying argparse4j-0.8.1.jar to /data/workspace/dev_testbed_blr3/java_sdk_client/collections/target/javaclient/lib/argparse4j-0.8.1.jar [INFO] Copying commons-lang3-3.5.jar to /data/workspace/dev_testbed_blr3/java_sdk_client/collections/target/javaclient/lib/commons-lang3-3.5.jar [INFO] Copying httpclient-4.5.12.jar to /data/workspace/dev_testbed_blr3/java_sdk_client/collections/target/javaclient/lib/httpclient-4.5.12.jar [INFO] Copying httpcore-4.4.13.jar to /data/workspace/dev_testbed_blr3/java_sdk_client/collections/target/javaclient/lib/httpcore-4.4.13.jar [INFO] Copying hamcrest-core-1.3.jar to /data/workspace/dev_testbed_blr3/java_sdk_client/collections/target/javaclient/lib/hamcrest-core-1.3.jar [INFO] Copying reactive-streams-1.0.2.jar to /data/workspace/dev_testbed_blr3/java_sdk_client/collections/target/javaclient/lib/reactive-streams-1.0.2.jar [INFO] Copying slf4j-api-1.7.30.jar to /data/workspace/dev_testbed_blr3/java_sdk_client/collections/target/javaclient/lib/slf4j-api-1.7.30.jar [INFO] Copying slf4j-log4j12-1.7.30.jar to /data/workspace/dev_testbed_blr3/java_sdk_client/collections/target/javaclient/lib/slf4j-log4j12-1.7.30.jar [INFO] Copying snakeyaml-1.23-android.jar to /data/workspace/dev_testbed_blr3/java_sdk_client/collections/target/javaclient/lib/snakeyaml-1.23-android.jar [INFO] [INFO] --- maven-resources-plugin:2.6:copy-resources (copy-resources) @ javaclient --- [WARNING] File encoding has not been set, using platform encoding UTF-8, i.e. build is platform dependent! [WARNING] Using platform encoding (UTF-8 actually) to copy filtered resources, i.e. build is platform dependent! [INFO] skip non existing resourceDirectory /data/workspace/dev_testbed_blr3/java_sdk_client/collections/resources [INFO] [INFO] --- maven-jar-plugin:3.2.0:jar (default-jar) @ javaclient --- [INFO] Building jar: /data/workspace/dev_testbed_blr3/java_sdk_client/collections/target/javaclient/javaclient.jar [INFO] [INFO] --- maven-install-plugin:2.3.1:install (default-install) @ javaclient --- [INFO] Installing /data/workspace/dev_testbed_blr3/java_sdk_client/collections/target/javaclient/javaclient.jar to /root/.m2/repository/com/couchbase/javaclient/javaclient/1.0-SNAPSHOT/javaclient-1.0-SNAPSHOT.jar [INFO] Installing /data/workspace/dev_testbed_blr3/java_sdk_client/collections/pom.xml to /root/.m2/repository/com/couchbase/javaclient/javaclient/1.0-SNAPSHOT/javaclient-1.0-SNAPSHOT.pom [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 2.151s [INFO] Finished at: Wed Apr 28 01:42:48 PDT 2021 [INFO] Final Memory: 24M/480M [INFO] ------------------------------------------------------------------------ + docker build -t jsc java_sdk_client/collections Sending build context to Docker daemon 22.77 MB Step 1/7 : FROM adoptopenjdk/openjdk12:latest ---> 38066eacb3b2 Step 2/7 : WORKDIR / ---> Using cache ---> 2312a6cb2b36 Step 3/7 : ADD target/javaclient/javaclient.jar /tmp/ ---> f2746c967098 Removing intermediate container 3b929feeb92c Step 4/7 : ADD target/javaclient/lib /tmp/lib/ ---> 626a199358e4 Removing intermediate container 23f089a93e5b Step 5/7 : ADD configure.sh /tmp ---> 59183c8027dd Removing intermediate container 6ca3bd0a36b7 Step 6/7 : EXPOSE 8080 ---> Running in 1c1b95e6ba75 ---> f335512b2943 Removing intermediate container 1c1b95e6ba75 Step 7/7 : ENTRYPOINT /tmp/configure.sh ---> Running in 5fac1c84ccfd ---> fdb7185fb78e Removing intermediate container 5fac1c84ccfd Successfully built fdb7185fb78e [dev_testbed_blr3] $ /bin/sh -xe /tmp/jenkins3178125861739116006.sh + echo '[global] username:root password:couchbase port:8091 index_port:9102 n1ql_port:8093 fts_port:8094 eventing_port:8096 [membase] rest_username:Administrator rest_password:password [servers] 1:_1 2:_2 3:_3 4:_4 5:_5 6:_6 7:_7 8:_8 #9:_9 #10:_10 #11:_11 #12:_12 [_1] ip:172.23.106.64 [_2] ip:172.23.106.67 [_3] ip:172.23.106.73 [_4] ip:172.23.106.74 [_5] ip:172.23.106.75 [_6] ip:172.23.106.76 [_7] ip:172.23.106.77 [_8] ip:172.23.106.78 [_9] ip:172.23.106.79 [_10] ip:172.23.106.80 [_11] ip:172.23.106.81 services:kv' + python3 scripts/new_install.py -i /tmp/win10-bucket-ops.ini -p timeout=1800,skip_local_download=False,get-cbcollect-info=True,version=7.0.0-5046,product=cb,debug_logs=True,url= 2021-04-28 01:42:49,938 - root - WARNING - URL: is not valid, will use version to locate build 2021-04-28 01:42:49,939 - root - INFO - SSH Connecting to 172.23.106.64 with username:root, attempt#1 of 5 2021-04-28 01:42:50,046 - root - INFO - SSH Connected to 172.23.106.64 as root 2021-04-28 01:42:50,308 - root - INFO - os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-04-28 01:42:50,630 - root - INFO - extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-04-28 01:42:50,632 - root - INFO - SSH Connecting to 172.23.106.67 with username:root, attempt#1 of 5 2021-04-28 01:42:50,736 - root - INFO - SSH Connected to 172.23.106.67 as root 2021-04-28 01:42:51,005 - root - INFO - os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-04-28 01:42:51,313 - root - INFO - extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-04-28 01:42:51,314 - root - INFO - SSH Connecting to 172.23.106.73 with username:root, attempt#1 of 5 2021-04-28 01:42:51,420 - root - INFO - SSH Connected to 172.23.106.73 as root 2021-04-28 01:42:51,713 - root - INFO - os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-04-28 01:42:52,031 - root - INFO - extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-04-28 01:42:52,032 - root - INFO - SSH Connecting to 172.23.106.74 with username:root, attempt#1 of 5 2021-04-28 01:42:52,138 - root - INFO - SSH Connected to 172.23.106.74 as root 2021-04-28 01:42:52,415 - root - INFO - os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-04-28 01:42:52,719 - root - INFO - extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-04-28 01:42:52,721 - root - INFO - SSH Connecting to 172.23.106.75 with username:root, attempt#1 of 5 2021-04-28 01:42:52,826 - root - INFO - SSH Connected to 172.23.106.75 as root 2021-04-28 01:42:53,099 - root - INFO - os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-04-28 01:42:53,406 - root - INFO - extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-04-28 01:42:53,408 - root - INFO - SSH Connecting to 172.23.106.76 with username:root, attempt#1 of 5 2021-04-28 01:42:53,511 - root - INFO - SSH Connected to 172.23.106.76 as root 2021-04-28 01:42:53,779 - root - INFO - os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-04-28 01:42:54,084 - root - INFO - extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-04-28 01:42:54,085 - root - INFO - SSH Connecting to 172.23.106.77 with username:root, attempt#1 of 5 2021-04-28 01:42:54,191 - root - INFO - SSH Connected to 172.23.106.77 as root 2021-04-28 01:42:54,476 - root - INFO - os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-04-28 01:42:54,779 - root - INFO - extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-04-28 01:42:54,780 - root - INFO - SSH Connecting to 172.23.106.78 with username:root, attempt#1 of 5 2021-04-28 01:42:54,887 - root - INFO - SSH Connected to 172.23.106.78 as root 2021-04-28 01:42:55,163 - root - INFO - os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-04-28 01:42:55,472 - root - INFO - extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-04-28 01:42:55,473 - root - INFO - SSH Connecting to 172.23.106.64 with username:root, attempt#1 of 5 2021-04-28 01:42:55,573 - root - INFO - SSH Connected to 172.23.106.64 as root 2021-04-28 01:42:55,829 - root - INFO - os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-04-28 01:42:56,157 - root - INFO - extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-04-28 01:42:56,159 - root - INFO - SSH Connecting to 172.23.106.67 with username:root, attempt#1 of 5 2021-04-28 01:42:56,261 - root - INFO - SSH Connected to 172.23.106.67 as root 2021-04-28 01:42:56,528 - root - INFO - os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-04-28 01:42:56,839 - root - INFO - extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-04-28 01:42:56,840 - root - INFO - SSH Connecting to 172.23.106.73 with username:root, attempt#1 of 5 2021-04-28 01:42:56,941 - root - INFO - SSH Connected to 172.23.106.73 as root 2021-04-28 01:42:57,217 - root - INFO - os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-04-28 01:42:57,540 - root - INFO - extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-04-28 01:42:57,541 - root - INFO - SSH Connecting to 172.23.106.74 with username:root, attempt#1 of 5 2021-04-28 01:42:57,644 - root - INFO - SSH Connected to 172.23.106.74 as root 2021-04-28 01:42:57,900 - root - INFO - os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-04-28 01:42:58,203 - root - INFO - extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-04-28 01:42:58,205 - root - INFO - SSH Connecting to 172.23.106.75 with username:root, attempt#1 of 5 2021-04-28 01:42:58,305 - root - INFO - SSH Connected to 172.23.106.75 as root 2021-04-28 01:42:58,559 - root - INFO - os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-04-28 01:42:58,863 - root - INFO - extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-04-28 01:42:58,865 - root - INFO - SSH Connecting to 172.23.106.76 with username:root, attempt#1 of 5 2021-04-28 01:42:58,968 - root - INFO - SSH Connected to 172.23.106.76 as root 2021-04-28 01:42:59,226 - root - INFO - os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-04-28 01:42:59,534 - root - INFO - extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-04-28 01:42:59,535 - root - INFO - SSH Connecting to 172.23.106.77 with username:root, attempt#1 of 5 2021-04-28 01:42:59,637 - root - INFO - SSH Connected to 172.23.106.77 as root 2021-04-28 01:42:59,895 - root - INFO - os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-04-28 01:43:00,199 - root - INFO - extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-04-28 01:43:00,201 - root - INFO - SSH Connecting to 172.23.106.78 with username:root, attempt#1 of 5 2021-04-28 01:43:00,304 - root - INFO - SSH Connected to 172.23.106.78 as root 2021-04-28 01:43:00,566 - root - INFO - os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-04-28 01:43:00,878 - root - INFO - extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-04-28 01:43:00,878 - root - INFO - Trying to check is this url alive: http://172.23.126.166/builds/latestbuilds/couchbase-server/cheshire-cat/5046/couchbase-server-enterprise-7.0.0-5046-centos7.x86_64.rpm 2021-04-28 01:43:00,881 - root - INFO - This url http://172.23.126.166/builds/latestbuilds/couchbase-server/cheshire-cat/5046/couchbase-server-enterprise-7.0.0-5046-centos7.x86_64.rpm is live 2021-04-28 01:43:00,882 - root - INFO - Trying to check is this url alive: http://172.23.126.166/builds/latestbuilds/couchbase-server/cheshire-cat/5046/couchbase-server-enterprise-7.0.0-5046-centos7.x86_64.rpm 2021-04-28 01:43:00,884 - root - INFO - This url http://172.23.126.166/builds/latestbuilds/couchbase-server/cheshire-cat/5046/couchbase-server-enterprise-7.0.0-5046-centos7.x86_64.rpm is live 2021-04-28 01:43:00,884 - root - INFO - Trying to check is this url alive: http://172.23.126.166/builds/latestbuilds/couchbase-server/cheshire-cat/5046/couchbase-server-enterprise-7.0.0-5046-centos7.x86_64.rpm 2021-04-28 01:43:00,886 - root - INFO - This url http://172.23.126.166/builds/latestbuilds/couchbase-server/cheshire-cat/5046/couchbase-server-enterprise-7.0.0-5046-centos7.x86_64.rpm is live 2021-04-28 01:43:00,886 - root - INFO - Trying to check is this url alive: http://172.23.126.166/builds/latestbuilds/couchbase-server/cheshire-cat/5046/couchbase-server-enterprise-7.0.0-5046-centos7.x86_64.rpm 2021-04-28 01:43:00,887 - root - INFO - This url http://172.23.126.166/builds/latestbuilds/couchbase-server/cheshire-cat/5046/couchbase-server-enterprise-7.0.0-5046-centos7.x86_64.rpm is live 2021-04-28 01:43:00,887 - root - INFO - Trying to check is this url alive: http://172.23.126.166/builds/latestbuilds/couchbase-server/cheshire-cat/5046/couchbase-server-enterprise-7.0.0-5046-centos7.x86_64.rpm 2021-04-28 01:43:00,889 - root - INFO - This url http://172.23.126.166/builds/latestbuilds/couchbase-server/cheshire-cat/5046/couchbase-server-enterprise-7.0.0-5046-centos7.x86_64.rpm is live 2021-04-28 01:43:00,889 - root - INFO - Trying to check is this url alive: http://172.23.126.166/builds/latestbuilds/couchbase-server/cheshire-cat/5046/couchbase-server-enterprise-7.0.0-5046-centos7.x86_64.rpm 2021-04-28 01:43:00,890 - root - INFO - This url http://172.23.126.166/builds/latestbuilds/couchbase-server/cheshire-cat/5046/couchbase-server-enterprise-7.0.0-5046-centos7.x86_64.rpm is live 2021-04-28 01:43:00,890 - root - INFO - Trying to check is this url alive: http://172.23.126.166/builds/latestbuilds/couchbase-server/cheshire-cat/5046/couchbase-server-enterprise-7.0.0-5046-centos7.x86_64.rpm 2021-04-28 01:43:00,891 - root - INFO - This url http://172.23.126.166/builds/latestbuilds/couchbase-server/cheshire-cat/5046/couchbase-server-enterprise-7.0.0-5046-centos7.x86_64.rpm is live 2021-04-28 01:43:00,891 - root - INFO - Trying to check is this url alive: http://172.23.126.166/builds/latestbuilds/couchbase-server/cheshire-cat/5046/couchbase-server-enterprise-7.0.0-5046-centos7.x86_64.rpm 2021-04-28 01:43:00,892 - root - INFO - This url http://172.23.126.166/builds/latestbuilds/couchbase-server/cheshire-cat/5046/couchbase-server-enterprise-7.0.0-5046-centos7.x86_64.rpm is live 2021-04-28 01:43:00,892 - root - INFO - Downloading build binary to /tmp/couchbase-server-enterprise-7.0.0-5046-centos7.x86_64.rpm.. 2021-04-28 01:43:00,900 - root - INFO - Copying /tmp/couchbase-server-enterprise-7.0.0-5046-centos7.x86_64.rpm to 172.23.106.64 2021-04-28 01:43:00,901 - root - INFO - Copying /tmp/couchbase-server-enterprise-7.0.0-5046-centos7.x86_64.rpm to 172.23.106.67 2021-04-28 01:43:00,902 - root - INFO - Copying /tmp/couchbase-server-enterprise-7.0.0-5046-centos7.x86_64.rpm to 172.23.106.73 2021-04-28 01:43:00,902 - root - INFO - Copying /tmp/couchbase-server-enterprise-7.0.0-5046-centos7.x86_64.rpm to 172.23.106.74 2021-04-28 01:43:00,903 - root - INFO - Copying /tmp/couchbase-server-enterprise-7.0.0-5046-centos7.x86_64.rpm to 172.23.106.75 2021-04-28 01:43:00,904 - root - INFO - Copying /tmp/couchbase-server-enterprise-7.0.0-5046-centos7.x86_64.rpm to 172.23.106.76 2021-04-28 01:43:00,904 - root - INFO - Copying /tmp/couchbase-server-enterprise-7.0.0-5046-centos7.x86_64.rpm to 172.23.106.77 2021-04-28 01:43:00,904 - root - INFO - Copying /tmp/couchbase-server-enterprise-7.0.0-5046-centos7.x86_64.rpm to 172.23.106.78 2021-04-28 01:45:25,984 - root - INFO - Done copying build to 172.23.106.78. 2021-04-28 01:45:25,995 - root - INFO - Done copying build to 172.23.106.74. 2021-04-28 01:45:26,007 - root - INFO - Done copying build to 172.23.106.67. 2021-04-28 01:45:26,056 - root - INFO - Done copying build to 172.23.106.64. 2021-04-28 01:45:26,086 - root - INFO - Done copying build to 172.23.106.75. 2021-04-28 01:45:26,091 - root - INFO - Done copying build to 172.23.106.73. 2021-04-28 01:45:26,117 - root - INFO - Done copying build to 172.23.106.77. 2021-04-28 01:45:26,120 - root - INFO - Done copying build to 172.23.106.76. 2021-04-28 01:45:26,121 - root - INFO - running command.raw on 172.23.106.64: ls -lh /tmp/couchbase-server-enterprise-7.0.0-5046-centos7.x86_64.rpm 2021-04-28 01:45:26,142 - root - INFO - command executed successfully with root 2021-04-28 01:45:26,142 - root - INFO - running command.raw on 172.23.106.64: curl -I http://172.23.126.166/builds/latestbuilds/couchbase-server/cheshire-cat/5046/couchbase-server-enterprise-7.0.0-5046-centos7.x86_64.rpm 2>&1 | grep Content-Length 2021-04-28 01:45:26,207 - root - INFO - command executed successfully with root 2021-04-28 01:45:26,207 - root - INFO - running command.raw on 172.23.106.64: cd /tmp/ && wc -c couchbase-server-enterprise-7.0.0-5046-centos7.x86_64.rpm 2021-04-28 01:45:26,264 - root - INFO - command executed successfully with root 2021-04-28 01:45:26,265 - root - INFO - running command.raw on 172.23.106.67: ls -lh /tmp/couchbase-server-enterprise-7.0.0-5046-centos7.x86_64.rpm 2021-04-28 01:45:26,286 - root - INFO - command executed successfully with root 2021-04-28 01:45:26,286 - root - INFO - running command.raw on 172.23.106.67: curl -I http://172.23.126.166/builds/latestbuilds/couchbase-server/cheshire-cat/5046/couchbase-server-enterprise-7.0.0-5046-centos7.x86_64.rpm 2>&1 | grep Content-Length 2021-04-28 01:45:26,347 - root - INFO - command executed successfully with root 2021-04-28 01:45:26,348 - root - INFO - running command.raw on 172.23.106.67: cd /tmp/ && wc -c couchbase-server-enterprise-7.0.0-5046-centos7.x86_64.rpm 2021-04-28 01:45:26,403 - root - INFO - command executed successfully with root 2021-04-28 01:45:26,403 - root - INFO - running command.raw on 172.23.106.73: ls -lh /tmp/couchbase-server-enterprise-7.0.0-5046-centos7.x86_64.rpm 2021-04-28 01:45:26,430 - root - INFO - command executed successfully with root 2021-04-28 01:45:26,431 - root - INFO - running command.raw on 172.23.106.73: curl -I http://172.23.126.166/builds/latestbuilds/couchbase-server/cheshire-cat/5046/couchbase-server-enterprise-7.0.0-5046-centos7.x86_64.rpm 2>&1 | grep Content-Length 2021-04-28 01:45:26,494 - root - INFO - command executed successfully with root 2021-04-28 01:45:26,494 - root - INFO - running command.raw on 172.23.106.73: cd /tmp/ && wc -c couchbase-server-enterprise-7.0.0-5046-centos7.x86_64.rpm 2021-04-28 01:45:26,550 - root - INFO - command executed successfully with root 2021-04-28 01:45:26,550 - root - INFO - running command.raw on 172.23.106.74: ls -lh /tmp/couchbase-server-enterprise-7.0.0-5046-centos7.x86_64.rpm 2021-04-28 01:45:26,571 - root - INFO - command executed successfully with root 2021-04-28 01:45:26,572 - root - INFO - running command.raw on 172.23.106.74: curl -I http://172.23.126.166/builds/latestbuilds/couchbase-server/cheshire-cat/5046/couchbase-server-enterprise-7.0.0-5046-centos7.x86_64.rpm 2>&1 | grep Content-Length 2021-04-28 01:45:26,633 - root - INFO - command executed successfully with root 2021-04-28 01:45:26,634 - root - INFO - running command.raw on 172.23.106.74: cd /tmp/ && wc -c couchbase-server-enterprise-7.0.0-5046-centos7.x86_64.rpm 2021-04-28 01:45:26,688 - root - INFO - command executed successfully with root 2021-04-28 01:45:26,688 - root - INFO - running command.raw on 172.23.106.75: ls -lh /tmp/couchbase-server-enterprise-7.0.0-5046-centos7.x86_64.rpm 2021-04-28 01:45:26,709 - root - INFO - command executed successfully with root 2021-04-28 01:45:26,709 - root - INFO - running command.raw on 172.23.106.75: curl -I http://172.23.126.166/builds/latestbuilds/couchbase-server/cheshire-cat/5046/couchbase-server-enterprise-7.0.0-5046-centos7.x86_64.rpm 2>&1 | grep Content-Length 2021-04-28 01:45:26,772 - root - INFO - command executed successfully with root 2021-04-28 01:45:26,772 - root - INFO - running command.raw on 172.23.106.75: cd /tmp/ && wc -c couchbase-server-enterprise-7.0.0-5046-centos7.x86_64.rpm 2021-04-28 01:45:26,826 - root - INFO - command executed successfully with root 2021-04-28 01:45:26,826 - root - INFO - running command.raw on 172.23.106.76: ls -lh /tmp/couchbase-server-enterprise-7.0.0-5046-centos7.x86_64.rpm 2021-04-28 01:45:26,846 - root - INFO - command executed successfully with root 2021-04-28 01:45:26,847 - root - INFO - running command.raw on 172.23.106.76: curl -I http://172.23.126.166/builds/latestbuilds/couchbase-server/cheshire-cat/5046/couchbase-server-enterprise-7.0.0-5046-centos7.x86_64.rpm 2>&1 | grep Content-Length 2021-04-28 01:45:26,909 - root - INFO - command executed successfully with root 2021-04-28 01:45:26,909 - root - INFO - running command.raw on 172.23.106.76: cd /tmp/ && wc -c couchbase-server-enterprise-7.0.0-5046-centos7.x86_64.rpm 2021-04-28 01:45:26,964 - root - INFO - command executed successfully with root 2021-04-28 01:45:26,964 - root - INFO - running command.raw on 172.23.106.77: ls -lh /tmp/couchbase-server-enterprise-7.0.0-5046-centos7.x86_64.rpm 2021-04-28 01:45:26,986 - root - INFO - command executed successfully with root 2021-04-28 01:45:26,986 - root - INFO - running command.raw on 172.23.106.77: curl -I http://172.23.126.166/builds/latestbuilds/couchbase-server/cheshire-cat/5046/couchbase-server-enterprise-7.0.0-5046-centos7.x86_64.rpm 2>&1 | grep Content-Length 2021-04-28 01:45:27,047 - root - INFO - command executed successfully with root 2021-04-28 01:45:27,047 - root - INFO - running command.raw on 172.23.106.77: cd /tmp/ && wc -c couchbase-server-enterprise-7.0.0-5046-centos7.x86_64.rpm 2021-04-28 01:45:27,103 - root - INFO - command executed successfully with root 2021-04-28 01:45:27,103 - root - INFO - running command.raw on 172.23.106.78: ls -lh /tmp/couchbase-server-enterprise-7.0.0-5046-centos7.x86_64.rpm 2021-04-28 01:45:27,125 - root - INFO - command executed successfully with root 2021-04-28 01:45:27,125 - root - INFO - running command.raw on 172.23.106.78: curl -I http://172.23.126.166/builds/latestbuilds/couchbase-server/cheshire-cat/5046/couchbase-server-enterprise-7.0.0-5046-centos7.x86_64.rpm 2>&1 | grep Content-Length 2021-04-28 01:45:27,188 - root - INFO - command executed successfully with root 2021-04-28 01:45:27,188 - root - INFO - running command.raw on 172.23.106.78: cd /tmp/ && wc -c couchbase-server-enterprise-7.0.0-5046-centos7.x86_64.rpm 2021-04-28 01:45:27,243 - root - INFO - command executed successfully with root 2021-04-28 01:45:27,243 - root - INFO - running command.raw on 172.23.106.64: systemctl stop couchbase-server; rpm -e couchbase-server; rm -rf /opt/couchbase; rm -rf /home/nonroot/opt/couchbase/ > /dev/null && echo 1 || echo 0 2021-04-28 01:45:27,244 - root - INFO - running command.raw on 172.23.106.67: systemctl stop couchbase-server; rpm -e couchbase-server; rm -rf /opt/couchbase; rm -rf /home/nonroot/opt/couchbase/ > /dev/null && echo 1 || echo 0 2021-04-28 01:45:27,245 - root - INFO - running command.raw on 172.23.106.73: systemctl stop couchbase-server; rpm -e couchbase-server; rm -rf /opt/couchbase; rm -rf /home/nonroot/opt/couchbase/ > /dev/null && echo 1 || echo 0 2021-04-28 01:45:27,246 - root - INFO - running command.raw on 172.23.106.74: systemctl stop couchbase-server; rpm -e couchbase-server; rm -rf /opt/couchbase; rm -rf /home/nonroot/opt/couchbase/ > /dev/null && echo 1 || echo 0 2021-04-28 01:45:27,247 - root - INFO - running command.raw on 172.23.106.75: systemctl stop couchbase-server; rpm -e couchbase-server; rm -rf /opt/couchbase; rm -rf /home/nonroot/opt/couchbase/ > /dev/null && echo 1 || echo 0 2021-04-28 01:45:27,248 - root - INFO - running command.raw on 172.23.106.76: systemctl stop couchbase-server; rpm -e couchbase-server; rm -rf /opt/couchbase; rm -rf /home/nonroot/opt/couchbase/ > /dev/null && echo 1 || echo 0 2021-04-28 01:45:27,248 - root - INFO - running command.raw on 172.23.106.77: systemctl stop couchbase-server; rpm -e couchbase-server; rm -rf /opt/couchbase; rm -rf /home/nonroot/opt/couchbase/ > /dev/null && echo 1 || echo 0 2021-04-28 01:45:27,250 - root - INFO - running command.raw on 172.23.106.78: systemctl stop couchbase-server; rpm -e couchbase-server; rm -rf /opt/couchbase; rm -rf /home/nonroot/opt/couchbase/ > /dev/null && echo 1 || echo 0 2021-04-28 01:45:29,822 - root - INFO - command executed with root but got an error ['warning: /opt/couchbase/var/lib/couchbase/ip_start saved as /opt/couchbase/var/lib/couchbase/ip_start.rpmsave', 'warning: file /opt/couchbase/var/lib/couchbase/ip: remove failed: No such file or directory', 'warning: /opt/couchbase/var/lib/couchbase/config/config.dat saved as /opt/couchbase/var/lib/couchbase/config/config.dat.rpmsave', 'warning: /opt/couchbase/etc/couchdb/local.ini saved as /opt ... 2021-04-28 01:45:29,902 - root - INFO - command executed with root but got an error ['warning: /opt/couchbase/var/lib/couchbase/ip_start saved as /opt/couchbase/var/lib/couchbase/ip_start.rpmsave', 'warning: file /opt/couchbase/var/lib/couchbase/ip: remove failed: No such file or directory', 'warning: /opt/couchbase/var/lib/couchbase/config/config.dat saved as /opt/couchbase/var/lib/couchbase/config/config.dat.rpmsave', 'warning: /opt/couchbase/etc/couchdb/local.ini saved as /opt ... 2021-04-28 01:45:29,972 - root - INFO - command executed with root but got an error ['warning: /opt/couchbase/var/lib/couchbase/ip_start saved as /opt/couchbase/var/lib/couchbase/ip_start.rpmsave', 'warning: file /opt/couchbase/var/lib/couchbase/ip: remove failed: No such file or directory', 'warning: /opt/couchbase/var/lib/couchbase/config/config.dat saved as /opt/couchbase/var/lib/couchbase/config/config.dat.rpmsave', 'warning: /opt/couchbase/etc/couchdb/local.ini saved as /opt ... 2021-04-28 01:45:30,079 - root - INFO - command executed with root but got an error ['warning: /opt/couchbase/var/lib/couchbase/ip_start saved as /opt/couchbase/var/lib/couchbase/ip_start.rpmsave', 'warning: file /opt/couchbase/var/lib/couchbase/ip: remove failed: No such file or directory', 'warning: /opt/couchbase/var/lib/couchbase/config/dist_cfg saved as /opt/couchbase/var/lib/couchbase/config/dist_cfg.rpmsave', 'warning: /opt/couchbase/var/lib/couchbase/config/config.dat sav ... 2021-04-28 01:45:30,168 - root - INFO - command executed with root but got an error ['warning: /opt/couchbase/var/lib/couchbase/ip_start saved as /opt/couchbase/var/lib/couchbase/ip_start.rpmsave', 'warning: file /opt/couchbase/var/lib/couchbase/ip: remove failed: No such file or directory', 'warning: /opt/couchbase/var/lib/couchbase/config/dist_cfg saved as /opt/couchbase/var/lib/couchbase/config/dist_cfg.rpmsave', 'warning: /opt/couchbase/var/lib/couchbase/config/config.dat sav ... 2021-04-28 01:45:30,251 - root - INFO - command executed with root but got an error ['warning: /opt/couchbase/var/lib/couchbase/ip_start saved as /opt/couchbase/var/lib/couchbase/ip_start.rpmsave', 'warning: file /opt/couchbase/var/lib/couchbase/ip: remove failed: No such file or directory', 'warning: /opt/couchbase/var/lib/couchbase/config/dist_cfg saved as /opt/couchbase/var/lib/couchbase/config/dist_cfg.rpmsave', 'warning: /opt/couchbase/var/lib/couchbase/config/config.dat sav ... 2021-04-28 01:45:30,262 - root - INFO - command executed with root but got an error ['warning: /opt/couchbase/var/lib/couchbase/ip_start saved as /opt/couchbase/var/lib/couchbase/ip_start.rpmsave', 'warning: file /opt/couchbase/var/lib/couchbase/ip: remove failed: No such file or directory', 'warning: /opt/couchbase/var/lib/couchbase/config/dist_cfg saved as /opt/couchbase/var/lib/couchbase/config/dist_cfg.rpmsave', 'warning: /opt/couchbase/var/lib/couchbase/config/config.dat sav ... 2021-04-28 01:45:30,923 - root - INFO - Done with uninstall on 172.23.106.74. 2021-04-28 01:45:30,923 - root - INFO - running command.raw on 172.23.106.74: yes | yum remove `rpm -qa | grep couchbase` 2021-04-28 01:45:30,995 - root - INFO - Done with uninstall on 172.23.106.73. 2021-04-28 01:45:30,995 - root - INFO - running command.raw on 172.23.106.73: yes | yum remove `rpm -qa | grep couchbase` 2021-04-28 01:45:31,027 - root - INFO - Done with uninstall on 172.23.106.67. 2021-04-28 01:45:31,028 - root - INFO - running command.raw on 172.23.106.67: yes | yum remove `rpm -qa | grep couchbase` 2021-04-28 01:45:31,171 - root - INFO - Done with uninstall on 172.23.106.75. 2021-04-28 01:45:31,172 - root - INFO - running command.raw on 172.23.106.75: yes | yum remove `rpm -qa | grep couchbase` 2021-04-28 01:45:31,213 - root - INFO - Done with uninstall on 172.23.106.77. 2021-04-28 01:45:31,213 - root - INFO - running command.raw on 172.23.106.77: yes | yum remove `rpm -qa | grep couchbase` 2021-04-28 01:45:31,325 - root - INFO - Done with uninstall on 172.23.106.76. 2021-04-28 01:45:31,325 - root - INFO - running command.raw on 172.23.106.76: yes | yum remove `rpm -qa | grep couchbase` 2021-04-28 01:45:31,327 - root - INFO - Done with uninstall on 172.23.106.78. 2021-04-28 01:45:31,327 - root - INFO - running command.raw on 172.23.106.78: yes | yum remove `rpm -qa | grep couchbase` 2021-04-28 01:45:31,459 - root - INFO - command executed with root but got an error ['warning: file /opt/couchbase/var/lib/couchbase/ip_start: remove failed: No such file or directory', 'warning: /opt/couchbase/var/lib/couchbase/ip saved as /opt/couchbase/var/lib/couchbase/ip.rpmsave', 'warning: /opt/couchbase/var/lib/couchbase/config/config.dat saved as /opt/couchbase/var/lib/couchbase/config/config.dat.rpmsave', 'warning: /opt/couchbase/etc/couchdb/local.ini saved as /opt/couch ... 2021-04-28 01:45:31,738 - root - INFO - command executed with root but got an error ['Error: Need to pass a list of pkgs to remove', ' Mini usage:', '', 'erase PACKAGE...', '', 'Remove a package or packages from your system', '', 'aliases: remove, autoremove, erase-n, erase-na, erase-nevra, autoremove-n, autoremove-na, autoremove-nevra, remove-n, remove-na, remove-nevra'] ... 2021-04-28 01:45:31,738 - root - INFO - Waiting 20s to remove previous yum repo on 172.23.106.74.. 2021-04-28 01:45:31,979 - root - INFO - command executed with root but got an error ['Error: Need to pass a list of pkgs to remove', ' Mini usage:', '', 'erase PACKAGE...', '', 'Remove a package or packages from your system', '', 'aliases: remove, autoremove, erase-n, erase-na, erase-nevra, autoremove-n, autoremove-na, autoremove-nevra, remove-n, remove-na, remove-nevra'] ... 2021-04-28 01:45:31,979 - root - INFO - Waiting 20s to remove previous yum repo on 172.23.106.75.. 2021-04-28 01:45:31,980 - root - INFO - command executed with root but got an error ['Error: Need to pass a list of pkgs to remove', ' Mini usage:', '', 'erase PACKAGE...', '', 'Remove a package or packages from your system', '', 'aliases: remove, autoremove, erase-n, erase-na, erase-nevra, autoremove-n, autoremove-na, autoremove-nevra, remove-n, remove-na, remove-nevra'] ... 2021-04-28 01:45:31,980 - root - INFO - Waiting 20s to remove previous yum repo on 172.23.106.73.. 2021-04-28 01:45:32,022 - root - INFO - command executed with root but got an error ['Error: Need to pass a list of pkgs to remove', ' Mini usage:', '', 'erase PACKAGE...', '', 'Remove a package or packages from your system', '', 'aliases: remove, autoremove, erase-n, erase-na, erase-nevra, autoremove-n, autoremove-na, autoremove-nevra, remove-n, remove-na, remove-nevra'] ... 2021-04-28 01:45:32,023 - root - INFO - Waiting 20s to remove previous yum repo on 172.23.106.67.. 2021-04-28 01:45:32,050 - root - INFO - command executed with root but got an error ['Error: Need to pass a list of pkgs to remove', ' Mini usage:', '', 'erase PACKAGE...', '', 'Remove a package or packages from your system', '', 'aliases: remove, autoremove, erase-n, erase-na, erase-nevra, autoremove-n, autoremove-na, autoremove-nevra, remove-n, remove-na, remove-nevra'] ... 2021-04-28 01:45:32,050 - root - INFO - Waiting 20s to remove previous yum repo on 172.23.106.77.. 2021-04-28 01:45:32,149 - root - INFO - command executed with root but got an error ['Error: Need to pass a list of pkgs to remove', ' Mini usage:', '', 'erase PACKAGE...', '', 'Remove a package or packages from your system', '', 'aliases: remove, autoremove, erase-n, erase-na, erase-nevra, autoremove-n, autoremove-na, autoremove-nevra, remove-n, remove-na, remove-nevra'] ... 2021-04-28 01:45:32,149 - root - INFO - Waiting 20s to remove previous yum repo on 172.23.106.76.. 2021-04-28 01:45:32,168 - root - INFO - command executed with root but got an error ['Error: Need to pass a list of pkgs to remove', ' Mini usage:', '', 'erase PACKAGE...', '', 'Remove a package or packages from your system', '', 'aliases: remove, autoremove, erase-n, erase-na, erase-nevra, autoremove-n, autoremove-na, autoremove-nevra, remove-n, remove-na, remove-nevra'] ... 2021-04-28 01:45:32,168 - root - INFO - Waiting 20s to remove previous yum repo on 172.23.106.78.. 2021-04-28 01:45:32,554 - root - INFO - Done with uninstall on 172.23.106.64. 2021-04-28 01:45:32,554 - root - INFO - running command.raw on 172.23.106.64: yes | yum remove `rpm -qa | grep couchbase` 2021-04-28 01:45:33,509 - root - INFO - command executed with root but got an error ['Error: Need to pass a list of pkgs to remove', ' Mini usage:', '', 'erase PACKAGE...', '', 'Remove a package or packages from your system', '', 'aliases: remove, autoremove, erase-n, erase-na, erase-nevra, autoremove-n, autoremove-na, autoremove-nevra, remove-n, remove-na, remove-nevra'] ... 2021-04-28 01:45:33,510 - root - INFO - Waiting 20s to remove previous yum repo on 172.23.106.64.. 2021-04-28 01:45:51,747 - root - INFO - running command.raw on 172.23.106.74: /sbin/sysctl vm.swappiness=0; echo never > /sys/kernel/mm/transparent_hugepage/enabled; echo never > /sys/kernel/mm/transparent_hugepage/defrag; 2021-04-28 01:45:51,766 - root - INFO - command executed successfully with root 2021-04-28 01:45:51,766 - root - INFO - running command.raw on 172.23.106.74: yes | yum localinstall -y /tmp/couchbase-server-enterprise-7.0.0-5046-centos7.x86_64.rpm > /dev/null && echo 1 || echo 0 2021-04-28 01:45:51,999 - root - INFO - running command.raw on 172.23.106.75: /sbin/sysctl vm.swappiness=0; echo never > /sys/kernel/mm/transparent_hugepage/enabled; echo never > /sys/kernel/mm/transparent_hugepage/defrag; 2021-04-28 01:45:52,001 - root - INFO - running command.raw on 172.23.106.73: /sbin/sysctl vm.swappiness=0; echo never > /sys/kernel/mm/transparent_hugepage/enabled; echo never > /sys/kernel/mm/transparent_hugepage/defrag; 2021-04-28 01:45:52,019 - root - INFO - command executed successfully with root 2021-04-28 01:45:52,019 - root - INFO - running command.raw on 172.23.106.75: yes | yum localinstall -y /tmp/couchbase-server-enterprise-7.0.0-5046-centos7.x86_64.rpm > /dev/null && echo 1 || echo 0 2021-04-28 01:45:52,021 - root - INFO - command executed successfully with root 2021-04-28 01:45:52,022 - root - INFO - running command.raw on 172.23.106.73: yes | yum localinstall -y /tmp/couchbase-server-enterprise-7.0.0-5046-centos7.x86_64.rpm > /dev/null && echo 1 || echo 0 2021-04-28 01:45:52,031 - root - INFO - running command.raw on 172.23.106.67: /sbin/sysctl vm.swappiness=0; echo never > /sys/kernel/mm/transparent_hugepage/enabled; echo never > /sys/kernel/mm/transparent_hugepage/defrag; 2021-04-28 01:45:52,050 - root - INFO - command executed successfully with root 2021-04-28 01:45:52,050 - root - INFO - running command.raw on 172.23.106.67: yes | yum localinstall -y /tmp/couchbase-server-enterprise-7.0.0-5046-centos7.x86_64.rpm > /dev/null && echo 1 || echo 0 2021-04-28 01:45:52,060 - root - INFO - running command.raw on 172.23.106.77: /sbin/sysctl vm.swappiness=0; echo never > /sys/kernel/mm/transparent_hugepage/enabled; echo never > /sys/kernel/mm/transparent_hugepage/defrag; 2021-04-28 01:45:52,079 - root - INFO - command executed successfully with root 2021-04-28 01:45:52,079 - root - INFO - running command.raw on 172.23.106.77: yes | yum localinstall -y /tmp/couchbase-server-enterprise-7.0.0-5046-centos7.x86_64.rpm > /dev/null && echo 1 || echo 0 2021-04-28 01:45:52,169 - root - INFO - running command.raw on 172.23.106.76: /sbin/sysctl vm.swappiness=0; echo never > /sys/kernel/mm/transparent_hugepage/enabled; echo never > /sys/kernel/mm/transparent_hugepage/defrag; 2021-04-28 01:45:52,188 - root - INFO - command executed successfully with root 2021-04-28 01:45:52,188 - root - INFO - running command.raw on 172.23.106.76: yes | yum localinstall -y /tmp/couchbase-server-enterprise-7.0.0-5046-centos7.x86_64.rpm > /dev/null && echo 1 || echo 0 2021-04-28 01:45:52,189 - root - INFO - running command.raw on 172.23.106.78: /sbin/sysctl vm.swappiness=0; echo never > /sys/kernel/mm/transparent_hugepage/enabled; echo never > /sys/kernel/mm/transparent_hugepage/defrag; 2021-04-28 01:45:52,208 - root - INFO - command executed successfully with root 2021-04-28 01:45:52,208 - root - INFO - running command.raw on 172.23.106.78: yes | yum localinstall -y /tmp/couchbase-server-enterprise-7.0.0-5046-centos7.x86_64.rpm > /dev/null && echo 1 || echo 0 2021-04-28 01:45:53,530 - root - INFO - running command.raw on 172.23.106.64: /sbin/sysctl vm.swappiness=0; echo never > /sys/kernel/mm/transparent_hugepage/enabled; echo never > /sys/kernel/mm/transparent_hugepage/defrag; 2021-04-28 01:45:53,551 - root - INFO - command executed successfully with root 2021-04-28 01:45:53,551 - root - INFO - running command.raw on 172.23.106.64: yes | yum localinstall -y /tmp/couchbase-server-enterprise-7.0.0-5046-centos7.x86_64.rpm > /dev/null && echo 1 || echo 0 2021-04-28 01:46:52,325 - root - INFO - command executed with root but got an error ['Warning: RPMDB altered outside of yum.'] ... 2021-04-28 01:46:52,325 - root - INFO - running command.raw on 172.23.106.74: systemctl -q is-active couchbase-server && echo 1 || echo 0 2021-04-28 01:46:52,355 - root - INFO - command executed successfully with root 2021-04-28 01:46:52,355 - root - INFO - Done with install on 172.23.106.74. 2021-04-28 01:46:52,355 - root - INFO - Waiting 60s for 172.23.106.74 to be initialized.. 2021-04-28 01:46:53,052 - root - INFO - command executed with root but got an error ['Warning: RPMDB altered outside of yum.'] ... 2021-04-28 01:46:53,052 - root - INFO - running command.raw on 172.23.106.73: systemctl -q is-active couchbase-server && echo 1 || echo 0 2021-04-28 01:46:53,087 - root - INFO - command executed successfully with root 2021-04-28 01:46:53,087 - root - INFO - Done with install on 172.23.106.73. 2021-04-28 01:46:53,088 - root - INFO - Waiting 60s for 172.23.106.73 to be initialized.. 2021-04-28 01:46:53,185 - root - INFO - command executed with root but got an error ['Warning: RPMDB altered outside of yum.'] ... 2021-04-28 01:46:53,185 - root - INFO - running command.raw on 172.23.106.75: systemctl -q is-active couchbase-server && echo 1 || echo 0 2021-04-28 01:46:53,215 - root - INFO - command executed successfully with root 2021-04-28 01:46:53,215 - root - INFO - Done with install on 172.23.106.75. 2021-04-28 01:46:53,215 - root - INFO - Waiting 60s for 172.23.106.75 to be initialized.. 2021-04-28 01:46:53,305 - root - INFO - command executed with root but got an error ['Warning: RPMDB altered outside of yum.'] ... 2021-04-28 01:46:53,305 - root - INFO - running command.raw on 172.23.106.67: systemctl -q is-active couchbase-server && echo 1 || echo 0 2021-04-28 01:46:53,335 - root - INFO - command executed successfully with root 2021-04-28 01:46:53,335 - root - INFO - Done with install on 172.23.106.67. 2021-04-28 01:46:53,335 - root - INFO - Waiting 60s for 172.23.106.67 to be initialized.. 2021-04-28 01:46:53,549 - root - INFO - command executed with root but got an error ['Warning: RPMDB altered outside of yum.'] ... 2021-04-28 01:46:53,549 - root - INFO - running command.raw on 172.23.106.76: systemctl -q is-active couchbase-server && echo 1 || echo 0 2021-04-28 01:46:53,619 - root - INFO - command executed successfully with root 2021-04-28 01:46:53,619 - root - INFO - Done with install on 172.23.106.76. 2021-04-28 01:46:53,619 - root - INFO - Waiting 60s for 172.23.106.76 to be initialized.. 2021-04-28 01:46:53,761 - root - INFO - command executed with root but got an error ['Warning: RPMDB altered outside of yum.'] ... 2021-04-28 01:46:53,761 - root - INFO - running command.raw on 172.23.106.77: systemctl -q is-active couchbase-server && echo 1 || echo 0 2021-04-28 01:46:53,826 - root - INFO - command executed successfully with root 2021-04-28 01:46:53,826 - root - INFO - Done with install on 172.23.106.77. 2021-04-28 01:46:53,826 - root - INFO - Waiting 60s for 172.23.106.77 to be initialized.. 2021-04-28 01:46:54,360 - root - INFO - command executed with root but got an error ['Warning: RPMDB altered outside of yum.'] ... 2021-04-28 01:46:54,360 - root - INFO - running command.raw on 172.23.106.78: systemctl -q is-active couchbase-server && echo 1 || echo 0 2021-04-28 01:46:54,392 - root - INFO - command executed successfully with root 2021-04-28 01:46:54,392 - root - INFO - Done with install on 172.23.106.78. 2021-04-28 01:46:54,392 - root - INFO - Waiting 60s for 172.23.106.78 to be initialized.. 2021-04-28 01:46:55,769 - root - INFO - command executed with root but got an error ['Warning: RPMDB altered outside of yum.'] ... 2021-04-28 01:46:55,769 - root - INFO - running command.raw on 172.23.106.64: systemctl -q is-active couchbase-server && echo 1 || echo 0 2021-04-28 01:46:55,800 - root - INFO - command executed successfully with root 2021-04-28 01:46:55,800 - root - INFO - Done with install on 172.23.106.64. 2021-04-28 01:46:55,800 - root - INFO - Waiting 60s for 172.23.106.64 to be initialized.. 2021-04-28 01:47:52,385 - root - INFO - running command.raw on 172.23.106.74: /opt/couchbase/bin/couchbase-cli node-init -c 172.23.106.74 -u Administrator -p password > /dev/null && echo 1 || echo 0; 2021-04-28 01:47:52,676 - root - INFO - command executed successfully with root 2021-04-28 01:47:52,677 - root - INFO - running command.raw on 172.23.106.74: /opt/couchbase/bin/couchbase-cli node-init -c 172.23.106.74 -u Administrator -p password > /dev/null && echo 1 || echo 0; 2021-04-28 01:47:52,879 - root - INFO - command executed successfully with root 2021-04-28 01:47:52,879 - root - INFO - running command.raw on 172.23.106.74: /opt/couchbase/bin/couchbase-cli node-init -c 172.23.106.74 -u Administrator -p password > /dev/null && echo 1 || echo 0; 2021-04-28 01:47:53,075 - root - INFO - command executed successfully with root 2021-04-28 01:47:53,075 - root - WARNING - Unable to init node 172.23.106.74 due to [] 2021-04-28 01:47:53,135 - root - INFO - running command.raw on 172.23.106.73: /opt/couchbase/bin/couchbase-cli node-init -c 172.23.106.73 -u Administrator -p password > /dev/null && echo 1 || echo 0; 2021-04-28 01:47:53,237 - root - INFO - running command.raw on 172.23.106.75: /opt/couchbase/bin/couchbase-cli node-init -c 172.23.106.75 -u Administrator -p password > /dev/null && echo 1 || echo 0; 2021-04-28 01:47:53,377 - root - INFO - running command.raw on 172.23.106.67: /opt/couchbase/bin/couchbase-cli node-init -c 172.23.106.67 -u Administrator -p password > /dev/null && echo 1 || echo 0; 2021-04-28 01:47:53,422 - root - INFO - command executed successfully with root 2021-04-28 01:47:53,422 - root - INFO - running command.raw on 172.23.106.73: /opt/couchbase/bin/couchbase-cli node-init -c 172.23.106.73 -u Administrator -p password > /dev/null && echo 1 || echo 0; 2021-04-28 01:47:53,528 - root - INFO - command executed successfully with root 2021-04-28 01:47:53,528 - root - INFO - running command.raw on 172.23.106.75: /opt/couchbase/bin/couchbase-cli node-init -c 172.23.106.75 -u Administrator -p password > /dev/null && echo 1 || echo 0; 2021-04-28 01:47:53,644 - root - INFO - running command.raw on 172.23.106.76: /opt/couchbase/bin/couchbase-cli node-init -c 172.23.106.76 -u Administrator -p password > /dev/null && echo 1 || echo 0; 2021-04-28 01:47:53,666 - root - INFO - command executed successfully with root 2021-04-28 01:47:53,666 - root - INFO - running command.raw on 172.23.106.73: /opt/couchbase/bin/couchbase-cli node-init -c 172.23.106.73 -u Administrator -p password > /dev/null && echo 1 || echo 0; 2021-04-28 01:47:53,669 - root - INFO - command executed successfully with root 2021-04-28 01:47:53,669 - root - INFO - running command.raw on 172.23.106.67: /opt/couchbase/bin/couchbase-cli node-init -c 172.23.106.67 -u Administrator -p password > /dev/null && echo 1 || echo 0; 2021-04-28 01:47:53,736 - root - INFO - command executed successfully with root 2021-04-28 01:47:53,737 - root - INFO - running command.raw on 172.23.106.75: /opt/couchbase/bin/couchbase-cli node-init -c 172.23.106.75 -u Administrator -p password > /dev/null && echo 1 || echo 0; 2021-04-28 01:47:53,845 - root - INFO - running command.raw on 172.23.106.77: /opt/couchbase/bin/couchbase-cli node-init -c 172.23.106.77 -u Administrator -p password > /dev/null && echo 1 || echo 0; 2021-04-28 01:47:53,870 - root - INFO - command executed successfully with root 2021-04-28 01:47:53,870 - root - WARNING - Unable to init node 172.23.106.73 due to [] 2021-04-28 01:47:53,880 - root - INFO - command executed successfully with root 2021-04-28 01:47:53,880 - root - INFO - running command.raw on 172.23.106.67: /opt/couchbase/bin/couchbase-cli node-init -c 172.23.106.67 -u Administrator -p password > /dev/null && echo 1 || echo 0; 2021-04-28 01:47:53,955 - root - INFO - command executed successfully with root 2021-04-28 01:47:53,956 - root - WARNING - Unable to init node 172.23.106.75 due to [] 2021-04-28 01:47:53,959 - root - INFO - command executed successfully with root 2021-04-28 01:47:53,960 - root - INFO - running command.raw on 172.23.106.76: /opt/couchbase/bin/couchbase-cli node-init -c 172.23.106.76 -u Administrator -p password > /dev/null && echo 1 || echo 0; 2021-04-28 01:47:54,086 - root - INFO - Setting KV memory quota as 2016 MB on 172.23.106.74 2021-04-28 01:47:54,086 - root - INFO - pools/default params : memoryQuota=2016 2021-04-28 01:47:54,091 - root - INFO - --> init_node_services(Administrator,password,None,8091,['kv']) 2021-04-28 01:47:54,091 - root - INFO - /node/controller/setupServices params on 172.23.106.74: 8091:hostname=None&user=Administrator&password=password&services=kv 2021-04-28 01:47:54,092 - root - INFO - command executed successfully with root 2021-04-28 01:47:54,092 - root - WARNING - Unable to init node 172.23.106.67 due to [] 2021-04-28 01:47:54,121 - root - INFO - --> in init_cluster...Administrator,password,8091 2021-04-28 01:47:54,121 - root - INFO - settings/web params on 172.23.106.74:8091:port=8091&username=Administrator&password=password 2021-04-28 01:47:54,147 - root - INFO - command executed successfully with root 2021-04-28 01:47:54,147 - root - INFO - running command.raw on 172.23.106.77: /opt/couchbase/bin/couchbase-cli node-init -c 172.23.106.77 -u Administrator -p password > /dev/null && echo 1 || echo 0; 2021-04-28 01:47:54,184 - root - INFO - --> status:True 2021-04-28 01:47:54,184 - root - INFO - Done with init on 172.23.106.74. 2021-04-28 01:47:54,185 - root - INFO - running command.raw on 172.23.106.74: ls -td /tmp/couchbase*.rpm | awk 'NR>2' | xargs rm -f 2021-04-28 01:47:54,197 - root - INFO - command executed successfully with root 2021-04-28 01:47:54,198 - root - INFO - running command.raw on 172.23.106.76: /opt/couchbase/bin/couchbase-cli node-init -c 172.23.106.76 -u Administrator -p password > /dev/null && echo 1 || echo 0; 2021-04-28 01:47:54,211 - root - INFO - command executed successfully with root 2021-04-28 01:47:54,212 - root - INFO - Done with cleanup on 172.23.106.74. 2021-04-28 01:47:54,351 - root - INFO - command executed successfully with root 2021-04-28 01:47:54,351 - root - INFO - running command.raw on 172.23.106.77: /opt/couchbase/bin/couchbase-cli node-init -c 172.23.106.77 -u Administrator -p password > /dev/null && echo 1 || echo 0; 2021-04-28 01:47:54,410 - root - INFO - running command.raw on 172.23.106.78: /opt/couchbase/bin/couchbase-cli node-init -c 172.23.106.78 -u Administrator -p password > /dev/null && echo 1 || echo 0; 2021-04-28 01:47:54,433 - root - INFO - command executed successfully with root 2021-04-28 01:47:54,433 - root - WARNING - Unable to init node 172.23.106.76 due to [] 2021-04-28 01:47:54,555 - root - INFO - command executed successfully with root 2021-04-28 01:47:54,555 - root - WARNING - Unable to init node 172.23.106.77 due to [] 2021-04-28 01:47:54,691 - root - INFO - command executed successfully with root 2021-04-28 01:47:54,691 - root - INFO - running command.raw on 172.23.106.78: /opt/couchbase/bin/couchbase-cli node-init -c 172.23.106.78 -u Administrator -p password > /dev/null && echo 1 || echo 0; 2021-04-28 01:47:54,880 - root - INFO - Setting KV memory quota as 2016 MB on 172.23.106.73 2021-04-28 01:47:54,880 - root - INFO - pools/default params : memoryQuota=2016 2021-04-28 01:47:54,884 - root - INFO - --> init_node_services(Administrator,password,None,8091,['kv']) 2021-04-28 01:47:54,885 - root - INFO - /node/controller/setupServices params on 172.23.106.73: 8091:hostname=None&user=Administrator&password=password&services=kv 2021-04-28 01:47:54,887 - root - INFO - command executed successfully with root 2021-04-28 01:47:54,888 - root - INFO - running command.raw on 172.23.106.78: /opt/couchbase/bin/couchbase-cli node-init -c 172.23.106.78 -u Administrator -p password > /dev/null && echo 1 || echo 0; 2021-04-28 01:47:54,916 - root - INFO - --> in init_cluster...Administrator,password,8091 2021-04-28 01:47:54,916 - root - INFO - settings/web params on 172.23.106.73:8091:port=8091&username=Administrator&password=password 2021-04-28 01:47:54,965 - root - INFO - Setting KV memory quota as 2016 MB on 172.23.106.75 2021-04-28 01:47:54,966 - root - INFO - pools/default params : memoryQuota=2016 2021-04-28 01:47:54,970 - root - INFO - --> init_node_services(Administrator,password,None,8091,['kv']) 2021-04-28 01:47:54,970 - root - INFO - /node/controller/setupServices params on 172.23.106.75: 8091:hostname=None&user=Administrator&password=password&services=kv 2021-04-28 01:47:54,978 - root - INFO - --> status:True 2021-04-28 01:47:54,978 - root - INFO - Done with init on 172.23.106.73. 2021-04-28 01:47:54,978 - root - INFO - running command.raw on 172.23.106.73: ls -td /tmp/couchbase*.rpm | awk 'NR>2' | xargs rm -f 2021-04-28 01:47:54,999 - root - INFO - command executed successfully with root 2021-04-28 01:47:54,999 - root - INFO - Done with cleanup on 172.23.106.73. 2021-04-28 01:47:55,000 - root - INFO - --> in init_cluster...Administrator,password,8091 2021-04-28 01:47:55,000 - root - INFO - settings/web params on 172.23.106.75:8091:port=8091&username=Administrator&password=password 2021-04-28 01:47:55,065 - root - INFO - --> status:True 2021-04-28 01:47:55,065 - root - INFO - Done with init on 172.23.106.75. 2021-04-28 01:47:55,065 - root - INFO - running command.raw on 172.23.106.75: ls -td /tmp/couchbase*.rpm | awk 'NR>2' | xargs rm -f 2021-04-28 01:47:55,084 - root - INFO - command executed successfully with root 2021-04-28 01:47:55,085 - root - INFO - Done with cleanup on 172.23.106.75. 2021-04-28 01:47:55,101 - root - INFO - Setting KV memory quota as 2016 MB on 172.23.106.67 2021-04-28 01:47:55,101 - root - INFO - pools/default params : memoryQuota=2016 2021-04-28 01:47:55,106 - root - INFO - --> init_node_services(Administrator,password,None,8091,['kv']) 2021-04-28 01:47:55,106 - root - INFO - /node/controller/setupServices params on 172.23.106.67: 8091:hostname=None&user=Administrator&password=password&services=kv 2021-04-28 01:47:55,129 - root - INFO - command executed successfully with root 2021-04-28 01:47:55,129 - root - WARNING - Unable to init node 172.23.106.78 due to [] 2021-04-28 01:47:55,136 - root - INFO - --> in init_cluster...Administrator,password,8091 2021-04-28 01:47:55,136 - root - INFO - settings/web params on 172.23.106.67:8091:port=8091&username=Administrator&password=password 2021-04-28 01:47:55,199 - root - INFO - --> status:True 2021-04-28 01:47:55,199 - root - INFO - Done with init on 172.23.106.67. 2021-04-28 01:47:55,199 - root - INFO - running command.raw on 172.23.106.67: ls -td /tmp/couchbase*.rpm | awk 'NR>2' | xargs rm -f 2021-04-28 01:47:55,220 - root - INFO - command executed successfully with root 2021-04-28 01:47:55,220 - root - INFO - Done with cleanup on 172.23.106.67. 2021-04-28 01:47:55,443 - root - INFO - Setting KV memory quota as 2016 MB on 172.23.106.76 2021-04-28 01:47:55,444 - root - INFO - pools/default params : memoryQuota=2016 2021-04-28 01:47:55,448 - root - INFO - --> init_node_services(Administrator,password,None,8091,['kv']) 2021-04-28 01:47:55,448 - root - INFO - /node/controller/setupServices params on 172.23.106.76: 8091:hostname=None&user=Administrator&password=password&services=kv 2021-04-28 01:47:55,478 - root - INFO - --> in init_cluster...Administrator,password,8091 2021-04-28 01:47:55,478 - root - INFO - settings/web params on 172.23.106.76:8091:port=8091&username=Administrator&password=password 2021-04-28 01:47:55,542 - root - INFO - --> status:True 2021-04-28 01:47:55,542 - root - INFO - Done with init on 172.23.106.76. 2021-04-28 01:47:55,542 - root - INFO - running command.raw on 172.23.106.76: ls -td /tmp/couchbase*.rpm | awk 'NR>2' | xargs rm -f 2021-04-28 01:47:55,565 - root - INFO - Setting KV memory quota as 2016 MB on 172.23.106.77 2021-04-28 01:47:55,565 - root - INFO - pools/default params : memoryQuota=2016 2021-04-28 01:47:55,567 - root - INFO - command executed successfully with root 2021-04-28 01:47:55,568 - root - INFO - Done with cleanup on 172.23.106.76. 2021-04-28 01:47:55,569 - root - INFO - --> init_node_services(Administrator,password,None,8091,['kv']) 2021-04-28 01:47:55,569 - root - INFO - /node/controller/setupServices params on 172.23.106.77: 8091:hostname=None&user=Administrator&password=password&services=kv 2021-04-28 01:47:55,598 - root - INFO - --> in init_cluster...Administrator,password,8091 2021-04-28 01:47:55,598 - root - INFO - settings/web params on 172.23.106.77:8091:port=8091&username=Administrator&password=password 2021-04-28 01:47:55,663 - root - INFO - --> status:True 2021-04-28 01:47:55,663 - root - INFO - Done with init on 172.23.106.77. 2021-04-28 01:47:55,664 - root - INFO - running command.raw on 172.23.106.77: ls -td /tmp/couchbase*.rpm | awk 'NR>2' | xargs rm -f 2021-04-28 01:47:55,688 - root - INFO - command executed successfully with root 2021-04-28 01:47:55,688 - root - INFO - Done with cleanup on 172.23.106.77. 2021-04-28 01:47:55,860 - root - INFO - running command.raw on 172.23.106.64: /opt/couchbase/bin/couchbase-cli node-init -c 172.23.106.64 -u Administrator -p password > /dev/null && echo 1 || echo 0; 2021-04-28 01:47:56,139 - root - INFO - Setting KV memory quota as 2016 MB on 172.23.106.78 2021-04-28 01:47:56,139 - root - INFO - pools/default params : memoryQuota=2016 2021-04-28 01:47:56,143 - root - INFO - --> init_node_services(Administrator,password,None,8091,['kv']) 2021-04-28 01:47:56,143 - root - INFO - /node/controller/setupServices params on 172.23.106.78: 8091:hostname=None&user=Administrator&password=password&services=kv 2021-04-28 01:47:56,147 - root - INFO - command executed successfully with root 2021-04-28 01:47:56,147 - root - INFO - running command.raw on 172.23.106.64: /opt/couchbase/bin/couchbase-cli node-init -c 172.23.106.64 -u Administrator -p password > /dev/null && echo 1 || echo 0; 2021-04-28 01:47:56,172 - root - INFO - --> in init_cluster...Administrator,password,8091 2021-04-28 01:47:56,173 - root - INFO - settings/web params on 172.23.106.78:8091:port=8091&username=Administrator&password=password 2021-04-28 01:47:56,236 - root - INFO - --> status:True 2021-04-28 01:47:56,236 - root - INFO - Done with init on 172.23.106.78. 2021-04-28 01:47:56,236 - root - INFO - running command.raw on 172.23.106.78: ls -td /tmp/couchbase*.rpm | awk 'NR>2' | xargs rm -f 2021-04-28 01:47:56,256 - root - INFO - command executed successfully with root 2021-04-28 01:47:56,257 - root - INFO - Done with cleanup on 172.23.106.78. 2021-04-28 01:47:56,351 - root - INFO - command executed successfully with root 2021-04-28 01:47:56,351 - root - INFO - running command.raw on 172.23.106.64: /opt/couchbase/bin/couchbase-cli node-init -c 172.23.106.64 -u Administrator -p password > /dev/null && echo 1 || echo 0; 2021-04-28 01:47:56,554 - root - INFO - command executed successfully with root 2021-04-28 01:47:56,554 - root - WARNING - Unable to init node 172.23.106.64 due to [] 2021-04-28 01:47:57,565 - root - INFO - Setting KV memory quota as 2016 MB on 172.23.106.64 2021-04-28 01:47:57,565 - root - INFO - pools/default params : memoryQuota=2016 2021-04-28 01:47:57,569 - root - INFO - --> init_node_services(Administrator,password,None,8091,['kv']) 2021-04-28 01:47:57,569 - root - INFO - /node/controller/setupServices params on 172.23.106.64: 8091:hostname=None&user=Administrator&password=password&services=kv 2021-04-28 01:47:57,599 - root - INFO - --> in init_cluster...Administrator,password,8091 2021-04-28 01:47:57,599 - root - INFO - settings/web params on 172.23.106.64:8091:port=8091&username=Administrator&password=password 2021-04-28 01:47:57,664 - root - INFO - --> status:True 2021-04-28 01:47:57,664 - root - INFO - Done with init on 172.23.106.64. 2021-04-28 01:47:57,664 - root - INFO - running command.raw on 172.23.106.64: ls -td /tmp/couchbase*.rpm | awk 'NR>2' | xargs rm -f 2021-04-28 01:47:57,693 - root - INFO - command executed successfully with root 2021-04-28 01:47:57,693 - root - INFO - Done with cleanup on 172.23.106.64. 2021-04-28 01:48:07,412 - root - INFO - ---------------------------------------------------------------------------------------------------- 2021-04-28 01:48:07,449 - root - INFO - cluster:C1 node:172.23.106.64:8091 version:7.0.0-5046-enterprise aFamily:inet services:['kv'] 2021-04-28 01:48:07,449 - root - INFO - cluster:C2 node:172.23.106.67:8091 version:7.0.0-5046-enterprise aFamily:inet services:['kv'] 2021-04-28 01:48:07,449 - root - INFO - cluster:C3 node:172.23.106.73:8091 version:7.0.0-5046-enterprise aFamily:inet services:['kv'] 2021-04-28 01:48:07,449 - root - INFO - cluster:C4 node:172.23.106.74:8091 version:7.0.0-5046-enterprise aFamily:inet services:['kv'] 2021-04-28 01:48:07,449 - root - INFO - cluster:C5 node:172.23.106.75:8091 version:7.0.0-5046-enterprise aFamily:inet services:['kv'] 2021-04-28 01:48:07,449 - root - INFO - cluster:C6 node:172.23.106.76:8091 version:7.0.0-5046-enterprise aFamily:inet services:['kv'] 2021-04-28 01:48:07,450 - root - INFO - cluster:C7 node:172.23.106.77:8091 version:7.0.0-5046-enterprise aFamily:inet services:['kv'] 2021-04-28 01:48:07,450 - root - INFO - cluster:C8 node:172.23.106.78:8091 version:7.0.0-5046-enterprise aFamily:inet services:['kv'] 2021-04-28 01:48:07,450 - root - INFO - ---------------------------------------------------------------------------------------------------- 2021-04-28 01:48:07,450 - root - INFO - ---------------------------------------------------------------------------------------------------- 2021-04-28 01:48:07,450 - root - INFO - ---------------------------------------------------------------------------------------------------- 2021-04-28 01:48:07,450 - root - INFO - INSTALL COMPLETED ON: 172.23.106.64 2021-04-28 01:48:07,450 - root - INFO - INSTALL COMPLETED ON: 172.23.106.67 2021-04-28 01:48:07,450 - root - INFO - INSTALL COMPLETED ON: 172.23.106.73 2021-04-28 01:48:07,450 - root - INFO - INSTALL COMPLETED ON: 172.23.106.74 2021-04-28 01:48:07,450 - root - INFO - INSTALL COMPLETED ON: 172.23.106.75 2021-04-28 01:48:07,450 - root - INFO - INSTALL COMPLETED ON: 172.23.106.76 2021-04-28 01:48:07,450 - root - INFO - INSTALL COMPLETED ON: 172.23.106.77 2021-04-28 01:48:07,450 - root - INFO - INSTALL COMPLETED ON: 172.23.106.78 2021-04-28 01:48:07,450 - root - INFO - ---------------------------------------------------------------------------------------------------- 2021-04-28 01:48:07,450 - root - INFO - TOTAL INSTALL TIME = 318 seconds + ./testrunner.py -i /tmp/win10-bucket-ops.ini -t eventing.eventing_upgrade.EventingUpgrade.test_online_upgrade_with_regular_rebalance_with_eventing,nodes_init=4,services_init=kv-eventing-index-n1ql,dataset=default,groups=simple,upgrade_test=True,initial_version=6.6.1-9213,doc-per-day=10,upgrade_version=7.0.0-5046 INFO:root:__main__ INFO:__main__:TestRunner: parsing args... INFO:__main__:Checking arguments... INFO:__main__:TestRunner: start... INFO:__main__:Global Test input params: INFO:__main__: Number of tests initially selected before GROUP filters: 1 INFO:__main__:--> Running test: eventing.eventing_upgrade.EventingUpgrade.test_online_upgrade_with_regular_rebalance_with_eventing,nodes_init=4,services_init=kv-eventing-index-n1ql,dataset=default,groups=simple,upgrade_test=True,initial_version=6.6.1-9213,doc-per-day=10,upgrade_version=7.0.0-5046 INFO:__main__:Logs folder: /data/workspace/dev_testbed_blr3/logs/testrunner-21-Apr-28_01-48-07/test_1 *** TestRunner *** {'cluster_name': 'win10-bucket-ops', 'ini': '/tmp/win10-bucket-ops.ini', 'num_nodes': 8} Logs will be stored at /data/workspace/dev_testbed_blr3/logs/testrunner-21-Apr-28_01-48-07/test_1 ./testrunner -i /tmp/win10-bucket-ops.ini -p -t eventing.eventing_upgrade.EventingUpgrade.test_online_upgrade_with_regular_rebalance_with_eventing,nodes_init=4,services_init=kv-eventing-index-n1ql,dataset=default,groups=simple,upgrade_test=True,initial_version=6.6.1-9213,doc-per-day=10,upgrade_version=7.0.0-5046 Test Input params: {'nodes_init': '4', 'services_init': 'kv-eventing-index-n1ql', 'dataset': 'default', 'groups': 'simple', 'upgrade_test': 'True', 'initial_version': '6.6.1-9213', 'doc-per-day': '10', 'upgrade_version': '7.0.0-5046', 'ini': '/tmp/win10-bucket-ops.ini', 'cluster_name': 'win10-bucket-ops', 'num_nodes': 8, 'case_number': 1, 'logs_folder': '/data/workspace/dev_testbed_blr3/logs/testrunner-21-Apr-28_01-48-07/test_1'} test_online_upgrade_with_regular_rebalance_with_eventing (eventing.eventing_upgrade.EventingUpgrade) ... 2021-04-28 01:48:07,865 - root - INFO - ============== EventingUpgrade setup has started ============== 2021-04-28 01:48:07,865 - root - INFO - ============== EventingBaseTest setup has started ============== 2021-04-28 01:48:07,865 - root - INFO - Starting Test: test_online_upgrade_with_regular_rebalance_with_eventing 2021-04-28 01:48:07,867 - root - INFO - SSH Connecting to 172.23.106.64 with username:root, attempt#1 of 5 2021-04-28 01:48:07,973 - root - INFO - SSH Connected to 172.23.106.64 as root 2021-04-28 01:48:08,231 - root - INFO - os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-04-28 01:48:08,550 - root - INFO - extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-04-28 01:48:08,555 - root - INFO - -->is_ns_server_running? 2021-04-28 01:48:08,566 - root - INFO - Node version in cluster 7.0.0-5046-enterprise 2021-04-28 01:48:08,572 - root - INFO - Node versions in cluster ['7.0.0-5046-enterprise'] 2021-04-28 01:48:08,572 - root - INFO - ============== basetestcase setup was started for test #1 test_online_upgrade_with_regular_rebalance_with_eventing============== 2021-04-28 01:48:08,584 - root - INFO - Could not find any buckets for node 172.23.106.64, nothing to delete 2021-04-28 01:48:08,590 - root - INFO - Could not find any buckets for node 172.23.106.67, nothing to delete 2021-04-28 01:48:08,595 - root - INFO - Could not find any buckets for node 172.23.106.73, nothing to delete 2021-04-28 01:48:08,601 - root - INFO - Could not find any buckets for node 172.23.106.74, nothing to delete 2021-04-28 01:48:08,606 - root - INFO - Could not find any buckets for node 172.23.106.75, nothing to delete 2021-04-28 01:48:08,613 - root - INFO - Could not find any buckets for node 172.23.106.76, nothing to delete 2021-04-28 01:48:08,618 - root - INFO - Could not find any buckets for node 172.23.106.77, nothing to delete 2021-04-28 01:48:08,624 - root - INFO - Could not find any buckets for node 172.23.106.78, nothing to delete 2021-04-28 01:48:08,627 - root - INFO - -->is_ns_server_running? 2021-04-28 01:50:08,695 - root - INFO - waiting for ns_server @ 172.23.106.64:8091 2021-04-28 01:50:08,696 - root - INFO - -->is_ns_server_running? 2021-04-28 01:50:08,698 - root - INFO - ns_server @ 172.23.106.64:8091 is running 2021-04-28 01:50:08,702 - root - INFO - waiting for ns_server @ 172.23.106.67:8091 2021-04-28 01:50:08,702 - root - INFO - -->is_ns_server_running? 2021-04-28 01:50:08,705 - root - INFO - ns_server @ 172.23.106.67:8091 is running 2021-04-28 01:50:08,708 - root - INFO - waiting for ns_server @ 172.23.106.73:8091 2021-04-28 01:50:08,708 - root - INFO - -->is_ns_server_running? 2021-04-28 01:50:08,711 - root - INFO - ns_server @ 172.23.106.73:8091 is running 2021-04-28 01:50:08,715 - root - INFO - waiting for ns_server @ 172.23.106.74:8091 2021-04-28 01:50:08,715 - root - INFO - -->is_ns_server_running? 2021-04-28 01:50:08,718 - root - INFO - ns_server @ 172.23.106.74:8091 is running 2021-04-28 01:50:08,721 - root - INFO - waiting for ns_server @ 172.23.106.75:8091 2021-04-28 01:50:08,721 - root - INFO - -->is_ns_server_running? 2021-04-28 01:50:08,724 - root - INFO - ns_server @ 172.23.106.75:8091 is running 2021-04-28 01:50:08,728 - root - INFO - waiting for ns_server @ 172.23.106.76:8091 2021-04-28 01:50:08,728 - root - INFO - -->is_ns_server_running? 2021-04-28 01:50:08,731 - root - INFO - ns_server @ 172.23.106.76:8091 is running 2021-04-28 01:50:08,734 - root - INFO - waiting for ns_server @ 172.23.106.77:8091 2021-04-28 01:50:08,734 - root - INFO - -->is_ns_server_running? 2021-04-28 01:50:08,737 - root - INFO - ns_server @ 172.23.106.77:8091 is running 2021-04-28 01:50:08,741 - root - INFO - waiting for ns_server @ 172.23.106.78:8091 2021-04-28 01:50:08,741 - root - INFO - -->is_ns_server_running? 2021-04-28 01:50:08,744 - root - INFO - ns_server @ 172.23.106.78:8091 is running 2021-04-28 01:50:08,744 - root - INFO - ============== EventingUpgrade tearDown has started ============== 2021-04-28 01:50:08,744 - root - INFO - ============== NewUpgradeBaseTest tearDown has started ============== 2021-04-28 01:50:08,745 - root - INFO - ============== EventingUpgrade tearDown has completed ============== 2021-04-28 01:50:08,745 - root - INFO - initializing cluster 2021-04-28 01:50:08,745 - root - INFO - service will set to first element 2021-04-28 01:50:08,745 - root - INFO - service will set to first element 2021-04-28 01:50:08,745 - root - INFO - service will set to first element 2021-04-28 01:50:08,745 - root - INFO - service will set to first element 2021-04-28 01:50:08,745 - root - INFO - service will set to first element 2021-04-28 01:50:08,745 - root - INFO - service will set to first element 2021-04-28 01:50:08,746 - root - INFO - service will set to first element 2021-04-28 01:50:08,746 - root - INFO - service will set to first element 2021-04-28 01:50:09,750 - root - INFO - server: ip:172.23.106.64 port:8091 ssh_username:root, nodes/self 2021-04-28 01:50:09,753 - root - INFO - {'uptime': '190', 'memoryTotal': 3945984000, 'memoryFree': 3415171072, 'mcdMemoryReserved': 3010, 'mcdMemoryAllocated': 3010, 'status': 'healthy', 'hostname': '172.23.106.64:8091', 'clusterCompatibility': 458752, 'clusterMembership': 'active', 'recoveryType': 'none', 'version': '7.0.0-5046-enterprise', 'os': 'x86_64-unknown-linux-gnu', 'ports': [], 'availableStorage': [], 'storage': [], 'memoryQuota': 2016, 'moxi': 11211, 'memcached': 11210, 'id': 'ns_1@cb.local', 'ip': '172.23.106.64', 'rest_username': '', 'rest_password': '', 'port': '8091', 'services': ['kv'], 'storageTotalRam': 3763, 'curr_items': 0} 2021-04-28 01:50:09,757 - root - INFO - pools/default params : memoryQuota=2016 2021-04-28 01:50:09,760 - root - INFO - --> init_node_services(Administrator,password,172.23.106.64,8091,['kv']) 2021-04-28 01:50:09,760 - root - INFO - /node/controller/setupServices params on 172.23.106.64: 8091:hostname=172.23.106.64&user=Administrator&password=password&services=kv 2021-04-28 01:50:09,763 - root - ERROR - POST http://172.23.106.64:8091//node/controller/setupServices body: hostname=172.23.106.64&user=Administrator&password=password&services=kv headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==', 'Accept': '*/*'} error: 400 reason: unknown b'["cannot change node services after cluster is provisioned"]' auth: Administrator:password 2021-04-28 01:50:09,766 - root - INFO - This node is already provisioned with services, we do not consider this as failure for test case 2021-04-28 01:50:09,766 - root - INFO - --> in init_cluster...Administrator,password,8091 2021-04-28 01:50:09,766 - root - INFO - settings/web params on 172.23.106.64:8091:port=8091&username=Administrator&password=password 2021-04-28 01:50:09,818 - root - INFO - --> status:True 2021-04-28 01:50:09,819 - root - INFO - SSH Connecting to 172.23.106.64 with username:root, attempt#1 of 5 2021-04-28 01:50:09,926 - root - INFO - SSH Connected to 172.23.106.64 as root 2021-04-28 01:50:10,182 - root - INFO - os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-04-28 01:50:10,500 - root - INFO - extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-04-28 01:50:10,508 - root - INFO - running command.raw on 172.23.106.64: curl --silent --show-error http://Administrator:password@localhost:8091/diag/eval -X POST -d 'ns_config:set(allow_nonlocal_eval, true).' 2021-04-28 01:50:10,569 - root - INFO - command executed successfully with root 2021-04-28 01:50:10,569 - root - INFO - ['ok'] 2021-04-28 01:50:10,572 - root - INFO - /diag/eval status on 172.23.106.64:8091: True content: [7,0] command: cluster_compat_mode:get_compat_version(). 2021-04-28 01:50:10,574 - root - INFO - /diag/eval status on 172.23.106.64:8091: True content: [7,0] command: cluster_compat_mode:get_compat_version(). 2021-04-28 01:50:10,578 - root - INFO - settings/indexes params : storageMode=plasma 2021-04-28 01:50:10,596 - root - INFO - server: ip:172.23.106.67 port:8091 ssh_username:root, nodes/self 2021-04-28 01:50:10,599 - root - INFO - {'uptime': '191', 'memoryTotal': 3945984000, 'memoryFree': 3397320704, 'mcdMemoryReserved': 3010, 'mcdMemoryAllocated': 3010, 'status': 'healthy', 'hostname': '172.23.106.67:8091', 'clusterCompatibility': 458752, 'clusterMembership': 'active', 'recoveryType': 'none', 'version': '7.0.0-5046-enterprise', 'os': 'x86_64-unknown-linux-gnu', 'ports': [], 'availableStorage': [], 'storage': [], 'memoryQuota': 2016, 'moxi': 11211, 'memcached': 11210, 'id': 'ns_1@cb.local', 'ip': '172.23.106.67', 'rest_username': '', 'rest_password': '', 'port': '8091', 'services': ['kv'], 'storageTotalRam': 3763, 'curr_items': 0} 2021-04-28 01:50:10,603 - root - INFO - pools/default params : memoryQuota=2016 2021-04-28 01:50:10,606 - root - INFO - --> init_node_services(Administrator,password,172.23.106.67,8091,['kv']) 2021-04-28 01:50:10,607 - root - INFO - /node/controller/setupServices params on 172.23.106.67: 8091:hostname=172.23.106.67&user=Administrator&password=password&services=kv 2021-04-28 01:50:10,609 - root - ERROR - POST http://172.23.106.67:8091//node/controller/setupServices body: hostname=172.23.106.67&user=Administrator&password=password&services=kv headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==', 'Accept': '*/*'} error: 400 reason: unknown b'["cannot change node services after cluster is provisioned"]' auth: Administrator:password 2021-04-28 01:50:10,609 - root - INFO - This node is already provisioned with services, we do not consider this as failure for test case 2021-04-28 01:50:10,610 - root - INFO - --> in init_cluster...Administrator,password,8091 2021-04-28 01:50:10,610 - root - INFO - settings/web params on 172.23.106.67:8091:port=8091&username=Administrator&password=password 2021-04-28 01:50:10,662 - root - INFO - --> status:True 2021-04-28 01:50:10,663 - root - INFO - SSH Connecting to 172.23.106.67 with username:root, attempt#1 of 5 2021-04-28 01:50:10,770 - root - INFO - SSH Connected to 172.23.106.67 as root 2021-04-28 01:50:11,039 - root - INFO - os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-04-28 01:50:11,336 - root - INFO - extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-04-28 01:50:11,343 - root - INFO - running command.raw on 172.23.106.67: curl --silent --show-error http://Administrator:password@localhost:8091/diag/eval -X POST -d 'ns_config:set(allow_nonlocal_eval, true).' 2021-04-28 01:50:11,401 - root - INFO - command executed successfully with root 2021-04-28 01:50:11,401 - root - INFO - ['ok'] 2021-04-28 01:50:11,403 - root - INFO - /diag/eval status on 172.23.106.67:8091: True content: [7,0] command: cluster_compat_mode:get_compat_version(). 2021-04-28 01:50:11,405 - root - INFO - /diag/eval status on 172.23.106.67:8091: True content: [7,0] command: cluster_compat_mode:get_compat_version(). 2021-04-28 01:50:11,409 - root - INFO - settings/indexes params : storageMode=plasma 2021-04-28 01:50:11,429 - root - INFO - server: ip:172.23.106.73 port:8091 ssh_username:root, nodes/self 2021-04-28 01:50:11,432 - root - INFO - {'uptime': '195', 'memoryTotal': 3945984000, 'memoryFree': 3390582784, 'mcdMemoryReserved': 3010, 'mcdMemoryAllocated': 3010, 'status': 'healthy', 'hostname': '172.23.106.73:8091', 'clusterCompatibility': 458752, 'clusterMembership': 'active', 'recoveryType': 'none', 'version': '7.0.0-5046-enterprise', 'os': 'x86_64-unknown-linux-gnu', 'ports': [], 'availableStorage': [], 'storage': [], 'memoryQuota': 2016, 'moxi': 11211, 'memcached': 11210, 'id': 'ns_1@cb.local', 'ip': '172.23.106.73', 'rest_username': '', 'rest_password': '', 'port': '8091', 'services': ['kv'], 'storageTotalRam': 3763, 'curr_items': 0} 2021-04-28 01:50:11,436 - root - INFO - pools/default params : memoryQuota=2016 2021-04-28 01:50:11,439 - root - INFO - --> init_node_services(Administrator,password,172.23.106.73,8091,['kv']) 2021-04-28 01:50:11,439 - root - INFO - /node/controller/setupServices params on 172.23.106.73: 8091:hostname=172.23.106.73&user=Administrator&password=password&services=kv 2021-04-28 01:50:11,441 - root - ERROR - POST http://172.23.106.73:8091//node/controller/setupServices body: hostname=172.23.106.73&user=Administrator&password=password&services=kv headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==', 'Accept': '*/*'} error: 400 reason: unknown b'["cannot change node services after cluster is provisioned"]' auth: Administrator:password 2021-04-28 01:50:11,441 - root - INFO - This node is already provisioned with services, we do not consider this as failure for test case 2021-04-28 01:50:11,442 - root - INFO - --> in init_cluster...Administrator,password,8091 2021-04-28 01:50:11,442 - root - INFO - settings/web params on 172.23.106.73:8091:port=8091&username=Administrator&password=password 2021-04-28 01:50:11,493 - root - INFO - --> status:True 2021-04-28 01:50:11,495 - root - INFO - SSH Connecting to 172.23.106.73 with username:root, attempt#1 of 5 2021-04-28 01:50:11,607 - root - INFO - SSH Connected to 172.23.106.73 as root 2021-04-28 01:50:11,896 - root - INFO - os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-04-28 01:50:12,212 - root - INFO - extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-04-28 01:50:12,220 - root - INFO - running command.raw on 172.23.106.73: curl --silent --show-error http://Administrator:password@localhost:8091/diag/eval -X POST -d 'ns_config:set(allow_nonlocal_eval, true).' 2021-04-28 01:50:12,279 - root - INFO - command executed successfully with root 2021-04-28 01:50:12,279 - root - INFO - ['ok'] 2021-04-28 01:50:12,282 - root - INFO - /diag/eval status on 172.23.106.73:8091: True content: [7,0] command: cluster_compat_mode:get_compat_version(). 2021-04-28 01:50:12,283 - root - INFO - /diag/eval status on 172.23.106.73:8091: True content: [7,0] command: cluster_compat_mode:get_compat_version(). 2021-04-28 01:50:12,287 - root - INFO - settings/indexes params : storageMode=plasma 2021-04-28 01:50:12,304 - root - INFO - server: ip:172.23.106.74 port:8091 ssh_username:root, nodes/self 2021-04-28 01:50:12,307 - root - INFO - {'uptime': '195', 'memoryTotal': 3945984000, 'memoryFree': 3383795712, 'mcdMemoryReserved': 3010, 'mcdMemoryAllocated': 3010, 'status': 'healthy', 'hostname': '172.23.106.74:8091', 'clusterCompatibility': 458752, 'clusterMembership': 'active', 'recoveryType': 'none', 'version': '7.0.0-5046-enterprise', 'os': 'x86_64-unknown-linux-gnu', 'ports': [], 'availableStorage': [], 'storage': [], 'memoryQuota': 2016, 'moxi': 11211, 'memcached': 11210, 'id': 'ns_1@cb.local', 'ip': '172.23.106.74', 'rest_username': '', 'rest_password': '', 'port': '8091', 'services': ['kv'], 'storageTotalRam': 3763, 'curr_items': 0} 2021-04-28 01:50:12,310 - root - INFO - pools/default params : memoryQuota=2016 2021-04-28 01:50:12,313 - root - INFO - --> init_node_services(Administrator,password,172.23.106.74,8091,['kv']) 2021-04-28 01:50:12,314 - root - INFO - /node/controller/setupServices params on 172.23.106.74: 8091:hostname=172.23.106.74&user=Administrator&password=password&services=kv 2021-04-28 01:50:12,316 - root - ERROR - POST http://172.23.106.74:8091//node/controller/setupServices body: hostname=172.23.106.74&user=Administrator&password=password&services=kv headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==', 'Accept': '*/*'} error: 400 reason: unknown b'["cannot change node services after cluster is provisioned"]' auth: Administrator:password 2021-04-28 01:50:12,316 - root - INFO - This node is already provisioned with services, we do not consider this as failure for test case 2021-04-28 01:50:12,316 - root - INFO - --> in init_cluster...Administrator,password,8091 2021-04-28 01:50:12,316 - root - INFO - settings/web params on 172.23.106.74:8091:port=8091&username=Administrator&password=password 2021-04-28 01:50:12,367 - root - INFO - --> status:True 2021-04-28 01:50:12,369 - root - INFO - SSH Connecting to 172.23.106.74 with username:root, attempt#1 of 5 2021-04-28 01:50:12,477 - root - INFO - SSH Connected to 172.23.106.74 as root 2021-04-28 01:50:12,747 - root - INFO - os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-04-28 01:50:13,050 - root - INFO - extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-04-28 01:50:13,057 - root - INFO - running command.raw on 172.23.106.74: curl --silent --show-error http://Administrator:password@localhost:8091/diag/eval -X POST -d 'ns_config:set(allow_nonlocal_eval, true).' 2021-04-28 01:50:13,115 - root - INFO - command executed successfully with root 2021-04-28 01:50:13,115 - root - INFO - ['ok'] 2021-04-28 01:50:13,117 - root - INFO - /diag/eval status on 172.23.106.74:8091: True content: [7,0] command: cluster_compat_mode:get_compat_version(). 2021-04-28 01:50:13,119 - root - INFO - /diag/eval status on 172.23.106.74:8091: True content: [7,0] command: cluster_compat_mode:get_compat_version(). 2021-04-28 01:50:13,126 - root - INFO - settings/indexes params : storageMode=plasma 2021-04-28 01:50:13,145 - root - INFO - server: ip:172.23.106.75 port:8091 ssh_username:root, nodes/self 2021-04-28 01:50:13,147 - root - INFO - {'uptime': '195', 'memoryTotal': 3945984000, 'memoryFree': 3407593472, 'mcdMemoryReserved': 3010, 'mcdMemoryAllocated': 3010, 'status': 'healthy', 'hostname': '172.23.106.75:8091', 'clusterCompatibility': 458752, 'clusterMembership': 'active', 'recoveryType': 'none', 'version': '7.0.0-5046-enterprise', 'os': 'x86_64-unknown-linux-gnu', 'ports': [], 'availableStorage': [], 'storage': [], 'memoryQuota': 2016, 'moxi': 11211, 'memcached': 11210, 'id': 'ns_1@cb.local', 'ip': '172.23.106.75', 'rest_username': '', 'rest_password': '', 'port': '8091', 'services': ['kv'], 'storageTotalRam': 3763, 'curr_items': 0} 2021-04-28 01:50:13,151 - root - INFO - pools/default params : memoryQuota=2016 2021-04-28 01:50:13,154 - root - INFO - --> init_node_services(Administrator,password,172.23.106.75,8091,['kv']) 2021-04-28 01:50:13,154 - root - INFO - /node/controller/setupServices params on 172.23.106.75: 8091:hostname=172.23.106.75&user=Administrator&password=password&services=kv 2021-04-28 01:50:13,156 - root - ERROR - POST http://172.23.106.75:8091//node/controller/setupServices body: hostname=172.23.106.75&user=Administrator&password=password&services=kv headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==', 'Accept': '*/*'} error: 400 reason: unknown b'["cannot change node services after cluster is provisioned"]' auth: Administrator:password 2021-04-28 01:50:13,157 - root - INFO - This node is already provisioned with services, we do not consider this as failure for test case 2021-04-28 01:50:13,157 - root - INFO - --> in init_cluster...Administrator,password,8091 2021-04-28 01:50:13,157 - root - INFO - settings/web params on 172.23.106.75:8091:port=8091&username=Administrator&password=password 2021-04-28 01:50:13,209 - root - INFO - --> status:True 2021-04-28 01:50:13,210 - root - INFO - SSH Connecting to 172.23.106.75 with username:root, attempt#1 of 5 2021-04-28 01:50:13,313 - root - INFO - SSH Connected to 172.23.106.75 as root 2021-04-28 01:50:13,584 - root - INFO - os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-04-28 01:50:13,884 - root - INFO - extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-04-28 01:50:13,892 - root - INFO - running command.raw on 172.23.106.75: curl --silent --show-error http://Administrator:password@localhost:8091/diag/eval -X POST -d 'ns_config:set(allow_nonlocal_eval, true).' 2021-04-28 01:50:13,951 - root - INFO - command executed successfully with root 2021-04-28 01:50:13,951 - root - INFO - ['ok'] 2021-04-28 01:50:13,954 - root - INFO - /diag/eval status on 172.23.106.75:8091: True content: [7,0] command: cluster_compat_mode:get_compat_version(). 2021-04-28 01:50:13,956 - root - INFO - /diag/eval status on 172.23.106.75:8091: True content: [7,0] command: cluster_compat_mode:get_compat_version(). 2021-04-28 01:50:13,961 - root - INFO - settings/indexes params : storageMode=plasma 2021-04-28 01:50:13,981 - root - INFO - server: ip:172.23.106.76 port:8091 ssh_username:root, nodes/self 2021-04-28 01:50:13,984 - root - INFO - {'uptime': '195', 'memoryTotal': 3945984000, 'memoryFree': 3385438208, 'mcdMemoryReserved': 3010, 'mcdMemoryAllocated': 3010, 'status': 'healthy', 'hostname': '172.23.106.76:8091', 'clusterCompatibility': 458752, 'clusterMembership': 'active', 'recoveryType': 'none', 'version': '7.0.0-5046-enterprise', 'os': 'x86_64-unknown-linux-gnu', 'ports': [], 'availableStorage': [], 'storage': [], 'memoryQuota': 2016, 'moxi': 11211, 'memcached': 11210, 'id': 'ns_1@cb.local', 'ip': '172.23.106.76', 'rest_username': '', 'rest_password': '', 'port': '8091', 'services': ['kv'], 'storageTotalRam': 3763, 'curr_items': 0} 2021-04-28 01:50:13,987 - root - INFO - pools/default params : memoryQuota=2016 2021-04-28 01:50:13,991 - root - INFO - --> init_node_services(Administrator,password,172.23.106.76,8091,['kv']) 2021-04-28 01:50:13,991 - root - INFO - /node/controller/setupServices params on 172.23.106.76: 8091:hostname=172.23.106.76&user=Administrator&password=password&services=kv 2021-04-28 01:50:13,993 - root - ERROR - POST http://172.23.106.76:8091//node/controller/setupServices body: hostname=172.23.106.76&user=Administrator&password=password&services=kv headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==', 'Accept': '*/*'} error: 400 reason: unknown b'["cannot change node services after cluster is provisioned"]' auth: Administrator:password 2021-04-28 01:50:13,994 - root - INFO - This node is already provisioned with services, we do not consider this as failure for test case 2021-04-28 01:50:13,994 - root - INFO - --> in init_cluster...Administrator,password,8091 2021-04-28 01:50:13,994 - root - INFO - settings/web params on 172.23.106.76:8091:port=8091&username=Administrator&password=password 2021-04-28 01:50:14,047 - root - INFO - --> status:True 2021-04-28 01:50:14,048 - root - INFO - SSH Connecting to 172.23.106.76 with username:root, attempt#1 of 5 2021-04-28 01:50:14,153 - root - INFO - SSH Connected to 172.23.106.76 as root 2021-04-28 01:50:14,413 - root - INFO - os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-04-28 01:50:14,719 - root - INFO - extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-04-28 01:50:14,727 - root - INFO - running command.raw on 172.23.106.76: curl --silent --show-error http://Administrator:password@localhost:8091/diag/eval -X POST -d 'ns_config:set(allow_nonlocal_eval, true).' 2021-04-28 01:50:14,785 - root - INFO - command executed successfully with root 2021-04-28 01:50:14,785 - root - INFO - ['ok'] 2021-04-28 01:50:14,788 - root - INFO - /diag/eval status on 172.23.106.76:8091: True content: [7,0] command: cluster_compat_mode:get_compat_version(). 2021-04-28 01:50:14,790 - root - INFO - /diag/eval status on 172.23.106.76:8091: True content: [7,0] command: cluster_compat_mode:get_compat_version(). 2021-04-28 01:50:14,794 - root - INFO - settings/indexes params : storageMode=plasma 2021-04-28 01:50:14,813 - root - INFO - server: ip:172.23.106.77 port:8091 ssh_username:root, nodes/self 2021-04-28 01:50:14,816 - root - INFO - {'uptime': '195', 'memoryTotal': 3945984000, 'memoryFree': 3360243712, 'mcdMemoryReserved': 3010, 'mcdMemoryAllocated': 3010, 'status': 'healthy', 'hostname': '172.23.106.77:8091', 'clusterCompatibility': 458752, 'clusterMembership': 'active', 'recoveryType': 'none', 'version': '7.0.0-5046-enterprise', 'os': 'x86_64-unknown-linux-gnu', 'ports': [], 'availableStorage': [], 'storage': [], 'memoryQuota': 2016, 'moxi': 11211, 'memcached': 11210, 'id': 'ns_1@cb.local', 'ip': '172.23.106.77', 'rest_username': '', 'rest_password': '', 'port': '8091', 'services': ['kv'], 'storageTotalRam': 3763, 'curr_items': 0} 2021-04-28 01:50:14,820 - root - INFO - pools/default params : memoryQuota=2016 2021-04-28 01:50:14,823 - root - INFO - --> init_node_services(Administrator,password,172.23.106.77,8091,['kv']) 2021-04-28 01:50:14,823 - root - INFO - /node/controller/setupServices params on 172.23.106.77: 8091:hostname=172.23.106.77&user=Administrator&password=password&services=kv 2021-04-28 01:50:14,826 - root - ERROR - POST http://172.23.106.77:8091//node/controller/setupServices body: hostname=172.23.106.77&user=Administrator&password=password&services=kv headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==', 'Accept': '*/*'} error: 400 reason: unknown b'["cannot change node services after cluster is provisioned"]' auth: Administrator:password 2021-04-28 01:50:14,826 - root - INFO - This node is already provisioned with services, we do not consider this as failure for test case 2021-04-28 01:50:14,826 - root - INFO - --> in init_cluster...Administrator,password,8091 2021-04-28 01:50:14,826 - root - INFO - settings/web params on 172.23.106.77:8091:port=8091&username=Administrator&password=password 2021-04-28 01:50:14,879 - root - INFO - --> status:True 2021-04-28 01:50:14,880 - root - INFO - SSH Connecting to 172.23.106.77 with username:root, attempt#1 of 5 2021-04-28 01:50:14,986 - root - INFO - SSH Connected to 172.23.106.77 as root 2021-04-28 01:50:15,256 - root - INFO - os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-04-28 01:50:15,560 - root - INFO - extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-04-28 01:50:15,567 - root - INFO - running command.raw on 172.23.106.77: curl --silent --show-error http://Administrator:password@localhost:8091/diag/eval -X POST -d 'ns_config:set(allow_nonlocal_eval, true).' 2021-04-28 01:50:15,626 - root - INFO - command executed successfully with root 2021-04-28 01:50:15,626 - root - INFO - ['ok'] 2021-04-28 01:50:15,629 - root - INFO - /diag/eval status on 172.23.106.77:8091: True content: [7,0] command: cluster_compat_mode:get_compat_version(). 2021-04-28 01:50:15,631 - root - INFO - /diag/eval status on 172.23.106.77:8091: True content: [7,0] command: cluster_compat_mode:get_compat_version(). 2021-04-28 01:50:15,636 - root - INFO - settings/indexes params : storageMode=plasma 2021-04-28 01:50:15,655 - root - INFO - server: ip:172.23.106.78 port:8091 ssh_username:root, nodes/self 2021-04-28 01:50:15,658 - root - INFO - {'uptime': '195', 'memoryTotal': 3945984000, 'memoryFree': 3367403520, 'mcdMemoryReserved': 3010, 'mcdMemoryAllocated': 3010, 'status': 'healthy', 'hostname': '172.23.106.78:8091', 'clusterCompatibility': 458752, 'clusterMembership': 'active', 'recoveryType': 'none', 'version': '7.0.0-5046-enterprise', 'os': 'x86_64-unknown-linux-gnu', 'ports': [], 'availableStorage': [], 'storage': [], 'memoryQuota': 2016, 'moxi': 11211, 'memcached': 11210, 'id': 'ns_1@cb.local', 'ip': '172.23.106.78', 'rest_username': '', 'rest_password': '', 'port': '8091', 'services': ['kv'], 'storageTotalRam': 3763, 'curr_items': 0} 2021-04-28 01:50:15,662 - root - INFO - pools/default params : memoryQuota=2016 2021-04-28 01:50:15,666 - root - INFO - --> init_node_services(Administrator,password,172.23.106.78,8091,['kv']) 2021-04-28 01:50:15,666 - root - INFO - /node/controller/setupServices params on 172.23.106.78: 8091:hostname=172.23.106.78&user=Administrator&password=password&services=kv 2021-04-28 01:50:15,668 - root - ERROR - POST http://172.23.106.78:8091//node/controller/setupServices body: hostname=172.23.106.78&user=Administrator&password=password&services=kv headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==', 'Accept': '*/*'} error: 400 reason: unknown b'["cannot change node services after cluster is provisioned"]' auth: Administrator:password 2021-04-28 01:50:15,668 - root - INFO - This node is already provisioned with services, we do not consider this as failure for test case 2021-04-28 01:50:15,668 - root - INFO - --> in init_cluster...Administrator,password,8091 2021-04-28 01:50:15,668 - root - INFO - settings/web params on 172.23.106.78:8091:port=8091&username=Administrator&password=password 2021-04-28 01:50:15,722 - root - INFO - --> status:True 2021-04-28 01:50:15,723 - root - INFO - SSH Connecting to 172.23.106.78 with username:root, attempt#1 of 5 2021-04-28 01:50:15,830 - root - INFO - SSH Connected to 172.23.106.78 as root 2021-04-28 01:50:16,104 - root - INFO - os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-04-28 01:50:16,411 - root - INFO - extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-04-28 01:50:16,420 - root - INFO - running command.raw on 172.23.106.78: curl --silent --show-error http://Administrator:password@localhost:8091/diag/eval -X POST -d 'ns_config:set(allow_nonlocal_eval, true).' 2021-04-28 01:50:16,477 - root - INFO - command executed successfully with root 2021-04-28 01:50:16,477 - root - INFO - ['ok'] 2021-04-28 01:50:16,480 - root - INFO - /diag/eval status on 172.23.106.78:8091: True content: [7,0] command: cluster_compat_mode:get_compat_version(). 2021-04-28 01:50:16,482 - root - INFO - /diag/eval status on 172.23.106.78:8091: True content: [7,0] command: cluster_compat_mode:get_compat_version(). 2021-04-28 01:50:16,485 - root - INFO - settings/indexes params : storageMode=plasma 2021-04-28 01:50:16,506 - root - INFO - **** add built-in 'cbadminbucket' user to node 172.23.106.64 **** 2021-04-28 01:50:16,517 - root - ERROR - DELETE http://172.23.106.64:8091/settings/rbac/users/local/cbadminbucket body: headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==', 'Accept': '*/*'} error: 404 reason: unknown b'"User was not found."' auth: Administrator:password 2021-04-28 01:50:16,520 - root - INFO - Exception while deleting user. Exception is -b'"User was not found."' 2021-04-28 01:50:16,584 - root - INFO - sleep for 5 secs. ... 2021-04-28 01:50:21,589 - root - INFO - **** add 'admin' role to 'cbadminbucket' user **** 2021-04-28 01:50:21,601 - root - INFO - done initializing cluster 2021-04-28 01:50:22,505 - root - INFO - adding node 172.23.106.67:8091 to cluster 2021-04-28 01:50:22,506 - root - INFO - adding remote node @172.23.106.67:8091 to this cluster @172.23.106.64:8091 2021-04-28 01:50:32,521 - root - INFO - rebalance progress took 10.02 seconds 2021-04-28 01:50:32,521 - root - INFO - sleep for 10 seconds after rebalance... 2021-04-28 01:50:45,938 - root - INFO - adding node 172.23.106.73:8091 to cluster 2021-04-28 01:50:45,938 - root - INFO - adding remote node @172.23.106.73:8091 to this cluster @172.23.106.64:8091 2021-04-28 01:50:55,954 - root - INFO - rebalance progress took 10.02 seconds 2021-04-28 01:50:55,955 - root - INFO - sleep for 10 seconds after rebalance... 2021-04-28 01:51:09,069 - root - INFO - adding node 172.23.106.74:8091 to cluster 2021-04-28 01:51:09,069 - root - INFO - adding remote node @172.23.106.74:8091 to this cluster @172.23.106.64:8091 2021-04-28 01:51:19,077 - root - INFO - rebalance progress took 10.01 seconds 2021-04-28 01:51:19,078 - root - INFO - sleep for 10 seconds after rebalance... 2021-04-28 01:53:31,883 - root - INFO - rebalance params : {'knownNodes': 'ns_1@172.23.106.64,ns_1@172.23.106.67,ns_1@172.23.106.73,ns_1@172.23.106.74', 'ejectedNodes': '', 'user': 'Administrator', 'password': 'password'} 2021-04-28 01:53:31,911 - root - INFO - rebalance operation started 2021-04-28 01:53:41,924 - root - INFO - Rebalance - status: none, progress: 100.00% 2021-04-28 01:53:41,930 - root - INFO - rebalancing was completed with progress: 100% in 10.018722295761108 sec 2021-04-28 01:53:41,931 - root - INFO - SSH Connecting to 172.23.106.64 with username:root, attempt#1 of 5 2021-04-28 01:53:42,031 - root - INFO - SSH Connected to 172.23.106.64 as root 2021-04-28 01:53:42,286 - root - INFO - os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-04-28 01:53:42,604 - root - INFO - extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-04-28 01:53:42,612 - root - INFO - running command.raw on 172.23.106.64: curl --silent --show-error http://Administrator:password@localhost:8091/diag/eval -X POST -d 'ns_config:set(allow_nonlocal_eval, true).' 2021-04-28 01:53:42,672 - root - INFO - command executed successfully with root 2021-04-28 01:53:42,672 - root - INFO - ['ok'] 2021-04-28 01:53:42,672 - root - INFO - Enabled diag/eval for non-local hosts from 172.23.106.64 2021-04-28 01:53:42,672 - root - INFO - ============== basetestcase setup was finished for test #1 test_online_upgrade_with_regular_rebalance_with_eventing ============== 2021-04-28 01:53:42,682 - root - INFO - SSH Connecting to 172.23.106.64 with username:root, attempt#1 of 5 2021-04-28 01:53:42,782 - root - INFO - SSH Connected to 172.23.106.64 as root 2021-04-28 01:53:43,034 - root - INFO - os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-04-28 01:53:43,352 - root - INFO - extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-04-28 01:53:43,353 - root - INFO - SSH Connecting to 172.23.106.67 with username:root, attempt#1 of 5 2021-04-28 01:53:43,459 - root - INFO - SSH Connected to 172.23.106.67 as root 2021-04-28 01:53:43,729 - root - INFO - os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-04-28 01:53:44,032 - root - INFO - extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-04-28 01:53:44,034 - root - INFO - SSH Connecting to 172.23.106.73 with username:root, attempt#1 of 5 2021-04-28 01:53:44,138 - root - INFO - SSH Connected to 172.23.106.73 as root 2021-04-28 01:53:44,432 - root - INFO - os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-04-28 01:53:44,751 - root - INFO - extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-04-28 01:53:44,752 - root - INFO - SSH Connecting to 172.23.106.74 with username:root, attempt#1 of 5 2021-04-28 01:53:44,858 - root - INFO - SSH Connected to 172.23.106.74 as root 2021-04-28 01:53:45,137 - root - INFO - os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-04-28 01:53:45,452 - root - INFO - extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-04-28 01:53:45,453 - root - INFO - SSH Connecting to 172.23.106.75 with username:root, attempt#1 of 5 2021-04-28 01:53:45,559 - root - INFO - SSH Connected to 172.23.106.75 as root 2021-04-28 01:53:45,828 - root - INFO - os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-04-28 01:53:46,135 - root - INFO - extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-04-28 01:53:46,137 - root - INFO - SSH Connecting to 172.23.106.76 with username:root, attempt#1 of 5 2021-04-28 01:53:46,239 - root - INFO - SSH Connected to 172.23.106.76 as root 2021-04-28 01:53:46,504 - root - INFO - os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-04-28 01:53:46,811 - root - INFO - extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-04-28 01:53:46,812 - root - INFO - SSH Connecting to 172.23.106.77 with username:root, attempt#1 of 5 2021-04-28 01:53:46,918 - root - INFO - SSH Connected to 172.23.106.77 as root 2021-04-28 01:53:47,197 - root - INFO - os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-04-28 01:53:47,502 - root - INFO - extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-04-28 01:53:47,504 - root - INFO - SSH Connecting to 172.23.106.78 with username:root, attempt#1 of 5 2021-04-28 01:53:47,609 - root - INFO - SSH Connected to 172.23.106.78 as root 2021-04-28 01:53:47,884 - root - INFO - os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-04-28 01:53:48,190 - root - INFO - extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-04-28 01:53:52,412 - root - INFO - ------- Cluster statistics ------- 2021-04-28 01:53:52,412 - root - INFO - 172.23.106.64:8091 => {'services': ['kv'], 'cpu_utilization': 2.506265664160401, 'mem_free': 3388854272, 'mem_total': 3945984000, 'swap_mem_used': 40370176, 'swap_mem_total': 2147479552} 2021-04-28 01:53:52,412 - root - INFO - 172.23.106.67:8091 => {'services': ['eventing'], 'cpu_utilization': 5.168088309081786, 'mem_free': 3377426432, 'mem_total': 3945984000, 'swap_mem_used': 0, 'swap_mem_total': 2147479552} 2021-04-28 01:53:52,412 - root - INFO - 172.23.106.73:8091 => {'services': ['index'], 'cpu_utilization': 4.767879548306148, 'mem_free': 3326574592, 'mem_total': 3945984000, 'swap_mem_used': 0, 'swap_mem_total': 2147479552} 2021-04-28 01:53:52,412 - root - INFO - 172.23.106.74:8091 => {'services': ['n1ql'], 'cpu_utilization': 2.211055276381909, 'mem_free': 3354771456, 'mem_total': 3945984000, 'swap_mem_used': 0, 'swap_mem_total': 2147479552} 2021-04-28 01:53:52,413 - root - INFO - --- End of cluster statistics --- 2021-04-28 01:53:52,414 - root - INFO - SSH Connecting to 172.23.106.64 with username:root, attempt#1 of 5 2021-04-28 01:53:52,515 - root - INFO - SSH Connected to 172.23.106.64 as root 2021-04-28 01:53:52,768 - root - INFO - os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-04-28 01:53:53,088 - root - INFO - extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-04-28 01:53:54,474 - root - INFO - list of n1ql nodes in cluster: [ip:172.23.106.74 port:8091 ssh_username:root] 2021-04-28 01:53:54,481 - root - INFO - list of eventing nodes in cluster: [ip:172.23.106.67 port:8091 ssh_username:root] 2021-04-28 01:53:54,486 - root - INFO - settings/indexes params : storageMode=plasma 2021-04-28 01:53:54,491 - root - INFO - Setting the min possible memory quota so that adding mode nodes to the cluster wouldn't be a problem. 2021-04-28 01:53:54,491 - root - INFO - pools/default params : memoryQuota=330 2021-04-28 01:53:54,497 - root - INFO - pools/default params : indexMemoryQuota=256 2021-04-28 01:53:54,502 - root - INFO - pools/default params : eventingMemoryQuota=512 2021-04-28 01:53:54,519 - root - INFO - list of n1ql nodes in cluster: [ip:172.23.106.74 port:8091 ssh_username:root] 2021-04-28 01:53:54,522 - root - INFO - ============== EventingBaseTest setup has completed ============== 2021-04-28 01:53:54,522 - root - INFO - ============== NewUpgradeBaseTest setup has started ============== 2021-04-28 01:53:54,523 - root - INFO - SSH Connecting to 172.23.106.64 with username:root, attempt#1 of 5 2021-04-28 01:53:54,624 - root - INFO - SSH Connected to 172.23.106.64 as root 2021-04-28 01:53:54,879 - root - INFO - os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-04-28 01:53:55,201 - root - INFO - extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-04-28 01:53:55,201 - root - INFO - ============== NewUpgradeBaseTest setup has completed ============== 2021-04-28 01:53:55,247 - root - INFO - ============== EventingUpgrade setup has completed ============== 2021-04-28 01:53:55,247 - root - INFO - will install 6.6.1-9213 on ['172.23.106.64', '172.23.106.67', '172.23.106.73', '172.23.106.74'] 2021-04-28 01:53:55,249 - root - INFO - SSH Connecting to 172.23.106.67 with username:root, attempt#1 of 5 2021-04-28 01:53:55,251 - root - INFO - SSH Connecting to 172.23.106.64 with username:root, attempt#1 of 5 2021-04-28 01:53:55,253 - root - INFO - SSH Connecting to 172.23.106.73 with username:root, attempt#1 of 5 2021-04-28 01:53:55,255 - root - INFO - SSH Connecting to 172.23.106.74 with username:root, attempt#1 of 5 2021-04-28 01:53:55,350 - root - INFO - SSH Connected to 172.23.106.64 as root 2021-04-28 01:53:55,356 - root - INFO - SSH Connected to 172.23.106.74 as root 2021-04-28 01:53:55,359 - root - INFO - SSH Connected to 172.23.106.67 as root 2021-04-28 01:53:55,361 - root - INFO - SSH Connected to 172.23.106.73 as root 2021-04-28 01:53:55,612 - root - INFO - os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-04-28 01:53:55,627 - root - INFO - os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-04-28 01:53:55,641 - root - INFO - os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-04-28 01:53:55,658 - root - INFO - os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-04-28 01:53:55,936 - root - INFO - extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-04-28 01:53:55,936 - root - INFO - 172.23.106.74 *****In couchbase uninstall**** 2021-04-28 01:53:55,937 - root - INFO - CentOS 2021-04-28 01:53:55,941 - root - INFO - extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-04-28 01:53:55,941 - root - INFO - 172.23.106.64 *****In couchbase uninstall**** 2021-04-28 01:53:55,942 - root - INFO - CentOS 2021-04-28 01:53:55,979 - root - INFO - extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-04-28 01:53:55,979 - root - INFO - 172.23.106.73 *****In couchbase uninstall**** 2021-04-28 01:53:55,980 - root - INFO - CentOS 2021-04-28 01:53:55,995 - root - INFO - extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-04-28 01:53:55,996 - root - INFO - 172.23.106.67 *****In couchbase uninstall**** 2021-04-28 01:53:55,996 - root - INFO - CentOS 2021-04-28 01:53:56,044 - root - INFO - running command.raw on 172.23.106.74: killall -9 rpm 2021-04-28 01:53:56,054 - root - INFO - running command.raw on 172.23.106.64: killall -9 rpm 2021-04-28 01:53:56,075 - root - INFO - command executed with root but got an error ['rpm: no process found'] ... 2021-04-28 01:53:56,075 - root - ERROR - rpm: no process found 2021-04-28 01:53:56,076 - root - INFO - running command.raw on 172.23.106.74: rm -f /var/lib/rpm/.rpm.lock 2021-04-28 01:53:56,087 - root - INFO - command executed with root but got an error ['rpm: no process found'] ... 2021-04-28 01:53:56,087 - root - ERROR - rpm: no process found 2021-04-28 01:53:56,087 - root - INFO - running command.raw on 172.23.106.64: rm -f /var/lib/rpm/.rpm.lock 2021-04-28 01:53:56,092 - root - INFO - running command.raw on 172.23.106.73: killall -9 rpm 2021-04-28 01:53:56,104 - root - INFO - running command.raw on 172.23.106.67: killall -9 rpm 2021-04-28 01:53:56,126 - root - INFO - command executed with root but got an error ['rpm: no process found'] ... 2021-04-28 01:53:56,127 - root - ERROR - rpm: no process found 2021-04-28 01:53:56,127 - root - INFO - running command.raw on 172.23.106.73: rm -f /var/lib/rpm/.rpm.lock 2021-04-28 01:53:56,132 - root - INFO - command executed successfully with root 2021-04-28 01:53:56,135 - root - INFO - command executed with root but got an error ['rpm: no process found'] ... 2021-04-28 01:53:56,135 - root - ERROR - rpm: no process found 2021-04-28 01:53:56,135 - root - INFO - running command.raw on 172.23.106.67: rm -f /var/lib/rpm/.rpm.lock 2021-04-28 01:53:56,144 - root - INFO - command executed successfully with root 2021-04-28 01:53:56,183 - root - INFO - command executed successfully with root 2021-04-28 01:53:56,190 - root - INFO - command executed successfully with root 2021-04-28 01:53:56,236 - root - INFO - running rpm -e to remove couchbase-server 2021-04-28 01:53:56,237 - root - INFO - running command.raw on 172.23.106.74: rpm -e couchbase-server 2021-04-28 01:53:56,255 - root - INFO - running rpm -e to remove couchbase-server 2021-04-28 01:53:56,255 - root - INFO - running command.raw on 172.23.106.64: rpm -e couchbase-server 2021-04-28 01:53:56,296 - root - INFO - running rpm -e to remove couchbase-server 2021-04-28 01:53:56,296 - root - INFO - running command.raw on 172.23.106.73: rpm -e couchbase-server 2021-04-28 01:53:56,297 - root - INFO - running rpm -e to remove couchbase-server 2021-04-28 01:53:56,298 - root - INFO - running command.raw on 172.23.106.67: rpm -e couchbase-server 2021-04-28 01:53:58,974 - root - INFO - command executed with root but got an error ['warning: /opt/couchbase/var/lib/couchbase/ip_start saved as /opt/couchbase/var/lib/couchbase/ip_start.rpmsave', 'warning: file /opt/couchbase/var/lib/couchbase/ip: remove failed: No such file or directory', 'warning: /opt/couchbase/var/lib/couchbase/config/config.dat saved as /opt/couchbase/var/lib/couchbase/config/config.dat.rpmsave', 'warning: /opt/couchbase/etc/couchdb/local.ini saved as /opt ... 2021-04-28 01:53:58,974 - root - INFO - running command.raw on 172.23.106.74: pkill -u couchbase 2021-04-28 01:53:58,997 - root - INFO - command executed successfully with root 2021-04-28 01:53:58,998 - root - INFO - running command.raw on 172.23.106.74: ps -ef | grep couchbase 2021-04-28 01:53:59,065 - root - INFO - command executed successfully with root 2021-04-28 01:53:59,118 - root - INFO - command executed with root but got an error ['warning: /opt/couchbase/var/lib/couchbase/ip_start saved as /opt/couchbase/var/lib/couchbase/ip_start.rpmsave', 'warning: file /opt/couchbase/var/lib/couchbase/ip: remove failed: No such file or directory', 'warning: /opt/couchbase/var/lib/couchbase/config/config.dat saved as /opt/couchbase/var/lib/couchbase/config/config.dat.rpmsave', 'warning: /opt/couchbase/etc/couchdb/local.ini saved as /opt ... 2021-04-28 01:53:59,118 - root - INFO - running command.raw on 172.23.106.73: pkill -u couchbase 2021-04-28 01:53:59,144 - root - INFO - command executed successfully with root 2021-04-28 01:53:59,144 - root - INFO - running command.raw on 172.23.106.73: ps -ef | grep couchbase 2021-04-28 01:53:59,159 - root - INFO - command executed with root but got an error ['warning: /opt/couchbase/var/lib/couchbase/ip_start saved as /opt/couchbase/var/lib/couchbase/ip_start.rpmsave', 'warning: file /opt/couchbase/var/lib/couchbase/ip: remove failed: No such file or directory', 'warning: /opt/couchbase/var/lib/couchbase/config/config.dat saved as /opt/couchbase/var/lib/couchbase/config/config.dat.rpmsave', 'warning: /opt/couchbase/etc/couchdb/local.ini saved as /opt ... 2021-04-28 01:53:59,159 - root - INFO - running command.raw on 172.23.106.67: pkill -u couchbase 2021-04-28 01:53:59,216 - root - INFO - command executed successfully with root 2021-04-28 01:53:59,218 - root - INFO - command executed successfully with root 2021-04-28 01:53:59,218 - root - INFO - running command.raw on 172.23.106.67: ps -ef | grep couchbase 2021-04-28 01:53:59,249 - root - INFO - command executed with root but got an error ['warning: file /opt/couchbase/var/lib/couchbase/ip_start: remove failed: No such file or directory', 'warning: /opt/couchbase/var/lib/couchbase/ip saved as /opt/couchbase/var/lib/couchbase/ip.rpmsave', 'warning: /opt/couchbase/var/lib/couchbase/config/config.dat saved as /opt/couchbase/var/lib/couchbase/config/config.dat.rpmsave', 'warning: /opt/couchbase/etc/couchdb/local.ini saved as /opt/couch ... 2021-04-28 01:53:59,249 - root - INFO - running command.raw on 172.23.106.64: pkill -u couchbase 2021-04-28 01:53:59,275 - root - INFO - command executed successfully with root 2021-04-28 01:53:59,275 - root - INFO - running command.raw on 172.23.106.64: ps -ef | grep couchbase 2021-04-28 01:53:59,287 - root - INFO - command executed successfully with root 2021-04-28 01:53:59,348 - root - INFO - command executed successfully with root 2021-04-28 01:54:00,507 - root - INFO - running command.raw on 172.23.106.74: ipcrm 2021-04-28 01:54:00,561 - root - INFO - command executed successfully with root 2021-04-28 01:54:00,701 - root - INFO - running command.raw on 172.23.106.73: ipcrm 2021-04-28 01:54:00,738 - root - INFO - running command.raw on 172.23.106.67: ipcrm 2021-04-28 01:54:00,757 - root - INFO - command executed successfully with root 2021-04-28 01:54:00,791 - root - INFO - command executed successfully with root 2021-04-28 01:54:00,870 - root - INFO - running command.raw on 172.23.106.64: ipcrm 2021-04-28 01:54:00,926 - root - INFO - command executed successfully with root thread uninstaller-thread-172.23.106.64 finished thread uninstaller-thread-172.23.106.67 finished thread uninstaller-thread-172.23.106.73 finished thread uninstaller-thread-172.23.106.74 finished 2021-04-28 01:54:00,928 - root - INFO - SSH Connecting to 172.23.106.64 with username:root, attempt#1 of 5 2021-04-28 01:54:01,027 - root - INFO - SSH Connected to 172.23.106.64 as root 2021-04-28 01:54:01,281 - root - INFO - os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-04-28 01:54:01,600 - root - INFO - extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-04-28 01:54:01,662 - root - INFO - SSH Connecting to 172.23.106.67 with username:root, attempt#1 of 5 2021-04-28 01:54:01,767 - root - INFO - SSH Connected to 172.23.106.67 as root 2021-04-28 01:54:02,038 - root - INFO - os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-04-28 01:54:02,344 - root - INFO - extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-04-28 01:54:02,366 - root - INFO - SSH Connecting to 172.23.106.73 with username:root, attempt#1 of 5 2021-04-28 01:54:02,471 - root - INFO - SSH Connected to 172.23.106.73 as root 2021-04-28 01:54:02,755 - root - INFO - os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-04-28 01:54:03,069 - root - INFO - extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-04-28 01:54:03,092 - root - INFO - SSH Connecting to 172.23.106.74 with username:root, attempt#1 of 5 2021-04-28 01:54:03,193 - root - INFO - SSH Connected to 172.23.106.74 as root 2021-04-28 01:54:03,460 - root - INFO - os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-04-28 01:54:03,766 - root - INFO - extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-04-28 01:54:03,787 - root - INFO - ********CouchbaseServerInstaller:install 2021-04-28 01:54:03,787 - root - INFO - ********CouchbaseServerInstaller:install 2021-04-28 01:54:03,788 - root - INFO - SSH Connecting to 172.23.106.67 with username:root, attempt#1 of 5 2021-04-28 01:54:03,789 - root - INFO - SSH Connecting to 172.23.106.64 with username:root, attempt#1 of 5 2021-04-28 01:54:03,790 - root - INFO - ********CouchbaseServerInstaller:install 2021-04-28 01:54:03,790 - root - INFO - ********CouchbaseServerInstaller:install 2021-04-28 01:54:03,792 - root - INFO - SSH Connecting to 172.23.106.73 with username:root, attempt#1 of 5 2021-04-28 01:54:03,793 - root - INFO - SSH Connecting to 172.23.106.74 with username:root, attempt#1 of 5 2021-04-28 01:54:03,893 - root - INFO - SSH Connected to 172.23.106.74 as root 2021-04-28 01:54:03,894 - root - INFO - SSH Connected to 172.23.106.64 as root 2021-04-28 01:54:03,894 - root - INFO - SSH Connected to 172.23.106.67 as root 2021-04-28 01:54:03,898 - root - INFO - SSH Connected to 172.23.106.73 as root 2021-04-28 01:54:04,151 - root - INFO - os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-04-28 01:54:04,166 - root - INFO - os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-04-28 01:54:04,167 - root - INFO - os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-04-28 01:54:04,187 - root - INFO - os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-04-28 01:54:04,473 - root - INFO - extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 --> remote server architecture_type..x86_64 *** OS version of this server 172.23.106.74 is centos 7 *** 2021-04-28 01:54:04,475 - root - INFO - SSH Connecting to 172.23.106.74 with username:root, attempt#1 of 5 2021-04-28 01:54:04,476 - root - INFO - extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 --> remote server architecture_type..x86_64 *** OS version of this server 172.23.106.64 is centos 7 *** 2021-04-28 01:54:04,478 - root - INFO - SSH Connecting to 172.23.106.64 with username:root, attempt#1 of 5 2021-04-28 01:54:04,505 - root - INFO - extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 --> remote server architecture_type..x86_64 *** OS version of this server 172.23.106.73 is centos 7 *** 2021-04-28 01:54:04,507 - root - INFO - SSH Connecting to 172.23.106.73 with username:root, attempt#1 of 5 2021-04-28 01:54:04,516 - root - INFO - extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 --> remote server architecture_type..x86_64 *** OS version of this server 172.23.106.67 is centos 7 *** 2021-04-28 01:54:04,518 - root - INFO - SSH Connecting to 172.23.106.67 with username:root, attempt#1 of 5 2021-04-28 01:54:04,577 - root - INFO - SSH Connected to 172.23.106.74 as root 2021-04-28 01:54:04,579 - root - INFO - SSH Connected to 172.23.106.64 as root 2021-04-28 01:54:04,609 - root - INFO - SSH Connected to 172.23.106.73 as root 2021-04-28 01:54:04,619 - root - INFO - SSH Connected to 172.23.106.67 as root 2021-04-28 01:54:04,835 - root - INFO - os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-04-28 01:54:04,846 - root - INFO - os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-04-28 01:54:04,897 - root - INFO - os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-04-28 01:54:04,899 - root - INFO - os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-04-28 01:54:05,157 - root - INFO - extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 check if http://172.23.126.166/builds/releases/6.6.1/couchbase-server-enterprise-6.6.1-centos7.x86_64.rpm is live 2021-04-28 01:54:05,158 - root - INFO - Trying to check is this url alive: http://172.23.126.166/builds/releases/6.6.1/couchbase-server-enterprise-6.6.1-centos7.x86_64.rpm 2021-04-28 01:54:05,160 - root - INFO - extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 check if http://172.23.126.166/builds/releases/6.6.1/couchbase-server-enterprise-6.6.1-centos7.x86_64.rpm is live 2021-04-28 01:54:05,161 - root - INFO - Trying to check is this url alive: http://172.23.126.166/builds/releases/6.6.1/couchbase-server-enterprise-6.6.1-centos7.x86_64.rpm 2021-04-28 01:54:05,174 - root - INFO - This url http://172.23.126.166/builds/releases/6.6.1/couchbase-server-enterprise-6.6.1-centos7.x86_64.rpm is live 2021-04-28 01:54:05,175 - root - INFO - SSH Connecting to 172.23.106.74 with username:root, attempt#1 of 5 2021-04-28 01:54:05,175 - root - INFO - This url http://172.23.126.166/builds/releases/6.6.1/couchbase-server-enterprise-6.6.1-centos7.x86_64.rpm is live 2021-04-28 01:54:05,177 - root - INFO - SSH Connecting to 172.23.106.64 with username:root, attempt#1 of 5 2021-04-28 01:54:05,209 - root - INFO - extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 check if http://172.23.126.166/builds/releases/6.6.1/couchbase-server-enterprise-6.6.1-centos7.x86_64.rpm is live 2021-04-28 01:54:05,209 - root - INFO - Trying to check is this url alive: http://172.23.126.166/builds/releases/6.6.1/couchbase-server-enterprise-6.6.1-centos7.x86_64.rpm 2021-04-28 01:54:05,211 - root - INFO - This url http://172.23.126.166/builds/releases/6.6.1/couchbase-server-enterprise-6.6.1-centos7.x86_64.rpm is live 2021-04-28 01:54:05,213 - root - INFO - SSH Connecting to 172.23.106.67 with username:root, attempt#1 of 5 2021-04-28 01:54:05,223 - root - INFO - extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 check if http://172.23.126.166/builds/releases/6.6.1/couchbase-server-enterprise-6.6.1-centos7.x86_64.rpm is live 2021-04-28 01:54:05,223 - root - INFO - Trying to check is this url alive: http://172.23.126.166/builds/releases/6.6.1/couchbase-server-enterprise-6.6.1-centos7.x86_64.rpm 2021-04-28 01:54:05,225 - root - INFO - This url http://172.23.126.166/builds/releases/6.6.1/couchbase-server-enterprise-6.6.1-centos7.x86_64.rpm is live 2021-04-28 01:54:05,226 - root - INFO - SSH Connecting to 172.23.106.73 with username:root, attempt#1 of 5 2021-04-28 01:54:05,276 - root - INFO - SSH Connected to 172.23.106.74 as root 2021-04-28 01:54:05,279 - root - INFO - SSH Connected to 172.23.106.64 as root 2021-04-28 01:54:05,315 - root - INFO - SSH Connected to 172.23.106.67 as root 2021-04-28 01:54:05,330 - root - INFO - SSH Connected to 172.23.106.73 as root 2021-04-28 01:54:05,540 - root - INFO - os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-04-28 01:54:05,549 - root - INFO - os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-04-28 01:54:05,591 - root - INFO - os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-04-28 01:54:05,618 - root - INFO - os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-04-28 01:54:05,862 - root - INFO - extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 Downloading the build...url : http://172.23.126.166/builds/releases/6.6.1/couchbase-server-enterprise-6.6.1-centos7.x86_64.rpm name : couchbase-server-enterprise_x86_64_6.6.1-9213.rpm product : couchbase-server-enterprise product_version : 6.6.1-9213 os : deliverable_type : rpm architecture_type : x86_64 2021-04-28 01:54:05,862 - root - INFO - running command.raw on 172.23.106.74: /sbin/iptables -F 2021-04-28 01:54:05,865 - root - INFO - extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 Downloading the build...url : http://172.23.126.166/builds/releases/6.6.1/couchbase-server-enterprise-6.6.1-centos7.x86_64.rpm name : couchbase-server-enterprise_x86_64_6.6.1-9213.rpm product : couchbase-server-enterprise product_version : 6.6.1-9213 os : deliverable_type : rpm architecture_type : x86_64 2021-04-28 01:54:05,866 - root - INFO - running command.raw on 172.23.106.64: /sbin/iptables -F 2021-04-28 01:54:05,882 - root - INFO - command executed successfully with root 2021-04-28 01:54:05,882 - root - INFO - running command.raw on 172.23.106.74: /sbin/iptables -t nat -F 2021-04-28 01:54:05,902 - root - INFO - extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 Downloading the build...url : http://172.23.126.166/builds/releases/6.6.1/couchbase-server-enterprise-6.6.1-centos7.x86_64.rpm name : couchbase-server-enterprise_x86_64_6.6.1-9213.rpm product : couchbase-server-enterprise product_version : 6.6.1-9213 os : deliverable_type : rpm architecture_type : x86_64 2021-04-28 01:54:05,903 - root - INFO - running command.raw on 172.23.106.67: /sbin/iptables -F 2021-04-28 01:54:05,921 - root - INFO - command executed successfully with root 2021-04-28 01:54:05,921 - root - INFO - running command.raw on 172.23.106.67: /sbin/iptables -t nat -F 2021-04-28 01:54:05,928 - root - INFO - command executed successfully with root 2021-04-28 01:54:05,928 - root - INFO - running command.raw on 172.23.106.64: /sbin/iptables -t nat -F 2021-04-28 01:54:05,938 - root - INFO - command executed successfully with root 2021-04-28 01:54:05,938 - root - INFO - SSH Connecting to 172.23.106.74 with username:root, attempt#1 of 5 2021-04-28 01:54:05,942 - root - INFO - extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 Downloading the build...url : http://172.23.126.166/builds/releases/6.6.1/couchbase-server-enterprise-6.6.1-centos7.x86_64.rpm name : couchbase-server-enterprise_x86_64_6.6.1-9213.rpm product : couchbase-server-enterprise product_version : 6.6.1-9213 os : deliverable_type : rpm architecture_type : x86_64 2021-04-28 01:54:05,943 - root - INFO - running command.raw on 172.23.106.73: /sbin/iptables -F 2021-04-28 01:54:05,963 - root - INFO - command executed successfully with root 2021-04-28 01:54:05,963 - root - INFO - running command.raw on 172.23.106.73: /sbin/iptables -t nat -F 2021-04-28 01:54:05,977 - root - INFO - command executed successfully with root 2021-04-28 01:54:05,977 - root - INFO - SSH Connecting to 172.23.106.67 with username:root, attempt#1 of 5 2021-04-28 01:54:05,984 - root - INFO - command executed successfully with root 2021-04-28 01:54:05,984 - root - INFO - SSH Connecting to 172.23.106.64 with username:root, attempt#1 of 5 2021-04-28 01:54:06,020 - root - INFO - command executed successfully with root 2021-04-28 01:54:06,020 - root - INFO - SSH Connecting to 172.23.106.73 with username:root, attempt#1 of 5 2021-04-28 01:54:06,040 - root - INFO - SSH Connected to 172.23.106.74 as root 2021-04-28 01:54:06,076 - root - INFO - SSH Connected to 172.23.106.67 as root 2021-04-28 01:54:06,084 - root - INFO - SSH Connected to 172.23.106.64 as root 2021-04-28 01:54:06,118 - root - INFO - SSH Connected to 172.23.106.73 as root 2021-04-28 01:54:06,313 - root - INFO - running command.raw on 172.23.106.74: cd /tmp;wget -nc -q -O couchbase-server-enterprise_x86_64_6.6.1-9213.rpm http://172.23.126.166/builds/releases/6.6.1/couchbase-server-enterprise-6.6.1-centos7.x86_64.rpm;cd /tmp;ls -lh 2021-04-28 01:54:06,356 - root - INFO - running command.raw on 172.23.106.67: cd /tmp;wget -nc -q -O couchbase-server-enterprise_x86_64_6.6.1-9213.rpm http://172.23.126.166/builds/releases/6.6.1/couchbase-server-enterprise-6.6.1-centos7.x86_64.rpm;cd /tmp;ls -lh 2021-04-28 01:54:06,361 - root - INFO - running command.raw on 172.23.106.64: cd /tmp;wget -nc -q -O couchbase-server-enterprise_x86_64_6.6.1-9213.rpm http://172.23.126.166/builds/releases/6.6.1/couchbase-server-enterprise-6.6.1-centos7.x86_64.rpm;cd /tmp;ls -lh 2021-04-28 01:54:06,376 - root - INFO - command executed successfully with root 2021-04-28 01:54:06,376 - root - INFO - total 361M 2021-04-28 01:54:06,376 - root - INFO - -rw-r--r--. 1 root root 361M Dec 16 01:59 couchbase-server-enterprise_x86_64_6.6.1-9213.rpm 2021-04-28 01:54:06,376 - root - INFO - drwx------. 3 root root 17 Dec 30 12:15 systemd-private-87b23c726f78425397d329e96b2525e6-ntpd.service-XWjTud 2021-04-28 01:54:06,414 - root - INFO - running command.raw on 172.23.106.73: cd /tmp;wget -nc -q -O couchbase-server-enterprise_x86_64_6.6.1-9213.rpm http://172.23.126.166/builds/releases/6.6.1/couchbase-server-enterprise-6.6.1-centos7.x86_64.rpm;cd /tmp;ls -lh 2021-04-28 01:54:06,419 - root - INFO - command executed successfully with root 2021-04-28 01:54:06,419 - root - INFO - total 361M 2021-04-28 01:54:06,419 - root - INFO - -rw-r--r--. 1 root root 361M Dec 16 01:59 couchbase-server-enterprise_x86_64_6.6.1-9213.rpm 2021-04-28 01:54:06,420 - root - INFO - drwx------. 3 root root 17 Feb 14 23:31 systemd-private-44fb7cd225544b82bd1ee85044c735bd-ntpd.service-YvbGZI 2021-04-28 01:54:06,424 - root - INFO - command executed successfully with root 2021-04-28 01:54:06,424 - root - INFO - total 361M 2021-04-28 01:54:06,424 - root - INFO - -rw-r--r--. 1 root root 361M Dec 16 01:59 couchbase-server-enterprise_x86_64_6.6.1-9213.rpm 2021-04-28 01:54:06,424 - root - INFO - drwx------. 3 root root 17 Dec 30 12:15 systemd-private-5a7ef6ce23df4c698305ac55070508a2-ntpd.service-0R8Zg1 2021-04-28 01:54:06,437 - root - INFO - *****install server *** 2021-04-28 01:54:06,437 - root - INFO - -->deliverable_type : rpm 2021-04-28 01:54:06,438 - root - INFO - /tmp/couchbase-server-enterprise_x86_64_6.6.1-9213.rpm or /tmp/couchbase-server-enterprise 2021-04-28 01:54:06,457 - root - INFO - --> execute_command: end... --> inside rpm check... 2021-04-28 01:54:06,457 - root - INFO - running command.raw on 172.23.106.74: cat /etc/redhat-release 2021-04-28 01:54:06,478 - root - INFO - command executed successfully with root 2021-04-28 01:54:06,478 - root - INFO - total 361M 2021-04-28 01:54:06,478 - root - INFO - -rw-r--r--. 1 root root 361M Dec 16 01:59 couchbase-server-enterprise_x86_64_6.6.1-9213.rpm 2021-04-28 01:54:06,478 - root - INFO - drwx------. 3 root root 17 Feb 14 23:33 systemd-private-14ec17851ff44c07bbf9443e01d76e82-ntpd.service-gcmmPn 2021-04-28 01:54:06,480 - root - INFO - *****install server *** 2021-04-28 01:54:06,480 - root - INFO - -->deliverable_type : rpm 2021-04-28 01:54:06,480 - root - INFO - /tmp/couchbase-server-enterprise_x86_64_6.6.1-9213.rpm or /tmp/couchbase-server-enterprise 2021-04-28 01:54:06,488 - root - INFO - *****install server *** 2021-04-28 01:54:06,488 - root - INFO - -->deliverable_type : rpm 2021-04-28 01:54:06,488 - root - INFO - /tmp/couchbase-server-enterprise_x86_64_6.6.1-9213.rpm or /tmp/couchbase-server-enterprise 2021-04-28 01:54:06,499 - root - INFO - --> execute_command: end... --> inside rpm check... 2021-04-28 01:54:06,499 - root - INFO - running command.raw on 172.23.106.67: cat /etc/redhat-release 2021-04-28 01:54:06,509 - root - INFO - --> execute_command: end... --> inside rpm check... 2021-04-28 01:54:06,509 - root - INFO - running command.raw on 172.23.106.64: cat /etc/redhat-release 2021-04-28 01:54:06,511 - root - INFO - command executed successfully with root 2021-04-28 01:54:06,511 - root - INFO - CentOS Linux release 7.6.1810 (Core) 2021-04-28 01:54:06,541 - root - INFO - *****install server *** 2021-04-28 01:54:06,541 - root - INFO - -->deliverable_type : rpm 2021-04-28 01:54:06,541 - root - INFO - /tmp/couchbase-server-enterprise_x86_64_6.6.1-9213.rpm or /tmp/couchbase-server-enterprise 2021-04-28 01:54:06,554 - root - INFO - command executed successfully with root 2021-04-28 01:54:06,554 - root - INFO - CentOS Linux release 7.6.1810 (Core) 2021-04-28 01:54:06,561 - root - INFO - --> execute_command: end... --> inside rpm check... 2021-04-28 01:54:06,561 - root - INFO - running command.raw on 172.23.106.73: cat /etc/redhat-release 2021-04-28 01:54:06,565 - root - INFO - command executed successfully with root 2021-04-28 01:54:06,565 - root - INFO - CentOS Linux release 7.6.1810 (Core) 2021-04-28 01:54:06,618 - root - INFO - command executed successfully with root 2021-04-28 01:54:06,618 - root - INFO - CentOS Linux release 7.6.1810 (Core) 2021-04-28 01:54:58,154 - root - INFO - If couchbase server is running with this error. Go to log_command_output to add error mesg to bypass it. 2021-04-28 01:54:58,154 - root - INFO - Warming: RPMDB altered outside of yum 2021-04-28 01:54:58,216 - root - INFO - Checking if the directory /tmp exists or not. 2021-04-28 01:54:58,217 - root - ERROR - Directory at /tmp DOES exist. Fx returns True 2021-04-28 01:54:58,298 - root - INFO - running command.raw on 172.23.106.74: sed -i 's/export PATH/export PATH\nexport CBFT_ENV_OPTIONS=bleveMaxResultWindow=10000000/' /opt/couchbase/bin/couchbase-server 2021-04-28 01:54:58,355 - root - INFO - command executed successfully with root 2021-04-28 01:54:58,355 - root - INFO - Checking for process beam.smp on linux 2021-04-28 01:54:58,434 - root - INFO - Starting couchbase server 2021-04-28 01:54:58,546 - root - INFO - Running systemd command on this server 2021-04-28 01:54:58,546 - root - INFO - running command.raw on 172.23.106.74: systemctl start couchbase-server.service 2021-04-28 01:54:58,572 - root - INFO - command executed successfully with root 2021-04-28 01:54:58,572 - root - INFO - 172.23.106.74:sleep for 5 secs. waiting for couchbase server to come up ... 2021-04-28 01:54:58,588 - root - INFO - If couchbase server is running with this error. Go to log_command_output to add error mesg to bypass it. 2021-04-28 01:54:58,588 - root - INFO - Warming: RPMDB altered outside of yum 2021-04-28 01:54:58,654 - root - INFO - Checking if the directory /tmp exists or not. 2021-04-28 01:54:58,655 - root - ERROR - Directory at /tmp DOES exist. Fx returns True 2021-04-28 01:54:58,738 - root - INFO - running command.raw on 172.23.106.73: sed -i 's/export PATH/export PATH\nexport CBFT_ENV_OPTIONS=bleveMaxResultWindow=10000000/' /opt/couchbase/bin/couchbase-server 2021-04-28 01:54:58,796 - root - INFO - command executed successfully with root 2021-04-28 01:54:58,796 - root - INFO - Checking for process beam.smp on linux 2021-04-28 01:54:58,875 - root - INFO - Starting couchbase server 2021-04-28 01:54:58,917 - root - INFO - If couchbase server is running with this error. Go to log_command_output to add error mesg to bypass it. 2021-04-28 01:54:58,917 - root - INFO - Warming: RPMDB altered outside of yum 2021-04-28 01:54:58,980 - root - INFO - Checking if the directory /tmp exists or not. 2021-04-28 01:54:58,982 - root - ERROR - Directory at /tmp DOES exist. Fx returns True 2021-04-28 01:54:58,987 - root - INFO - Running systemd command on this server 2021-04-28 01:54:58,988 - root - INFO - running command.raw on 172.23.106.73: systemctl start couchbase-server.service 2021-04-28 01:54:59,014 - root - INFO - command executed successfully with root 2021-04-28 01:54:59,015 - root - INFO - 172.23.106.73:sleep for 5 secs. waiting for couchbase server to come up ... 2021-04-28 01:54:59,060 - root - INFO - running command.raw on 172.23.106.67: sed -i 's/export PATH/export PATH\nexport CBFT_ENV_OPTIONS=bleveMaxResultWindow=10000000/' /opt/couchbase/bin/couchbase-server 2021-04-28 01:54:59,117 - root - INFO - command executed successfully with root 2021-04-28 01:54:59,117 - root - INFO - Checking for process beam.smp on linux 2021-04-28 01:54:59,193 - root - INFO - Starting couchbase server 2021-04-28 01:54:59,220 - root - INFO - If couchbase server is running with this error. Go to log_command_output to add error mesg to bypass it. 2021-04-28 01:54:59,220 - root - INFO - Warming: RPMDB altered outside of yum 2021-04-28 01:54:59,248 - root - INFO - Checking if the directory /tmp exists or not. 2021-04-28 01:54:59,249 - root - ERROR - Directory at /tmp DOES exist. Fx returns True 2021-04-28 01:54:59,298 - root - INFO - Running systemd command on this server 2021-04-28 01:54:59,298 - root - INFO - running command.raw on 172.23.106.67: systemctl start couchbase-server.service 2021-04-28 01:54:59,323 - root - INFO - command executed successfully with root 2021-04-28 01:54:59,323 - root - INFO - 172.23.106.67:sleep for 5 secs. waiting for couchbase server to come up ... 2021-04-28 01:54:59,330 - root - INFO - running command.raw on 172.23.106.64: sed -i 's/export PATH/export PATH\nexport CBFT_ENV_OPTIONS=bleveMaxResultWindow=10000000/' /opt/couchbase/bin/couchbase-server 2021-04-28 01:54:59,387 - root - INFO - command executed successfully with root 2021-04-28 01:54:59,387 - root - INFO - Checking for process beam.smp on linux 2021-04-28 01:54:59,465 - root - INFO - Starting couchbase server 2021-04-28 01:54:59,577 - root - INFO - Running systemd command on this server 2021-04-28 01:54:59,577 - root - INFO - running command.raw on 172.23.106.64: systemctl start couchbase-server.service 2021-04-28 01:54:59,602 - root - INFO - command executed successfully with root 2021-04-28 01:54:59,602 - root - INFO - 172.23.106.64:sleep for 5 secs. waiting for couchbase server to come up ... 2021-04-28 01:55:03,578 - root - INFO - running command.raw on 172.23.106.74: systemctl status couchbase-server.service | grep ExecStop=/opt/couchbase/bin/couchbase-server 2021-04-28 01:55:03,613 - root - INFO - command executed successfully with root 2021-04-28 01:55:03,613 - root - INFO - Couchbase server status: [] 2021-04-28 01:55:03,613 - root - INFO - Checking for process beam.smp on linux 2021-04-28 01:55:03,691 - root - INFO - process beam.smp is running on 172.23.106.74: with pid 112244 2021-04-28 01:55:03,747 - root - INFO - wait 30 seconds for Couchbase server to start 2021-04-28 01:55:04,020 - root - INFO - running command.raw on 172.23.106.73: systemctl status couchbase-server.service | grep ExecStop=/opt/couchbase/bin/couchbase-server 2021-04-28 01:55:04,052 - root - INFO - command executed successfully with root 2021-04-28 01:55:04,052 - root - INFO - Couchbase server status: [] 2021-04-28 01:55:04,052 - root - INFO - Checking for process beam.smp on linux 2021-04-28 01:55:04,124 - root - INFO - process beam.smp is running on 172.23.106.73: with pid 121781 2021-04-28 01:55:04,179 - root - INFO - wait 30 seconds for Couchbase server to start 2021-04-28 01:55:04,328 - root - INFO - running command.raw on 172.23.106.67: systemctl status couchbase-server.service | grep ExecStop=/opt/couchbase/bin/couchbase-server 2021-04-28 01:55:04,356 - root - INFO - command executed successfully with root 2021-04-28 01:55:04,356 - root - INFO - Couchbase server status: [] 2021-04-28 01:55:04,356 - root - INFO - Checking for process beam.smp on linux 2021-04-28 01:55:04,428 - root - INFO - process beam.smp is running on 172.23.106.67: with pid 116270 2021-04-28 01:55:04,482 - root - INFO - wait 30 seconds for Couchbase server to start 2021-04-28 01:55:04,607 - root - INFO - running command.raw on 172.23.106.64: systemctl status couchbase-server.service | grep ExecStop=/opt/couchbase/bin/couchbase-server 2021-04-28 01:55:04,641 - root - INFO - command executed successfully with root 2021-04-28 01:55:04,641 - root - INFO - Couchbase server status: [] 2021-04-28 01:55:04,641 - root - INFO - Checking for process beam.smp on linux 2021-04-28 01:55:04,713 - root - INFO - process beam.smp is running on 172.23.106.64: with pid 1571 2021-04-28 01:55:04,770 - root - INFO - wait 30 seconds for Couchbase server to start thread installer-thread-172.23.106.64 finished thread installer-thread-172.23.106.67 finished thread installer-thread-172.23.106.73 finished thread installer-thread-172.23.106.74 finished 2021-04-28 01:55:34,781 - root - INFO - SSH Connecting to 172.23.106.64 with username:root, attempt#1 of 5 2021-04-28 01:55:34,783 - root - INFO - SSH Connecting to 172.23.106.67 with username:root, attempt#1 of 5 2021-04-28 01:55:34,788 - root - INFO - SSH Connecting to 172.23.106.74 with username:root, attempt#1 of 5 2021-04-28 01:55:34,788 - root - INFO - SSH Connecting to 172.23.106.73 with username:root, attempt#1 of 5 2021-04-28 01:55:34,886 - root - INFO - SSH Connected to 172.23.106.64 as root 2021-04-28 01:55:34,890 - root - INFO - SSH Connected to 172.23.106.67 as root 2021-04-28 01:55:34,899 - root - INFO - SSH Connected to 172.23.106.73 as root 2021-04-28 01:55:34,899 - root - INFO - SSH Connected to 172.23.106.74 as root 2021-04-28 01:55:35,165 - root - INFO - os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-04-28 01:55:35,170 - root - INFO - os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-04-28 01:55:35,173 - root - INFO - os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-04-28 01:55:35,187 - root - INFO - os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-04-28 01:55:35,481 - root - INFO - extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-04-28 01:55:35,484 - root - INFO - extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-04-28 01:55:35,493 - root - INFO - extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-04-28 01:55:35,507 - root - INFO - extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-04-28 01:55:35,509 - root - INFO - 172.23.106.67 **** The linux version file /opt/couchbase/ VERSION.txt exists 2021-04-28 01:55:35,510 - root - INFO - creating RestConnection(ip:172.23.106.67 port:8091 ssh_username:root) and the server type is 2021-04-28 01:55:35,510 - root - INFO - 172.23.106.74 **** The linux version file /opt/couchbase/ VERSION.txt exists 2021-04-28 01:55:35,511 - root - INFO - creating RestConnection(ip:172.23.106.74 port:8091 ssh_username:root) and the server type is 2021-04-28 01:55:35,518 - root - INFO - Done creating rest connection 2021-04-28 01:55:35,519 - root - INFO - Done creating rest connection 2021-04-28 01:55:35,559 - root - INFO - 172.23.106.64 **** The linux version file /opt/couchbase/ VERSION.txt exists 2021-04-28 01:55:35,559 - root - INFO - creating RestConnection(ip:172.23.106.64 port:8091 ssh_username:root) and the server type is 2021-04-28 01:55:35,565 - root - INFO - Done creating rest connection 2021-04-28 01:55:35,571 - root - INFO - 172.23.106.73 **** The linux version file /opt/couchbase/ VERSION.txt exists 2021-04-28 01:55:35,571 - root - INFO - creating RestConnection(ip:172.23.106.73 port:8091 ssh_username:root) and the server type is 2021-04-28 01:55:35,579 - root - INFO - Done creating rest connection 2021-04-28 01:55:39,532 - root - INFO - --> rest.get_nodes_self().mcdMemoryReserved 2021-04-28 01:55:39,532 - root - INFO - --> info.mcdMemoryReserved=3010,CLUSTER_QUOTA_RATIO=0.67 2021-04-28 01:55:39,533 - root - INFO - quota for kv: 2015 MB 2021-04-28 01:55:39,533 - root - INFO - pools/default params : memoryQuota=2015 2021-04-28 01:55:39,534 - root - INFO - --> rest.get_nodes_self().mcdMemoryReserved 2021-04-28 01:55:39,534 - root - INFO - --> info.mcdMemoryReserved=3010,CLUSTER_QUOTA_RATIO=0.67 2021-04-28 01:55:39,534 - root - INFO - quota for kv: 2015 MB 2021-04-28 01:55:39,534 - root - INFO - pools/default params : memoryQuota=2015 2021-04-28 01:55:39,539 - root - INFO - --> init_node_services(Administrator,password,127.0.0.1,8091,['kv']) 2021-04-28 01:55:39,539 - root - INFO - /node/controller/setupServices params on 172.23.106.67: 8091:hostname=127.0.0.1%3A8091&user=Administrator&password=password&services=kv 2021-04-28 01:55:39,540 - root - INFO - --> init_node_services(Administrator,password,127.0.0.1,8091,['kv']) 2021-04-28 01:55:39,540 - root - INFO - /node/controller/setupServices params on 172.23.106.74: 8091:hostname=127.0.0.1%3A8091&user=Administrator&password=password&services=kv 2021-04-28 01:55:39,546 - root - INFO - --> in init_cluster...Administrator,password,8091 2021-04-28 01:55:39,546 - root - INFO - settings/web params on 172.23.106.67:8091:port=8091&username=Administrator&password=password 2021-04-28 01:55:39,547 - root - INFO - --> in init_cluster...Administrator,password,8091 2021-04-28 01:55:39,547 - root - INFO - settings/web params on 172.23.106.74:8091:port=8091&username=Administrator&password=password 2021-04-28 01:55:39,576 - root - INFO - --> rest.get_nodes_self().mcdMemoryReserved 2021-04-28 01:55:39,576 - root - INFO - --> info.mcdMemoryReserved=3010,CLUSTER_QUOTA_RATIO=0.67 2021-04-28 01:55:39,576 - root - INFO - quota for kv: 2015 MB 2021-04-28 01:55:39,576 - root - INFO - pools/default params : memoryQuota=2015 2021-04-28 01:55:39,582 - root - INFO - --> init_node_services(Administrator,password,127.0.0.1,8091,['kv']) 2021-04-28 01:55:39,582 - root - INFO - /node/controller/setupServices params on 172.23.106.64: 8091:hostname=127.0.0.1%3A8091&user=Administrator&password=password&services=kv 2021-04-28 01:55:39,588 - root - INFO - --> in init_cluster...Administrator,password,8091 2021-04-28 01:55:39,588 - root - INFO - settings/web params on 172.23.106.64:8091:port=8091&username=Administrator&password=password 2021-04-28 01:55:39,590 - root - INFO - --> status:True 2021-04-28 01:55:39,591 - root - INFO - --> status:True 2021-04-28 01:55:39,592 - root - INFO - --> rest.get_nodes_self().mcdMemoryReserved 2021-04-28 01:55:39,592 - root - INFO - --> info.mcdMemoryReserved=3010,CLUSTER_QUOTA_RATIO=0.67 2021-04-28 01:55:39,592 - root - INFO - quota for kv: 2015 MB 2021-04-28 01:55:39,592 - root - INFO - pools/default params : memoryQuota=2015 2021-04-28 01:55:39,595 - root - INFO - -->is_ns_server_running? 2021-04-28 01:55:39,596 - root - INFO - -->is_ns_server_running? 2021-04-28 01:55:39,598 - root - INFO - --> init_node_services(Administrator,password,127.0.0.1,8091,['kv']) 2021-04-28 01:55:39,598 - root - INFO - /node/controller/setupServices params on 172.23.106.73: 8091:hostname=127.0.0.1%3A8091&user=Administrator&password=password&services=kv 2021-04-28 01:55:39,604 - root - INFO - --> in init_cluster...Administrator,password,8091 2021-04-28 01:55:39,604 - root - INFO - settings/web params on 172.23.106.73:8091:port=8091&username=Administrator&password=password 2021-04-28 01:55:39,606 - root - INFO - Node version in cluster 6.6.1-9213-enterprise 2021-04-28 01:55:39,606 - root - INFO - ***** set NS_SERVER_CBAUTH env in linux ***** 2021-04-28 01:55:39,608 - root - INFO - Node version in cluster 6.6.1-9213-enterprise 2021-04-28 01:55:39,608 - root - INFO - ***** set NS_SERVER_CBAUTH env in linux ***** 2021-04-28 01:55:39,632 - root - INFO - --> status:True 2021-04-28 01:55:39,637 - root - INFO - -->is_ns_server_running? 2021-04-28 01:55:39,645 - root - INFO - --> status:True 2021-04-28 01:55:39,650 - root - INFO - -->is_ns_server_running? 2021-04-28 01:55:39,653 - root - INFO - Node version in cluster 6.6.1-9213-enterprise 2021-04-28 01:55:39,653 - root - INFO - ***** set NS_SERVER_CBAUTH env in linux ***** 2021-04-28 01:55:39,662 - root - INFO - Node version in cluster 6.6.1-9213-enterprise 2021-04-28 01:55:39,662 - root - INFO - ***** set NS_SERVER_CBAUTH env in linux ***** 2021-04-28 01:55:39,840 - root - INFO - check if man installed on vm? 2021-04-28 01:55:39,841 - root - INFO - running command.raw on 172.23.106.74: rpm -qa | grep 'man-db' 2021-04-28 01:55:39,842 - root - INFO - check if man installed on vm? 2021-04-28 01:55:39,843 - root - INFO - running command.raw on 172.23.106.67: rpm -qa | grep 'man-db' 2021-04-28 01:55:39,895 - root - INFO - check if man installed on vm? 2021-04-28 01:55:39,895 - root - INFO - running command.raw on 172.23.106.64: rpm -qa | grep 'man-db' 2021-04-28 01:55:39,911 - root - INFO - check if man installed on vm? 2021-04-28 01:55:39,911 - root - INFO - running command.raw on 172.23.106.73: rpm -qa | grep 'man-db' 2021-04-28 01:55:40,455 - root - INFO - command executed successfully with root 2021-04-28 01:55:40,455 - root - INFO - ---> Run command unzip to check if it is ready on server 172.23.106.74 2021-04-28 01:55:40,455 - root - INFO - running command.raw on 172.23.106.74: unzip 2021-04-28 01:55:40,512 - root - INFO - command executed successfully with root 2021-04-28 01:55:40,512 - root - INFO - unzip command is ready 2021-04-28 01:55:40,512 - root - INFO - Check if ntp is installed 2021-04-28 01:55:40,512 - root - INFO - This OS version centos 7 2021-04-28 01:55:40,513 - root - INFO - running command.raw on 172.23.106.74: systemctl status ntpd 2021-04-28 01:55:40,579 - root - INFO - command executed successfully with root 2021-04-28 01:55:40,579 - root - INFO - running command.raw on 172.23.106.74: ntpstat 2021-04-28 01:55:40,635 - root - INFO - command executed successfully with root 2021-04-28 01:55:40,635 - root - INFO - running command.raw on 172.23.106.74: timedatectl status 2021-04-28 01:55:40,643 - root - INFO - command executed successfully with root 2021-04-28 01:55:40,644 - root - INFO - ---> Run command unzip to check if it is ready on server 172.23.106.67 2021-04-28 01:55:40,644 - root - INFO - running command.raw on 172.23.106.67: unzip 2021-04-28 01:55:40,661 - root - INFO - command executed successfully with root 2021-04-28 01:55:40,661 - root - INFO - unzip command is ready 2021-04-28 01:55:40,661 - root - INFO - Check if ntp is installed 2021-04-28 01:55:40,661 - root - INFO - This OS version centos 7 2021-04-28 01:55:40,662 - root - INFO - running command.raw on 172.23.106.67: systemctl status ntpd 2021-04-28 01:55:40,673 - root - INFO - command executed successfully with root 2021-04-28 01:55:40,673 - root - INFO - ---> Run command unzip to check if it is ready on server 172.23.106.64 2021-04-28 01:55:40,673 - root - INFO - running command.raw on 172.23.106.64: unzip 2021-04-28 01:55:40,700 - root - INFO - command executed successfully with root 2021-04-28 01:55:40,700 - root - INFO - ---> Run command unzip to check if it is ready on server 172.23.106.73 2021-04-28 01:55:40,700 - root - INFO - running command.raw on 172.23.106.73: unzip 2021-04-28 01:55:40,718 - root - INFO - command executed successfully with root 2021-04-28 01:55:40,718 - root - INFO - running command.raw on 172.23.106.74: date 2021-04-28 01:55:40,729 - root - INFO - command executed successfully with root 2021-04-28 01:55:40,729 - root - INFO - running command.raw on 172.23.106.67: ntpstat 2021-04-28 01:55:40,731 - root - INFO - command executed successfully with root 2021-04-28 01:55:40,731 - root - INFO - unzip command is ready 2021-04-28 01:55:40,731 - root - INFO - Check if ntp is installed 2021-04-28 01:55:40,732 - root - INFO - This OS version centos 7 2021-04-28 01:55:40,732 - root - INFO - running command.raw on 172.23.106.64: systemctl status ntpd 2021-04-28 01:55:40,758 - root - INFO - command executed successfully with root 2021-04-28 01:55:40,758 - root - INFO - unzip command is ready 2021-04-28 01:55:40,758 - root - INFO - Check if ntp is installed 2021-04-28 01:55:40,758 - root - INFO - This OS version centos 7 2021-04-28 01:55:40,758 - root - INFO - running command.raw on 172.23.106.73: systemctl status ntpd 2021-04-28 01:55:40,773 - root - INFO - command executed successfully with root 2021-04-28 01:55:40,784 - root - INFO - command executed successfully with root 2021-04-28 01:55:40,784 - root - INFO - running command.raw on 172.23.106.67: timedatectl status 2021-04-28 01:55:40,798 - root - INFO - command executed successfully with root 2021-04-28 01:55:40,799 - root - INFO - running command.raw on 172.23.106.64: ntpstat 2021-04-28 01:55:40,826 - root - INFO - command executed successfully with root 2021-04-28 01:55:40,826 - root - INFO - running command.raw on 172.23.106.73: ntpstat 2021-04-28 01:55:40,854 - root - INFO - command executed successfully with root 2021-04-28 01:55:40,855 - root - INFO - running command.raw on 172.23.106.64: timedatectl status 2021-04-28 01:55:40,868 - root - INFO - command executed successfully with root 2021-04-28 01:55:40,868 - root - INFO - running command.raw on 172.23.106.67: date 2021-04-28 01:55:40,883 - root - INFO - command executed successfully with root 2021-04-28 01:55:40,883 - root - INFO - running command.raw on 172.23.106.73: timedatectl status 2021-04-28 01:55:40,885 - root - INFO - ['Wed Apr 28 01:55:40 PDT 2021'] IP: 172.23.106.74 2021-04-28 01:55:40,923 - root - INFO - command executed successfully with root 2021-04-28 01:55:40,936 - root - INFO - command executed successfully with root 2021-04-28 01:55:40,937 - root - INFO - running command.raw on 172.23.106.64: date 2021-04-28 01:55:40,971 - root - INFO - command executed successfully with root 2021-04-28 01:55:40,971 - root - INFO - running command.raw on 172.23.106.73: date 2021-04-28 01:55:40,993 - root - INFO - command executed successfully with root 2021-04-28 01:55:41,028 - root - INFO - command executed successfully with root 2021-04-28 01:55:41,038 - root - INFO - ['Wed Apr 28 01:55:41 PDT 2021'] IP: 172.23.106.67 2021-04-28 01:55:41,106 - root - INFO - ['Wed Apr 28 01:55:41 PDT 2021'] IP: 172.23.106.64 thread initializer-thread-172.23.106.64 finished thread initializer-thread-172.23.106.67 finished 2021-04-28 01:55:41,143 - root - INFO - ['Wed Apr 28 01:55:41 PDT 2021'] IP: 172.23.106.73 thread initializer-thread-172.23.106.73 finished thread initializer-thread-172.23.106.74 finished 2021-04-28 01:55:41,145 - root - INFO - SSH Connecting to 172.23.106.64 with username:root, attempt#1 of 5 2021-04-28 01:55:41,242 - root - INFO - SSH Connected to 172.23.106.64 as root 2021-04-28 01:55:41,493 - root - INFO - os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-04-28 01:55:41,810 - root - INFO - extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-04-28 01:55:41,812 - root - INFO - SSH Connecting to 172.23.106.64 with username:root, attempt#1 of 5 2021-04-28 01:55:41,911 - root - INFO - SSH Connected to 172.23.106.64 as root 2021-04-28 01:55:42,160 - root - INFO - os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-04-28 01:55:42,478 - root - INFO - extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-04-28 01:55:42,504 - root - INFO - 172.23.106.64 **** The linux version file /opt/couchbase/ VERSION.txt exists 2021-04-28 01:55:42,504 - root - INFO - sleep for 5 secs. sleep 5 seconds to let cb up completely ... 2021-04-28 01:55:47,514 - root - INFO - -->is_ns_server_running? 2021-04-28 01:55:47,518 - root - INFO - SSH Connecting to 172.23.106.67 with username:root, attempt#1 of 5 2021-04-28 01:55:47,619 - root - INFO - SSH Connected to 172.23.106.67 as root 2021-04-28 01:55:47,879 - root - INFO - os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-04-28 01:55:48,184 - root - INFO - extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-04-28 01:55:48,207 - root - INFO - 172.23.106.67 **** The linux version file /opt/couchbase/ VERSION.txt exists 2021-04-28 01:55:48,207 - root - INFO - sleep for 5 secs. sleep 5 seconds to let cb up completely ... 2021-04-28 01:55:53,216 - root - INFO - -->is_ns_server_running? 2021-04-28 01:55:53,220 - root - INFO - SSH Connecting to 172.23.106.73 with username:root, attempt#1 of 5 2021-04-28 01:55:53,320 - root - INFO - SSH Connected to 172.23.106.73 as root 2021-04-28 01:55:53,596 - root - INFO - os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-04-28 01:55:53,910 - root - INFO - extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-04-28 01:55:53,975 - root - INFO - 172.23.106.73 **** The linux version file /opt/couchbase/ VERSION.txt exists 2021-04-28 01:55:53,975 - root - INFO - sleep for 5 secs. sleep 5 seconds to let cb up completely ... 2021-04-28 01:55:58,983 - root - INFO - -->is_ns_server_running? 2021-04-28 01:55:58,987 - root - INFO - SSH Connecting to 172.23.106.74 with username:root, attempt#1 of 5 2021-04-28 01:55:59,089 - root - INFO - SSH Connected to 172.23.106.74 as root 2021-04-28 01:55:59,352 - root - INFO - os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-04-28 01:55:59,657 - root - INFO - extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-04-28 01:55:59,680 - root - INFO - 172.23.106.74 **** The linux version file /opt/couchbase/ VERSION.txt exists 2021-04-28 01:55:59,680 - root - INFO - sleep for 5 secs. sleep 5 seconds to let cb up completely ... 2021-04-28 01:56:04,689 - root - INFO - -->is_ns_server_running? 2021-04-28 01:56:04,693 - root - INFO - will install 7.0.0-5046 on ['172.23.106.75', '172.23.106.76', '172.23.106.77', '172.23.106.78'] 2021-04-28 01:56:04,695 - root - INFO - SSH Connecting to 172.23.106.78 with username:root, attempt#1 of 5 2021-04-28 01:56:04,696 - root - INFO - SSH Connecting to 172.23.106.77 with username:root, attempt#1 of 5 2021-04-28 01:56:04,697 - root - INFO - SSH Connecting to 172.23.106.76 with username:root, attempt#1 of 5 2021-04-28 01:56:04,698 - root - INFO - SSH Connecting to 172.23.106.75 with username:root, attempt#1 of 5 2021-04-28 01:56:04,803 - root - INFO - SSH Connected to 172.23.106.77 as root 2021-04-28 01:56:04,806 - root - INFO - SSH Connected to 172.23.106.76 as root 2021-04-28 01:56:04,806 - root - INFO - SSH Connected to 172.23.106.78 as root 2021-04-28 01:56:04,806 - root - INFO - SSH Connected to 172.23.106.75 as root 2021-04-28 01:56:05,078 - root - INFO - os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-04-28 01:56:05,095 - root - INFO - os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-04-28 01:56:05,098 - root - INFO - os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-04-28 01:56:05,100 - root - INFO - os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-04-28 01:56:05,393 - root - INFO - extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-04-28 01:56:05,394 - root - INFO - 172.23.106.76 *****In couchbase uninstall**** 2021-04-28 01:56:05,394 - root - INFO - CentOS 2021-04-28 01:56:05,410 - root - INFO - extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-04-28 01:56:05,410 - root - INFO - 172.23.106.75 *****In couchbase uninstall**** 2021-04-28 01:56:05,411 - root - INFO - CentOS 2021-04-28 01:56:05,412 - root - INFO - extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-04-28 01:56:05,413 - root - INFO - extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-04-28 01:56:05,414 - root - INFO - 172.23.106.78 *****In couchbase uninstall**** 2021-04-28 01:56:05,414 - root - INFO - 172.23.106.77 *****In couchbase uninstall**** 2021-04-28 01:56:05,414 - root - INFO - CentOS 2021-04-28 01:56:05,415 - root - INFO - CentOS 2021-04-28 01:56:05,505 - root - INFO - running command.raw on 172.23.106.76: killall -9 rpm 2021-04-28 01:56:05,523 - root - INFO - running command.raw on 172.23.106.78: killall -9 rpm 2021-04-28 01:56:05,526 - root - INFO - running command.raw on 172.23.106.77: killall -9 rpm 2021-04-28 01:56:05,537 - root - INFO - command executed with root but got an error ['rpm: no process found'] ... 2021-04-28 01:56:05,538 - root - ERROR - rpm: no process found 2021-04-28 01:56:05,538 - root - INFO - running command.raw on 172.23.106.76: rm -f /var/lib/rpm/.rpm.lock 2021-04-28 01:56:05,556 - root - INFO - running command.raw on 172.23.106.75: killall -9 rpm 2021-04-28 01:56:05,561 - root - INFO - command executed with root but got an error ['rpm: no process found'] ... 2021-04-28 01:56:05,562 - root - ERROR - rpm: no process found 2021-04-28 01:56:05,562 - root - INFO - running command.raw on 172.23.106.78: rm -f /var/lib/rpm/.rpm.lock 2021-04-28 01:56:05,563 - root - INFO - command executed with root but got an error ['rpm: no process found'] ... 2021-04-28 01:56:05,563 - root - ERROR - rpm: no process found 2021-04-28 01:56:05,563 - root - INFO - running command.raw on 172.23.106.77: rm -f /var/lib/rpm/.rpm.lock 2021-04-28 01:56:05,587 - root - INFO - command executed with root but got an error ['rpm: no process found'] ... 2021-04-28 01:56:05,587 - root - ERROR - rpm: no process found 2021-04-28 01:56:05,587 - root - INFO - running command.raw on 172.23.106.75: rm -f /var/lib/rpm/.rpm.lock 2021-04-28 01:56:05,593 - root - INFO - command executed successfully with root 2021-04-28 01:56:05,616 - root - INFO - command executed successfully with root 2021-04-28 01:56:05,617 - root - INFO - command executed successfully with root 2021-04-28 01:56:05,642 - root - INFO - command executed successfully with root 2021-04-28 01:56:05,701 - root - INFO - running rpm -e to remove couchbase-server 2021-04-28 01:56:05,701 - root - INFO - running command.raw on 172.23.106.76: rpm -e couchbase-server 2021-04-28 01:56:05,726 - root - INFO - running rpm -e to remove couchbase-server 2021-04-28 01:56:05,727 - root - INFO - running command.raw on 172.23.106.77: rpm -e couchbase-server 2021-04-28 01:56:05,749 - root - INFO - running rpm -e to remove couchbase-server 2021-04-28 01:56:05,749 - root - INFO - running command.raw on 172.23.106.75: rpm -e couchbase-server 2021-04-28 01:56:05,758 - root - INFO - running rpm -e to remove couchbase-server 2021-04-28 01:56:05,758 - root - INFO - running command.raw on 172.23.106.78: rpm -e couchbase-server 2021-04-28 01:56:08,476 - root - INFO - command executed with root but got an error ['warning: file /opt/couchbase/var/lib/couchbase/ip_start: remove failed: No such file or directory', 'warning: /opt/couchbase/var/lib/couchbase/ip saved as /opt/couchbase/var/lib/couchbase/ip.rpmsave', 'warning: /opt/couchbase/var/lib/couchbase/config/config.dat saved as /opt/couchbase/var/lib/couchbase/config/config.dat.rpmsave', 'warning: /opt/couchbase/etc/couchdb/local.ini saved as /opt/couch ... 2021-04-28 01:56:08,477 - root - INFO - running command.raw on 172.23.106.78: pkill -u couchbase 2021-04-28 01:56:08,504 - root - INFO - command executed with root but got an error ['warning: file /opt/couchbase/var/lib/couchbase/ip_start: remove failed: No such file or directory', 'warning: /opt/couchbase/var/lib/couchbase/ip saved as /opt/couchbase/var/lib/couchbase/ip.rpmsave', 'warning: /opt/couchbase/var/lib/couchbase/config/config.dat saved as /opt/couchbase/var/lib/couchbase/config/config.dat.rpmsave', 'warning: /opt/couchbase/etc/couchdb/local.ini saved as /opt/couch ... 2021-04-28 01:56:08,504 - root - INFO - running command.raw on 172.23.106.76: pkill -u couchbase 2021-04-28 01:56:08,527 - root - INFO - command executed with root but got an error ['warning: file /opt/couchbase/var/lib/couchbase/ip_start: remove failed: No such file or directory', 'warning: /opt/couchbase/var/lib/couchbase/ip saved as /opt/couchbase/var/lib/couchbase/ip.rpmsave', 'warning: /opt/couchbase/var/lib/couchbase/config/config.dat saved as /opt/couchbase/var/lib/couchbase/config/config.dat.rpmsave', 'warning: /opt/couchbase/etc/couchdb/local.ini saved as /opt/couch ... 2021-04-28 01:56:08,527 - root - INFO - running command.raw on 172.23.106.77: pkill -u couchbase 2021-04-28 01:56:08,528 - root - INFO - command executed successfully with root 2021-04-28 01:56:08,528 - root - INFO - running command.raw on 172.23.106.76: ps -ef | grep couchbase 2021-04-28 01:56:08,540 - root - INFO - command executed successfully with root 2021-04-28 01:56:08,540 - root - INFO - running command.raw on 172.23.106.78: ps -ef | grep couchbase 2021-04-28 01:56:08,549 - root - INFO - command executed successfully with root 2021-04-28 01:56:08,549 - root - INFO - running command.raw on 172.23.106.77: ps -ef | grep couchbase 2021-04-28 01:56:08,569 - root - INFO - command executed with root but got an error ['warning: file /opt/couchbase/var/lib/couchbase/ip_start: remove failed: No such file or directory', 'warning: /opt/couchbase/var/lib/couchbase/ip saved as /opt/couchbase/var/lib/couchbase/ip.rpmsave', 'warning: /opt/couchbase/var/lib/couchbase/config/config.dat saved as /opt/couchbase/var/lib/couchbase/config/config.dat.rpmsave', 'warning: /opt/couchbase/etc/couchdb/local.ini saved as /opt/couch ... 2021-04-28 01:56:08,570 - root - INFO - running command.raw on 172.23.106.75: pkill -u couchbase 2021-04-28 01:56:08,592 - root - INFO - command executed successfully with root 2021-04-28 01:56:08,592 - root - INFO - running command.raw on 172.23.106.75: ps -ef | grep couchbase 2021-04-28 01:56:08,597 - root - INFO - command executed successfully with root 2021-04-28 01:56:08,611 - root - INFO - command executed successfully with root 2021-04-28 01:56:08,620 - root - INFO - command executed successfully with root 2021-04-28 01:56:08,666 - root - INFO - command executed successfully with root 2021-04-28 01:56:10,046 - root - INFO - running command.raw on 172.23.106.76: ipcrm 2021-04-28 01:56:10,056 - root - INFO - running command.raw on 172.23.106.78: ipcrm 2021-04-28 01:56:10,068 - root - INFO - running command.raw on 172.23.106.77: ipcrm 2021-04-28 01:56:10,100 - root - INFO - command executed successfully with root 2021-04-28 01:56:10,110 - root - INFO - command executed successfully with root 2021-04-28 01:56:10,114 - root - INFO - running command.raw on 172.23.106.75: ipcrm 2021-04-28 01:56:10,124 - root - INFO - command executed successfully with root 2021-04-28 01:56:10,168 - root - INFO - command executed successfully with root thread uninstaller-thread-172.23.106.75 finished thread uninstaller-thread-172.23.106.76 finished thread uninstaller-thread-172.23.106.77 finished thread uninstaller-thread-172.23.106.78 finished 2021-04-28 01:56:10,169 - root - INFO - SSH Connecting to 172.23.106.75 with username:root, attempt#1 of 5 2021-04-28 01:56:10,269 - root - INFO - SSH Connected to 172.23.106.75 as root 2021-04-28 01:56:10,538 - root - INFO - os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-04-28 01:56:10,835 - root - INFO - extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-04-28 01:56:10,855 - root - INFO - SSH Connecting to 172.23.106.76 with username:root, attempt#1 of 5 2021-04-28 01:56:10,957 - root - INFO - SSH Connected to 172.23.106.76 as root 2021-04-28 01:56:11,214 - root - INFO - os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-04-28 01:56:11,515 - root - INFO - extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-04-28 01:56:11,537 - root - INFO - SSH Connecting to 172.23.106.77 with username:root, attempt#1 of 5 2021-04-28 01:56:11,637 - root - INFO - SSH Connected to 172.23.106.77 as root 2021-04-28 01:56:11,903 - root - INFO - os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-04-28 01:56:12,202 - root - INFO - extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-04-28 01:56:12,223 - root - INFO - SSH Connecting to 172.23.106.78 with username:root, attempt#1 of 5 2021-04-28 01:56:12,327 - root - INFO - SSH Connected to 172.23.106.78 as root 2021-04-28 01:56:12,598 - root - INFO - os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-04-28 01:56:12,907 - root - INFO - extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-04-28 01:56:12,927 - root - INFO - ********CouchbaseServerInstaller:install 2021-04-28 01:56:12,928 - root - INFO - SSH Connecting to 172.23.106.75 with username:root, attempt#1 of 5 2021-04-28 01:56:12,929 - root - INFO - ********CouchbaseServerInstaller:install 2021-04-28 01:56:12,929 - root - INFO - ********CouchbaseServerInstaller:install 2021-04-28 01:56:12,930 - root - INFO - SSH Connecting to 172.23.106.76 with username:root, attempt#1 of 5 2021-04-28 01:56:12,932 - root - INFO - SSH Connecting to 172.23.106.77 with username:root, attempt#1 of 5 2021-04-28 01:56:12,932 - root - INFO - ********CouchbaseServerInstaller:install 2021-04-28 01:56:12,934 - root - INFO - SSH Connecting to 172.23.106.78 with username:root, attempt#1 of 5 2021-04-28 01:56:13,031 - root - INFO - SSH Connected to 172.23.106.75 as root 2021-04-28 01:56:13,033 - root - INFO - SSH Connected to 172.23.106.76 as root 2021-04-28 01:56:13,034 - root - INFO - SSH Connected to 172.23.106.77 as root 2021-04-28 01:56:13,034 - root - INFO - SSH Connected to 172.23.106.78 as root 2021-04-28 01:56:13,292 - root - INFO - os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-04-28 01:56:13,313 - root - INFO - os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-04-28 01:56:13,316 - root - INFO - os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-04-28 01:56:13,317 - root - INFO - os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-04-28 01:56:13,624 - root - INFO - extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 --> remote server architecture_type..x86_64 *** OS version of this server 172.23.106.78 is centos 7 *** 2021-04-28 01:56:13,625 - root - INFO - SSH Connecting to 172.23.106.78 with username:root, attempt#1 of 5 2021-04-28 01:56:13,628 - root - INFO - extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 --> remote server architecture_type..x86_64 *** OS version of this server 172.23.106.77 is centos 7 *** 2021-04-28 01:56:13,629 - root - INFO - SSH Connecting to 172.23.106.77 with username:root, attempt#1 of 5 2021-04-28 01:56:13,631 - root - INFO - extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 --> remote server architecture_type..x86_64 *** OS version of this server 172.23.106.75 is centos 7 *** 2021-04-28 01:56:13,633 - root - INFO - SSH Connecting to 172.23.106.75 with username:root, attempt#1 of 5 2021-04-28 01:56:13,642 - root - INFO - extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 --> remote server architecture_type..x86_64 *** OS version of this server 172.23.106.76 is centos 7 *** 2021-04-28 01:56:13,644 - root - INFO - SSH Connecting to 172.23.106.76 with username:root, attempt#1 of 5 2021-04-28 01:56:13,729 - root - INFO - SSH Connected to 172.23.106.78 as root 2021-04-28 01:56:13,731 - root - INFO - SSH Connected to 172.23.106.77 as root 2021-04-28 01:56:13,734 - root - INFO - SSH Connected to 172.23.106.75 as root 2021-04-28 01:56:13,743 - root - INFO - SSH Connected to 172.23.106.76 as root 2021-04-28 01:56:14,012 - root - INFO - os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-04-28 01:56:14,015 - root - INFO - os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-04-28 01:56:14,016 - root - INFO - os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-04-28 01:56:14,022 - root - INFO - os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-04-28 01:56:14,322 - root - INFO - extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 check if http://172.23.126.166/builds/latestbuilds/couchbase-server/cheshire-cat/5046/couchbase-server-enterprise-7.0.0-5046-centos7.x86_64.rpm is live 2021-04-28 01:56:14,322 - root - INFO - Trying to check is this url alive: http://172.23.126.166/builds/latestbuilds/couchbase-server/cheshire-cat/5046/couchbase-server-enterprise-7.0.0-5046-centos7.x86_64.rpm 2021-04-28 01:56:14,324 - root - INFO - extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-04-28 01:56:14,325 - root - INFO - This url http://172.23.126.166/builds/latestbuilds/couchbase-server/cheshire-cat/5046/couchbase-server-enterprise-7.0.0-5046-centos7.x86_64.rpm is live check if http://172.23.126.166/builds/latestbuilds/couchbase-server/cheshire-cat/5046/couchbase-server-enterprise-7.0.0-5046-centos7.x86_64.rpm is live 2021-04-28 01:56:14,325 - root - INFO - Trying to check is this url alive: http://172.23.126.166/builds/latestbuilds/couchbase-server/cheshire-cat/5046/couchbase-server-enterprise-7.0.0-5046-centos7.x86_64.rpm 2021-04-28 01:56:14,326 - root - INFO - SSH Connecting to 172.23.106.76 with username:root, attempt#1 of 5 2021-04-28 01:56:14,327 - root - INFO - extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 check if http://172.23.126.166/builds/latestbuilds/couchbase-server/cheshire-cat/5046/couchbase-server-enterprise-7.0.0-5046-centos7.x86_64.rpm is live 2021-04-28 01:56:14,327 - root - INFO - Trying to check is this url alive: http://172.23.126.166/builds/latestbuilds/couchbase-server/cheshire-cat/5046/couchbase-server-enterprise-7.0.0-5046-centos7.x86_64.rpm 2021-04-28 01:56:14,329 - root - INFO - This url http://172.23.126.166/builds/latestbuilds/couchbase-server/cheshire-cat/5046/couchbase-server-enterprise-7.0.0-5046-centos7.x86_64.rpm is live 2021-04-28 01:56:14,330 - root - INFO - SSH Connecting to 172.23.106.77 with username:root, attempt#1 of 5 2021-04-28 01:56:14,331 - root - INFO - This url http://172.23.126.166/builds/latestbuilds/couchbase-server/cheshire-cat/5046/couchbase-server-enterprise-7.0.0-5046-centos7.x86_64.rpm is live 2021-04-28 01:56:14,332 - root - INFO - SSH Connecting to 172.23.106.78 with username:root, attempt#1 of 5 2021-04-28 01:56:14,334 - root - INFO - extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 check if http://172.23.126.166/builds/latestbuilds/couchbase-server/cheshire-cat/5046/couchbase-server-enterprise-7.0.0-5046-centos7.x86_64.rpm is live 2021-04-28 01:56:14,335 - root - INFO - Trying to check is this url alive: http://172.23.126.166/builds/latestbuilds/couchbase-server/cheshire-cat/5046/couchbase-server-enterprise-7.0.0-5046-centos7.x86_64.rpm 2021-04-28 01:56:14,336 - root - INFO - This url http://172.23.126.166/builds/latestbuilds/couchbase-server/cheshire-cat/5046/couchbase-server-enterprise-7.0.0-5046-centos7.x86_64.rpm is live 2021-04-28 01:56:14,337 - root - INFO - SSH Connecting to 172.23.106.75 with username:root, attempt#1 of 5 2021-04-28 01:56:14,427 - root - INFO - SSH Connected to 172.23.106.76 as root 2021-04-28 01:56:14,432 - root - INFO - SSH Connected to 172.23.106.78 as root 2021-04-28 01:56:14,434 - root - INFO - SSH Connected to 172.23.106.77 as root 2021-04-28 01:56:14,441 - root - INFO - SSH Connected to 172.23.106.75 as root 2021-04-28 01:56:14,685 - root - INFO - os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-04-28 01:56:14,710 - root - INFO - os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-04-28 01:56:14,711 - root - INFO - os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-04-28 01:56:14,719 - root - INFO - os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-04-28 01:56:15,018 - root - INFO - extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 Downloading the build...url : http://172.23.126.166/builds/latestbuilds/couchbase-server/cheshire-cat/5046/couchbase-server-enterprise-7.0.0-5046-centos7.x86_64.rpm name : couchbase-server-enterprise-7.0.0-5046-centos7.x86_64.rpm product : product_version : 7.0.0-5046 os : deliverable_type : rpm architecture_type : x86_64 toy : toy- 2021-04-28 01:56:15,019 - root - INFO - running command.raw on 172.23.106.77: /sbin/iptables -F 2021-04-28 01:56:15,022 - root - INFO - extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 Downloading the build...url : http://172.23.126.166/builds/latestbuilds/couchbase-server/cheshire-cat/5046/couchbase-server-enterprise-7.0.0-5046-centos7.x86_64.rpm name : couchbase-server-enterprise-7.0.0-5046-centos7.x86_64.rpm product : product_version : 7.0.0-5046 os : deliverable_type : rpm architecture_type : x86_64 toy : toy- 2021-04-28 01:56:15,022 - root - INFO - running command.raw on 172.23.106.78: /sbin/iptables -F 2021-04-28 01:56:15,026 - root - INFO - extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 Downloading the build...url : http://172.23.126.166/builds/latestbuilds/couchbase-server/cheshire-cat/5046/couchbase-server-enterprise-7.0.0-5046-centos7.x86_64.rpm name : couchbase-server-enterprise-7.0.0-5046-centos7.x86_64.rpm product : product_version : 7.0.0-5046 os : deliverable_type : rpm architecture_type : x86_64 toy : toy- 2021-04-28 01:56:15,027 - root - INFO - running command.raw on 172.23.106.75: /sbin/iptables -F 2021-04-28 01:56:15,037 - root - INFO - extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 Downloading the build...url : http://172.23.126.166/builds/latestbuilds/couchbase-server/cheshire-cat/5046/couchbase-server-enterprise-7.0.0-5046-centos7.x86_64.rpm name : couchbase-server-enterprise-7.0.0-5046-centos7.x86_64.rpm product : product_version : 7.0.0-5046 os : deliverable_type : rpm architecture_type : x86_64 toy : toy- 2021-04-28 01:56:15,038 - root - INFO - running command.raw on 172.23.106.76: /sbin/iptables -F 2021-04-28 01:56:15,039 - root - INFO - command executed successfully with root 2021-04-28 01:56:15,039 - root - INFO - running command.raw on 172.23.106.77: /sbin/iptables -t nat -F 2021-04-28 01:56:15,056 - root - INFO - command executed successfully with root 2021-04-28 01:56:15,056 - root - INFO - running command.raw on 172.23.106.76: /sbin/iptables -t nat -F 2021-04-28 01:56:15,079 - root - INFO - command executed successfully with root 2021-04-28 01:56:15,079 - root - INFO - running command.raw on 172.23.106.78: /sbin/iptables -t nat -F 2021-04-28 01:56:15,082 - root - INFO - command executed successfully with root 2021-04-28 01:56:15,082 - root - INFO - running command.raw on 172.23.106.75: /sbin/iptables -t nat -F 2021-04-28 01:56:15,094 - root - INFO - command executed successfully with root 2021-04-28 01:56:15,094 - root - INFO - SSH Connecting to 172.23.106.77 with username:root, attempt#1 of 5 2021-04-28 01:56:15,110 - root - INFO - command executed successfully with root 2021-04-28 01:56:15,111 - root - INFO - SSH Connecting to 172.23.106.76 with username:root, attempt#1 of 5 2021-04-28 01:56:15,133 - root - INFO - command executed successfully with root 2021-04-28 01:56:15,134 - root - INFO - SSH Connecting to 172.23.106.78 with username:root, attempt#1 of 5 2021-04-28 01:56:15,137 - root - INFO - command executed successfully with root 2021-04-28 01:56:15,137 - root - INFO - SSH Connecting to 172.23.106.75 with username:root, attempt#1 of 5 2021-04-28 01:56:15,195 - root - INFO - SSH Connected to 172.23.106.77 as root 2021-04-28 01:56:15,211 - root - INFO - SSH Connected to 172.23.106.76 as root 2021-04-28 01:56:15,236 - root - INFO - SSH Connected to 172.23.106.78 as root 2021-04-28 01:56:15,237 - root - INFO - SSH Connected to 172.23.106.75 as root 2021-04-28 01:56:15,390 - root - INFO - running command.raw on 172.23.106.76: cd /tmp;wget -nc -q -O couchbase-server-enterprise-7.0.0-5046-centos7.x86_64.rpm http://172.23.126.166/builds/latestbuilds/couchbase-server/cheshire-cat/5046/couchbase-server-enterprise-7.0.0-5046-centos7.x86_64.rpm;cd /tmp;ls -lh 2021-04-28 01:56:15,390 - root - INFO - running command.raw on 172.23.106.77: cd /tmp;wget -nc -q -O couchbase-server-enterprise-7.0.0-5046-centos7.x86_64.rpm http://172.23.126.166/builds/latestbuilds/couchbase-server/cheshire-cat/5046/couchbase-server-enterprise-7.0.0-5046-centos7.x86_64.rpm;cd /tmp;ls -lh 2021-04-28 01:56:15,430 - root - INFO - running command.raw on 172.23.106.78: cd /tmp;wget -nc -q -O couchbase-server-enterprise-7.0.0-5046-centos7.x86_64.rpm http://172.23.126.166/builds/latestbuilds/couchbase-server/cheshire-cat/5046/couchbase-server-enterprise-7.0.0-5046-centos7.x86_64.rpm;cd /tmp;ls -lh 2021-04-28 01:56:15,432 - root - INFO - running command.raw on 172.23.106.75: cd /tmp;wget -nc -q -O couchbase-server-enterprise-7.0.0-5046-centos7.x86_64.rpm http://172.23.126.166/builds/latestbuilds/couchbase-server/cheshire-cat/5046/couchbase-server-enterprise-7.0.0-5046-centos7.x86_64.rpm;cd /tmp;ls -lh 2021-04-28 01:56:15,450 - root - INFO - command executed successfully with root 2021-04-28 01:56:15,451 - root - INFO - total 450M 2021-04-28 01:56:15,451 - root - INFO - -rw-r--r--. 1 root root 450M Apr 28 01:45 couchbase-server-enterprise-7.0.0-5046-centos7.x86_64.rpm 2021-04-28 01:56:15,451 - root - INFO - drwx------. 3 root root 17 Dec 30 12:15 systemd-private-f3599990727a4bc3919a1d2add63bcdf-ntpd.service-rovbVW 2021-04-28 01:56:15,452 - root - INFO - command executed successfully with root 2021-04-28 01:56:15,452 - root - INFO - total 450M 2021-04-28 01:56:15,452 - root - INFO - -rw-r--r--. 1 root root 450M Apr 28 01:45 couchbase-server-enterprise-7.0.0-5046-centos7.x86_64.rpm 2021-04-28 01:56:15,452 - root - INFO - drwx------. 3 root root 17 Dec 30 12:15 systemd-private-2d113fb2bf2248ee8fd25ec1079a6104-ntpd.service-p0cbI1 2021-04-28 01:56:15,489 - root - INFO - command executed successfully with root 2021-04-28 01:56:15,489 - root - INFO - total 450M 2021-04-28 01:56:15,489 - root - INFO - -rw-r--r--. 1 root root 450M Apr 28 01:45 couchbase-server-enterprise-7.0.0-5046-centos7.x86_64.rpm 2021-04-28 01:56:15,489 - root - INFO - drwx------. 3 root root 17 Dec 30 12:15 systemd-private-aadb504c8d4844308658e8e1f1a8132e-ntpd.service-moCHwU 2021-04-28 01:56:15,493 - root - INFO - command executed successfully with root 2021-04-28 01:56:15,493 - root - INFO - total 450M 2021-04-28 01:56:15,493 - root - INFO - -rw-r--r--. 1 root root 450M Apr 28 01:45 couchbase-server-enterprise-7.0.0-5046-centos7.x86_64.rpm 2021-04-28 01:56:15,493 - root - INFO - drwx------. 3 root root 17 Dec 30 12:15 systemd-private-95ad56093ad84fa1a5d98e1bcfe95623-ntpd.service-9rlQCJ 2021-04-28 01:56:15,512 - root - INFO - *****install server *** 2021-04-28 01:56:15,512 - root - INFO - -->deliverable_type : rpm 2021-04-28 01:56:15,512 - root - INFO - /tmp/couchbase-server-enterprise-7.0.0-5046-centos7.x86_64.rpm or /tmp/ 2021-04-28 01:56:15,515 - root - INFO - *****install server *** 2021-04-28 01:56:15,515 - root - INFO - -->deliverable_type : rpm 2021-04-28 01:56:15,516 - root - INFO - /tmp/couchbase-server-enterprise-7.0.0-5046-centos7.x86_64.rpm or /tmp/ 2021-04-28 01:56:15,530 - root - INFO - --> execute_command: end... --> inside rpm check... 2021-04-28 01:56:15,530 - root - INFO - running command.raw on 172.23.106.76: cat /etc/redhat-release 2021-04-28 01:56:15,551 - root - INFO - *****install server *** 2021-04-28 01:56:15,551 - root - INFO - -->deliverable_type : rpm 2021-04-28 01:56:15,551 - root - INFO - /tmp/couchbase-server-enterprise-7.0.0-5046-centos7.x86_64.rpm or /tmp/ 2021-04-28 01:56:15,554 - root - INFO - *****install server *** 2021-04-28 01:56:15,554 - root - INFO - -->deliverable_type : rpm 2021-04-28 01:56:15,554 - root - INFO - /tmp/couchbase-server-enterprise-7.0.0-5046-centos7.x86_64.rpm or /tmp/ 2021-04-28 01:56:15,572 - root - INFO - --> execute_command: end... --> inside rpm check... 2021-04-28 01:56:15,572 - root - INFO - running command.raw on 172.23.106.77: cat /etc/redhat-release 2021-04-28 01:56:15,573 - root - INFO - --> execute_command: end... --> inside rpm check... 2021-04-28 01:56:15,574 - root - INFO - running command.raw on 172.23.106.75: cat /etc/redhat-release 2021-04-28 01:56:15,583 - root - INFO - command executed successfully with root 2021-04-28 01:56:15,584 - root - INFO - CentOS Linux release 7.6.1810 (Core) 2021-04-28 01:56:15,608 - root - INFO - --> execute_command: end... --> inside rpm check... 2021-04-28 01:56:15,609 - root - INFO - running command.raw on 172.23.106.78: cat /etc/redhat-release 2021-04-28 01:56:15,626 - root - INFO - command executed successfully with root 2021-04-28 01:56:15,627 - root - INFO - CentOS Linux release 7.6.1810 (Core) 2021-04-28 01:56:15,628 - root - INFO - command executed successfully with root 2021-04-28 01:56:15,629 - root - INFO - CentOS Linux release 7.6.1810 (Core) 2021-04-28 01:56:15,663 - root - INFO - command executed successfully with root 2021-04-28 01:56:15,663 - root - INFO - CentOS Linux release 7.6.1810 (Core) 2021-04-28 01:57:15,713 - root - INFO - If couchbase server is running with this error. Go to log_command_output to add error mesg to bypass it. 2021-04-28 01:57:15,713 - root - INFO - Warming: RPMDB altered outside of yum 2021-04-28 01:57:15,737 - root - INFO - Checking if the directory /tmp exists or not. 2021-04-28 01:57:15,738 - root - ERROR - Directory at /tmp DOES exist. Fx returns True 2021-04-28 01:57:15,817 - root - INFO - running command.raw on 172.23.106.76: sed -i 's/export PATH/export PATH\nexport CBFT_ENV_OPTIONS=bleveMaxResultWindow=10000000/' /opt/couchbase/bin/couchbase-server 2021-04-28 01:57:15,874 - root - INFO - command executed successfully with root 2021-04-28 01:57:15,874 - root - INFO - Checking for process beam.smp on linux 2021-04-28 01:57:15,886 - root - INFO - If couchbase server is running with this error. Go to log_command_output to add error mesg to bypass it. 2021-04-28 01:57:15,887 - root - INFO - Warming: RPMDB altered outside of yum 2021-04-28 01:57:15,909 - root - INFO - Checking if the directory /tmp exists or not. 2021-04-28 01:57:15,910 - root - ERROR - Directory at /tmp DOES exist. Fx returns True 2021-04-28 01:57:15,938 - root - INFO - If couchbase server is running with this error. Go to log_command_output to add error mesg to bypass it. 2021-04-28 01:57:15,938 - root - INFO - Warming: RPMDB altered outside of yum 2021-04-28 01:57:15,946 - root - INFO - If couchbase server is running with this error. Go to log_command_output to add error mesg to bypass it. 2021-04-28 01:57:15,946 - root - INFO - Warming: RPMDB altered outside of yum 2021-04-28 01:57:15,952 - root - INFO - Starting couchbase server 2021-04-28 01:57:15,964 - root - INFO - Checking if the directory /tmp exists or not. 2021-04-28 01:57:15,965 - root - ERROR - Directory at /tmp DOES exist. Fx returns True 2021-04-28 01:57:15,969 - root - INFO - Checking if the directory /tmp exists or not. 2021-04-28 01:57:15,970 - root - ERROR - Directory at /tmp DOES exist. Fx returns True 2021-04-28 01:57:15,986 - root - INFO - running command.raw on 172.23.106.75: sed -i 's/export PATH/export PATH\nexport CBFT_ENV_OPTIONS=bleveMaxResultWindow=10000000/' /opt/couchbase/bin/couchbase-server 2021-04-28 01:57:16,041 - root - INFO - command executed successfully with root 2021-04-28 01:57:16,041 - root - INFO - Checking for process beam.smp on linux 2021-04-28 01:57:16,043 - root - INFO - running command.raw on 172.23.106.78: sed -i 's/export PATH/export PATH\nexport CBFT_ENV_OPTIONS=bleveMaxResultWindow=10000000/' /opt/couchbase/bin/couchbase-server 2021-04-28 01:57:16,046 - root - INFO - running command.raw on 172.23.106.77: sed -i 's/export PATH/export PATH\nexport CBFT_ENV_OPTIONS=bleveMaxResultWindow=10000000/' /opt/couchbase/bin/couchbase-server 2021-04-28 01:57:16,058 - root - INFO - Running systemd command on this server 2021-04-28 01:57:16,058 - root - INFO - running command.raw on 172.23.106.76: systemctl start couchbase-server.service 2021-04-28 01:57:16,082 - root - INFO - command executed successfully with root 2021-04-28 01:57:16,083 - root - INFO - 172.23.106.76:sleep for 5 secs. waiting for couchbase server to come up ... 2021-04-28 01:57:16,099 - root - INFO - command executed successfully with root 2021-04-28 01:57:16,099 - root - INFO - Checking for process beam.smp on linux 2021-04-28 01:57:16,102 - root - INFO - command executed successfully with root 2021-04-28 01:57:16,102 - root - INFO - Checking for process beam.smp on linux 2021-04-28 01:57:16,118 - root - INFO - Starting couchbase server 2021-04-28 01:57:16,176 - root - INFO - Starting couchbase server 2021-04-28 01:57:16,181 - root - INFO - Starting couchbase server 2021-04-28 01:57:16,264 - root - INFO - Running systemd command on this server 2021-04-28 01:57:16,264 - root - INFO - running command.raw on 172.23.106.75: systemctl start couchbase-server.service 2021-04-28 01:57:16,284 - root - INFO - Running systemd command on this server 2021-04-28 01:57:16,284 - root - INFO - running command.raw on 172.23.106.78: systemctl start couchbase-server.service 2021-04-28 01:57:16,289 - root - INFO - command executed successfully with root 2021-04-28 01:57:16,289 - root - INFO - 172.23.106.75:sleep for 5 secs. waiting for couchbase server to come up ... 2021-04-28 01:57:16,290 - root - INFO - Running systemd command on this server 2021-04-28 01:57:16,290 - root - INFO - running command.raw on 172.23.106.77: systemctl start couchbase-server.service 2021-04-28 01:57:16,310 - root - INFO - command executed successfully with root 2021-04-28 01:57:16,311 - root - INFO - 172.23.106.78:sleep for 5 secs. waiting for couchbase server to come up ... 2021-04-28 01:57:16,317 - root - INFO - command executed successfully with root 2021-04-28 01:57:16,317 - root - INFO - 172.23.106.77:sleep for 5 secs. waiting for couchbase server to come up ... 2021-04-28 01:57:21,088 - root - INFO - running command.raw on 172.23.106.76: systemctl status couchbase-server.service | grep ExecStop=/opt/couchbase/bin/couchbase-server 2021-04-28 01:57:21,119 - root - INFO - command executed successfully with root 2021-04-28 01:57:21,119 - root - INFO - Couchbase server status: [] 2021-04-28 01:57:21,119 - root - INFO - Checking for process beam.smp on linux 2021-04-28 01:57:21,189 - root - INFO - process beam.smp is running on 172.23.106.76: with pid 26265 2021-04-28 01:57:21,245 - root - INFO - wait 30 seconds for Couchbase server to start 2021-04-28 01:57:21,294 - root - INFO - running command.raw on 172.23.106.75: systemctl status couchbase-server.service | grep ExecStop=/opt/couchbase/bin/couchbase-server 2021-04-28 01:57:21,316 - root - INFO - running command.raw on 172.23.106.78: systemctl status couchbase-server.service | grep ExecStop=/opt/couchbase/bin/couchbase-server 2021-04-28 01:57:21,318 - root - INFO - running command.raw on 172.23.106.77: systemctl status couchbase-server.service | grep ExecStop=/opt/couchbase/bin/couchbase-server 2021-04-28 01:57:21,343 - root - INFO - command executed successfully with root 2021-04-28 01:57:21,343 - root - INFO - Couchbase server status: [] 2021-04-28 01:57:21,344 - root - INFO - Checking for process beam.smp on linux 2021-04-28 01:57:21,346 - root - INFO - command executed successfully with root 2021-04-28 01:57:21,347 - root - INFO - Couchbase server status: [] 2021-04-28 01:57:21,347 - root - INFO - Checking for process beam.smp on linux 2021-04-28 01:57:21,352 - root - INFO - command executed successfully with root 2021-04-28 01:57:21,352 - root - INFO - Couchbase server status: [] 2021-04-28 01:57:21,352 - root - INFO - Checking for process beam.smp on linux 2021-04-28 01:57:21,384 - root - INFO - process beam.smp is running on 172.23.106.75: with pid 77853 2021-04-28 01:57:21,415 - root - INFO - process beam.smp is running on 172.23.106.78: with pid 49121 2021-04-28 01:57:21,422 - root - INFO - process beam.smp is running on 172.23.106.77: with pid 15326 2021-04-28 01:57:21,440 - root - INFO - wait 30 seconds for Couchbase server to start 2021-04-28 01:57:21,469 - root - INFO - wait 30 seconds for Couchbase server to start 2021-04-28 01:57:21,477 - root - INFO - wait 30 seconds for Couchbase server to start thread installer-thread-172.23.106.75 finished thread installer-thread-172.23.106.76 finished thread installer-thread-172.23.106.77 finished thread installer-thread-172.23.106.78 finished 2021-04-28 01:57:51,501 - root - INFO - SSH Connecting to 172.23.106.75 with username:root, attempt#1 of 5 2021-04-28 01:57:51,503 - root - INFO - SSH Connecting to 172.23.106.76 with username:root, attempt#1 of 5 2021-04-28 01:57:51,505 - root - INFO - SSH Connecting to 172.23.106.77 with username:root, attempt#1 of 5 2021-04-28 01:57:51,506 - root - INFO - SSH Connecting to 172.23.106.78 with username:root, attempt#1 of 5 2021-04-28 01:57:51,607 - root - INFO - SSH Connected to 172.23.106.76 as root 2021-04-28 01:57:51,609 - root - INFO - SSH Connected to 172.23.106.75 as root 2021-04-28 01:57:51,615 - root - INFO - SSH Connected to 172.23.106.77 as root 2021-04-28 01:57:51,617 - root - INFO - SSH Connected to 172.23.106.78 as root 2021-04-28 01:57:51,869 - root - INFO - os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-04-28 01:57:51,892 - root - INFO - os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-04-28 01:57:51,894 - root - INFO - os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-04-28 01:57:51,899 - root - INFO - os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-04-28 01:57:52,181 - root - INFO - extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-04-28 01:57:52,202 - root - INFO - extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-04-28 01:57:52,206 - root - INFO - 172.23.106.76 **** The linux version file /opt/couchbase/ VERSION.txt exists 2021-04-28 01:57:52,206 - root - INFO - creating RestConnection(ip:172.23.106.76 port:8091 ssh_username:root) and the server type is 2021-04-28 01:57:52,214 - root - INFO - Done creating rest connection 2021-04-28 01:57:52,214 - root - INFO - extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-04-28 01:57:52,216 - root - INFO - extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-04-28 01:57:52,240 - root - INFO - 172.23.106.75 **** The linux version file /opt/couchbase/ VERSION.txt exists 2021-04-28 01:57:52,240 - root - INFO - creating RestConnection(ip:172.23.106.75 port:8091 ssh_username:root) and the server type is 2021-04-28 01:57:52,248 - root - INFO - Done creating rest connection 2021-04-28 01:57:52,265 - root - INFO - 172.23.106.77 **** The linux version file /opt/couchbase/ VERSION.txt exists 2021-04-28 01:57:52,265 - root - INFO - creating RestConnection(ip:172.23.106.77 port:8091 ssh_username:root) and the server type is 2021-04-28 01:57:52,271 - root - INFO - Done creating rest connection 2021-04-28 01:57:52,276 - root - INFO - 172.23.106.78 **** The linux version file /opt/couchbase/ VERSION.txt exists 2021-04-28 01:57:52,276 - root - INFO - creating RestConnection(ip:172.23.106.78 port:8091 ssh_username:root) and the server type is 2021-04-28 01:57:52,285 - root - INFO - Done creating rest connection 2021-04-28 01:57:56,228 - root - INFO - --> rest.get_nodes_self().mcdMemoryReserved 2021-04-28 01:57:56,228 - root - INFO - --> info.mcdMemoryReserved=3010,CLUSTER_QUOTA_RATIO=0.67 2021-04-28 01:57:56,228 - root - INFO - quota for kv: 2015 MB 2021-04-28 01:57:56,228 - root - INFO - pools/default params : memoryQuota=2015 2021-04-28 01:57:56,234 - root - INFO - --> init_node_services(Administrator,password,127.0.0.1,8091,['kv']) 2021-04-28 01:57:56,234 - root - INFO - /node/controller/setupServices params on 172.23.106.76: 8091:hostname=127.0.0.1%3A8091&user=Administrator&password=password&services=kv 2021-04-28 01:57:56,260 - root - INFO - --> rest.get_nodes_self().mcdMemoryReserved 2021-04-28 01:57:56,260 - root - INFO - --> info.mcdMemoryReserved=3010,CLUSTER_QUOTA_RATIO=0.67 2021-04-28 01:57:56,260 - root - INFO - quota for kv: 2015 MB 2021-04-28 01:57:56,260 - root - INFO - pools/default params : memoryQuota=2015 2021-04-28 01:57:56,263 - root - INFO - --> in init_cluster...Administrator,password,8091 2021-04-28 01:57:56,263 - root - INFO - settings/web params on 172.23.106.76:8091:port=8091&username=Administrator&password=password 2021-04-28 01:57:56,266 - root - INFO - --> init_node_services(Administrator,password,127.0.0.1,8091,['kv']) 2021-04-28 01:57:56,266 - root - INFO - /node/controller/setupServices params on 172.23.106.75: 8091:hostname=127.0.0.1%3A8091&user=Administrator&password=password&services=kv 2021-04-28 01:57:56,282 - root - INFO - --> rest.get_nodes_self().mcdMemoryReserved 2021-04-28 01:57:56,282 - root - INFO - --> info.mcdMemoryReserved=3010,CLUSTER_QUOTA_RATIO=0.67 2021-04-28 01:57:56,282 - root - INFO - quota for kv: 2015 MB 2021-04-28 01:57:56,282 - root - INFO - pools/default params : memoryQuota=2015 2021-04-28 01:57:56,288 - root - INFO - --> init_node_services(Administrator,password,127.0.0.1,8091,['kv']) 2021-04-28 01:57:56,288 - root - INFO - /node/controller/setupServices params on 172.23.106.77: 8091:hostname=127.0.0.1%3A8091&user=Administrator&password=password&services=kv 2021-04-28 01:57:56,295 - root - INFO - --> in init_cluster...Administrator,password,8091 2021-04-28 01:57:56,295 - root - INFO - settings/web params on 172.23.106.75:8091:port=8091&username=Administrator&password=password 2021-04-28 01:57:56,299 - root - INFO - --> rest.get_nodes_self().mcdMemoryReserved 2021-04-28 01:57:56,299 - root - INFO - --> info.mcdMemoryReserved=3010,CLUSTER_QUOTA_RATIO=0.67 2021-04-28 01:57:56,299 - root - INFO - quota for kv: 2015 MB 2021-04-28 01:57:56,299 - root - INFO - pools/default params : memoryQuota=2015 2021-04-28 01:57:56,305 - root - INFO - --> init_node_services(Administrator,password,127.0.0.1,8091,['kv']) 2021-04-28 01:57:56,305 - root - INFO - /node/controller/setupServices params on 172.23.106.78: 8091:hostname=127.0.0.1%3A8091&user=Administrator&password=password&services=kv 2021-04-28 01:57:56,317 - root - INFO - --> in init_cluster...Administrator,password,8091 2021-04-28 01:57:56,318 - root - INFO - settings/web params on 172.23.106.77:8091:port=8091&username=Administrator&password=password 2021-04-28 01:57:56,331 - root - INFO - --> status:True 2021-04-28 01:57:56,334 - root - INFO - --> in init_cluster...Administrator,password,8091 2021-04-28 01:57:56,334 - root - INFO - settings/web params on 172.23.106.78:8091:port=8091&username=Administrator&password=password 2021-04-28 01:57:56,335 - root - INFO - -->is_ns_server_running? 2021-04-28 01:57:56,348 - root - INFO - Node version in cluster 7.0.0-5046-enterprise 2021-04-28 01:57:56,348 - root - INFO - ***** set NS_SERVER_CBAUTH env in linux ***** 2021-04-28 01:57:56,360 - root - INFO - --> status:True 2021-04-28 01:57:56,367 - root - INFO - -->is_ns_server_running? 2021-04-28 01:57:56,378 - root - INFO - Node version in cluster 7.0.0-5046-enterprise 2021-04-28 01:57:56,378 - root - INFO - ***** set NS_SERVER_CBAUTH env in linux ***** 2021-04-28 01:57:56,382 - root - INFO - --> status:True 2021-04-28 01:57:56,386 - root - INFO - -->is_ns_server_running? 2021-04-28 01:57:56,397 - root - INFO - --> status:True 2021-04-28 01:57:56,402 - root - INFO - Node version in cluster 7.0.0-5046-enterprise 2021-04-28 01:57:56,402 - root - INFO - ***** set NS_SERVER_CBAUTH env in linux ***** 2021-04-28 01:57:56,403 - root - INFO - -->is_ns_server_running? 2021-04-28 01:57:56,413 - root - INFO - Node version in cluster 7.0.0-5046-enterprise 2021-04-28 01:57:56,414 - root - INFO - ***** set NS_SERVER_CBAUTH env in linux ***** 2021-04-28 01:57:56,584 - root - INFO - check if man installed on vm? 2021-04-28 01:57:56,584 - root - INFO - running command.raw on 172.23.106.76: rpm -qa | grep 'man-db' 2021-04-28 01:57:56,620 - root - INFO - check if man installed on vm? 2021-04-28 01:57:56,620 - root - INFO - running command.raw on 172.23.106.75: rpm -qa | grep 'man-db' 2021-04-28 01:57:56,640 - root - INFO - check if man installed on vm? 2021-04-28 01:57:56,640 - root - INFO - running command.raw on 172.23.106.77: rpm -qa | grep 'man-db' 2021-04-28 01:57:56,648 - root - INFO - check if man installed on vm? 2021-04-28 01:57:56,648 - root - INFO - running command.raw on 172.23.106.78: rpm -qa | grep 'man-db' 2021-04-28 01:57:57,205 - root - INFO - command executed successfully with root 2021-04-28 01:57:57,206 - root - INFO - ---> Run command unzip to check if it is ready on server 172.23.106.76 2021-04-28 01:57:57,206 - root - INFO - running command.raw on 172.23.106.76: unzip 2021-04-28 01:57:57,224 - root - INFO - command executed successfully with root 2021-04-28 01:57:57,224 - root - INFO - unzip command is ready 2021-04-28 01:57:57,224 - root - INFO - Check if ntp is installed 2021-04-28 01:57:57,224 - root - INFO - This OS version centos 7 2021-04-28 01:57:57,224 - root - INFO - running command.raw on 172.23.106.76: systemctl status ntpd 2021-04-28 01:57:57,232 - root - INFO - command executed successfully with root 2021-04-28 01:57:57,233 - root - INFO - ---> Run command unzip to check if it is ready on server 172.23.106.75 2021-04-28 01:57:57,233 - root - INFO - running command.raw on 172.23.106.75: unzip 2021-04-28 01:57:57,250 - root - INFO - command executed successfully with root 2021-04-28 01:57:57,250 - root - INFO - unzip command is ready 2021-04-28 01:57:57,250 - root - INFO - Check if ntp is installed 2021-04-28 01:57:57,250 - root - INFO - This OS version centos 7 2021-04-28 01:57:57,250 - root - INFO - running command.raw on 172.23.106.75: systemctl status ntpd 2021-04-28 01:57:57,261 - root - INFO - command executed successfully with root 2021-04-28 01:57:57,261 - root - INFO - ---> Run command unzip to check if it is ready on server 172.23.106.77 2021-04-28 01:57:57,261 - root - INFO - running command.raw on 172.23.106.77: unzip 2021-04-28 01:57:57,291 - root - INFO - command executed successfully with root 2021-04-28 01:57:57,291 - root - INFO - ---> Run command unzip to check if it is ready on server 172.23.106.78 2021-04-28 01:57:57,291 - root - INFO - running command.raw on 172.23.106.78: unzip 2021-04-28 01:57:57,293 - root - INFO - command executed successfully with root 2021-04-28 01:57:57,293 - root - INFO - running command.raw on 172.23.106.76: ntpstat 2021-04-28 01:57:57,318 - root - INFO - command executed successfully with root 2021-04-28 01:57:57,318 - root - INFO - unzip command is ready 2021-04-28 01:57:57,318 - root - INFO - Check if ntp is installed 2021-04-28 01:57:57,318 - root - INFO - This OS version centos 7 2021-04-28 01:57:57,318 - root - INFO - running command.raw on 172.23.106.77: systemctl status ntpd 2021-04-28 01:57:57,319 - root - INFO - command executed successfully with root 2021-04-28 01:57:57,319 - root - INFO - running command.raw on 172.23.106.75: ntpstat 2021-04-28 01:57:57,347 - root - INFO - command executed successfully with root 2021-04-28 01:57:57,347 - root - INFO - running command.raw on 172.23.106.76: timedatectl status 2021-04-28 01:57:57,348 - root - INFO - command executed successfully with root 2021-04-28 01:57:57,349 - root - INFO - unzip command is ready 2021-04-28 01:57:57,349 - root - INFO - Check if ntp is installed 2021-04-28 01:57:57,349 - root - INFO - This OS version centos 7 2021-04-28 01:57:57,349 - root - INFO - running command.raw on 172.23.106.78: systemctl status ntpd 2021-04-28 01:57:57,374 - root - INFO - command executed successfully with root 2021-04-28 01:57:57,374 - root - INFO - running command.raw on 172.23.106.75: timedatectl status 2021-04-28 01:57:57,386 - root - INFO - command executed successfully with root 2021-04-28 01:57:57,386 - root - INFO - running command.raw on 172.23.106.77: ntpstat 2021-04-28 01:57:57,416 - root - INFO - command executed successfully with root 2021-04-28 01:57:57,416 - root - INFO - running command.raw on 172.23.106.78: ntpstat 2021-04-28 01:57:57,431 - root - INFO - command executed successfully with root 2021-04-28 01:57:57,431 - root - INFO - running command.raw on 172.23.106.76: date 2021-04-28 01:57:57,442 - root - INFO - command executed successfully with root 2021-04-28 01:57:57,442 - root - INFO - running command.raw on 172.23.106.77: timedatectl status 2021-04-28 01:57:57,457 - root - INFO - command executed successfully with root 2021-04-28 01:57:57,457 - root - INFO - running command.raw on 172.23.106.75: date 2021-04-28 01:57:57,470 - root - INFO - command executed successfully with root 2021-04-28 01:57:57,471 - root - INFO - running command.raw on 172.23.106.78: timedatectl status 2021-04-28 01:57:57,486 - root - INFO - command executed successfully with root 2021-04-28 01:57:57,512 - root - INFO - command executed successfully with root 2021-04-28 01:57:57,525 - root - INFO - command executed successfully with root 2021-04-28 01:57:57,525 - root - INFO - running command.raw on 172.23.106.77: date 2021-04-28 01:57:57,556 - root - INFO - command executed successfully with root 2021-04-28 01:57:57,556 - root - INFO - running command.raw on 172.23.106.78: date 2021-04-28 01:57:57,582 - root - INFO - command executed successfully with root 2021-04-28 01:57:57,599 - root - INFO - ['Wed Apr 28 01:57:57 PDT 2021'] IP: 172.23.106.76 2021-04-28 01:57:57,612 - root - INFO - command executed successfully with root 2021-04-28 01:57:57,622 - root - INFO - ['Wed Apr 28 01:57:57 PDT 2021'] IP: 172.23.106.75 thread initializer-thread-172.23.106.75 finished thread initializer-thread-172.23.106.76 finished 2021-04-28 01:57:57,695 - root - INFO - ['Wed Apr 28 01:57:57 PDT 2021'] IP: 172.23.106.77 thread initializer-thread-172.23.106.77 finished 2021-04-28 01:57:57,724 - root - INFO - ['Wed Apr 28 01:57:57 PDT 2021'] IP: 172.23.106.78 thread initializer-thread-172.23.106.78 finished 2021-04-28 01:57:57,726 - root - INFO - SSH Connecting to 172.23.106.75 with username:root, attempt#1 of 5 2021-04-28 01:57:57,829 - root - INFO - SSH Connected to 172.23.106.75 as root 2021-04-28 01:57:58,086 - root - INFO - os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-04-28 01:57:58,386 - root - INFO - extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-04-28 01:57:58,388 - root - INFO - SSH Connecting to 172.23.106.75 with username:root, attempt#1 of 5 2021-04-28 01:57:58,489 - root - INFO - SSH Connected to 172.23.106.75 as root 2021-04-28 01:57:58,749 - root - INFO - os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-04-28 01:57:59,056 - root - INFO - extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-04-28 01:57:59,080 - root - INFO - 172.23.106.75 **** The linux version file /opt/couchbase/ VERSION.txt exists 2021-04-28 01:57:59,080 - root - INFO - sleep for 5 secs. sleep 5 seconds to let cb up completely ... 2021-04-28 01:58:04,089 - root - INFO - -->is_ns_server_running? 2021-04-28 01:58:04,093 - root - INFO - SSH Connecting to 172.23.106.76 with username:root, attempt#1 of 5 2021-04-28 01:58:04,197 - root - INFO - SSH Connected to 172.23.106.76 as root 2021-04-28 01:58:04,445 - root - INFO - os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-04-28 01:58:04,747 - root - INFO - extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-04-28 01:58:04,770 - root - INFO - 172.23.106.76 **** The linux version file /opt/couchbase/ VERSION.txt exists 2021-04-28 01:58:04,770 - root - INFO - sleep for 5 secs. sleep 5 seconds to let cb up completely ... 2021-04-28 01:58:09,779 - root - INFO - -->is_ns_server_running? 2021-04-28 01:58:09,784 - root - INFO - SSH Connecting to 172.23.106.77 with username:root, attempt#1 of 5 2021-04-28 01:58:09,886 - root - INFO - SSH Connected to 172.23.106.77 as root 2021-04-28 01:58:10,147 - root - INFO - os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-04-28 01:58:10,446 - root - INFO - extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-04-28 01:58:10,469 - root - INFO - 172.23.106.77 **** The linux version file /opt/couchbase/ VERSION.txt exists 2021-04-28 01:58:10,469 - root - INFO - sleep for 5 secs. sleep 5 seconds to let cb up completely ... 2021-04-28 01:58:15,478 - root - INFO - -->is_ns_server_running? 2021-04-28 01:58:15,482 - root - INFO - SSH Connecting to 172.23.106.78 with username:root, attempt#1 of 5 2021-04-28 01:58:15,585 - root - INFO - SSH Connected to 172.23.106.78 as root 2021-04-28 01:58:15,852 - root - INFO - os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-04-28 01:58:16,155 - root - INFO - extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-04-28 01:58:16,179 - root - INFO - 172.23.106.78 **** The linux version file /opt/couchbase/ VERSION.txt exists 2021-04-28 01:58:16,179 - root - INFO - sleep for 5 secs. sleep 5 seconds to let cb up completely ... 2021-04-28 01:58:21,189 - root - INFO - -->is_ns_server_running? 2021-04-28 01:58:21,706 - root - INFO - server: ip:172.23.106.64 port:8091 ssh_username:root, nodes/self 2021-04-28 01:58:21,709 - root - INFO - {'uptime': '197', 'memoryTotal': 3945984000, 'memoryFree': 3490254848, 'mcdMemoryReserved': 3010, 'mcdMemoryAllocated': 3010, 'status': 'healthy', 'hostname': '172.23.106.64:8091', 'clusterCompatibility': 393222, 'clusterMembership': 'active', 'recoveryType': 'none', 'version': '6.6.1-9213-enterprise', 'os': 'x86_64-unknown-linux-gnu', 'ports': [], 'availableStorage': [], 'storage': [], 'memoryQuota': 2015, 'moxi': 11211, 'memcached': 11210, 'id': 'ns_1@cb.local', 'ip': '172.23.106.64', 'rest_username': '', 'rest_password': '', 'port': '8091', 'services': ['kv'], 'storageTotalRam': 3763, 'curr_items': 0} 2021-04-28 01:58:21,712 - root - INFO - pools/default params : memoryQuota=2016 2021-04-28 01:58:21,715 - root - INFO - --> init_node_services(Administrator,password,172.23.106.64,8091,['kv']) 2021-04-28 01:58:21,715 - root - INFO - /node/controller/setupServices params on 172.23.106.64: 8091:hostname=172.23.106.64&user=Administrator&password=password&services=kv 2021-04-28 01:58:21,717 - root - ERROR - POST http://172.23.106.64:8091//node/controller/setupServices body: hostname=172.23.106.64&user=Administrator&password=password&services=kv headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==', 'Accept': '*/*'} error: 400 reason: unknown b'["cannot change node services after cluster is provisioned"]' auth: Administrator:password 2021-04-28 01:58:21,717 - root - INFO - This node is already provisioned with services, we do not consider this as failure for test case 2021-04-28 01:58:21,717 - root - INFO - --> in init_cluster...Administrator,password,8091 2021-04-28 01:58:21,718 - root - INFO - settings/web params on 172.23.106.64:8091:port=8091&username=Administrator&password=password 2021-04-28 01:58:21,753 - root - INFO - --> status:True 2021-04-28 01:58:21,754 - root - INFO - SSH Connecting to 172.23.106.64 with username:root, attempt#1 of 5 2021-04-28 01:58:21,859 - root - INFO - SSH Connected to 172.23.106.64 as root 2021-04-28 01:58:22,114 - root - INFO - os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-04-28 01:58:22,430 - root - INFO - extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-04-28 01:58:22,436 - root - INFO - running command.raw on 172.23.106.64: curl --silent --show-error http://Administrator:password@localhost:8091/diag/eval -X POST -d 'ns_config:set(allow_nonlocal_eval, true).' 2021-04-28 01:58:22,497 - root - INFO - command executed successfully with root 2021-04-28 01:58:22,497 - root - INFO - ['ok'] 2021-04-28 01:58:22,499 - root - INFO - /diag/eval status on 172.23.106.64:8091: True content: [6,6] command: cluster_compat_mode:get_compat_version(). 2021-04-28 01:58:22,501 - root - INFO - /diag/eval status on 172.23.106.64:8091: True content: [6,6] command: cluster_compat_mode:get_compat_version(). 2021-04-28 01:58:22,504 - root - INFO - settings/indexes params : storageMode=plasma 2021-04-28 01:58:22,520 - root - INFO - server: ip:172.23.106.67 port:8091 ssh_username:root, nodes/self 2021-04-28 01:58:22,523 - root - INFO - {'uptime': '197', 'memoryTotal': 3945984000, 'memoryFree': 3478392832, 'mcdMemoryReserved': 3010, 'mcdMemoryAllocated': 3010, 'status': 'healthy', 'hostname': '172.23.106.67:8091', 'clusterCompatibility': 393222, 'clusterMembership': 'active', 'recoveryType': 'none', 'version': '6.6.1-9213-enterprise', 'os': 'x86_64-unknown-linux-gnu', 'ports': [], 'availableStorage': [], 'storage': [], 'memoryQuota': 2015, 'moxi': 11211, 'memcached': 11210, 'id': 'ns_1@cb.local', 'ip': '172.23.106.67', 'rest_username': '', 'rest_password': '', 'port': '8091', 'services': ['kv'], 'storageTotalRam': 3763, 'curr_items': 0} 2021-04-28 01:58:22,527 - root - INFO - pools/default params : memoryQuota=2016 2021-04-28 01:58:22,530 - root - INFO - --> init_node_services(Administrator,password,172.23.106.67,8091,['eventing']) 2021-04-28 01:58:22,530 - root - INFO - /node/controller/setupServices params on 172.23.106.67: 8091:hostname=172.23.106.67&user=Administrator&password=password&services=eventing 2021-04-28 01:58:22,532 - root - ERROR - POST http://172.23.106.67:8091//node/controller/setupServices body: hostname=172.23.106.67&user=Administrator&password=password&services=eventing headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==', 'Accept': '*/*'} error: 400 reason: unknown b'["cannot change node services after cluster is provisioned"]' auth: Administrator:password 2021-04-28 01:58:22,532 - root - INFO - This node is already provisioned with services, we do not consider this as failure for test case 2021-04-28 01:58:22,532 - root - INFO - --> in init_cluster...Administrator,password,8091 2021-04-28 01:58:22,533 - root - INFO - settings/web params on 172.23.106.67:8091:port=8091&username=Administrator&password=password 2021-04-28 01:58:22,568 - root - INFO - --> status:True 2021-04-28 01:58:22,570 - root - INFO - SSH Connecting to 172.23.106.67 with username:root, attempt#1 of 5 2021-04-28 01:58:22,674 - root - INFO - SSH Connected to 172.23.106.67 as root 2021-04-28 01:58:22,933 - root - INFO - os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-04-28 01:58:23,237 - root - INFO - extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-04-28 01:58:23,244 - root - INFO - running command.raw on 172.23.106.67: curl --silent --show-error http://Administrator:password@localhost:8091/diag/eval -X POST -d 'ns_config:set(allow_nonlocal_eval, true).' 2021-04-28 01:58:23,303 - root - INFO - command executed successfully with root 2021-04-28 01:58:23,303 - root - INFO - ['ok'] 2021-04-28 01:58:23,305 - root - INFO - /diag/eval status on 172.23.106.67:8091: True content: [6,6] command: cluster_compat_mode:get_compat_version(). 2021-04-28 01:58:23,307 - root - INFO - /diag/eval status on 172.23.106.67:8091: True content: [6,6] command: cluster_compat_mode:get_compat_version(). 2021-04-28 01:58:23,311 - root - INFO - settings/indexes params : storageMode=plasma 2021-04-28 01:58:23,327 - root - INFO - server: ip:172.23.106.73 port:8091 ssh_username:root, nodes/self 2021-04-28 01:58:23,330 - root - INFO - {'uptime': '198', 'memoryTotal': 3945984000, 'memoryFree': 3475582976, 'mcdMemoryReserved': 3010, 'mcdMemoryAllocated': 3010, 'status': 'healthy', 'hostname': '172.23.106.73:8091', 'clusterCompatibility': 393222, 'clusterMembership': 'active', 'recoveryType': 'none', 'version': '6.6.1-9213-enterprise', 'os': 'x86_64-unknown-linux-gnu', 'ports': [], 'availableStorage': [], 'storage': [], 'memoryQuota': 2015, 'moxi': 11211, 'memcached': 11210, 'id': 'ns_1@cb.local', 'ip': '172.23.106.73', 'rest_username': '', 'rest_password': '', 'port': '8091', 'services': ['kv'], 'storageTotalRam': 3763, 'curr_items': 0} 2021-04-28 01:58:23,333 - root - INFO - quota for index service will be 256 MB 2021-04-28 01:58:23,333 - root - INFO - set index quota to node 172.23.106.73 2021-04-28 01:58:23,333 - root - INFO - pools/default params : indexMemoryQuota=256 2021-04-28 01:58:23,336 - root - INFO - pools/default params : memoryQuota=1760 2021-04-28 01:58:23,340 - root - INFO - --> init_node_services(Administrator,password,172.23.106.73,8091,['index']) 2021-04-28 01:58:23,340 - root - INFO - /node/controller/setupServices params on 172.23.106.73: 8091:hostname=172.23.106.73&user=Administrator&password=password&services=index 2021-04-28 01:58:23,342 - root - ERROR - POST http://172.23.106.73:8091//node/controller/setupServices body: hostname=172.23.106.73&user=Administrator&password=password&services=index headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==', 'Accept': '*/*'} error: 400 reason: unknown b'["cannot change node services after cluster is provisioned"]' auth: Administrator:password 2021-04-28 01:58:23,342 - root - INFO - This node is already provisioned with services, we do not consider this as failure for test case 2021-04-28 01:58:23,342 - root - INFO - --> in init_cluster...Administrator,password,8091 2021-04-28 01:58:23,342 - root - INFO - settings/web params on 172.23.106.73:8091:port=8091&username=Administrator&password=password 2021-04-28 01:58:23,379 - root - INFO - --> status:True 2021-04-28 01:58:23,380 - root - INFO - SSH Connecting to 172.23.106.73 with username:root, attempt#1 of 5 2021-04-28 01:58:23,483 - root - INFO - SSH Connected to 172.23.106.73 as root 2021-04-28 01:58:23,762 - root - INFO - os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-04-28 01:58:24,075 - root - INFO - extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-04-28 01:58:24,082 - root - INFO - running command.raw on 172.23.106.73: curl --silent --show-error http://Administrator:password@localhost:8091/diag/eval -X POST -d 'ns_config:set(allow_nonlocal_eval, true).' 2021-04-28 01:58:24,142 - root - INFO - command executed successfully with root 2021-04-28 01:58:24,142 - root - INFO - ['ok'] 2021-04-28 01:58:24,146 - root - INFO - /diag/eval status on 172.23.106.73:8091: True content: [6,6] command: cluster_compat_mode:get_compat_version(). 2021-04-28 01:58:24,148 - root - INFO - /diag/eval status on 172.23.106.73:8091: True content: [6,6] command: cluster_compat_mode:get_compat_version(). 2021-04-28 01:58:24,152 - root - INFO - settings/indexes params : storageMode=plasma 2021-04-28 01:58:24,167 - root - INFO - server: ip:172.23.106.74 port:8091 ssh_username:root, nodes/self 2021-04-28 01:58:24,170 - root - INFO - {'uptime': '203', 'memoryTotal': 3945984000, 'memoryFree': 3461402624, 'mcdMemoryReserved': 3010, 'mcdMemoryAllocated': 3010, 'status': 'healthy', 'hostname': '172.23.106.74:8091', 'clusterCompatibility': 393222, 'clusterMembership': 'active', 'recoveryType': 'none', 'version': '6.6.1-9213-enterprise', 'os': 'x86_64-unknown-linux-gnu', 'ports': [], 'availableStorage': [], 'storage': [], 'memoryQuota': 2015, 'moxi': 11211, 'memcached': 11210, 'id': 'ns_1@cb.local', 'ip': '172.23.106.74', 'rest_username': '', 'rest_password': '', 'port': '8091', 'services': ['kv'], 'storageTotalRam': 3763, 'curr_items': 0} 2021-04-28 01:58:24,174 - root - INFO - pools/default params : memoryQuota=2016 2021-04-28 01:58:24,177 - root - INFO - --> init_node_services(Administrator,password,172.23.106.74,8091,['n1ql']) 2021-04-28 01:58:24,177 - root - INFO - /node/controller/setupServices params on 172.23.106.74: 8091:hostname=172.23.106.74&user=Administrator&password=password&services=n1ql 2021-04-28 01:58:24,179 - root - ERROR - POST http://172.23.106.74:8091//node/controller/setupServices body: hostname=172.23.106.74&user=Administrator&password=password&services=n1ql headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==', 'Accept': '*/*'} error: 400 reason: unknown b'["cannot change node services after cluster is provisioned"]' auth: Administrator:password 2021-04-28 01:58:24,179 - root - INFO - This node is already provisioned with services, we do not consider this as failure for test case 2021-04-28 01:58:24,179 - root - INFO - --> in init_cluster...Administrator,password,8091 2021-04-28 01:58:24,179 - root - INFO - settings/web params on 172.23.106.74:8091:port=8091&username=Administrator&password=password 2021-04-28 01:58:24,215 - root - INFO - --> status:True 2021-04-28 01:58:24,217 - root - INFO - SSH Connecting to 172.23.106.74 with username:root, attempt#1 of 5 2021-04-28 01:58:24,320 - root - INFO - SSH Connected to 172.23.106.74 as root 2021-04-28 01:58:24,580 - root - INFO - os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2021-04-28 01:58:24,889 - root - INFO - extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2021-04-28 01:58:24,896 - root - INFO - running command.raw on 172.23.106.74: curl --silent --show-error http://Administrator:password@localhost:8091/diag/eval -X POST -d 'ns_config:set(allow_nonlocal_eval, true).' 2021-04-28 01:58:24,954 - root - INFO - command executed successfully with root 2021-04-28 01:58:24,955 - root - INFO - ['ok'] 2021-04-28 01:58:24,957 - root - INFO - /diag/eval status on 172.23.106.74:8091: True content: [6,6] command: cluster_compat_mode:get_compat_version(). 2021-04-28 01:58:24,958 - root - INFO - /diag/eval status on 172.23.106.74:8091: True content: [6,6] command: cluster_compat_mode:get_compat_version(). 2021-04-28 01:58:24,963 - root - INFO - settings/indexes params : storageMode=plasma 2021-04-28 01:58:24,983 - root - INFO - **** add built-in 'cbadminbucket' user to node 172.23.106.64 **** 2021-04-28 01:58:24,991 - root - ERROR - DELETE http://172.23.106.64:8091/settings/rbac/users/local/cbadminbucket body: headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==', 'Accept': '*/*'} error: 404 reason: unknown b'"User was not found."' auth: Administrator:password 2021-04-28 01:58:24,992 - root - INFO - Exception while deleting user. Exception is -b'"User was not found."' 2021-04-28 01:58:25,037 - root - INFO - sleep for 5 secs. ... 2021-04-28 01:58:30,043 - root - INFO - **** add 'admin' role to 'cbadminbucket' user **** 2021-04-28 01:58:30,053 - root - INFO - sleep for 7 secs. wait to make sure node is ready ... 2021-04-28 01:58:37,991 - root - INFO - adding node 172.23.106.67:8091 to cluster 2021-04-28 01:58:37,991 - root - INFO - adding remote node @172.23.106.67:8091 to this cluster @172.23.106.64:8091 2021-04-28 01:58:48,007 - root - INFO - rebalance progress took 10.02 seconds 2021-04-28 01:58:48,007 - root - INFO - sleep for 10 seconds after rebalance... 2021-04-28 02:01:00,522 - root - INFO - rebalance params : {'knownNodes': 'ns_1@172.23.106.64,ns_1@172.23.106.67', 'ejectedNodes': '', 'user': 'Administrator', 'password': 'password'} 2021-04-28 02:01:00,528 - root - INFO - rebalance operation started 2021-04-28 02:01:10,540 - root - INFO - Rebalance - status: none, progress: 100.00% 2021-04-28 02:01:10,544 - root - INFO - rebalancing was completed with progress: 100% in 10.015916585922241 sec 2021-04-28 02:01:10,544 - root - INFO - sleep for 10 secs. ... 2021-04-28 02:01:21,556 - root - INFO - adding node 172.23.106.73:8091 to cluster 2021-04-28 02:01:21,557 - root - INFO - adding remote node @172.23.106.73:8091 to this cluster @172.23.106.64:8091 2021-04-28 02:01:31,572 - root - INFO - rebalance progress took 10.02 seconds 2021-04-28 02:01:31,572 - root - INFO - sleep for 10 seconds after rebalance... 2021-04-28 02:03:43,670 - root - INFO - rebalance params : {'knownNodes': 'ns_1@172.23.106.64,ns_1@172.23.106.67,ns_1@172.23.106.73', 'ejectedNodes': '', 'user': 'Administrator', 'password': 'password'} 2021-04-28 02:03:43,675 - root - INFO - rebalance operation started 2021-04-28 02:03:53,688 - root - INFO - Rebalance - status: none, progress: 100.00% 2021-04-28 02:03:53,693 - root - INFO - rebalancing was completed with progress: 100% in 10.018305778503418 sec 2021-04-28 02:03:53,693 - root - INFO - sleep for 10 secs. ... 2021-04-28 02:04:04,705 - root - INFO - adding node 172.23.106.74:8091 to cluster 2021-04-28 02:04:04,706 - root - INFO - adding remote node @172.23.106.74:8091 to this cluster @172.23.106.64:8091 2021-04-28 02:04:14,721 - root - INFO - rebalance progress took 10.02 seconds 2021-04-28 02:04:14,721 - root - INFO - sleep for 10 seconds after rebalance... 2021-04-28 02:06:26,892 - root - INFO - rebalance params : {'knownNodes': 'ns_1@172.23.106.64,ns_1@172.23.106.67,ns_1@172.23.106.73,ns_1@172.23.106.74', 'ejectedNodes': '', 'user': 'Administrator', 'password': 'password'} 2021-04-28 02:06:26,896 - root - INFO - rebalance operation started 2021-04-28 02:06:36,910 - root - INFO - Rebalance - status: none, progress: 100.00% 2021-04-28 02:06:36,916 - root - INFO - rebalancing was completed with progress: 100% in 10.019137620925903 sec 2021-04-28 02:06:36,916 - root - INFO - sleep for 10 secs. ... 2021-04-28 02:06:46,999 - root - INFO - Node versions in cluster ['6.6.1-9213-enterprise', '6.6.1-9213-enterprise', '6.6.1-9213-enterprise', '6.6.1-9213-enterprise'] 2021-04-28 02:06:47,005 - root - INFO - list of kv nodes in cluster: [ip:172.23.106.64 port:8091 ssh_username:root] 2021-04-28 02:06:47,005 - root - INFO - pools/default params : memoryQuota=1000 2021-04-28 02:06:47,011 - root - ERROR - DELETE http://172.23.106.64:8091/pools/default/buckets/default body: headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==', 'Accept': '*/*'} error: 404 reason: status: 404, content: b'Requested resource not found.\r\n' b'Requested resource not found.\r\n' auth: Administrator:password 2021-04-28 02:06:47,014 - root - INFO - Create the required buckets in the initial version 2021-04-28 02:06:48,006 - root - INFO - http://172.23.106.64:8091/pools/default/buckets with param: name=src_bucket&ramQuotaMB=100&replicaNumber=1&bucketType=membase&replicaIndex=1&threadsNumber=3&flushEnabled=1&evictionPolicy=valueOnly&compressionMode=passive&storageBackend=couchstore 2021-04-28 02:06:48,011 - root - INFO - 0.00 seconds to create bucket src_bucket 2021-04-28 02:06:48,011 - root - INFO - waiting for memcached bucket : src_bucket in 172.23.106.64 to accept set ops 2021-04-28 02:06:48,625 - root - INFO - creating direct client 172.23.106.64:11210 src_bucket 2021-04-28 02:06:48,734 - root - INFO - creating direct client 172.23.106.64:11210 src_bucket 2021-04-28 02:06:48,768 - root - WARNING - vbucket map not ready after try 0 2021-04-28 02:06:48,768 - root - INFO - waiting for memcached bucket : src_bucket in 172.23.106.64 to accept set ops 2021-04-28 02:06:48,868 - root - INFO - creating direct client 172.23.106.64:11210 src_bucket 2021-04-28 02:06:48,927 - root - INFO - creating direct client 172.23.106.64:11210 src_bucket 2021-04-28 02:06:49,009 - root - WARNING - vbucket map not ready after try 1 2021-04-28 02:06:49,009 - root - INFO - waiting for memcached bucket : src_bucket in 172.23.106.64 to accept set ops 2021-04-28 02:06:49,067 - root - INFO - creating direct client 172.23.106.64:11210 src_bucket 2021-04-28 02:06:49,177 - root - INFO - creating direct client 172.23.106.64:11210 src_bucket 2021-04-28 02:06:49,214 - root - WARNING - vbucket map not ready after try 2 2021-04-28 02:06:49,214 - root - INFO - waiting for memcached bucket : src_bucket in 172.23.106.64 to accept set ops 2021-04-28 02:06:49,321 - root - INFO - creating direct client 172.23.106.64:11210 src_bucket 2021-04-28 02:06:49,388 - root - INFO - creating direct client 172.23.106.64:11210 src_bucket 2021-04-28 02:06:49,425 - root - WARNING - vbucket map not ready after try 3 2021-04-28 02:06:49,426 - root - INFO - waiting for memcached bucket : src_bucket in 172.23.106.64 to accept set ops 2021-04-28 02:06:49,531 - root - INFO - creating direct client 172.23.106.64:11210 src_bucket 2021-04-28 02:06:49,646 - root - INFO - creating direct client 172.23.106.64:11210 src_bucket 2021-04-28 02:06:49,685 - root - WARNING - vbucket map not ready after try 4 2021-04-28 02:06:49,685 - root - INFO - waiting for memcached bucket : src_bucket in 172.23.106.64 to accept set ops 2021-04-28 02:06:49,757 - root - INFO - creating direct client 172.23.106.64:11210 src_bucket 2021-04-28 02:06:49,864 - root - INFO - creating direct client 172.23.106.64:11210 src_bucket 2021-04-28 02:06:49,902 - root - WARNING - vbucket map not ready after try 5 2021-04-28 02:06:49,915 - root - INFO - sleep for 60 secs. ... 2021-04-28 02:07:49,986 - root - INFO - http://172.23.106.64:8091/pools/default/buckets with param: name=dst_bucket&ramQuotaMB=100&replicaNumber=1&bucketType=membase&replicaIndex=1&threadsNumber=3&flushEnabled=1&evictionPolicy=valueOnly&compressionMode=passive&storageBackend=couchstore 2021-04-28 02:07:49,992 - root - INFO - 0.01 seconds to create bucket dst_bucket 2021-04-28 02:07:49,992 - root - INFO - waiting for memcached bucket : dst_bucket in 172.23.106.64 to accept set ops 2021-04-28 02:07:50,673 - root - INFO - creating direct client 172.23.106.64:11210 dst_bucket 2021-04-28 02:07:50,809 - root - INFO - creating direct client 172.23.106.64:11210 dst_bucket 2021-04-28 02:07:50,847 - root - WARNING - vbucket map not ready after try 0 2021-04-28 02:07:50,847 - root - INFO - waiting for memcached bucket : dst_bucket in 172.23.106.64 to accept set ops 2021-04-28 02:07:50,959 - root - INFO - creating direct client 172.23.106.64:11210 dst_bucket 2021-04-28 02:07:51,024 - root - INFO - creating direct client 172.23.106.64:11210 dst_bucket 2021-04-28 02:07:51,062 - root - WARNING - vbucket map not ready after try 1 2021-04-28 02:07:51,063 - root - INFO - waiting for memcached bucket : dst_bucket in 172.23.106.64 to accept set ops 2021-04-28 02:07:51,169 - root - INFO - creating direct client 172.23.106.64:11210 dst_bucket 2021-04-28 02:07:51,299 - root - INFO - creating direct client 172.23.106.64:11210 dst_bucket 2021-04-28 02:07:51,337 - root - WARNING - vbucket map not ready after try 2 2021-04-28 02:07:51,337 - root - INFO - waiting for memcached bucket : dst_bucket in 172.23.106.64 to accept set ops 2021-04-28 02:07:51,464 - root - INFO - creating direct client 172.23.106.64:11210 dst_bucket 2021-04-28 02:07:51,533 - root - INFO - creating direct client 172.23.106.64:11210 dst_bucket 2021-04-28 02:07:51,572 - root - WARNING - vbucket map not ready after try 3 2021-04-28 02:07:51,572 - root - INFO - waiting for memcached bucket : dst_bucket in 172.23.106.64 to accept set ops 2021-04-28 02:07:51,700 - root - INFO - creating direct client 172.23.106.64:11210 dst_bucket 2021-04-28 02:07:51,832 - root - INFO - creating direct client 172.23.106.64:11210 dst_bucket 2021-04-28 02:07:51,868 - root - WARNING - vbucket map not ready after try 4 2021-04-28 02:07:51,868 - root - INFO - waiting for memcached bucket : dst_bucket in 172.23.106.64 to accept set ops 2021-04-28 02:07:51,931 - root - INFO - creating direct client 172.23.106.64:11210 dst_bucket 2021-04-28 02:07:52,042 - root - INFO - creating direct client 172.23.106.64:11210 dst_bucket 2021-04-28 02:07:52,080 - root - WARNING - vbucket map not ready after try 5 2021-04-28 02:07:52,081 - root - INFO - sleep for 60 secs. ... 2021-04-28 02:08:52,171 - root - INFO - http://172.23.106.64:8091/pools/default/buckets with param: name=metadata&ramQuotaMB=100&replicaNumber=1&bucketType=membase&replicaIndex=1&threadsNumber=3&flushEnabled=1&evictionPolicy=valueOnly&compressionMode=passive&storageBackend=couchstore 2021-04-28 02:08:52,177 - root - INFO - 0.01 seconds to create bucket metadata 2021-04-28 02:08:52,178 - root - INFO - waiting for memcached bucket : metadata in 172.23.106.64 to accept set ops 2021-04-28 02:08:52,375 - root - INFO - creating direct client 172.23.106.64:11210 metadata 2021-04-28 02:08:52,447 - root - INFO - creating direct client 172.23.106.64:11210 metadata 2021-04-28 02:08:52,555 - root - WARNING - vbucket map not ready after try 0 2021-04-28 02:08:52,555 - root - INFO - waiting for memcached bucket : metadata in 172.23.106.64 to accept set ops 2021-04-28 02:08:52,619 - root - INFO - creating direct client 172.23.106.64:11210 metadata 2021-04-28 02:08:52,738 - root - INFO - creating direct client 172.23.106.64:11210 metadata 2021-04-28 02:08:52,777 - root - WARNING - vbucket map not ready after try 1 2021-04-28 02:08:52,778 - root - INFO - waiting for memcached bucket : metadata in 172.23.106.64 to accept set ops 2021-04-28 02:08:52,888 - root - INFO - creating direct client 172.23.106.64:11210 metadata 2021-04-28 02:08:52,969 - root - INFO - creating direct client 172.23.106.64:11210 metadata 2021-04-28 02:08:53,055 - root - WARNING - vbucket map not ready after try 2 2021-04-28 02:08:53,055 - root - INFO - waiting for memcached bucket : metadata in 172.23.106.64 to accept set ops 2021-04-28 02:08:53,120 - root - INFO - creating direct client 172.23.106.64:11210 metadata 2021-04-28 02:08:53,235 - root - INFO - creating direct client 172.23.106.64:11210 metadata 2021-04-28 02:08:53,273 - root - WARNING - vbucket map not ready after try 3 2021-04-28 02:08:53,273 - root - INFO - waiting for memcached bucket : metadata in 172.23.106.64 to accept set ops 2021-04-28 02:08:53,383 - root - INFO - creating direct client 172.23.106.64:11210 metadata 2021-04-28 02:08:53,462 - root - INFO - creating direct client 172.23.106.64:11210 metadata 2021-04-28 02:08:53,551 - root - WARNING - vbucket map not ready after try 4 2021-04-28 02:08:53,551 - root - INFO - waiting for memcached bucket : metadata in 172.23.106.64 to accept set ops 2021-04-28 02:08:53,614 - root - INFO - creating direct client 172.23.106.64:11210 metadata 2021-04-28 02:08:53,728 - root - INFO - creating direct client 172.23.106.64:11210 metadata 2021-04-28 02:08:53,769 - root - WARNING - vbucket map not ready after try 5 2021-04-28 02:08:53,769 - root - INFO - sleep for 60 secs. ... 2021-04-28 02:09:53,858 - root - INFO - http://172.23.106.64:8091/pools/default/buckets with param: name=dst_bucket1&ramQuotaMB=100&replicaNumber=1&bucketType=membase&replicaIndex=1&threadsNumber=3&flushEnabled=1&evictionPolicy=valueOnly&compressionMode=passive&storageBackend=couchstore 2021-04-28 02:09:53,864 - root - INFO - 0.01 seconds to create bucket dst_bucket1 2021-04-28 02:09:53,864 - root - INFO - waiting for memcached bucket : dst_bucket1 in 172.23.106.64 to accept set ops 2021-04-28 02:09:54,521 - root - INFO - creating direct client 172.23.106.64:11210 dst_bucket1 2021-04-28 02:09:54,611 - root - INFO - creating direct client 172.23.106.64:11210 dst_bucket1 2021-04-28 02:09:54,701 - root - WARNING - vbucket map not ready after try 0 2021-04-28 02:09:54,701 - root - INFO - waiting for memcached bucket : dst_bucket1 in 172.23.106.64 to accept set ops 2021-04-28 02:09:54,774 - root - INFO - creating direct client 172.23.106.64:11210 dst_bucket1 2021-04-28 02:09:54,906 - root - INFO - creating direct client 172.23.106.64:11210 dst_bucket1 2021-04-28 02:09:54,951 - root - WARNING - vbucket map not ready after try 1 2021-04-28 02:09:54,951 - root - INFO - waiting for memcached bucket : dst_bucket1 in 172.23.106.64 to accept set ops 2021-04-28 02:09:55,069 - root - INFO - creating direct client 172.23.106.64:11210 dst_bucket1 2021-04-28 02:09:55,144 - root - INFO - creating direct client 172.23.106.64:11210 dst_bucket1 2021-04-28 02:09:55,231 - root - WARNING - vbucket map not ready after try 2 2021-04-28 02:09:55,231 - root - INFO - waiting for memcached bucket : dst_bucket1 in 172.23.106.64 to accept set ops 2021-04-28 02:09:55,301 - root - INFO - creating direct client 172.23.106.64:11210 dst_bucket1 2021-04-28 02:09:55,425 - root - INFO - creating direct client 172.23.106.64:11210 dst_bucket1 2021-04-28 02:09:55,469 - root - WARNING - vbucket map not ready after try 3 2021-04-28 02:09:55,469 - root - INFO - waiting for memcached bucket : dst_bucket1 in 172.23.106.64 to accept set ops 2021-04-28 02:09:55,586 - root - INFO - creating direct client 172.23.106.64:11210 dst_bucket1 2021-04-28 02:09:55,674 - root - INFO - creating direct client 172.23.106.64:11210 dst_bucket1 2021-04-28 02:09:55,762 - root - WARNING - vbucket map not ready after try 4 2021-04-28 02:09:55,763 - root - INFO - waiting for memcached bucket : dst_bucket1 in 172.23.106.64 to accept set ops 2021-04-28 02:09:55,838 - root - INFO - creating direct client 172.23.106.64:11210 dst_bucket1 2021-04-28 02:09:55,960 - root - INFO - creating direct client 172.23.106.64:11210 dst_bucket1 2021-04-28 02:09:56,017 - root - WARNING - vbucket map not ready after try 5 2021-04-28 02:09:56,017 - root - INFO - sleep for 60 secs. ... 2021-04-28 02:10:56,108 - root - INFO - http://172.23.106.64:8091/pools/default/buckets with param: name=dst_bucket_curl&ramQuotaMB=100&replicaNumber=1&bucketType=membase&replicaIndex=1&threadsNumber=3&flushEnabled=1&evictionPolicy=valueOnly&compressionMode=passive&storageBackend=couchstore 2021-04-28 02:10:56,114 - root - INFO - 0.01 seconds to create bucket dst_bucket_curl 2021-04-28 02:10:56,114 - root - INFO - waiting for memcached bucket : dst_bucket_curl in 172.23.106.64 to accept set ops 2021-04-28 02:10:56,796 - root - INFO - creating direct client 172.23.106.64:11210 dst_bucket_curl 2021-04-28 02:10:56,876 - root - INFO - creating direct client 172.23.106.64:11210 dst_bucket_curl 2021-04-28 02:10:57,002 - root - WARNING - vbucket map not ready after try 0 2021-04-28 02:10:57,002 - root - INFO - waiting for memcached bucket : dst_bucket_curl in 172.23.106.64 to accept set ops 2021-04-28 02:10:57,079 - root - INFO - creating direct client 172.23.106.64:11210 dst_bucket_curl 2021-04-28 02:10:57,222 - root - INFO - creating direct client 172.23.106.64:11210 dst_bucket_curl 2021-04-28 02:10:57,272 - root - WARNING - vbucket map not ready after try 1 2021-04-28 02:10:57,272 - root - INFO - waiting for memcached bucket : dst_bucket_curl in 172.23.106.64 to accept set ops 2021-04-28 02:10:57,395 - root - INFO - creating direct client 172.23.106.64:11210 dst_bucket_curl 2021-04-28 02:10:57,470 - root - INFO - creating direct client 172.23.106.64:11210 dst_bucket_curl 2021-04-28 02:10:57,512 - root - WARNING - vbucket map not ready after try 2 2021-04-28 02:10:57,512 - root - INFO - waiting for memcached bucket : dst_bucket_curl in 172.23.106.64 to accept set ops 2021-04-28 02:10:57,644 - root - INFO - creating direct client 172.23.106.64:11210 dst_bucket_curl 2021-04-28 02:10:57,763 - root - INFO - creating direct client 172.23.106.64:11210 dst_bucket_curl 2021-04-28 02:10:57,806 - root - WARNING - vbucket map not ready after try 3 2021-04-28 02:10:57,806 - root - INFO - waiting for memcached bucket : dst_bucket_curl in 172.23.106.64 to accept set ops 2021-04-28 02:10:57,923 - root - INFO - creating direct client 172.23.106.64:11210 dst_bucket_curl 2021-04-28 02:10:58,008 - root - INFO - creating direct client 172.23.106.64:11210 dst_bucket_curl 2021-04-28 02:10:58,051 - root - WARNING - vbucket map not ready after try 4 2021-04-28 02:10:58,051 - root - INFO - waiting for memcached bucket : dst_bucket_curl in 172.23.106.64 to accept set ops 2021-04-28 02:10:58,167 - root - INFO - creating direct client 172.23.106.64:11210 dst_bucket_curl 2021-04-28 02:10:58,304 - root - INFO - creating direct client 172.23.106.64:11210 dst_bucket_curl 2021-04-28 02:10:58,347 - root - WARNING - vbucket map not ready after try 5 2021-04-28 02:10:58,347 - root - INFO - sleep for 60 secs. ... 2021-04-28 02:11:58,442 - root - INFO - http://172.23.106.64:8091/pools/default/buckets with param: name=source_bucket_mutation&ramQuotaMB=100&replicaNumber=1&bucketType=membase&replicaIndex=1&threadsNumber=3&flushEnabled=1&evictionPolicy=valueOnly&compressionMode=passive&storageBackend=couchstore 2021-04-28 02:11:58,449 - root - INFO - 0.01 seconds to create bucket source_bucket_mutation 2021-04-28 02:11:58,449 - root - INFO - waiting for memcached bucket : source_bucket_mutation in 172.23.106.64 to accept set ops 2021-04-28 02:11:59,141 - root - INFO - creating direct client 172.23.106.64:11210 source_bucket_mutation 2021-04-28 02:11:59,224 - root - INFO - creating direct client 172.23.106.64:11210 source_bucket_mutation 2021-04-28 02:11:59,268 - root - WARNING - vbucket map not ready after try 0 2021-04-28 02:11:59,269 - root - INFO - waiting for memcached bucket : source_bucket_mutation in 172.23.106.64 to accept set ops 2021-04-28 02:11:59,411 - root - INFO - creating direct client 172.23.106.64:11210 source_bucket_mutation 2021-04-28 02:11:59,558 - root - INFO - creating direct client 172.23.106.64:11210 source_bucket_mutation 2021-04-28 02:11:59,611 - root - WARNING - vbucket map not ready after try 1 2021-04-28 02:11:59,611 - root - INFO - waiting for memcached bucket : source_bucket_mutation in 172.23.106.64 to accept set ops 2021-04-28 02:11:59,687 - root - INFO - creating direct client 172.23.106.64:11210 source_bucket_mutation 2021-04-28 02:11:59,825 - root - INFO - creating direct client 172.23.106.64:11210 source_bucket_mutation 2021-04-28 02:11:59,883 - root - WARNING - vbucket map not ready after try 2 2021-04-28 02:11:59,883 - root - INFO - waiting for memcached bucket : source_bucket_mutation in 172.23.106.64 to accept set ops 2021-04-28 02:12:00,007 - root - INFO - creating direct client 172.23.106.64:11210 source_bucket_mutation 2021-04-28 02:12:00,088 - root - INFO - creating direct client 172.23.106.64:11210 source_bucket_mutation 2021-04-28 02:12:00,198 - root - WARNING - vbucket map not ready after try 3 2021-04-28 02:12:00,198 - root - INFO - waiting for memcached bucket : source_bucket_mutation in 172.23.106.64 to accept set ops 2021-04-28 02:12:00,277 - root - INFO - creating direct client 172.23.106.64:11210 source_bucket_mutation 2021-04-28 02:12:00,411 - root - INFO - creating direct client 172.23.106.64:11210 source_bucket_mutation 2021-04-28 02:12:00,460 - root - WARNING - vbucket map not ready after try 4 2021-04-28 02:12:00,460 - root - INFO - waiting for memcached bucket : source_bucket_mutation in 172.23.106.64 to accept set ops 2021-04-28 02:12:00,613 - root - INFO - creating direct client 172.23.106.64:11210 source_bucket_mutation 2021-04-28 02:12:00,693 - root - INFO - creating direct client 172.23.106.64:11210 source_bucket_mutation 2021-04-28 02:12:00,784 - root - WARNING - vbucket map not ready after try 5 2021-04-28 02:12:00,784 - root - INFO - sleep for 60 secs. ... 2021-04-28 02:13:00,878 - root - INFO - http://172.23.106.64:8091/pools/default/buckets with param: name=n1ql_op_dst&ramQuotaMB=100&replicaNumber=1&bucketType=membase&replicaIndex=1&threadsNumber=3&flushEnabled=1&evictionPolicy=valueOnly&compressionMode=passive&storageBackend=couchstore 2021-04-28 02:13:00,884 - root - INFO - 0.01 seconds to create bucket n1ql_op_dst 2021-04-28 02:13:00,885 - root - INFO - waiting for memcached bucket : n1ql_op_dst in 172.23.106.64 to accept set ops 2021-04-28 02:13:01,539 - root - INFO - creating direct client 172.23.106.64:11210 n1ql_op_dst 2021-04-28 02:13:01,662 - root - INFO - creating direct client 172.23.106.64:11210 n1ql_op_dst 2021-04-28 02:13:01,707 - root - WARNING - vbucket map not ready after try 0 2021-04-28 02:13:01,707 - root - INFO - waiting for memcached bucket : n1ql_op_dst in 172.23.106.64 to accept set ops 2021-04-28 02:13:01,854 - root - INFO - creating direct client 172.23.106.64:11210 n1ql_op_dst 2021-04-28 02:13:01,937 - root - INFO - creating direct client 172.23.106.64:11210 n1ql_op_dst 2021-04-28 02:13:02,054 - root - WARNING - vbucket map not ready after try 1 2021-04-28 02:13:02,054 - root - INFO - waiting for memcached bucket : n1ql_op_dst in 172.23.106.64 to accept set ops 2021-04-28 02:13:02,152 - root - INFO - creating direct client 172.23.106.64:11210 n1ql_op_dst 2021-04-28 02:13:02,295 - root - INFO - creating direct client 172.23.106.64:11210 n1ql_op_dst 2021-04-28 02:13:02,338 - root - WARNING - vbucket map not ready after try 2 2021-04-28 02:13:02,338 - root - INFO - waiting for memcached bucket : n1ql_op_dst in 172.23.106.64 to accept set ops 2021-04-28 02:13:02,481 - root - INFO - creating direct client 172.23.106.64:11210 n1ql_op_dst 2021-04-28 02:13:02,563 - root - INFO - creating direct client 172.23.106.64:11210 n1ql_op_dst 2021-04-28 02:13:02,669 - root - WARNING - vbucket map not ready after try 3 2021-04-28 02:13:02,669 - root - INFO - waiting for memcached bucket : n1ql_op_dst in 172.23.106.64 to accept set ops 2021-04-28 02:13:02,746 - root - INFO - creating direct client 172.23.106.64:11210 n1ql_op_dst 2021-04-28 02:13:02,889 - root - INFO - creating direct client 172.23.106.64:11210 n1ql_op_dst 2021-04-28 02:13:02,933 - root - WARNING - vbucket map not ready after try 4 2021-04-28 02:13:02,933 - root - INFO - waiting for memcached bucket : n1ql_op_dst in 172.23.106.64 to accept set ops 2021-04-28 02:13:03,069 - root - INFO - creating direct client 172.23.106.64:11210 n1ql_op_dst 2021-04-28 02:13:03,150 - root - INFO - creating direct client 172.23.106.64:11210 n1ql_op_dst 2021-04-28 02:13:03,259 - root - WARNING - vbucket map not ready after try 5 2021-04-28 02:13:03,834 - root - INFO - create 20160.0 to src_bucket documents... 2021-04-28 02:13:03,890 - root - INFO - creating direct client 172.23.106.64:11210 src_bucket 2021-04-28 02:13:18,038 - root - INFO - LOAD IS FINISHED 2021-04-28 02:13:18,065 - root - INFO - list of eventing nodes in cluster: [ip:172.23.106.67 port:8091 ssh_username:root] 2021-04-28 02:13:18,071 - root - INFO - Deploy the function in the initial version 2021-04-28 02:13:19,164 - root - INFO - saving function bucket_op 2021-04-28 02:13:20,264 - root - INFO - saving function timers 2021-04-28 02:13:21,415 - root - INFO - list of eventing nodes in cluster: [ip:172.23.106.67 port:8091 ssh_username:root] 2021-04-28 02:13:21,420 - root - INFO - sleep for 20 secs. Waiting for timers to deployed... ... 2021-04-28 02:13:41,463 - root - INFO - sleep for 20 secs. Waiting for timers to deployed... ... 2021-04-28 02:14:01,524 - root - INFO - list of eventing nodes in cluster: [ip:172.23.106.67 port:8091 ssh_username:root] 2021-04-28 02:14:01,550 - root - INFO - Stats for Node 172.23.106.67 is [ { "dcp_feed_boundary": "everything", "event_processing_stats": { "adhoc_timer_response_received": 2, "agg_messages_sent_to_worker": 21625, "agg_queue_memory": 0, "agg_queue_memory_cap": 53477376, "agg_queue_size": 0, "agg_queue_size_cap": 300000, "agg_timer_feedback_queue_cap": 1500, "dcp_mutation": 20160, "dcp_mutation_sent_to_worker": 20160, "dcp_snapshot": 1024, "dcp_stream_req_counter": 1024, "dcp_streamreq": 1024, "execution_stats": 84, "failure_stats": 84, "latency_stats": 84, "lcb_exception_stats": 84, "log_level": 3, "num_processed_events": 20160, "processed_events_size": 13934248, "thr_count": 3, "thr_map": 3, "timer_responses_received": 20160, "v8_init": 3, "v8_load": 3 }, "events_remaining": { "dcp_backlog": 0 }, "execution_stats": { "agg_queue_memory": 0, "agg_queue_size": 0, "curl": { "delete": 0, "get": 0, "head": 0, "post": 0, "put": 0 }, "dcp_delete_msg_counter": 0, "dcp_delete_parse_failure": 0, "dcp_mutation_msg_counter": 20160, "dcp_mutation_parse_failure": 0, "enqueued_dcp_delete_msg_counter": 0, "enqueued_dcp_mutation_msg_counter": 20160, "enqueued_timer_msg_counter": 0, "feedback_queue_size": 0, "filtered_dcp_delete_counter": 0, "filtered_dcp_mutation_counter": 0, "lcb_retry_failure": 0, "messages_parsed": 21613, "num_processed_events": 20160, "on_delete_failure": 0, "on_delete_success": 0, "on_update_failure": 0, "on_update_success": 20160, "processed_events_size": 13934248, "timer_cancel_counter": 0, "timer_create_counter": 20160, "timer_create_failure": 0, "timer_msg_counter": 0, "timer_responses_sent": 0, "timestamp": { "118126": "2021-04-28T09:14:00Z", "118131": "2021-04-28T09:14:00Z", "118142": "2021-04-28T09:14:00Z" }, "uv_msg_parse_failure": 0, "uv_try_write_failure_counter": 0 }, "failure_stats": { "app_worker_setting_events_lost": 0, "bkt_ops_cas_mismatch_count": 0, "bucket_op_exception_count": 0, "checkpoint_failure_count": 0, "curl_non_200_response": 0, "dcp_events_lost": 0, "debugger_events_lost": 0, "delete_events_lost": 0, "mutation_events_lost": 0, "n1ql_op_exception_count": 0, "timeout_count": 0, "timer_callback_missing_counter": 0, "timer_context_size_exceeded_counter": 0, "timer_events_lost": 0, "timestamp": { "118126": "2021-04-28T09:14:00Z", "118131": "2021-04-28T09:14:00Z", "118142": "2021-04-28T09:14:00Z" }, "v8worker_events_lost": 0 }, "function_id": 2335275409, "function_name": "timers", "gocb_creds_request_counter": 8, "internal_vb_distribution_stats": { "worker_timers_0": "[0-341]", "worker_timers_1": "[342-682]", "worker_timers_2": "[683-1023]" }, "latency_percentile_stats": { "100": 22300, "50": 900, "80": 1200, "90": 1400, "95": 1700, "99": 3500 }, "lcb_creds_request_counter": 24, "lcb_exception_stats": {}, "metastore_stats": { "metastore_delete_err": 0, "metastore_deletes": 0, "metastore_not_found": 0, "metastore_scan": 0, "metastore_scan_due": 0, "metastore_scan_err": 0, "metastore_set": 0, "metastore_set_err": 0 }, "planner_stats": [ { "host_name": "172.23.106.67:8096", "start_vb": 0, "vb_count": 1024 } ], "vb_distribution_stats_from_metadata": {}, "worker_pids": { "worker_timers_0": 118126, "worker_timers_1": 118131, "worker_timers_2": 118142 } } ] 2021-04-28 02:14:02,338 - root - INFO - Node versions in cluster ['6.6.1-9213-enterprise', '6.6.1-9213-enterprise', '6.6.1-9213-enterprise', '6.6.1-9213-enterprise'] 2021-04-28 02:14:02,338 - root - INFO - adding node 172.23.106.75:8091 to cluster 2021-04-28 02:14:02,338 - root - INFO - adding remote node @172.23.106.75:8091 to this cluster @172.23.106.64:8091 2021-04-28 02:14:12,354 - root - INFO - rebalance progress took 10.02 seconds 2021-04-28 02:14:12,354 - root - INFO - sleep for 10 seconds after rebalance... 2021-04-28 02:14:25,517 - root - INFO - adding node 172.23.106.76:8091 to cluster 2021-04-28 02:14:25,517 - root - INFO - adding remote node @172.23.106.76:8091 to this cluster @172.23.106.64:8091 2021-04-28 02:14:35,534 - root - INFO - rebalance progress took 10.02 seconds 2021-04-28 02:14:35,534 - root - INFO - sleep for 10 seconds after rebalance... 2021-04-28 02:14:48,768 - root - INFO - adding node 172.23.106.77:8091 to cluster 2021-04-28 02:14:48,768 - root - INFO - adding remote node @172.23.106.77:8091 to this cluster @172.23.106.64:8091 2021-04-28 02:14:58,790 - root - INFO - rebalance progress took 10.02 seconds 2021-04-28 02:14:58,790 - root - INFO - sleep for 10 seconds after rebalance... 2021-04-28 02:15:11,866 - root - INFO - adding node 172.23.106.78:8091 to cluster 2021-04-28 02:15:11,866 - root - INFO - adding remote node @172.23.106.78:8091 to this cluster @172.23.106.64:8091 2021-04-28 02:15:21,882 - root - INFO - rebalance progress took 10.02 seconds 2021-04-28 02:15:21,882 - root - INFO - sleep for 10 seconds after rebalance... 2021-04-28 02:17:35,221 - root - INFO - rebalance params : {'knownNodes': 'ns_1@172.23.106.64,ns_1@172.23.106.67,ns_1@172.23.106.73,ns_1@172.23.106.74,ns_1@172.23.106.75,ns_1@172.23.106.76,ns_1@172.23.106.77,ns_1@172.23.106.78', 'ejectedNodes': 'ns_1@172.23.106.64,ns_1@172.23.106.67,ns_1@172.23.106.73,ns_1@172.23.106.74', 'user': 'Administrator', 'password': 'password'} 2021-04-28 02:17:35,239 - root - INFO - rebalance operation started 2021-04-28 02:17:45,291 - root - INFO - rebalance percentage : 0.00 % 2021-04-28 02:17:45,291 - root - INFO - Rebalance - status: running, progress: 0.00% 2021-04-28 02:18:05,335 - root - INFO - rebalance percentage : 1.00 % 2021-04-28 02:18:05,335 - root - INFO - Rebalance - status: running, progress: 1.00% 2021-04-28 02:18:25,396 - root - INFO - rebalance percentage : 2.00 % 2021-04-28 02:18:25,396 - root - INFO - Rebalance - status: running, progress: 2.00% 2021-04-28 02:18:45,484 - root - INFO - rebalance percentage : 3.00 % 2021-04-28 02:18:45,484 - root - INFO - Rebalance - status: running, progress: 3.00% 2021-04-28 02:19:05,569 - root - INFO - rebalance percentage : 3.00 % 2021-04-28 02:19:05,569 - root - INFO - Rebalance - status: running, progress: 3.00% 2021-04-28 02:19:25,633 - root - INFO - rebalance percentage : 4.00 % 2021-04-28 02:19:25,633 - root - INFO - Rebalance - status: running, progress: 4.00% 2021-04-28 02:19:45,712 - root - INFO - rebalance percentage : 5.00 % 2021-04-28 02:19:45,712 - root - INFO - Rebalance - status: running, progress: 5.00% 2021-04-28 02:20:05,750 - root - INFO - rebalance percentage : 6.00 % 2021-04-28 02:20:05,750 - root - INFO - Rebalance - status: running, progress: 6.00% 2021-04-28 02:20:28,777 - root - INFO - rebalance percentage : 7.00 % 2021-04-28 02:20:28,778 - root - INFO - Rebalance - status: running, progress: 7.00% 2021-04-28 02:20:48,834 - root - INFO - rebalance percentage : 8.00 % 2021-04-28 02:20:48,834 - root - INFO - Rebalance - status: running, progress: 8.00% 2021-04-28 02:21:08,880 - root - INFO - rebalance percentage : 9.00 % 2021-04-28 02:21:08,880 - root - INFO - Rebalance - status: running, progress: 9.00% 2021-04-28 02:21:28,948 - root - INFO - rebalance percentage : 10.00 % 2021-04-28 02:21:28,948 - root - INFO - Rebalance - status: running, progress: 10.00% 2021-04-28 02:21:48,997 - root - INFO - rebalance percentage : 10.00 % 2021-04-28 02:21:48,998 - root - INFO - Rebalance - status: running, progress: 10.00% 2021-04-28 02:22:09,044 - root - INFO - rebalance percentage : 11.00 % 2021-04-28 02:22:09,045 - root - INFO - Rebalance - status: running, progress: 11.00% 2021-04-28 02:22:29,137 - root - INFO - rebalance percentage : 12.00 % 2021-04-28 02:22:29,137 - root - INFO - Rebalance - status: running, progress: 12.00% 2021-04-28 02:22:49,175 - root - INFO - rebalance percentage : 13.00 % 2021-04-28 02:22:49,175 - root - INFO - Rebalance - status: running, progress: 13.00% 2021-04-28 02:23:09,277 - root - INFO - rebalance percentage : 14.00 % 2021-04-28 02:23:09,277 - root - INFO - Rebalance - status: running, progress: 14.00% 2021-04-28 02:23:29,353 - root - INFO - rebalance percentage : 14.00 % 2021-04-28 02:23:29,353 - root - INFO - Rebalance - status: running, progress: 14.00% 2021-04-28 02:23:49,447 - root - INFO - rebalance percentage : 15.00 % 2021-04-28 02:23:49,447 - root - INFO - Rebalance - status: running, progress: 15.00% 2021-04-28 02:24:09,517 - root - INFO - rebalance percentage : 16.00 % 2021-04-28 02:24:09,517 - root - INFO - Rebalance - status: running, progress: 16.00% 2021-04-28 02:24:29,591 - root - INFO - rebalance percentage : 17.00 % 2021-04-28 02:24:29,591 - root - INFO - Rebalance - status: running, progress: 17.00% 2021-04-28 02:24:49,642 - root - INFO - rebalance percentage : 17.00 % 2021-04-28 02:24:49,642 - root - INFO - Rebalance - status: running, progress: 17.00% 2021-04-28 02:25:09,718 - root - INFO - rebalance percentage : 18.00 % 2021-04-28 02:25:09,718 - root - INFO - Rebalance - status: running, progress: 18.00% 2021-04-28 02:25:29,763 - root - INFO - rebalance percentage : 19.00 % 2021-04-28 02:25:29,763 - root - INFO - Rebalance - status: running, progress: 19.00% 2021-04-28 02:25:49,844 - root - INFO - rebalance percentage : 20.00 % 2021-04-28 02:25:49,845 - root - INFO - Rebalance - status: running, progress: 20.00% 2021-04-28 02:26:09,884 - root - INFO - rebalance percentage : 20.00 % 2021-04-28 02:26:09,884 - root - INFO - Rebalance - status: running, progress: 20.00% 2021-04-28 02:26:29,967 - root - INFO - rebalance percentage : 21.00 % 2021-04-28 02:26:29,967 - root - INFO - Rebalance - status: running, progress: 21.00% 2021-04-28 02:26:50,009 - root - INFO - rebalance percentage : 22.00 % 2021-04-28 02:26:50,009 - root - INFO - Rebalance - status: running, progress: 22.00% 2021-04-28 02:27:10,089 - root - INFO - rebalance percentage : 23.00 % 2021-04-28 02:27:10,089 - root - INFO - Rebalance - status: running, progress: 23.00% 2021-04-28 02:27:30,173 - root - INFO - rebalance percentage : 23.00 % 2021-04-28 02:27:30,173 - root - INFO - Rebalance - status: running, progress: 23.00% 2021-04-28 02:27:50,244 - root - INFO - rebalance percentage : 24.00 % 2021-04-28 02:27:50,244 - root - INFO - Rebalance - status: running, progress: 24.00% 2021-04-28 02:28:10,288 - root - INFO - rebalance percentage : 50.00 % 2021-04-28 02:28:10,288 - root - INFO - Rebalance - status: running, progress: 50.00% 2021-04-28 02:28:30,324 - root - INFO - rebalance percentage : 50.00 % 2021-04-28 02:28:30,324 - root - INFO - Rebalance - status: running, progress: 50.00% 2021-04-28 02:28:50,361 - root - INFO - Rebalance - status: none, progress: 100.00% 2021-04-28 02:28:50,363 - root - ERROR - GET http://172.23.106.64:8091/pools/default body: headers: {'Content-Type': 'application/json', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==', 'Accept': '*/*'} error: 404 reason: unknown b'"unknown pool"' auth: Administrator:password http://172.23.106.64:8091/pools/default with status False: unknown pool 2021-04-28 02:28:50,394 - root - INFO - rebalancing was completed with progress: 100% in 675.154655456543 sec 2021-04-28 02:28:50,394 - root - INFO - sleep for 600 secs. ... 2021-04-28 02:38:50,421 - root - INFO - Rebalance in all 7.0.0-5046 nodes 2021-04-28 02:38:50,421 - root - INFO - Rebalanced out all old version nodes 2021-04-28 02:38:50,430 - root - INFO - /diag/eval status on 172.23.106.75:8091: True content: 'ns_1@172.23.106.75' command: mb_master:master_node(). 2021-04-28 02:38:50,436 - root - INFO - 7.0.0-5046 node 172.23.106.75 becomes the master 2021-04-28 02:38:50,457 - root - INFO - list of eventing nodes in cluster: [ip:172.23.106.76 port:8091 ssh_username:root] 2021-04-28 02:38:50,476 - root - INFO - **** add built-in 'cbadminbucket' user to node 172.23.106.75 **** 2021-04-28 02:38:50,584 - root - INFO - sleep for 5 secs. ... 2021-04-28 02:38:55,590 - root - INFO - **** add 'admin' role to 'cbadminbucket' user **** 2021-04-28 02:38:55,638 - root - INFO - list of eventing nodes in cluster: [ip:172.23.106.76 port:8091 ssh_username:root] 2021-04-28 02:38:55,649 - root - INFO - sleep for 20 secs. Waiting for bucket_op to undeployed... ... 2021-04-28 02:39:15,688 - root - INFO - sleep for 20 secs. Waiting for bucket_op to undeployed... ... 2021-04-28 02:39:35,750 - root - INFO - list of eventing nodes in cluster: [ip:172.23.106.76 port:8091 ssh_username:root] 2021-04-28 02:39:35,757 - root - INFO - sleep for 20 secs. Waiting for timers to deployed... ... 2021-04-28 02:39:55,782 - root - INFO - sleep for 20 secs. Waiting for timers to deployed... ... 2021-04-28 02:40:17,163 - root - INFO - list of eventing nodes in cluster: [ip:172.23.106.76 port:8091 ssh_username:root] 2021-04-28 02:40:17,174 - root - INFO - sleep for 20 secs. Waiting for bucket_op to deployed... ... 2021-04-28 02:40:37,197 - root - INFO - sleep for 20 secs. Waiting for bucket_op to deployed... ... 2021-04-28 02:40:57,266 - root - INFO - list of eventing nodes in cluster: [ip:172.23.106.76 port:8091 ssh_username:root] 2021-04-28 02:40:57,280 - root - INFO - list of n1ql nodes in cluster: [ip:172.23.106.78 port:8091 ssh_username:root] 2021-04-28 02:40:57,291 - root - INFO - RUN QUERY create primary index on dst_bucket._default._default 2021-04-28 02:40:57,301 - root - INFO - query params : statement=create+primary+index+on+dst_bucket._default._default 2021-04-28 02:40:59,588 - root - INFO - TOTAL ELAPSED TIME: 2.284707393s 2021-04-28 02:40:59,642 - root - INFO - list of n1ql nodes in cluster: [ip:172.23.106.78 port:8091 ssh_username:root] 2021-04-28 02:40:59,649 - root - INFO - query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket._default._default 2021-04-28 02:40:59,663 - root - INFO - Final docs count... Current : 20160 Expected : 20160 2021-04-28 02:40:59,697 - root - INFO - list of eventing nodes in cluster: [ip:172.23.106.76 port:8091 ssh_username:root] 2021-04-28 02:40:59,717 - root - INFO - list of n1ql nodes in cluster: [ip:172.23.106.78 port:8091 ssh_username:root] 2021-04-28 02:40:59,724 - root - INFO - RUN QUERY create primary index on dst_bucket1._default._default 2021-04-28 02:40:59,730 - root - INFO - query params : statement=create+primary+index+on+dst_bucket1._default._default 2021-04-28 02:41:02,173 - root - INFO - TOTAL ELAPSED TIME: 2.440474122s 2021-04-28 02:41:02,218 - root - INFO - list of n1ql nodes in cluster: [ip:172.23.106.78 port:8091 ssh_username:root] 2021-04-28 02:41:02,232 - root - INFO - query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket1._default._default 2021-04-28 02:41:02,248 - root - INFO - sleep for 30 secs. Waiting for handler code dst_bucket1._default._default to complete bucket operations... Current : 19581 Expected : 20160 ... 2021-04-28 02:41:32,306 - root - INFO - list of n1ql nodes in cluster: [ip:172.23.106.78 port:8091 ssh_username:root] 2021-04-28 02:41:32,314 - root - INFO - query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket1._default._default 2021-04-28 02:41:32,338 - root - INFO - list of n1ql nodes in cluster: [ip:172.23.106.78 port:8091 ssh_username:root] 2021-04-28 02:41:32,345 - root - INFO - query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket1._default._default 2021-04-28 02:41:32,349 - root - INFO - sleep for 30 secs. Waiting for handler code dst_bucket1._default._default to complete bucket operations... Current : 19581 Expected : 20160 ... 2021-04-28 02:42:02,390 - root - INFO - list of n1ql nodes in cluster: [ip:172.23.106.78 port:8091 ssh_username:root] 2021-04-28 02:42:02,399 - root - INFO - query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket1._default._default 2021-04-28 02:42:02,429 - root - INFO - list of n1ql nodes in cluster: [ip:172.23.106.78 port:8091 ssh_username:root] 2021-04-28 02:42:02,437 - root - INFO - query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket1._default._default 2021-04-28 02:42:02,441 - root - INFO - sleep for 30 secs. Waiting for handler code dst_bucket1._default._default to complete bucket operations... Current : 19581 Expected : 20160 ... 2021-04-28 02:42:32,497 - root - INFO - list of n1ql nodes in cluster: [ip:172.23.106.78 port:8091 ssh_username:root] 2021-04-28 02:42:32,507 - root - INFO - query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket1._default._default 2021-04-28 02:42:32,533 - root - INFO - list of n1ql nodes in cluster: [ip:172.23.106.78 port:8091 ssh_username:root] 2021-04-28 02:42:32,540 - root - INFO - query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket1._default._default 2021-04-28 02:42:32,544 - root - INFO - sleep for 30 secs. Waiting for handler code dst_bucket1._default._default to complete bucket operations... Current : 19581 Expected : 20160 ... 2021-04-28 02:43:02,591 - root - INFO - list of n1ql nodes in cluster: [ip:172.23.106.78 port:8091 ssh_username:root] 2021-04-28 02:43:02,601 - root - INFO - query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket1._default._default 2021-04-28 02:43:02,629 - root - INFO - list of n1ql nodes in cluster: [ip:172.23.106.78 port:8091 ssh_username:root] 2021-04-28 02:43:02,637 - root - INFO - query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket1._default._default 2021-04-28 02:43:02,642 - root - INFO - sleep for 30 secs. Waiting for handler code dst_bucket1._default._default to complete bucket operations... Current : 19581 Expected : 20160 ... 2021-04-28 02:43:32,697 - root - INFO - list of n1ql nodes in cluster: [ip:172.23.106.78 port:8091 ssh_username:root] 2021-04-28 02:43:32,707 - root - INFO - query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket1._default._default 2021-04-28 02:43:32,736 - root - INFO - list of n1ql nodes in cluster: [ip:172.23.106.78 port:8091 ssh_username:root] 2021-04-28 02:43:32,747 - root - INFO - query params : statement=select+RAW%28count%28%2A%29%29+from+dst_bucket1._default._default 2021-04-28 02:43:32,753 - root - INFO - sleep for 30 secs. Waiting for handler code dst_bucket1._default._default to complete bucket operations... Current : 19581 Expected : 20160 ... Build was aborted Aborted by vikas chaudhary Archiving artifacts *** TestRunner *** Recording test results ERROR: Step ‘Publish JUnit test result report’ failed: Test reports were found but none of them are new. Did tests run? For example, /data/workspace/dev_testbed_blr3/java_sdk_client/collections/pom.xml is 1 mo 12 days old [description-setter] Description set: [BFA] Scanning build for known causes... [BFA] No failure causes found [BFA] Done. 0s Notifying upstream projects of job completion Finished: ABORTED