Started by remote host 172.23.105.174 [EnvInject] - Loading node environment variables. Building remotely on slv-sc2402-32g-12c-col (P0 jython_slave) in workspace /data/workspace/debian-p0-query-vset00-00-upgrade_7.1_n1ql [WS-CLEANUP] Deleting project workspace... [WS-CLEANUP] Done Running Prebuild steps [debian-p0-query-vset00-00-upgrade_7.1_n1ql] $ /bin/sh -xe /tmp/jenkins7425220406534347245.sh ++ echo query-upgrade_7.1_n1ql-Jan-29-11:35:25-7.6.0-2054 ++ awk '{split($0,r,"-");print r[1],r[2]}' + desc='query upgrade_7.1_n1ql' + echo Desc: 7.6.0-2054 - query upgrade_7.1_n1ql - debian Desc: 7.6.0-2054 - query upgrade_7.1_n1ql - debian + echo newState=available + newState=available Success build forhudson.tasks.Shell@32c2140e [description-setter] Description set: 7.6.0-2054 - query upgrade_7.1_n1ql - debian Success build forhudson.plugins.descriptionsetter.DescriptionSetterBuilder@336be394 [EnvInject] - Injecting environment variables from a build step. [EnvInject] - Injecting as environment variables the properties file path 'propfile' [EnvInject] - Variables injected successfully. Success build fororg.jenkinsci.plugins.envinject.EnvInjectBuilder@1c7b250d Cloning the remote Git repository Cloning repository https://github.com/couchbase/testrunner > /usr/bin/git init /data/workspace/debian-p0-query-vset00-00-upgrade_7.1_n1ql # timeout=10 Fetching upstream changes from https://github.com/couchbase/testrunner > /usr/bin/git --version # timeout=10 > /usr/bin/git fetch --tags --progress https://github.com/couchbase/testrunner +refs/heads/*:refs/remotes/origin/* # timeout=30 > /usr/bin/git config remote.origin.url https://github.com/couchbase/testrunner # timeout=10 > /usr/bin/git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10 > /usr/bin/git config remote.origin.url https://github.com/couchbase/testrunner # timeout=10 Fetching upstream changes from https://github.com/couchbase/testrunner > /usr/bin/git fetch --tags --progress https://github.com/couchbase/testrunner +refs/heads/*:refs/remotes/origin/* # timeout=30 > /usr/bin/git rev-parse origin/master^{commit} # timeout=10 Checking out Revision f7128876a4d24e1a9636bd81fea6fcd81b1d6efe (origin/master) > /usr/bin/git config core.sparsecheckout # timeout=10 > /usr/bin/git checkout -f f7128876a4d24e1a9636bd81fea6fcd81b1d6efe > /usr/bin/git rev-list f003b588a3469649e202c03d3e29723af7b31f68 # timeout=10 > /usr/bin/git tag -a -f -m Jenkins Build #671842 jenkins-test_suite_executor-671842 # timeout=10 No emails were triggered. [debian-p0-query-vset00-00-upgrade_7.1_n1ql] $ /bin/sh -xe /tmp/jenkins5435023176715780370.sh + echo Desc: query-upgrade_7.1_n1ql-Jan-29-11:35:25-7.6.0-2054 Desc: query-upgrade_7.1_n1ql-Jan-29-11:35:25-7.6.0-2054 [description-setter] Could not determine description. [debian-p0-query-vset00-00-upgrade_7.1_n1ql] $ /bin/bash /tmp/jenkins9171687339659700338.sh python3 scripts/rerun_jobs.py 7.6.0-2054 --executor_jenkins_job --manual_run [EnvInject] - Injecting environment variables from a build step. [EnvInject] - Injecting as environment variables the properties file path 'rerun_props_file' [EnvInject] - Variables injected successfully. [debian-p0-query-vset00-00-upgrade_7.1_n1ql] $ /bin/bash /tmp/jenkins4344642654085175884.sh Set ALLOW_HTP to False so test could run. Submodule 'java_sdk_client' (https://github.com/couchbaselabs/java_sdk_client) registered for path 'java_sdk_client' Submodule 'lib/capellaAPI' (https://github.com/couchbaselabs/CapellaRESTAPIs) registered for path 'lib/capellaAPI' Submodule 'magma_loader/DocLoader' (https://github.com/couchbaselabs/DocLoader.git) registered for path 'magma_loader/DocLoader' Cloning into 'java_sdk_client'... Submodule path 'java_sdk_client': checked out 'de89b059ce28567dbac18afb032271a4eaa674ff' Cloning into 'lib/capellaAPI'... Submodule path 'lib/capellaAPI': checked out 'e98eb8225e9a714c77c670e807049f4480f05389' Cloning into 'magma_loader/DocLoader'... Submodule path 'magma_loader/DocLoader': checked out '0f5f758a9a89ecb5bc4ac20e5d4a15c704ec89f7' en_US.UTF-8 the major release is 7 "172.23.107.26","172.23.104.102","172.23.106.112","172.23.105.33" Searching for httplib2 Best match: httplib2 0.17.0 Adding httplib2 0.17.0 to easy-install.pth file Using /usr/local/lib/python3.7/site-packages Processing dependencies for httplib2 Finished processing dependencies for httplib2 Searching for ground Best match: ground 8.2.0 Adding ground 8.2.0 to easy-install.pth file Using /usr/local/lib/python3.7/site-packages Processing dependencies for ground Finished processing dependencies for ground Searching for hypothesis_geometry Best match: hypothesis-geometry 7.3.0 Adding hypothesis-geometry 7.3.0 to easy-install.pth file Using /usr/local/lib/python3.7/site-packages Processing dependencies for hypothesis_geometry Finished processing dependencies for hypothesis_geometry Searching for argparse Best match: argparse 1.4.0 Processing argparse-1.4.0-py3.7.egg argparse 1.4.0 is already the active version in easy-install.pth Using /usr/local/lib/python3.7/site-packages/argparse-1.4.0-py3.7.egg Processing dependencies for argparse Finished processing dependencies for argparse Searching for psycopg2 Reading https://pypi.org/simple/psycopg2/ Downloading https://files.pythonhosted.org/packages/c9/5e/dc6acaf46d78979d6b03458b7a1618a68e152a6776fce95daac5e0f0301b/psycopg2-2.9.9.tar.gz#sha256=d1454bde93fb1e224166811694d600e746430c006fbb031ea06ecc2ea41bf156 Best match: psycopg2 2.9.9 Processing psycopg2-2.9.9.tar.gz Writing /tmp/easy_install-y52fviab/psycopg2-2.9.9/setup.cfg Running psycopg2-2.9.9/setup.py -q bdist_egg --dist-dir /tmp/easy_install-y52fviab/psycopg2-2.9.9/egg-dist-tmp-slt89f8r Error: pg_config executable not found. pg_config is required to build psycopg2 from source. Please add the directory containing pg_config to the $PATH or specify the full executable path with the option: python setup.py build_ext --pg-config /path/to/pg_config build ... or with the pg_config option in 'setup.cfg'. If you prefer to avoid building psycopg2 from source, please install the PyPI 'psycopg2-binary' package instead. For further information please check the 'doc/src/install.rst' file (also at ). error: Setup script exited with 1 centos Loaded plugins: fastestmirror Loading mirror speeds from cached hostfile * base: mirror.sfo12.us.leaseweb.net * extras: centos-distro.1gservers.com * updates: mirror.keystealth.org Package 2:docker-1.13.1-209.git7d71120.el7.centos.x86_64 already installed and latest version Nothing to do Using default tag: latest Trying to pull repository docker.io/jamesdbloom/mockserver ... latest: Pulling from docker.io/jamesdbloom/mockserver Digest: sha256:bc41010b9e920f5e1b75a226e8eda05eb4eb9843dad042e9e3950d1b7de823f4 Status: Image is up to date for docker.io/jamesdbloom/mockserver:latest [global] username:root password:couchbase index_port:9102 n1ql_port:8093 [membase] rest_username:Administrator rest_password:password [servers] 1:_1 2:_2 3:_3 4:_4 [_1] ip:dynamic port:8091 services:kv,index,n1ql,fts [_2] ip:dynamic port:8091 services:kv,index,n1ql,fts [_3] ip:dynamic port:8091 services:kv,index,n1ql,fts [_4] ip:dynamic port:8091 services:kv,index,n1ql,fts python3 scripts/populateIni.py -s "172.23.107.26","172.23.104.102","172.23.106.112","172.23.105.33" -d None -a None -i /data/workspace/debian-p0-query-vset00-00-upgrade_7.1_n1ql/testexec_reformat.100814.ini -p debian -o /data/workspace/debian-p0-query-vset00-00-upgrade_7.1_n1ql/testexec.100814.ini -k {} /usr/local/lib/python3.7/site-packages/paramiko/transport.py:219: CryptographyDeprecationWarning: Blowfish has been deprecated "class": algorithms.Blowfish, INFO:root:SSH Connecting to 172.23.107.26 with username:root, attempt#1 of 5 INFO:root:SSH Connecting to 172.23.104.102 with username:root, attempt#1 of 5 INFO:root:SSH Connecting to 172.23.106.112 with username:root, attempt#1 of 5 INFO:root:SSH Connecting to 172.23.105.33 with username:root, attempt#1 of 5 INFO:root:SSH Connected to 172.23.105.33 as root INFO:root:SSH Connected to 172.23.104.102 as root INFO:root:SSH Connected to 172.23.107.26 as root INFO:root:SSH Connected to 172.23.106.112 as root INFO:root:os_distro: Ubuntu, os_version: debian 10, is_linux_distro: True INFO:root:os_distro: Ubuntu, os_version: debian 11, is_linux_distro: True INFO:root:os_distro: Ubuntu, os_version: debian 11, is_linux_distro: True INFO:root:os_distro: Ubuntu, os_version: debian 11, is_linux_distro: True INFO:root:extract_remote_info-->distribution_type: Ubuntu, distribution_version: debian 10 INFO:root:running command.raw on 172.23.105.33: sh -c 'if [[ "$OSTYPE" == "darwin"* ]]; then sysctl hw.memsize|grep -Eo [0-9]; else grep MemTotal /proc/meminfo|grep -Eo [0-9]; fi' INFO:root:command executed with root but got an error ['sh: 1: [[: not found'] ... INFO:root:extract_remote_info-->distribution_type: Ubuntu, distribution_version: debian 11 INFO:root:extract_remote_info-->distribution_type: Ubuntu, distribution_version: debian 11 INFO:root:running command.raw on 172.23.106.112: sh -c 'if [[ "$OSTYPE" == "darwin"* ]]; then sysctl hw.memsize|grep -Eo [0-9]; else grep MemTotal /proc/meminfo|grep -Eo [0-9]; fi' INFO:root:running command.raw on 172.23.107.26: sh -c 'if [[ "$OSTYPE" == "darwin"* ]]; then sysctl hw.memsize|grep -Eo [0-9]; else grep MemTotal /proc/meminfo|grep -Eo [0-9]; fi' INFO:root:command executed with root but got an error ['sh: 1: [[: not found'] ... INFO:root:command executed with root but got an error ['sh: 1: [[: not found'] ... INFO:root:extract_remote_info-->distribution_type: Ubuntu, distribution_version: debian 11 INFO:root:running command.raw on 172.23.104.102: sh -c 'if [[ "$OSTYPE" == "darwin"* ]]; then sysctl hw.memsize|grep -Eo [0-9]; else grep MemTotal /proc/meminfo|grep -Eo [0-9]; fi' INFO:root:command executed with root but got an error ['sh: 1: [[: not found'] ... in main the ini file is /data/workspace/debian-p0-query-vset00-00-upgrade_7.1_n1ql/testexec_reformat.100814.ini the given server info is "172.23.107.26","172.23.104.102","172.23.106.112","172.23.105.33" Collecting memory info from 172.23.105.33 sh: 1: [[: not found Collecting memory info from 172.23.106.112 Collecting memory info from 172.23.107.26 sh: 1: [[: not found sh: 1: [[: not found Collecting memory info from 172.23.104.102 sh: 1: [[: not found the servers memory info is [('172.23.106.112', 3987524), ('172.23.107.26', 3987524), ('172.23.104.102', 3987524), ('172.23.105.33', 4225072)] [global] username:root password:couchbase index_port:9102 n1ql_port:8093 [membase] rest_username:Administrator rest_password:password [servers] 1:_1 2:_2 3:_3 4:_4 [_1] ip:172.23.106.112 port:8091 services:kv,index,n1ql,fts [_2] ip:172.23.107.26 port:8091 services:kv,index,n1ql,fts [_3] ip:172.23.104.102 port:8091 services:kv,index,n1ql,fts [_4] ip:172.23.105.33 port:8091 services:kv,index,n1ql,fts extra install is Local time: Mon 2024-01-29 11:35:49 PST Universal time: Mon 2024-01-29 19:35:49 UTC RTC time: Mon 2024-01-29 19:35:49 Time zone: America/Los_Angeles (PST, -0800) NTP enabled: no NTP synchronized: yes RTC in local TZ: no DST active: no Last DST change: DST ended at Sun 2023-11-05 01:59:59 PDT Sun 2023-11-05 01:00:00 PST Next DST change: DST begins (the clock jumps one hour forward) at Sun 2024-03-10 01:59:59 PST Sun 2024-03-10 03:00:00 PDT python3 scripts/ssh.py -i /data/workspace/debian-p0-query-vset00-00-upgrade_7.1_n1ql/testexec_root.100814.ini iptables -F /usr/local/lib/python3.7/site-packages/paramiko/transport.py:219: CryptographyDeprecationWarning: Blowfish has been deprecated "class": algorithms.Blowfish, INFO:root:SSH Connecting to 172.23.106.112 with username:root, attempt#1 of 5 INFO:root:SSH Connecting to 172.23.107.26 with username:root, attempt#1 of 5 INFO:root:SSH Connecting to 172.23.104.102 with username:root, attempt#1 of 5 INFO:root:SSH Connecting to 172.23.105.33 with username:root, attempt#1 of 5 INFO:root:SSH Connected to 172.23.105.33 as root INFO:root:SSH Connected to 172.23.106.112 as root INFO:root:SSH Connected to 172.23.107.26 as root INFO:root:SSH Connected to 172.23.104.102 as root INFO:root:os_distro: Ubuntu, os_version: debian 10, is_linux_distro: True INFO:root:os_distro: Ubuntu, os_version: debian 11, is_linux_distro: True INFO:root:os_distro: Ubuntu, os_version: debian 11, is_linux_distro: True INFO:root:os_distro: Ubuntu, os_version: debian 11, is_linux_distro: True INFO:root:extract_remote_info-->distribution_type: Ubuntu, distribution_version: debian 10 INFO:root:running command.raw on 172.23.105.33: iptables -F INFO:root:command executed successfully with root INFO:root:extract_remote_info-->distribution_type: Ubuntu, distribution_version: debian 11 INFO:root:running command.raw on 172.23.106.112: iptables -F INFO:root:command executed successfully with root INFO:root:extract_remote_info-->distribution_type: Ubuntu, distribution_version: debian 11 INFO:root:running command.raw on 172.23.104.102: iptables -F INFO:root:extract_remote_info-->distribution_type: Ubuntu, distribution_version: debian 11 INFO:root:running command.raw on 172.23.107.26: iptables -F INFO:root:command executed successfully with root INFO:root:command executed successfully with root 172.23.105.33 172.23.106.112 172.23.104.102 172.23.107.26 Initial version: 7.1.6-3955 python3 scripts/new_install.py -i /data/workspace/debian-p0-query-vset00-00-upgrade_7.1_n1ql/testexec.100814.ini -p timeout=1800,skip_local_download=False,get-cbcollect-info=True,version=7.1.6-3955,product=cb,debug_logs=True,ntp=True,url=,cb_non_package_installer_url= /usr/local/lib/python3.7/site-packages/paramiko/transport.py:219: CryptographyDeprecationWarning: Blowfish has been deprecated "class": algorithms.Blowfish, 2024-01-29 11:35:51,049 - root - WARNING - URL: is not valid, will use version to locate build 2024-01-29 11:35:51,049 - root - WARNING - URL: is not valid, will use default url to locate installer 2024-01-29 11:35:51,050 - root - INFO - SSH Connecting to 172.23.106.112 with username:root, attempt#1 of 5 2024-01-29 11:35:51,238 - root - INFO - SSH Connected to 172.23.106.112 as root 2024-01-29 11:35:51,401 - root - INFO - os_distro: Ubuntu, os_version: debian 11, is_linux_distro: True 2024-01-29 11:35:51,696 - root - INFO - extract_remote_info-->distribution_type: Ubuntu, distribution_version: debian 11 2024-01-29 11:35:51,698 - root - INFO - SSH Connecting to 172.23.107.26 with username:root, attempt#1 of 5 2024-01-29 11:35:51,874 - root - INFO - SSH Connected to 172.23.107.26 as root 2024-01-29 11:35:52,055 - root - INFO - os_distro: Ubuntu, os_version: debian 11, is_linux_distro: True 2024-01-29 11:35:52,356 - root - INFO - extract_remote_info-->distribution_type: Ubuntu, distribution_version: debian 11 2024-01-29 11:35:52,358 - root - INFO - SSH Connecting to 172.23.104.102 with username:root, attempt#1 of 5 2024-01-29 11:35:52,538 - root - INFO - SSH Connected to 172.23.104.102 as root 2024-01-29 11:35:52,693 - root - INFO - os_distro: Ubuntu, os_version: debian 11, is_linux_distro: True 2024-01-29 11:35:52,995 - root - INFO - extract_remote_info-->distribution_type: Ubuntu, distribution_version: debian 11 2024-01-29 11:35:52,996 - root - INFO - SSH Connecting to 172.23.105.33 with username:root, attempt#1 of 5 2024-01-29 11:35:53,069 - root - INFO - SSH Connected to 172.23.105.33 as root 2024-01-29 11:35:53,198 - root - INFO - os_distro: Ubuntu, os_version: debian 10, is_linux_distro: True 2024-01-29 11:35:53,463 - root - INFO - extract_remote_info-->distribution_type: Ubuntu, distribution_version: debian 10 2024-01-29 11:35:53,465 - root - INFO - SSH Connecting to 172.23.106.112 with username:root, attempt#1 of 5 2024-01-29 11:35:53,646 - root - INFO - SSH Connected to 172.23.106.112 as root 2024-01-29 11:35:53,800 - root - INFO - os_distro: Ubuntu, os_version: debian 11, is_linux_distro: True 2024-01-29 11:35:54,101 - root - INFO - extract_remote_info-->distribution_type: Ubuntu, distribution_version: debian 11 2024-01-29 11:35:54,103 - root - INFO - SSH Connecting to 172.23.107.26 with username:root, attempt#1 of 5 2024-01-29 11:35:54,290 - root - INFO - SSH Connected to 172.23.107.26 as root 2024-01-29 11:35:54,450 - root - INFO - os_distro: Ubuntu, os_version: debian 11, is_linux_distro: True 2024-01-29 11:35:54,739 - root - INFO - extract_remote_info-->distribution_type: Ubuntu, distribution_version: debian 11 2024-01-29 11:35:54,741 - root - INFO - SSH Connecting to 172.23.104.102 with username:root, attempt#1 of 5 2024-01-29 11:35:54,867 - root - INFO - SSH Connected to 172.23.104.102 as root 2024-01-29 11:35:55,024 - root - INFO - os_distro: Ubuntu, os_version: debian 11, is_linux_distro: True 2024-01-29 11:35:55,319 - root - INFO - extract_remote_info-->distribution_type: Ubuntu, distribution_version: debian 11 2024-01-29 11:35:55,320 - root - INFO - SSH Connecting to 172.23.105.33 with username:root, attempt#1 of 5 2024-01-29 11:35:55,393 - root - INFO - SSH Connected to 172.23.105.33 as root 2024-01-29 11:35:55,514 - root - INFO - os_distro: Ubuntu, os_version: debian 10, is_linux_distro: True 2024-01-29 11:35:55,768 - root - INFO - extract_remote_info-->distribution_type: Ubuntu, distribution_version: debian 10 2024-01-29 11:35:55,769 - root - INFO - Check if ntp is installed 2024-01-29 11:35:55,770 - root - INFO - Check if ntp is installed 2024-01-29 11:35:55,770 - root - INFO - This OS version debian 11 2024-01-29 11:35:55,770 - root - INFO - Check if ntp is installed 2024-01-29 11:35:55,770 - root - INFO - will add install in other os later, no set do install 2024-01-29 11:35:55,770 - root - INFO - This OS version debian 11 2024-01-29 11:35:55,771 - root - INFO - Check if ntp is installed 2024-01-29 11:35:55,771 - root - INFO - This OS version debian 11 2024-01-29 11:35:55,772 - root - INFO - will add install in other os later, no set do install 2024-01-29 11:35:55,772 - root - INFO - will add install in other os later, no set do install 2024-01-29 11:35:55,773 - root - INFO - This OS version debian 10 2024-01-29 11:35:55,773 - root - INFO - running command.raw on 172.23.105.33: systemctl status ntp 2024-01-29 11:35:55,785 - root - INFO - ['Tue 30 Jan 2024 08:35:55 AM +13'] IP: 172.23.106.112 2024-01-29 11:35:55,787 - root - INFO - ['Mon 29 Jan 2024 11:35:55 AM PST'] IP: 172.23.107.26 2024-01-29 11:35:55,787 - root - INFO - ['Mon 29 Jan 2024 11:35:55 AM PST'] IP: 172.23.104.102 2024-01-29 11:35:55,821 - root - INFO - command executed successfully with root 2024-01-29 11:35:55,821 - root - INFO - running command.raw on 172.23.105.33: date 2024-01-29 11:35:55,871 - root - INFO - command executed successfully with root 2024-01-29 11:35:55,919 - root - INFO - ['Mon 29 Jan 2024 11:35:55 AM PST'] IP: 172.23.105.33 2024-01-29 11:35:55,920 - root - INFO - Trying to check is this url alive: http://172.23.126.166/builds/latestbuilds/couchbase-server/neo/3955/couchbase-server-enterprise_7.1.6-linux_amd64.deb 2024-01-29 11:35:56,008 - root - INFO - 172.23.106.112:sleep for 10 secs. Waiting for 10 seconds to try to reach url once again. Build server might be too busy. Error msg: {1} ... 2024-01-29 11:36:06,020 - root - INFO - 172.23.106.112:sleep for 10 secs. Waiting for 10 seconds to try to reach url once again. Build server might be too busy. Error msg: {1} ... 2024-01-29 11:36:16,031 - root - INFO - 172.23.106.112:sleep for 10 secs. Waiting for 10 seconds to try to reach url once again. Build server might be too busy. Error msg: {1} ... 2024-01-29 11:36:26,040 - root - INFO - Trying to check is this url alive: http://172.23.126.166/builds/releases/7.1.6/couchbase-server-enterprise_7.1.6-linux_amd64.deb 2024-01-29 11:36:26,102 - root - INFO - This url http://172.23.126.166/builds/releases/7.1.6/couchbase-server-enterprise_7.1.6-linux_amd64.deb is live 2024-01-29 11:36:26,102 - root - INFO - Trying to check is this url alive: http://172.23.126.166/builds/latestbuilds/couchbase-server/neo/3955/couchbase-server-enterprise_7.1.6-linux_amd64.deb 2024-01-29 11:36:26,104 - root - INFO - 172.23.107.26:sleep for 10 secs. Waiting for 10 seconds to try to reach url once again. Build server might be too busy. Error msg: {1} ... 2024-01-29 11:36:36,115 - root - INFO - 172.23.107.26:sleep for 10 secs. Waiting for 10 seconds to try to reach url once again. Build server might be too busy. Error msg: {1} ... 2024-01-29 11:36:46,126 - root - INFO - 172.23.107.26:sleep for 10 secs. Waiting for 10 seconds to try to reach url once again. Build server might be too busy. Error msg: {1} ... 2024-01-29 11:36:56,136 - root - INFO - Trying to check is this url alive: http://172.23.126.166/builds/releases/7.1.6/couchbase-server-enterprise_7.1.6-linux_amd64.deb 2024-01-29 11:36:56,138 - root - INFO - This url http://172.23.126.166/builds/releases/7.1.6/couchbase-server-enterprise_7.1.6-linux_amd64.deb is live 2024-01-29 11:36:56,138 - root - INFO - Trying to check is this url alive: http://172.23.126.166/builds/latestbuilds/couchbase-server/neo/3955/couchbase-server-enterprise_7.1.6-linux_amd64.deb 2024-01-29 11:36:56,139 - root - INFO - 172.23.104.102:sleep for 10 secs. Waiting for 10 seconds to try to reach url once again. Build server might be too busy. Error msg: {1} ... 2024-01-29 11:37:06,148 - root - INFO - 172.23.104.102:sleep for 10 secs. Waiting for 10 seconds to try to reach url once again. Build server might be too busy. Error msg: {1} ... 2024-01-29 11:37:16,159 - root - INFO - 172.23.104.102:sleep for 10 secs. Waiting for 10 seconds to try to reach url once again. Build server might be too busy. Error msg: {1} ... 2024-01-29 11:37:26,169 - root - INFO - Trying to check is this url alive: http://172.23.126.166/builds/releases/7.1.6/couchbase-server-enterprise_7.1.6-linux_amd64.deb 2024-01-29 11:37:26,171 - root - INFO - This url http://172.23.126.166/builds/releases/7.1.6/couchbase-server-enterprise_7.1.6-linux_amd64.deb is live 2024-01-29 11:37:26,171 - root - INFO - Trying to check is this url alive: http://172.23.126.166/builds/latestbuilds/couchbase-server/neo/3955/couchbase-server-enterprise_7.1.6-linux_amd64.deb 2024-01-29 11:37:26,172 - root - INFO - 172.23.105.33:sleep for 10 secs. Waiting for 10 seconds to try to reach url once again. Build server might be too busy. Error msg: {1} ... 2024-01-29 11:37:36,184 - root - INFO - 172.23.105.33:sleep for 10 secs. Waiting for 10 seconds to try to reach url once again. Build server might be too busy. Error msg: {1} ... 2024-01-29 11:37:46,195 - root - INFO - 172.23.105.33:sleep for 10 secs. Waiting for 10 seconds to try to reach url once again. Build server might be too busy. Error msg: {1} ... 2024-01-29 11:37:56,206 - root - INFO - Trying to check is this url alive: http://172.23.126.166/builds/releases/7.1.6/couchbase-server-enterprise_7.1.6-linux_amd64.deb 2024-01-29 11:37:56,207 - root - INFO - This url http://172.23.126.166/builds/releases/7.1.6/couchbase-server-enterprise_7.1.6-linux_amd64.deb is live 2024-01-29 11:37:56,208 - root - INFO - running command.raw on 172.23.106.112: rm -rf /tmp/tmp* ; rm -rf /tmp/cbbackupmgr-staging;rm -rf /tmp/entbackup*;systemctl -q stop couchbase-server;umount -a -t nfs,nfs4 -f -l;service ntp restart ; apt-get purge -y 'couchbase*' > /dev/null; sleep 10;dpkg --purge $(dpkg -l | grep couchbase | awk '{print $2}' | xargs echo); sleep 10; rm /var/lib/dpkg/info/couchbase-server*; sleep 10;kill -9 `ps -ef |egrep couchbase|cut -f3 -d' '`;rm -rf /opt/couchbase > /dev/null && echo 1 || echo 0; dpkg -P couchbase-server; rm -rf /var/lib/dpkg/info/couchbase-server*;dpkg --configure -a; apt-get update; journalctl --vacuum-size=100M; journalctl --vacuum-time=10d; grep 'kernel.dmesg_restrict=0' /etc/sysctl.conf || (echo 'kernel.dmesg_restrict=0' >> /etc/sysctl.conf && service procps restart) ; rm -rf/opt/couchbase 2024-01-29 11:37:56,208 - root - INFO - running command.raw on 172.23.107.26: rm -rf /tmp/tmp* ; rm -rf /tmp/cbbackupmgr-staging;rm -rf /tmp/entbackup*;systemctl -q stop couchbase-server;umount -a -t nfs,nfs4 -f -l;service ntp restart ; apt-get purge -y 'couchbase*' > /dev/null; sleep 10;dpkg --purge $(dpkg -l | grep couchbase | awk '{print $2}' | xargs echo); sleep 10; rm /var/lib/dpkg/info/couchbase-server*; sleep 10;kill -9 `ps -ef |egrep couchbase|cut -f3 -d' '`;rm -rf /opt/couchbase > /dev/null && echo 1 || echo 0; dpkg -P couchbase-server; rm -rf /var/lib/dpkg/info/couchbase-server*;dpkg --configure -a; apt-get update; journalctl --vacuum-size=100M; journalctl --vacuum-time=10d; grep 'kernel.dmesg_restrict=0' /etc/sysctl.conf || (echo 'kernel.dmesg_restrict=0' >> /etc/sysctl.conf && service procps restart) ; rm -rf/opt/couchbase 2024-01-29 11:37:56,209 - root - INFO - running command.raw on 172.23.104.102: rm -rf /tmp/tmp* ; rm -rf /tmp/cbbackupmgr-staging;rm -rf /tmp/entbackup*;systemctl -q stop couchbase-server;umount -a -t nfs,nfs4 -f -l;service ntp restart ; apt-get purge -y 'couchbase*' > /dev/null; sleep 10;dpkg --purge $(dpkg -l | grep couchbase | awk '{print $2}' | xargs echo); sleep 10; rm /var/lib/dpkg/info/couchbase-server*; sleep 10;kill -9 `ps -ef |egrep couchbase|cut -f3 -d' '`;rm -rf /opt/couchbase > /dev/null && echo 1 || echo 0; dpkg -P couchbase-server; rm -rf /var/lib/dpkg/info/couchbase-server*;dpkg --configure -a; apt-get update; journalctl --vacuum-size=100M; journalctl --vacuum-time=10d; grep 'kernel.dmesg_restrict=0' /etc/sysctl.conf || (echo 'kernel.dmesg_restrict=0' >> /etc/sysctl.conf && service procps restart) ; rm -rf/opt/couchbase 2024-01-29 11:37:56,210 - root - INFO - running command.raw on 172.23.105.33: rm -rf /tmp/tmp* ; rm -rf /tmp/cbbackupmgr-staging;rm -rf /tmp/entbackup*;systemctl -q stop couchbase-server;umount -a -t nfs,nfs4 -f -l;service ntp restart ; apt-get purge -y 'couchbase*' > /dev/null; sleep 10;dpkg --purge $(dpkg -l | grep couchbase | awk '{print $2}' | xargs echo); sleep 10; rm /var/lib/dpkg/info/couchbase-server*; sleep 10;kill -9 `ps -ef |egrep couchbase|cut -f3 -d' '`;rm -rf /opt/couchbase > /dev/null && echo 1 || echo 0; dpkg -P couchbase-server; rm -rf /var/lib/dpkg/info/couchbase-server*;dpkg --configure -a; apt-get update; journalctl --vacuum-size=100M; journalctl --vacuum-time=10d; grep 'kernel.dmesg_restrict=0' /etc/sysctl.conf || (echo 'kernel.dmesg_restrict=0' >> /etc/sysctl.conf && service procps restart) ; rm -rf/opt/couchbase 2024-01-29 11:38:31,376 - root - INFO - command executed with root but got an error ['dpkg: error: --purge needs at least one package name argument', '', 'Type dpkg --help for help about installing and deinstalling packages [*];', "Use 'apt' or 'aptitude' for user-friendly package management;", 'Type dpkg -Dhelp for a list of dpkg debug flag values;', 'Type dpkg --force-help for a list of forcing options;', 'Type dpkg-deb --help for help about manipulating *.deb files;', '', "Opt ... 2024-01-29 11:38:31,376 - root - INFO - Waiting 10s for uninstall to complete on 172.23.105.33.. 2024-01-29 11:38:31,549 - root - INFO - command executed with root but got an error ['dpkg: error: --purge needs at least one package name argument', '', 'Type dpkg --help for help about installing and deinstalling packages [*];', "Use 'apt' or 'aptitude' for user-friendly package management;", 'Type dpkg -Dhelp for a list of dpkg debug flag values;', 'Type dpkg --force-help for a list of forcing options;', 'Type dpkg-deb --help for help about manipulating *.deb files;', '', "Opt ... 2024-01-29 11:38:31,549 - root - INFO - Waiting 10s for uninstall to complete on 172.23.104.102.. 2024-01-29 11:38:31,645 - root - INFO - command executed with root but got an error ['dpkg: error: --purge needs at least one package name argument', '', 'Type dpkg --help for help about installing and deinstalling packages [*];', "Use 'apt' or 'aptitude' for user-friendly package management;", 'Type dpkg -Dhelp for a list of dpkg debug flag values;', 'Type dpkg --force-help for a list of forcing options;', 'Type dpkg-deb --help for help about manipulating *.deb files;', '', "Opt ... 2024-01-29 11:38:31,646 - root - INFO - Waiting 10s for uninstall to complete on 172.23.106.112.. 2024-01-29 11:38:31,675 - root - INFO - command executed with root but got an error ['dpkg: error: --purge needs at least one package name argument', '', 'Type dpkg --help for help about installing and deinstalling packages [*];', "Use 'apt' or 'aptitude' for user-friendly package management;", 'Type dpkg -Dhelp for a list of dpkg debug flag values;', 'Type dpkg --force-help for a list of forcing options;', 'Type dpkg-deb --help for help about manipulating *.deb files;', '', "Opt ... 2024-01-29 11:38:31,675 - root - INFO - Waiting 10s for uninstall to complete on 172.23.107.26.. 2024-01-29 11:38:42,189 - root - INFO - Done with uninstall on 172.23.105.33. 2024-01-29 11:38:42,546 - root - INFO - Done with uninstall on 172.23.104.102. 2024-01-29 11:38:42,661 - root - INFO - Done with uninstall on 172.23.106.112. 2024-01-29 11:38:42,683 - root - INFO - Done with uninstall on 172.23.107.26. 2024-01-29 11:38:46,256 - root - INFO - running command.raw on 172.23.106.112: cd /tmp/; wget -Nq -O couchbase-server-enterprise_7.1.6-linux_amd64.deb http://172.23.126.166/builds/releases/7.1.6/couchbase-server-enterprise_7.1.6-linux_amd64.deb 2024-01-29 11:38:56,727 - root - INFO - command executed successfully with root 2024-01-29 11:38:56,729 - root - INFO - running command.raw on 172.23.106.112: cd /tmp/ && wc -c couchbase-server-enterprise_7.1.6-linux_amd64.deb 2024-01-29 11:38:56,735 - root - INFO - command executed successfully with root 2024-01-29 11:38:56,736 - root - INFO - running command.raw on 172.23.106.112: ls -lh /tmp/couchbase-server-enterprise_7.1.6-linux_amd64.deb 2024-01-29 11:38:56,791 - root - INFO - command executed successfully with root 2024-01-29 11:38:56,791 - root - INFO - running command.raw on 172.23.107.26: cd /tmp/; wget -Nq -O couchbase-server-enterprise_7.1.6-linux_amd64.deb http://172.23.126.166/builds/releases/7.1.6/couchbase-server-enterprise_7.1.6-linux_amd64.deb 2024-01-29 11:38:57,906 - root - INFO - command executed successfully with root 2024-01-29 11:38:57,908 - root - INFO - running command.raw on 172.23.107.26: cd /tmp/ && wc -c couchbase-server-enterprise_7.1.6-linux_amd64.deb 2024-01-29 11:38:57,915 - root - INFO - command executed successfully with root 2024-01-29 11:38:57,916 - root - INFO - running command.raw on 172.23.107.26: ls -lh /tmp/couchbase-server-enterprise_7.1.6-linux_amd64.deb 2024-01-29 11:38:57,973 - root - INFO - command executed successfully with root 2024-01-29 11:38:57,974 - root - INFO - running command.raw on 172.23.104.102: cd /tmp/; wget -Nq -O couchbase-server-enterprise_7.1.6-linux_amd64.deb http://172.23.126.166/builds/releases/7.1.6/couchbase-server-enterprise_7.1.6-linux_amd64.deb 2024-01-29 11:38:59,058 - root - INFO - command executed successfully with root 2024-01-29 11:38:59,059 - root - INFO - running command.raw on 172.23.104.102: cd /tmp/ && wc -c couchbase-server-enterprise_7.1.6-linux_amd64.deb 2024-01-29 11:38:59,068 - root - INFO - command executed successfully with root 2024-01-29 11:38:59,068 - root - INFO - running command.raw on 172.23.104.102: ls -lh /tmp/couchbase-server-enterprise_7.1.6-linux_amd64.deb 2024-01-29 11:38:59,125 - root - INFO - command executed successfully with root 2024-01-29 11:38:59,126 - root - INFO - running command.raw on 172.23.105.33: cd /tmp/; wget -Nq -O couchbase-server-enterprise_7.1.6-linux_amd64.deb http://172.23.126.166/builds/releases/7.1.6/couchbase-server-enterprise_7.1.6-linux_amd64.deb 2024-01-29 11:39:00,387 - root - INFO - command executed successfully with root 2024-01-29 11:39:00,388 - root - INFO - running command.raw on 172.23.105.33: cd /tmp/ && wc -c couchbase-server-enterprise_7.1.6-linux_amd64.deb 2024-01-29 11:39:00,395 - root - INFO - command executed successfully with root 2024-01-29 11:39:00,395 - root - INFO - running command.raw on 172.23.105.33: ls -lh /tmp/couchbase-server-enterprise_7.1.6-linux_amd64.deb 2024-01-29 11:39:00,450 - root - INFO - command executed successfully with root 2024-01-29 11:39:00,450 - root - INFO - running command.raw on 172.23.106.112: rm -f /etc/couchbase.d/config_profile 2024-01-29 11:39:00,451 - root - INFO - running command.raw on 172.23.107.26: rm -f /etc/couchbase.d/config_profile 2024-01-29 11:39:00,451 - root - INFO - running command.raw on 172.23.104.102: rm -f /etc/couchbase.d/config_profile 2024-01-29 11:39:00,452 - root - INFO - running command.raw on 172.23.105.33: rm -f /etc/couchbase.d/config_profile 2024-01-29 11:39:00,463 - root - INFO - command executed successfully with root 2024-01-29 11:39:00,463 - root - INFO - running command.raw on 172.23.106.112: DEBIAN_FRONTEND='noninteractive' apt-get -y -f install /tmp/couchbase-server-enterprise_7.1.6-linux_amd64.deb > /dev/null && echo 1 || echo 0 2024-01-29 11:39:00,468 - root - INFO - command executed successfully with root 2024-01-29 11:39:00,468 - root - INFO - running command.raw on 172.23.107.26: DEBIAN_FRONTEND='noninteractive' apt-get -y -f install /tmp/couchbase-server-enterprise_7.1.6-linux_amd64.deb > /dev/null && echo 1 || echo 0 2024-01-29 11:39:00,469 - root - INFO - command executed successfully with root 2024-01-29 11:39:00,469 - root - INFO - running command.raw on 172.23.104.102: DEBIAN_FRONTEND='noninteractive' apt-get -y -f install /tmp/couchbase-server-enterprise_7.1.6-linux_amd64.deb > /dev/null && echo 1 || echo 0 2024-01-29 11:39:00,497 - root - INFO - command executed successfully with root 2024-01-29 11:39:00,498 - root - INFO - running command.raw on 172.23.105.33: DEBIAN_FRONTEND='noninteractive' apt-get -y -f install /tmp/couchbase-server-enterprise_7.1.6-linux_amd64.deb > /dev/null && echo 1 || echo 0 2024-01-29 11:39:44,765 - root - INFO - command executed successfully with root 2024-01-29 11:39:44,765 - root - INFO - running command.raw on 172.23.106.112: usermod -aG adm couchbase && systemctl -q is-active couchbase-server.service && echo 1 || echo 0 2024-01-29 11:39:44,837 - root - INFO - command executed successfully with root 2024-01-29 11:39:44,837 - root - INFO - running command.raw on 172.23.107.26: usermod -aG adm couchbase && systemctl -q is-active couchbase-server.service && echo 1 || echo 0 2024-01-29 11:39:44,843 - root - INFO - command executed successfully with root 2024-01-29 11:39:44,843 - root - INFO - Done with install on 172.23.106.112. 2024-01-29 11:39:44,843 - root - INFO - Waiting for couchbase to be reachable 2024-01-29 11:39:44,844 - root - ERROR - socket error while connecting to http://172.23.106.112:8091/pools/default error [Errno 111] Connection refused 2024-01-29 11:39:44,912 - root - INFO - command executed successfully with root 2024-01-29 11:39:44,913 - root - INFO - Done with install on 172.23.107.26. 2024-01-29 11:39:44,913 - root - INFO - Waiting for couchbase to be reachable 2024-01-29 11:39:44,915 - root - ERROR - socket error while connecting to http://172.23.107.26:8091/pools/default error [Errno 111] Connection refused 2024-01-29 11:39:45,274 - root - INFO - command executed successfully with root 2024-01-29 11:39:45,274 - root - INFO - running command.raw on 172.23.104.102: usermod -aG adm couchbase && systemctl -q is-active couchbase-server.service && echo 1 || echo 0 2024-01-29 11:39:45,297 - root - INFO - command executed successfully with root 2024-01-29 11:39:45,298 - root - INFO - Done with install on 172.23.104.102. 2024-01-29 11:39:45,298 - root - INFO - Waiting for couchbase to be reachable 2024-01-29 11:39:45,299 - root - ERROR - socket error while connecting to http://172.23.104.102:8091/pools/default error [Errno 111] Connection refused 2024-01-29 11:39:47,847 - root - ERROR - socket error while connecting to http://172.23.106.112:8091/pools/default error [Errno 111] Connection refused 2024-01-29 11:39:47,919 - root - ERROR - socket error while connecting to http://172.23.107.26:8091/pools/default error [Errno 111] Connection refused 2024-01-29 11:39:48,303 - root - ERROR - socket error while connecting to http://172.23.104.102:8091/pools/default error [Errno 111] Connection refused 2024-01-29 11:39:53,858 - root - ERROR - GET http://172.23.106.112:8091/pools/default body: headers: {'Content-Type': 'application/json', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==', 'Accept': '*/*'} error: 404 reason: unknown b'"unknown pool"' auth: Administrator:password http://172.23.106.112:8091/pools/default with status False: unknown pool 2024-01-29 11:39:53,861 - root - INFO - running command.raw on 172.23.106.112: /opt/couchbase/bin/couchbase-cli node-init -c 172.23.106.112 -u Administrator -p password > /dev/null && echo 1 || echo 0; 2024-01-29 11:39:53,928 - root - ERROR - GET http://172.23.107.26:8091/pools/default body: headers: {'Content-Type': 'application/json', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==', 'Accept': '*/*'} error: 404 reason: unknown b'"unknown pool"' auth: Administrator:password http://172.23.107.26:8091/pools/default with status False: unknown pool 2024-01-29 11:39:53,928 - root - INFO - running command.raw on 172.23.107.26: /opt/couchbase/bin/couchbase-cli node-init -c 172.23.107.26 -u Administrator -p password > /dev/null && echo 1 || echo 0; 2024-01-29 11:39:54,313 - root - ERROR - GET http://172.23.104.102:8091/pools/default body: headers: {'Content-Type': 'application/json', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==', 'Accept': '*/*'} error: 404 reason: unknown b'"unknown pool"' auth: Administrator:password http://172.23.104.102:8091/pools/default with status False: unknown pool 2024-01-29 11:39:54,313 - root - INFO - running command.raw on 172.23.104.102: /opt/couchbase/bin/couchbase-cli node-init -c 172.23.104.102 -u Administrator -p password > /dev/null && echo 1 || echo 0; 2024-01-29 11:39:54,845 - root - INFO - command executed successfully with root 2024-01-29 11:39:54,849 - root - ERROR - GET http://172.23.107.26:8091/pools/default body: headers: {'Content-Type': 'application/json', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==', 'Accept': '*/*'} error: 404 reason: unknown b'"unknown pool"' auth: Administrator:password http://172.23.107.26:8091/pools/default with status False: unknown pool 2024-01-29 11:39:54,881 - root - INFO - command executed successfully with root 2024-01-29 11:39:54,884 - root - ERROR - GET http://172.23.106.112:8091/pools/default body: headers: {'Content-Type': 'application/json', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==', 'Accept': '*/*'} error: 404 reason: unknown b'"unknown pool"' auth: Administrator:password http://172.23.106.112:8091/pools/default with status False: unknown pool 2024-01-29 11:39:55,275 - root - INFO - command executed successfully with root 2024-01-29 11:39:55,279 - root - ERROR - GET http://172.23.104.102:8091/pools/default body: headers: {'Content-Type': 'application/json', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==', 'Accept': '*/*'} error: 404 reason: unknown b'"unknown pool"' auth: Administrator:password http://172.23.104.102:8091/pools/default with status False: unknown pool 2024-01-29 11:39:55,557 - root - INFO - command executed successfully with root 2024-01-29 11:39:55,557 - root - INFO - running command.raw on 172.23.105.33: usermod -aG adm couchbase && systemctl -q is-active couchbase-server.service && echo 1 || echo 0 2024-01-29 11:39:55,571 - root - INFO - command executed successfully with root 2024-01-29 11:39:55,571 - root - INFO - Done with install on 172.23.105.33. 2024-01-29 11:39:55,571 - root - INFO - Waiting for couchbase to be reachable 2024-01-29 11:39:55,572 - root - ERROR - socket error while connecting to http://172.23.105.33:8091/pools/default error [Errno 111] Connection refused 2024-01-29 11:39:55,865 - root - INFO - Setting INDEX memory quota as 256 MB on 172.23.107.26 2024-01-29 11:39:55,865 - root - INFO - pools/default params : indexMemoryQuota=256 2024-01-29 11:39:55,878 - root - INFO - Setting FTS memory quota as 512 MB on 172.23.107.26 2024-01-29 11:39:55,878 - root - INFO - pools/default params : ftsMemoryQuota=512 2024-01-29 11:39:55,885 - root - INFO - Setting KV memory quota as 1840 MB on 172.23.107.26 2024-01-29 11:39:55,885 - root - INFO - pools/default params : memoryQuota=1840 2024-01-29 11:39:55,889 - root - INFO - --> init_node_services(Administrator,password,None,8091,['kv', 'index', 'n1ql', 'fts']) 2024-01-29 11:39:55,889 - root - INFO - node/controller/setupServices params on 172.23.107.26: 8091:hostname=None&user=Administrator&password=password&services=kv%2Cindex%2Cn1ql%2Cfts 2024-01-29 11:39:55,896 - root - INFO - Setting INDEX memory quota as 256 MB on 172.23.106.112 2024-01-29 11:39:55,896 - root - INFO - pools/default params : indexMemoryQuota=256 2024-01-29 11:39:55,912 - root - INFO - Setting FTS memory quota as 512 MB on 172.23.106.112 2024-01-29 11:39:55,912 - root - INFO - pools/default params : ftsMemoryQuota=512 2024-01-29 11:39:55,919 - root - INFO - Setting KV memory quota as 1840 MB on 172.23.106.112 2024-01-29 11:39:55,919 - root - INFO - pools/default params : memoryQuota=1840 2024-01-29 11:39:55,921 - root - INFO - settings/indexes params : storageMode=plasma 2024-01-29 11:39:55,923 - root - INFO - --> init_node_services(Administrator,password,None,8091,['kv', 'index', 'n1ql', 'fts']) 2024-01-29 11:39:55,923 - root - INFO - node/controller/setupServices params on 172.23.106.112: 8091:hostname=None&user=Administrator&password=password&services=kv%2Cindex%2Cn1ql%2Cfts 2024-01-29 11:39:55,929 - root - INFO - --> in init_cluster...Administrator,password,8091 2024-01-29 11:39:55,929 - root - INFO - settings/web params on 172.23.107.26:8091:port=8091&username=Administrator&password=password 2024-01-29 11:39:55,961 - root - INFO - settings/indexes params : storageMode=plasma 2024-01-29 11:39:55,971 - root - INFO - --> in init_cluster...Administrator,password,8091 2024-01-29 11:39:55,971 - root - INFO - settings/web params on 172.23.106.112:8091:port=8091&username=Administrator&password=password 2024-01-29 11:39:55,976 - root - INFO - --> status:True 2024-01-29 11:39:55,976 - root - INFO - Done with init on 172.23.107.26. 2024-01-29 11:39:55,977 - root - INFO - running command.raw on 172.23.107.26: ls -td /tmp/couchbase*.deb | awk 'NR>2' | xargs rm -f 2024-01-29 11:39:56,004 - root - INFO - command executed successfully with root 2024-01-29 11:39:56,004 - root - INFO - Done with cleanup on 172.23.107.26. 2024-01-29 11:39:56,026 - root - INFO - --> status:True 2024-01-29 11:39:56,026 - root - INFO - Done with init on 172.23.106.112. 2024-01-29 11:39:56,027 - root - INFO - running command.raw on 172.23.106.112: ls -td /tmp/couchbase*.deb | awk 'NR>2' | xargs rm -f 2024-01-29 11:39:56,047 - root - INFO - command executed successfully with root 2024-01-29 11:39:56,047 - root - INFO - Done with cleanup on 172.23.106.112. 2024-01-29 11:39:56,288 - root - INFO - Setting INDEX memory quota as 256 MB on 172.23.104.102 2024-01-29 11:39:56,288 - root - INFO - pools/default params : indexMemoryQuota=256 2024-01-29 11:39:56,303 - root - INFO - Setting FTS memory quota as 512 MB on 172.23.104.102 2024-01-29 11:39:56,303 - root - INFO - pools/default params : ftsMemoryQuota=512 2024-01-29 11:39:56,309 - root - INFO - Setting KV memory quota as 1840 MB on 172.23.104.102 2024-01-29 11:39:56,309 - root - INFO - pools/default params : memoryQuota=1840 2024-01-29 11:39:56,312 - root - INFO - --> init_node_services(Administrator,password,None,8091,['kv', 'index', 'n1ql', 'fts']) 2024-01-29 11:39:56,312 - root - INFO - node/controller/setupServices params on 172.23.104.102: 8091:hostname=None&user=Administrator&password=password&services=kv%2Cindex%2Cn1ql%2Cfts 2024-01-29 11:39:56,347 - root - INFO - settings/indexes params : storageMode=plasma 2024-01-29 11:39:56,355 - root - INFO - --> in init_cluster...Administrator,password,8091 2024-01-29 11:39:56,355 - root - INFO - settings/web params on 172.23.104.102:8091:port=8091&username=Administrator&password=password 2024-01-29 11:39:56,414 - root - INFO - --> status:True 2024-01-29 11:39:56,414 - root - INFO - Done with init on 172.23.104.102. 2024-01-29 11:39:56,414 - root - INFO - running command.raw on 172.23.104.102: ls -td /tmp/couchbase*.deb | awk 'NR>2' | xargs rm -f 2024-01-29 11:39:56,439 - root - INFO - command executed successfully with root 2024-01-29 11:39:56,442 - root - INFO - Done with cleanup on 172.23.104.102. 2024-01-29 11:39:58,576 - root - ERROR - socket error while connecting to http://172.23.105.33:8091/pools/default error [Errno 111] Connection refused 2024-01-29 11:40:04,580 - root - ERROR - GET http://172.23.105.33:8091/pools/default body: headers: {'Content-Type': 'application/json', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==', 'Accept': '*/*'} error: 404 reason: unknown b'"unknown pool"' auth: Administrator:password http://172.23.105.33:8091/pools/default with status False: unknown pool 2024-01-29 11:40:04,580 - root - INFO - running command.raw on 172.23.105.33: /opt/couchbase/bin/couchbase-cli node-init -c 172.23.105.33 -u Administrator -p password > /dev/null && echo 1 || echo 0; 2024-01-29 11:40:05,778 - root - INFO - command executed successfully with root 2024-01-29 11:40:05,781 - root - ERROR - GET http://172.23.105.33:8091/pools/default body: headers: {'Content-Type': 'application/json', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==', 'Accept': '*/*'} error: 404 reason: unknown b'"unknown pool"' auth: Administrator:password http://172.23.105.33:8091/pools/default with status False: unknown pool 2024-01-29 11:40:06,787 - root - INFO - Setting INDEX memory quota as 256 MB on 172.23.105.33 2024-01-29 11:40:06,788 - root - INFO - pools/default params : indexMemoryQuota=256 2024-01-29 11:40:06,795 - root - INFO - Setting FTS memory quota as 512 MB on 172.23.105.33 2024-01-29 11:40:06,795 - root - INFO - pools/default params : ftsMemoryQuota=512 2024-01-29 11:40:06,799 - root - INFO - Setting KV memory quota as 1996 MB on 172.23.105.33 2024-01-29 11:40:06,799 - root - INFO - pools/default params : memoryQuota=1996 2024-01-29 11:40:06,804 - root - INFO - --> init_node_services(Administrator,password,None,8091,['kv', 'index', 'n1ql', 'fts']) 2024-01-29 11:40:06,804 - root - INFO - node/controller/setupServices params on 172.23.105.33: 8091:hostname=None&user=Administrator&password=password&services=kv%2Cindex%2Cn1ql%2Cfts 2024-01-29 11:40:06,835 - root - INFO - settings/indexes params : storageMode=plasma 2024-01-29 11:40:06,845 - root - INFO - --> in init_cluster...Administrator,password,8091 2024-01-29 11:40:06,845 - root - INFO - settings/web params on 172.23.105.33:8091:port=8091&username=Administrator&password=password 2024-01-29 11:40:06,900 - root - INFO - --> status:True 2024-01-29 11:40:06,900 - root - INFO - Done with init on 172.23.105.33. 2024-01-29 11:40:06,901 - root - INFO - running command.raw on 172.23.105.33: ls -td /tmp/couchbase*.deb | awk 'NR>2' | xargs rm -f 2024-01-29 11:40:06,924 - root - INFO - command executed successfully with root 2024-01-29 11:40:06,925 - root - INFO - Done with cleanup on 172.23.105.33. 2024-01-29 11:40:10,516 - root - INFO - ---------------------------------------------------------------------------------------------------- 2024-01-29 11:40:10,546 - root - INFO - cluster:C1 node:172.23.106.112:8091 version:7.1.6-3955-enterprise aFamily:inet services:['fts', 'index', 'kv', 'n1ql'] 2024-01-29 11:40:10,546 - root - INFO - cluster:C2 node:172.23.107.26:8091 version:7.1.6-3955-enterprise aFamily:inet services:['fts', 'index', 'kv', 'n1ql'] 2024-01-29 11:40:10,546 - root - INFO - cluster:C3 node:172.23.104.102:8091 version:7.1.6-3955-enterprise aFamily:inet services:['fts', 'index', 'kv', 'n1ql'] 2024-01-29 11:40:10,546 - root - INFO - cluster:C4 node:172.23.105.33:8091 version:7.1.6-3955-enterprise aFamily:inet services:['fts', 'index', 'kv', 'n1ql'] 2024-01-29 11:40:10,546 - root - INFO - ---------------------------------------------------------------------------------------------------- 2024-01-29 11:40:10,546 - root - INFO - ---------------------------------------------------------------------------------------------------- 2024-01-29 11:40:10,547 - root - INFO - ---------------------------------------------------------------------------------------------------- 2024-01-29 11:40:10,547 - root - INFO - INSTALL COMPLETED ON: 172.23.106.112 2024-01-29 11:40:10,547 - root - INFO - INSTALL COMPLETED ON: 172.23.107.26 2024-01-29 11:40:10,547 - root - INFO - INSTALL COMPLETED ON: 172.23.104.102 2024-01-29 11:40:10,547 - root - INFO - INSTALL COMPLETED ON: 172.23.105.33 2024-01-29 11:40:10,547 - root - INFO - ---------------------------------------------------------------------------------------------------- 2024-01-29 11:40:10,547 - root - INFO - TOTAL INSTALL TIME = 259 seconds success /usr/local/lib/python3.7/site-packages/paramiko/transport.py:219: CryptographyDeprecationWarning: Blowfish has been deprecated "class": algorithms.Blowfish, INFO:root:SSH Connecting to 172.23.106.112 with username:root, attempt#1 of 5 INFO:root:SSH Connecting to 172.23.104.102 with username:root, attempt#1 of 5 INFO:root:SSH Connecting to 172.23.107.26 with username:root, attempt#1 of 5 INFO:root:SSH Connecting to 172.23.105.33 with username:root, attempt#1 of 5 INFO:root:SSH Connected to 172.23.105.33 as root INFO:root:SSH Connected to 172.23.107.26 as root INFO:root:SSH Connected to 172.23.106.112 as root INFO:root:SSH Connected to 172.23.104.102 as root INFO:root:os_distro: Ubuntu, os_version: debian 10, is_linux_distro: True INFO:root:os_distro: Ubuntu, os_version: debian 11, is_linux_distro: True INFO:root:os_distro: Ubuntu, os_version: debian 11, is_linux_distro: True INFO:root:os_distro: Ubuntu, os_version: debian 11, is_linux_distro: True INFO:root:extract_remote_info-->distribution_type: Ubuntu, distribution_version: debian 10 INFO:root:running command.raw on 172.23.105.33: iptables -F INFO:root:command executed successfully with root INFO:root:extract_remote_info-->distribution_type: Ubuntu, distribution_version: debian 11 INFO:root:running command.raw on 172.23.107.26: iptables -F INFO:root:extract_remote_info-->distribution_type: Ubuntu, distribution_version: debian 11 INFO:root:running command.raw on 172.23.104.102: iptables -F INFO:root:extract_remote_info-->distribution_type: Ubuntu, distribution_version: debian 11 INFO:root:running command.raw on 172.23.106.112: iptables -F INFO:root:command executed successfully with root INFO:root:command executed successfully with root INFO:root:command executed successfully with root 172.23.105.33 172.23.107.26 172.23.104.102 172.23.106.112 Need to set ALLOW_HTP back to True to do git pull branch Submodule path 'java_sdk_client': checked out 'de89b059ce28567dbac18afb032271a4eaa674ff' Submodule path 'lib/capellaAPI': checked out 'e98eb8225e9a714c77c670e807049f4480f05389' Submodule path 'magma_loader/DocLoader': checked out '0f5f758a9a89ecb5bc4ac20e5d4a15c704ec89f7' Requirement already satisfied: pip in /usr/local/lib/python3.7/site-packages (23.3.2) WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv Requirement already satisfied: swig in /usr/local/lib/python3.7/site-packages (4.1.1.post1) WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv Requirement already satisfied: boto3 in /usr/local/lib/python3.7/site-packages (1.17.112) Requirement already satisfied: faiss-cpu in /usr/local/lib/python3.7/site-packages (1.7.4) Requirement already satisfied: wget in /usr/local/lib/python3.7/site-packages (3.2) Requirement already satisfied: h5py in /usr/local/lib/python3.7/site-packages (3.8.0) Requirement already satisfied: botocore<1.21.0,>=1.20.112 in /usr/local/lib/python3.7/site-packages (from boto3) (1.20.112) Requirement already satisfied: jmespath<1.0.0,>=0.7.1 in /usr/local/lib/python3.7/site-packages (from boto3) (0.10.0) Requirement already satisfied: s3transfer<0.5.0,>=0.4.0 in /usr/local/lib/python3.7/site-packages (from boto3) (0.4.2) Requirement already satisfied: numpy>=1.14.5 in /usr/local/lib/python3.7/site-packages (from h5py) (1.21.6) Requirement already satisfied: python-dateutil<3.0.0,>=2.1 in /usr/local/lib/python3.7/site-packages (from botocore<1.21.0,>=1.20.112->boto3) (2.8.1) Requirement already satisfied: urllib3<1.27,>=1.25.4 in /usr/local/lib/python3.7/site-packages (from botocore<1.21.0,>=1.20.112->boto3) (1.25.8) Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.7/site-packages (from python-dateutil<3.0.0,>=2.1->botocore<1.21.0,>=1.20.112->boto3) (1.14.0) WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv Loaded plugins: fastestmirror Loading mirror speeds from cached hostfile * base: mirror.sfo12.us.leaseweb.net * extras: centos-distro.1gservers.com * updates: mirror.keystealth.org Package maven-3.0.5-17.el7.noarch already installed and latest version Nothing to do find: ‘/root/jenkins/workspace/’: No such file or directory find: ‘/root/workspace/*/logs/*’: No such file or directory find: ‘/root/workspace/’: No such file or directory gsi_type=plasma,nodes_init=4,initial_version=7.1.6-3955,upgrade_version=7.6.0-2054,GROUP=N1QL,get-cbcollect-info=True,get-cbcollect-info=True,bucket_storage=couchstore,sirius_url=http://172.23.120.103:4000 python3: no process found python3 testrunner.py -i /data/workspace/debian-p0-query-vset00-00-upgrade_7.1_n1ql/testexec.100814.ini -c conf/tuq/py-n1ql-upgrade.conf -p gsi_type=plasma,nodes_init=4,initial_version=7.1.6-3955,upgrade_version=7.6.0-2054,GROUP=N1QL,get-cbcollect-info=True,get-cbcollect-info=True,bucket_storage=couchstore,sirius_url=http://172.23.120.103:4000 -d failed=http://qa.sc.couchbase.com/job/test_suite_executor/671419/ INFO:__main__:Checking arguments... INFO:__main__:Conf filename: conf/tuq/py-n1ql-upgrade.conf INFO:__main__:Test prefix: tuqquery.n1ql_upgrade.QueriesUpgradeTests INFO:__main__:Downloading http://qa.sc.couchbase.com/job/test_suite_executor/671419//testReport/api/xml?pretty=true to logs/test_suite_executor_671419__testresult.xml INFO:__main__:Loading result data from logs/test_suite_executor_671419__testresult.xml INFO:__main__:-- logs/test_suite_executor_671419__testresult.xml -- INFO:__main__:TestRunner: start... INFO:__main__:Global Test input params: INFO:__main__: Number of tests initially selected before GROUP filters: 1 INFO:__main__:--> Running test: tuqquery.n1ql_upgrade.QueriesUpgradeTests.test_upgrade,standard_buckets=1,dgm_run=true,upgrade_type=offline,feature=n1ql,services_init=kv:n1ql:index-kv:n1ql:index-kv:n1ql:index-kv:n1ql:index,array_indexing=false,dataset=default,doc-per-day=1,GROUP=N1QL,gsi_type=plasma,nodes_init=4,initial_version=7.1.6-3955,upgrade_version=7.6.0-2054,get-cbcollect-info=True,bucket_storage=couchstore,sirius_url=http://172.23.120.103:4000 INFO:__main__:Logs folder: /data/workspace/debian-p0-query-vset00-00-upgrade_7.1_n1ql/logs/testrunner-24-Jan-29_11-40-24/test_1 *** TestRunner *** {'GROUP': 'N1QL', 'bucket_storage': 'couchstore', 'cluster_name': 'testexec.100814', 'conf_file': 'conf/tuq/py-n1ql-upgrade.conf', 'get-cbcollect-info': 'True', 'gsi_type': 'plasma', 'ini': '/data/workspace/debian-p0-query-vset00-00-upgrade_7.1_n1ql/testexec.100814.ini', 'initial_version': '7.1.6-3955', 'nodes_init': '4', 'num_nodes': 4, 'sirius_url': 'http://172.23.120.103:4000', 'spec': 'py-n1ql-upgrade', 'upgrade_version': '7.6.0-2054'} Only cases in GROUPs 'N1QL' will be executed Logs will be stored at /data/workspace/debian-p0-query-vset00-00-upgrade_7.1_n1ql/logs/testrunner-24-Jan-29_11-40-24/test_1 ./testrunner -i /data/workspace/debian-p0-query-vset00-00-upgrade_7.1_n1ql/testexec.100814.ini -p gsi_type=plasma,nodes_init=4,initial_version=7.1.6-3955,upgrade_version=7.6.0-2054,GROUP=N1QL,get-cbcollect-info=True,get-cbcollect-info=True,bucket_storage=couchstore,sirius_url=http://172.23.120.103:4000 -t tuqquery.n1ql_upgrade.QueriesUpgradeTests.test_upgrade,standard_buckets=1,dgm_run=true,upgrade_type=offline,feature=n1ql,services_init=kv:n1ql:index-kv:n1ql:index-kv:n1ql:index-kv:n1ql:index,array_indexing=false,dataset=default,doc-per-day=1,GROUP=N1QL,gsi_type=plasma,nodes_init=4,initial_version=7.1.6-3955,upgrade_version=7.6.0-2054,get-cbcollect-info=True,bucket_storage=couchstore,sirius_url=http://172.23.120.103:4000 Test Input params: {'standard_buckets': '1', 'dgm_run': 'true', 'upgrade_type': 'offline', 'feature': 'n1ql', 'services_init': 'kv:n1ql:index-kv:n1ql:index-kv:n1ql:index-kv:n1ql:index', 'array_indexing': 'false', 'dataset': 'default', 'doc-per-day': '1', 'GROUP': 'N1QL', 'gsi_type': 'plasma', 'nodes_init': '4', 'initial_version': '7.1.6-3955', 'upgrade_version': '7.6.0-2054', 'get-cbcollect-info': 'True', 'bucket_storage': 'couchstore', 'sirius_url': 'http://172.23.120.103:4000', 'ini': '/data/workspace/debian-p0-query-vset00-00-upgrade_7.1_n1ql/testexec.100814.ini', 'cluster_name': 'testexec.100814', 'spec': 'py-n1ql-upgrade', 'conf_file': 'conf/tuq/py-n1ql-upgrade.conf', 'num_nodes': 4, 'case_number': 1, 'total_testcases': 1, 'last_case_fail': 'False', 'teardown_run': 'False', 'logs_folder': '/data/workspace/debian-p0-query-vset00-00-upgrade_7.1_n1ql/logs/testrunner-24-Jan-29_11-40-24/test_1'} /usr/local/lib/python3.7/site-packages/paramiko/transport.py:219: CryptographyDeprecationWarning: Blowfish has been deprecated "class": algorithms.Blowfish, pytests/security/SecretsMasterBase.py:65: DeprecationWarning: invalid escape sequence \. status, ipAddress = subprocess.getstatusoutput("ifconfig eth0 | grep -Eo 'inet (addr:)?([0-9]*\.){3}[0-9]*' | awk '{print $2}'") /data/workspace/debian-p0-query-vset00-00-upgrade_7.1_n1ql/scripts/install.py:1044: DeprecationWarning: invalid escape sequence \d if re.match('\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}', server.ip): pytests/tuqquery/flex_index_phase1.py:3: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3,and in 3.9 it will stop working from collections import Mapping, Sequence, Set, deque pytests/clitest/cli_base.py:209: DeprecationWarning: invalid escape sequence \ cygwin_format = "/cygdrive/c/Program\ Files" test_upgrade (tuqquery.n1ql_upgrade.QueriesUpgradeTests) ... 2024-01-29 11:40:25,662 - root - INFO - SSH Connecting to 172.23.106.112 with username:root, attempt#1 of 5 2024-01-29 11:40:25,853 - root - INFO - SSH Connected to 172.23.106.112 as root 2024-01-29 11:40:26,009 - root - INFO - os_distro: Ubuntu, os_version: debian 11, is_linux_distro: True 2024-01-29 11:40:26,307 - root - INFO - extract_remote_info-->distribution_type: Ubuntu, distribution_version: debian 11 2024-01-29 11:40:26,316 - root - INFO - -->is_ns_server_running? 2024-01-29 11:40:26,331 - root - INFO - Node version in cluster 7.1.6-3955-enterprise 2024-01-29 11:40:26,331 - root - INFO - ============== basetestcase setup was started for test #1 test_upgrade============== 2024-01-29 11:40:26,331 - root - INFO - ============== QueriesUpgradeTests tearDown has started ============== 2024-01-29 11:40:26,334 - root - INFO - ============== QueriesUpgradeTests tearDown has completed ============== 2024-01-29 11:40:26,334 - root - INFO - ============== QueryTests tearDown has started ============== 2024-01-29 11:40:26,334 - root - INFO - ============== QueryTests tearDown has completed ============== 2024-01-29 11:40:26,334 - root - INFO - ============== NewUpgradeBaseTest tearDown has started ============== 2024-01-29 11:40:26,342 - root - INFO - ------- Cluster statistics ------- 2024-01-29 11:40:26,342 - root - INFO - 172.23.106.112:8091 => {'services': ['fts', 'index', 'kv', 'n1ql'], 'cpu_utilization': 1.552146508574124, 'mem_free': 3413737472, 'mem_total': 4083224576, 'swap_mem_used': 9175040, 'swap_mem_total': 1027600384} 2024-01-29 11:40:26,342 - root - INFO - --- End of cluster statistics --- 2024-01-29 11:40:26,346 - root - INFO - SSH Connecting to 172.23.106.112 with username:root, attempt#1 of 5 2024-01-29 11:40:26,528 - root - INFO - SSH Connected to 172.23.106.112 as root 2024-01-29 11:40:26,685 - root - INFO - os_distro: Ubuntu, os_version: debian 11, is_linux_distro: True 2024-01-29 11:40:26,984 - root - INFO - extract_remote_info-->distribution_type: Ubuntu, distribution_version: debian 11 2024-01-29 11:40:29,877 - root - INFO - ============== basetestcase cleanup was started for test #1 test_upgrade ============== 2024-01-29 11:40:29,893 - root - INFO - Could not find any buckets for node 172.23.106.112, nothing to delete 2024-01-29 11:40:29,893 - root - INFO - Removing user 'clientuser'... 2024-01-29 11:40:29,900 - root - ERROR - DELETE http://172.23.106.112:8091/settings/rbac/users/local/clientuser body: headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==', 'Accept': '*/*'} error: 404 reason: unknown b'"User was not found."' auth: Administrator:password 2024-01-29 11:40:29,906 - root - INFO - b'"User was not found."' 2024-01-29 11:40:29,908 - root - INFO - -->is_ns_server_running? 2024-01-29 11:40:29,927 - root - INFO - waiting for ns_server @ 172.23.106.112:8091 2024-01-29 11:40:29,927 - root - INFO - -->is_ns_server_running? 2024-01-29 11:40:29,929 - root - INFO - ns_server @ 172.23.106.112:8091 is running 2024-01-29 11:40:29,929 - root - INFO - ============== basetestcase cleanup was finished for test #1 test_upgrade ============== 2024-01-29 11:40:29,929 - root - INFO - closing all ssh connections 2024-01-29 11:40:29,929 - root - INFO - closing all memcached connections Cluster instance shutdown with force 2024-01-29 11:40:29,934 - root - INFO - will install 7.1.6-3955 on ['172.23.106.112', '172.23.107.26', '172.23.104.102', '172.23.105.33'] 2024-01-29 11:40:29,936 - root - INFO - SSH Connecting to 172.23.106.112 with username:root, attempt#1 of 5 2024-01-29 11:40:29,937 - root - INFO - SSH Connecting to 172.23.107.26 with username:root, attempt#1 of 5 2024-01-29 11:40:29,939 - root - INFO - SSH Connecting to 172.23.105.33 with username:root, attempt#1 of 5 2024-01-29 11:40:29,941 - root - INFO - SSH Connecting to 172.23.104.102 with username:root, attempt#1 of 5 2024-01-29 11:40:30,015 - root - INFO - SSH Connected to 172.23.105.33 as root 2024-01-29 11:40:30,120 - root - INFO - SSH Connected to 172.23.106.112 as root 2024-01-29 11:40:30,127 - root - INFO - SSH Connected to 172.23.107.26 as root 2024-01-29 11:40:30,127 - root - INFO - SSH Connected to 172.23.104.102 as root 2024-01-29 11:40:30,185 - root - INFO - os_distro: Ubuntu, os_version: debian 10, is_linux_distro: True 2024-01-29 11:40:30,276 - root - INFO - os_distro: Ubuntu, os_version: debian 11, is_linux_distro: True 2024-01-29 11:40:30,284 - root - INFO - os_distro: Ubuntu, os_version: debian 11, is_linux_distro: True 2024-01-29 11:40:30,378 - root - INFO - os_distro: Ubuntu, os_version: debian 11, is_linux_distro: True 2024-01-29 11:40:30,438 - root - INFO - extract_remote_info-->distribution_type: Ubuntu, distribution_version: debian 10 2024-01-29 11:40:30,439 - root - INFO - 172.23.105.33 *****In couchbase uninstall**** 2024-01-29 11:40:30,439 - root - INFO - Ubuntu 2024-01-29 11:40:30,539 - root - INFO - running command.raw on 172.23.105.33: dpkg -r couchbase-server;dpkg --purge couchbase-server; 2024-01-29 11:40:30,576 - root - INFO - extract_remote_info-->distribution_type: Ubuntu, distribution_version: debian 11 2024-01-29 11:40:30,577 - root - INFO - 172.23.106.112 *****In couchbase uninstall**** 2024-01-29 11:40:30,577 - root - INFO - Ubuntu 2024-01-29 11:40:30,592 - root - INFO - extract_remote_info-->distribution_type: Ubuntu, distribution_version: debian 11 2024-01-29 11:40:30,592 - root - INFO - 172.23.107.26 *****In couchbase uninstall**** 2024-01-29 11:40:30,593 - root - INFO - Ubuntu 2024-01-29 11:40:30,668 - root - INFO - running command.raw on 172.23.106.112: dpkg -r couchbase-server;dpkg --purge couchbase-server; 2024-01-29 11:40:30,679 - root - INFO - extract_remote_info-->distribution_type: Ubuntu, distribution_version: debian 11 2024-01-29 11:40:30,680 - root - INFO - 172.23.104.102 *****In couchbase uninstall**** 2024-01-29 11:40:30,680 - root - INFO - Ubuntu 2024-01-29 11:40:30,730 - root - INFO - running command.raw on 172.23.107.26: dpkg -r couchbase-server;dpkg --purge couchbase-server; 2024-01-29 11:40:30,810 - root - INFO - running command.raw on 172.23.104.102: dpkg -r couchbase-server;dpkg --purge couchbase-server; 2024-01-29 11:40:34,024 - root - INFO - command executed with root but got an error ["dpkg: warning: while removing couchbase-server, directory '/opt/couchbase/var/lib/couchbase/logs' not empty so not removed", "dpkg: warning: while removing couchbase-server, directory '/opt/couchbase/var/lib/couchbase/config' not empty so not removed", "dpkg: warning: while removing couchbase-server, directory '/opt/couchbase/etc/couchdb' not empty so not removed"] ... 2024-01-29 11:40:34,024 - root - ERROR - dpkg: warning: while removing couchbase-server, directory '/opt/couchbase/var/lib/couchbase/logs' not empty so not removed 2024-01-29 11:40:34,024 - root - ERROR - dpkg: warning: while removing couchbase-server, directory '/opt/couchbase/var/lib/couchbase/config' not empty so not removed 2024-01-29 11:40:34,024 - root - ERROR - dpkg: warning: while removing couchbase-server, directory '/opt/couchbase/etc/couchdb' not empty so not removed 2024-01-29 11:40:34,160 - root - INFO - command executed with root but got an error ["dpkg: warning: while removing couchbase-server, directory '/opt/couchbase/var/lib/couchbase/logs' not empty so not removed", "dpkg: warning: while removing couchbase-server, directory '/opt/couchbase/var/lib/couchbase/config' not empty so not removed", "dpkg: warning: while removing couchbase-server, directory '/opt/couchbase/etc/couchdb' not empty so not removed"] ... 2024-01-29 11:40:34,161 - root - ERROR - dpkg: warning: while removing couchbase-server, directory '/opt/couchbase/var/lib/couchbase/logs' not empty so not removed 2024-01-29 11:40:34,161 - root - ERROR - dpkg: warning: while removing couchbase-server, directory '/opt/couchbase/var/lib/couchbase/config' not empty so not removed 2024-01-29 11:40:34,161 - root - ERROR - dpkg: warning: while removing couchbase-server, directory '/opt/couchbase/etc/couchdb' not empty so not removed 2024-01-29 11:40:34,516 - root - INFO - command executed with root but got an error ["dpkg: warning: while removing couchbase-server, directory '/opt/couchbase/var/lib/couchbase/logs' not empty so not removed", "dpkg: warning: while removing couchbase-server, directory '/opt/couchbase/var/lib/couchbase/config' not empty so not removed", "dpkg: warning: while removing couchbase-server, directory '/opt/couchbase/etc/couchdb' not empty so not removed"] ... 2024-01-29 11:40:34,516 - root - ERROR - dpkg: warning: while removing couchbase-server, directory '/opt/couchbase/var/lib/couchbase/logs' not empty so not removed 2024-01-29 11:40:34,516 - root - ERROR - dpkg: warning: while removing couchbase-server, directory '/opt/couchbase/var/lib/couchbase/config' not empty so not removed 2024-01-29 11:40:34,517 - root - ERROR - dpkg: warning: while removing couchbase-server, directory '/opt/couchbase/etc/couchdb' not empty so not removed 2024-01-29 11:40:34,872 - root - INFO - command executed with root but got an error ["dpkg: warning: while removing couchbase-server, directory '/opt/couchbase/var/lib/couchbase/logs' not empty so not removed", "dpkg: warning: while removing couchbase-server, directory '/opt/couchbase/var/lib/couchbase/config' not empty so not removed", "dpkg: warning: while removing couchbase-server, directory '/opt/couchbase/etc/couchdb' not empty so not removed"] ... 2024-01-29 11:40:34,873 - root - ERROR - dpkg: warning: while removing couchbase-server, directory '/opt/couchbase/var/lib/couchbase/logs' not empty so not removed 2024-01-29 11:40:34,873 - root - ERROR - dpkg: warning: while removing couchbase-server, directory '/opt/couchbase/var/lib/couchbase/config' not empty so not removed 2024-01-29 11:40:34,873 - root - ERROR - dpkg: warning: while removing couchbase-server, directory '/opt/couchbase/etc/couchdb' not empty so not removed 2024-01-29 11:40:35,204 - root - INFO - running command.raw on 172.23.105.33: ipcrm 2024-01-29 11:40:35,253 - root - INFO - command executed successfully with root 2024-01-29 11:40:35,613 - root - INFO - running command.raw on 172.23.106.112: ipcrm 2024-01-29 11:40:35,666 - root - INFO - command executed successfully with root thread uninstaller-thread-172.23.106.112 finished 2024-01-29 11:40:35,929 - root - INFO - running command.raw on 172.23.107.26: ipcrm 2024-01-29 11:40:35,984 - root - INFO - command executed successfully with root thread uninstaller-thread-172.23.107.26 finished 2024-01-29 11:40:36,325 - root - INFO - running command.raw on 172.23.104.102: ipcrm 2024-01-29 11:40:36,383 - root - INFO - command executed successfully with root thread uninstaller-thread-172.23.104.102 finished thread uninstaller-thread-172.23.105.33 finished 2024-01-29 11:40:36,385 - root - INFO - SSH Connecting to 172.23.106.112 with username:root, attempt#1 of 5 2024-01-29 11:40:36,568 - root - INFO - SSH Connected to 172.23.106.112 as root 2024-01-29 11:40:36,724 - root - INFO - os_distro: Ubuntu, os_version: debian 11, is_linux_distro: True 2024-01-29 11:40:37,018 - root - INFO - extract_remote_info-->distribution_type: Ubuntu, distribution_version: debian 11 2024-01-29 11:40:37,079 - root - INFO - SSH Connecting to 172.23.107.26 with username:root, attempt#1 of 5 2024-01-29 11:40:37,260 - root - INFO - SSH Connected to 172.23.107.26 as root 2024-01-29 11:40:37,422 - root - INFO - os_distro: Ubuntu, os_version: debian 11, is_linux_distro: True 2024-01-29 11:40:37,717 - root - INFO - extract_remote_info-->distribution_type: Ubuntu, distribution_version: debian 11 2024-01-29 11:40:37,731 - root - INFO - SSH Connecting to 172.23.104.102 with username:root, attempt#1 of 5 2024-01-29 11:40:37,914 - root - INFO - SSH Connected to 172.23.104.102 as root 2024-01-29 11:40:38,067 - root - INFO - os_distro: Ubuntu, os_version: debian 11, is_linux_distro: True 2024-01-29 11:40:38,359 - root - INFO - extract_remote_info-->distribution_type: Ubuntu, distribution_version: debian 11 2024-01-29 11:40:38,377 - root - INFO - SSH Connecting to 172.23.105.33 with username:root, attempt#1 of 5 2024-01-29 11:40:38,451 - root - INFO - SSH Connected to 172.23.105.33 as root 2024-01-29 11:40:38,576 - root - INFO - os_distro: Ubuntu, os_version: debian 10, is_linux_distro: True 2024-01-29 11:40:38,830 - root - INFO - extract_remote_info-->distribution_type: Ubuntu, distribution_version: debian 10 2024-01-29 11:40:38,838 - root - INFO - ********CouchbaseServerInstaller:install 2024-01-29 11:40:38,838 - root - INFO - ********CouchbaseServerInstaller:install 2024-01-29 11:40:38,839 - root - INFO - ********CouchbaseServerInstaller:install 2024-01-29 11:40:38,840 - root - INFO - SSH Connecting to 172.23.107.26 with username:root, attempt#1 of 5 2024-01-29 11:40:38,841 - root - INFO - ********CouchbaseServerInstaller:install 2024-01-29 11:40:38,841 - root - INFO - SSH Connecting to 172.23.105.33 with username:root, attempt#1 of 5 2024-01-29 11:40:38,842 - root - INFO - SSH Connecting to 172.23.106.112 with username:root, attempt#1 of 5 2024-01-29 11:40:38,844 - root - INFO - SSH Connecting to 172.23.104.102 with username:root, attempt#1 of 5 2024-01-29 11:40:38,915 - root - INFO - SSH Connected to 172.23.105.33 as root 2024-01-29 11:40:39,027 - root - INFO - SSH Connected to 172.23.104.102 as root 2024-01-29 11:40:39,030 - root - INFO - SSH Connected to 172.23.106.112 as root 2024-01-29 11:40:39,032 - root - INFO - SSH Connected to 172.23.107.26 as root 2024-01-29 11:40:39,038 - root - INFO - os_distro: Ubuntu, os_version: debian 10, is_linux_distro: True 2024-01-29 11:40:39,198 - root - INFO - os_distro: Ubuntu, os_version: debian 11, is_linux_distro: True 2024-01-29 11:40:39,200 - root - INFO - os_distro: Ubuntu, os_version: debian 11, is_linux_distro: True 2024-01-29 11:40:39,202 - root - INFO - os_distro: Ubuntu, os_version: debian 11, is_linux_distro: True 2024-01-29 11:40:39,298 - root - INFO - extract_remote_info-->distribution_type: Ubuntu, distribution_version: debian 10 --> remote server architecture_type..x86_64 *** OS version of this server 172.23.105.33 is debian 10 *** 2024-01-29 11:40:39,300 - root - INFO - SSH Connecting to 172.23.105.33 with username:root, attempt#1 of 5 2024-01-29 11:40:39,371 - root - INFO - SSH Connected to 172.23.105.33 as root 2024-01-29 11:40:39,492 - root - INFO - os_distro: Ubuntu, os_version: debian 10, is_linux_distro: True 2024-01-29 11:40:39,502 - root - INFO - extract_remote_info-->distribution_type: Ubuntu, distribution_version: debian 11 2024-01-29 11:40:39,504 - root - INFO - extract_remote_info-->distribution_type: Ubuntu, distribution_version: debian 11 --> remote server architecture_type..x86_64 *** OS version of this server 172.23.104.102 is debian 11 *** 2024-01-29 11:40:39,505 - root - INFO - SSH Connecting to 172.23.104.102 with username:root, attempt#1 of 5 --> remote server architecture_type..x86_64 *** OS version of this server 172.23.106.112 is debian 11 *** 2024-01-29 11:40:39,509 - root - INFO - SSH Connecting to 172.23.106.112 with username:root, attempt#1 of 5 2024-01-29 11:40:39,509 - root - INFO - extract_remote_info-->distribution_type: Ubuntu, distribution_version: debian 11 --> remote server architecture_type..x86_64 *** OS version of this server 172.23.107.26 is debian 11 *** 2024-01-29 11:40:39,512 - root - INFO - SSH Connecting to 172.23.107.26 with username:root, attempt#1 of 5 2024-01-29 11:40:39,686 - root - INFO - SSH Connected to 172.23.104.102 as root 2024-01-29 11:40:39,695 - root - INFO - SSH Connected to 172.23.106.112 as root 2024-01-29 11:40:39,696 - root - INFO - SSH Connected to 172.23.107.26 as root 2024-01-29 11:40:39,750 - root - INFO - extract_remote_info-->distribution_type: Ubuntu, distribution_version: debian 10 check if http://172.23.126.166/builds/releases/7.1.6/couchbase-server-enterprise_7.1.6-linux_amd64.deb is live 2024-01-29 11:40:39,751 - root - INFO - Trying to check is this url alive: http://172.23.126.166/builds/releases/7.1.6/couchbase-server-enterprise_7.1.6-linux_amd64.deb 2024-01-29 11:40:39,752 - root - INFO - This url http://172.23.126.166/builds/releases/7.1.6/couchbase-server-enterprise_7.1.6-linux_amd64.deb is live 2024-01-29 11:40:39,753 - root - INFO - SSH Connecting to 172.23.105.33 with username:root, attempt#1 of 5 2024-01-29 11:40:39,823 - root - INFO - SSH Connected to 172.23.105.33 as root 2024-01-29 11:40:39,853 - root - INFO - os_distro: Ubuntu, os_version: debian 11, is_linux_distro: True 2024-01-29 11:40:39,862 - root - INFO - os_distro: Ubuntu, os_version: debian 11, is_linux_distro: True 2024-01-29 11:40:39,864 - root - INFO - os_distro: Ubuntu, os_version: debian 11, is_linux_distro: True 2024-01-29 11:40:39,950 - root - INFO - os_distro: Ubuntu, os_version: debian 10, is_linux_distro: True 2024-01-29 11:40:40,154 - root - INFO - extract_remote_info-->distribution_type: Ubuntu, distribution_version: debian 11 check if http://172.23.126.166/builds/releases/7.1.6/couchbase-server-enterprise_7.1.6-linux_amd64.deb is live 2024-01-29 11:40:40,156 - root - INFO - Trying to check is this url alive: http://172.23.126.166/builds/releases/7.1.6/couchbase-server-enterprise_7.1.6-linux_amd64.deb 2024-01-29 11:40:40,160 - root - INFO - This url http://172.23.126.166/builds/releases/7.1.6/couchbase-server-enterprise_7.1.6-linux_amd64.deb is live 2024-01-29 11:40:40,161 - root - INFO - SSH Connecting to 172.23.104.102 with username:root, attempt#1 of 5 2024-01-29 11:40:40,167 - root - INFO - extract_remote_info-->distribution_type: Ubuntu, distribution_version: debian 11 check if http://172.23.126.166/builds/releases/7.1.6/couchbase-server-enterprise_7.1.6-linux_amd64.deb is live 2024-01-29 11:40:40,167 - root - INFO - Trying to check is this url alive: http://172.23.126.166/builds/releases/7.1.6/couchbase-server-enterprise_7.1.6-linux_amd64.deb 2024-01-29 11:40:40,168 - root - INFO - extract_remote_info-->distribution_type: Ubuntu, distribution_version: debian 11 check if http://172.23.126.166/builds/releases/7.1.6/couchbase-server-enterprise_7.1.6-linux_amd64.deb is live 2024-01-29 11:40:40,169 - root - INFO - Trying to check is this url alive: http://172.23.126.166/builds/releases/7.1.6/couchbase-server-enterprise_7.1.6-linux_amd64.deb 2024-01-29 11:40:40,170 - root - INFO - This url http://172.23.126.166/builds/releases/7.1.6/couchbase-server-enterprise_7.1.6-linux_amd64.deb is live 2024-01-29 11:40:40,171 - root - INFO - SSH Connecting to 172.23.107.26 with username:root, attempt#1 of 5 2024-01-29 11:40:40,172 - root - INFO - This url http://172.23.126.166/builds/releases/7.1.6/couchbase-server-enterprise_7.1.6-linux_amd64.deb is live 2024-01-29 11:40:40,173 - root - INFO - SSH Connecting to 172.23.106.112 with username:root, attempt#1 of 5 2024-01-29 11:40:40,203 - root - INFO - extract_remote_info-->distribution_type: Ubuntu, distribution_version: debian 10 Downloading the build...url : http://172.23.126.166/builds/releases/7.1.6/couchbase-server-enterprise_7.1.6-linux_amd64.deb name : couchbase-server-enterprise_7.1.6-linux_amd64.deb product : couchbase-server-enterprise product_version : 7.1.6-3955 os : deliverable_type : deb architecture_type : x86_64 2024-01-29 11:40:40,203 - root - INFO - running command.raw on 172.23.105.33: /sbin/iptables -F 2024-01-29 11:40:40,250 - root - INFO - command executed successfully with root 2024-01-29 11:40:40,250 - root - INFO - running command.raw on 172.23.105.33: /sbin/iptables -t nat -F 2024-01-29 11:40:40,300 - root - INFO - command executed successfully with root 2024-01-29 11:40:40,300 - root - INFO - running command.raw on 172.23.105.33: nft flush ruleset 2024-01-29 11:40:40,341 - root - INFO - SSH Connected to 172.23.104.102 as root 2024-01-29 11:40:40,349 - root - INFO - command executed with root but got an error ['bash: nft: command not found'] ... 2024-01-29 11:40:40,349 - root - INFO - SSH Connecting to 172.23.105.33 with username:root, attempt#1 of 5 2024-01-29 11:40:40,356 - root - INFO - SSH Connected to 172.23.107.26 as root 2024-01-29 11:40:40,358 - root - INFO - SSH Connected to 172.23.106.112 as root 2024-01-29 11:40:40,419 - root - INFO - SSH Connected to 172.23.105.33 as root 2024-01-29 11:40:40,503 - root - INFO - os_distro: Ubuntu, os_version: debian 11, is_linux_distro: True 2024-01-29 11:40:40,511 - root - INFO - os_distro: Ubuntu, os_version: debian 11, is_linux_distro: True 2024-01-29 11:40:40,516 - root - INFO - os_distro: Ubuntu, os_version: debian 11, is_linux_distro: True 2024-01-29 11:40:40,568 - root - INFO - running command.raw on 172.23.105.33: cd /tmp;wget -nc -q -O couchbase-server-enterprise_7.1.6-linux_amd64.deb http://172.23.126.166/builds/releases/7.1.6/couchbase-server-enterprise_7.1.6-linux_amd64.deb;cd /tmp;ls -lh 2024-01-29 11:40:40,622 - root - INFO - command executed successfully with root 2024-01-29 11:40:40,672 - root - INFO - *****install server *** 2024-01-29 11:40:40,673 - root - INFO - -->deliverable_type : deb 2024-01-29 11:40:40,673 - root - INFO - /tmp/couchbase-server-enterprise_7.1.6-linux_amd64.deb or /tmp/couchbase-server-enterprise 2024-01-29 11:40:40,723 - root - INFO - --> execute_command: end... 2024-01-29 11:40:40,805 - root - INFO - extract_remote_info-->distribution_type: Ubuntu, distribution_version: debian 11 Downloading the build...url : http://172.23.126.166/builds/releases/7.1.6/couchbase-server-enterprise_7.1.6-linux_amd64.deb name : couchbase-server-enterprise_7.1.6-linux_amd64.deb product : couchbase-server-enterprise product_version : 7.1.6-3955 os : deliverable_type : deb architecture_type : x86_64 2024-01-29 11:40:40,806 - root - INFO - running command.raw on 172.23.107.26: /sbin/iptables -F 2024-01-29 11:40:40,810 - root - INFO - extract_remote_info-->distribution_type: Ubuntu, distribution_version: debian 11 Downloading the build...url : http://172.23.126.166/builds/releases/7.1.6/couchbase-server-enterprise_7.1.6-linux_amd64.deb name : couchbase-server-enterprise_7.1.6-linux_amd64.deb product : couchbase-server-enterprise product_version : 7.1.6-3955 os : deliverable_type : deb architecture_type : x86_64 2024-01-29 11:40:40,810 - root - INFO - running command.raw on 172.23.106.112: /sbin/iptables -F 2024-01-29 11:40:40,816 - root - INFO - command executed successfully with root 2024-01-29 11:40:40,816 - root - INFO - running command.raw on 172.23.107.26: /sbin/iptables -t nat -F 2024-01-29 11:40:40,819 - root - INFO - extract_remote_info-->distribution_type: Ubuntu, distribution_version: debian 11 Downloading the build...url : http://172.23.126.166/builds/releases/7.1.6/couchbase-server-enterprise_7.1.6-linux_amd64.deb name : couchbase-server-enterprise_7.1.6-linux_amd64.deb product : couchbase-server-enterprise product_version : 7.1.6-3955 os : deliverable_type : deb architecture_type : x86_64 2024-01-29 11:40:40,820 - root - INFO - running command.raw on 172.23.104.102: /sbin/iptables -F 2024-01-29 11:40:40,821 - root - INFO - command executed successfully with root 2024-01-29 11:40:40,822 - root - INFO - running command.raw on 172.23.106.112: /sbin/iptables -t nat -F 2024-01-29 11:40:40,874 - root - INFO - command executed successfully with root 2024-01-29 11:40:40,875 - root - INFO - running command.raw on 172.23.104.102: /sbin/iptables -t nat -F 2024-01-29 11:40:40,876 - root - INFO - command executed successfully with root 2024-01-29 11:40:40,876 - root - INFO - command executed successfully with root 2024-01-29 11:40:40,877 - root - INFO - running command.raw on 172.23.107.26: nft flush ruleset 2024-01-29 11:40:40,878 - root - INFO - running command.raw on 172.23.106.112: nft flush ruleset 2024-01-29 11:40:40,931 - root - INFO - command executed successfully with root 2024-01-29 11:40:40,931 - root - INFO - running command.raw on 172.23.104.102: nft flush ruleset 2024-01-29 11:40:40,934 - root - INFO - command executed successfully with root 2024-01-29 11:40:40,935 - root - INFO - SSH Connecting to 172.23.106.112 with username:root, attempt#1 of 5 2024-01-29 11:40:40,942 - root - INFO - command executed successfully with root 2024-01-29 11:40:40,942 - root - INFO - SSH Connecting to 172.23.107.26 with username:root, attempt#1 of 5 2024-01-29 11:40:41,000 - root - INFO - command executed successfully with root 2024-01-29 11:40:41,000 - root - INFO - SSH Connecting to 172.23.104.102 with username:root, attempt#1 of 5 2024-01-29 11:40:41,110 - root - INFO - SSH Connected to 172.23.106.112 as root 2024-01-29 11:40:41,125 - root - INFO - SSH Connected to 172.23.107.26 as root 2024-01-29 11:40:41,181 - root - INFO - SSH Connected to 172.23.104.102 as root 2024-01-29 11:40:41,319 - root - INFO - running command.raw on 172.23.106.112: cd /tmp;wget -nc -q -O couchbase-server-enterprise_7.1.6-linux_amd64.deb http://172.23.126.166/builds/releases/7.1.6/couchbase-server-enterprise_7.1.6-linux_amd64.deb;cd /tmp;ls -lh 2024-01-29 11:40:41,332 - root - INFO - running command.raw on 172.23.107.26: cd /tmp;wget -nc -q -O couchbase-server-enterprise_7.1.6-linux_amd64.deb http://172.23.126.166/builds/releases/7.1.6/couchbase-server-enterprise_7.1.6-linux_amd64.deb;cd /tmp;ls -lh 2024-01-29 11:40:41,383 - root - INFO - command executed successfully with root 2024-01-29 11:40:41,389 - root - INFO - running command.raw on 172.23.104.102: cd /tmp;wget -nc -q -O couchbase-server-enterprise_7.1.6-linux_amd64.deb http://172.23.126.166/builds/releases/7.1.6/couchbase-server-enterprise_7.1.6-linux_amd64.deb;cd /tmp;ls -lh 2024-01-29 11:40:41,397 - root - INFO - command executed successfully with root 2024-01-29 11:40:41,449 - root - INFO - *****install server *** 2024-01-29 11:40:41,449 - root - INFO - -->deliverable_type : deb 2024-01-29 11:40:41,449 - root - INFO - /tmp/couchbase-server-enterprise_7.1.6-linux_amd64.deb or /tmp/couchbase-server-enterprise 2024-01-29 11:40:41,451 - root - INFO - command executed successfully with root 2024-01-29 11:40:41,458 - root - INFO - --> execute_command: end... 2024-01-29 11:40:41,461 - root - INFO - *****install server *** 2024-01-29 11:40:41,461 - root - INFO - -->deliverable_type : deb 2024-01-29 11:40:41,462 - root - INFO - /tmp/couchbase-server-enterprise_7.1.6-linux_amd64.deb or /tmp/couchbase-server-enterprise 2024-01-29 11:40:41,470 - root - INFO - --> execute_command: end... 2024-01-29 11:40:41,517 - root - INFO - *****install server *** 2024-01-29 11:40:41,518 - root - INFO - -->deliverable_type : deb 2024-01-29 11:40:41,518 - root - INFO - /tmp/couchbase-server-enterprise_7.1.6-linux_amd64.deb or /tmp/couchbase-server-enterprise 2024-01-29 11:40:41,526 - root - INFO - --> execute_command: end... 2024-01-29 11:41:20,806 - root - INFO - If couchbase server is running with this error. Go to log_command_output to add error mesg to bypass it. 2024-01-29 11:41:20,806 - root - INFO - Created symlink /etc/systemd/system/multi-user.target.wants/couchbase-server.service → /lib/systemd/system/couchbase-server.service. 2024-01-29 11:41:20,806 - root - INFO - This message comes only in debian8 and debian9 during installation. This can be ignored. 2024-01-29 11:41:20,841 - root - INFO - If couchbase server is running with this error. Go to log_command_output to add error mesg to bypass it. 2024-01-29 11:41:20,841 - root - INFO - Created symlink /etc/systemd/system/multi-user.target.wants/couchbase-server.service → /lib/systemd/system/couchbase-server.service. 2024-01-29 11:41:20,841 - root - INFO - This message comes only in debian8 and debian9 during installation. This can be ignored. 2024-01-29 11:41:20,854 - root - INFO - Checking if the directory /tmp exists or not. 2024-01-29 11:41:20,856 - root - ERROR - Directory at /tmp DOES exist. Fx returns True 2024-01-29 11:41:20,868 - root - INFO - Checking if the directory /tmp exists or not. 2024-01-29 11:41:20,869 - root - ERROR - Directory at /tmp DOES exist. Fx returns True 2024-01-29 11:41:20,925 - root - INFO - running command.raw on 172.23.106.112: sed -i 's/export PATH/export PATH\nexport CBFT_ENV_OPTIONS=bleveMaxResultWindow=10000000/' /opt/couchbase/bin/couchbase-server 2024-01-29 11:41:20,939 - root - INFO - running command.raw on 172.23.107.26: sed -i 's/export PATH/export PATH\nexport CBFT_ENV_OPTIONS=bleveMaxResultWindow=10000000/' /opt/couchbase/bin/couchbase-server 2024-01-29 11:41:20,984 - root - INFO - command executed successfully with root 2024-01-29 11:41:20,984 - root - INFO - Checking for process beam.smp on linux 2024-01-29 11:41:20,998 - root - INFO - command executed successfully with root 2024-01-29 11:41:20,999 - root - INFO - Checking for process beam.smp on linux 2024-01-29 11:41:21,066 - root - INFO - Starting couchbase server 2024-01-29 11:41:21,337 - root - INFO - Running systemd command on this server 2024-01-29 11:41:21,338 - root - INFO - running command.raw on 172.23.107.26: systemctl start couchbase-server.service 2024-01-29 11:41:21,340 - root - INFO - If couchbase server is running with this error. Go to log_command_output to add error mesg to bypass it. 2024-01-29 11:41:21,340 - root - INFO - Created symlink /etc/systemd/system/multi-user.target.wants/couchbase-server.service → /lib/systemd/system/couchbase-server.service. 2024-01-29 11:41:21,340 - root - INFO - This message comes only in debian8 and debian9 during installation. This can be ignored. 2024-01-29 11:41:21,355 - root - INFO - Checking if the directory /tmp exists or not. 2024-01-29 11:41:21,356 - root - INFO - command executed successfully with root 2024-01-29 11:41:21,356 - root - INFO - 172.23.107.26:sleep for 5 secs. waiting for couchbase server to come up ... 2024-01-29 11:41:21,364 - root - ERROR - Directory at /tmp DOES exist. Fx returns True 2024-01-29 11:41:21,418 - root - INFO - Starting couchbase server 2024-01-29 11:41:21,441 - root - INFO - running command.raw on 172.23.104.102: sed -i 's/export PATH/export PATH\nexport CBFT_ENV_OPTIONS=bleveMaxResultWindow=10000000/' /opt/couchbase/bin/couchbase-server 2024-01-29 11:41:21,496 - root - INFO - command executed successfully with root 2024-01-29 11:41:21,497 - root - INFO - Checking for process beam.smp on linux 2024-01-29 11:41:21,509 - root - INFO - Running systemd command on this server 2024-01-29 11:41:21,510 - root - INFO - running command.raw on 172.23.106.112: systemctl start couchbase-server.service 2024-01-29 11:41:21,523 - root - INFO - command executed successfully with root 2024-01-29 11:41:21,524 - root - INFO - 172.23.106.112:sleep for 5 secs. waiting for couchbase server to come up ... 2024-01-29 11:41:21,563 - root - INFO - Starting couchbase server 2024-01-29 11:41:21,654 - root - INFO - Running systemd command on this server 2024-01-29 11:41:21,654 - root - INFO - running command.raw on 172.23.104.102: systemctl start couchbase-server.service 2024-01-29 11:41:21,674 - root - INFO - command executed successfully with root 2024-01-29 11:41:21,674 - root - INFO - 172.23.104.102:sleep for 5 secs. waiting for couchbase server to come up ... 2024-01-29 11:41:26,361 - root - INFO - running command.raw on 172.23.107.26: systemctl status couchbase-server.service | grep ExecStop=/opt/couchbase/bin/couchbase-server 2024-01-29 11:41:26,410 - root - INFO - command executed successfully with root 2024-01-29 11:41:26,410 - root - INFO - Couchbase server status: [] 2024-01-29 11:41:26,410 - root - INFO - Checking for process beam.smp on linux 2024-01-29 11:41:26,487 - root - INFO - process beam.smp is running on 172.23.107.26: with pid 741906 2024-01-29 11:41:26,526 - root - INFO - running command.raw on 172.23.106.112: systemctl status couchbase-server.service | grep ExecStop=/opt/couchbase/bin/couchbase-server 2024-01-29 11:41:26,535 - root - INFO - wait 30 seconds for Couchbase server to start 2024-01-29 11:41:26,564 - root - INFO - command executed successfully with root 2024-01-29 11:41:26,565 - root - INFO - Couchbase server status: [] 2024-01-29 11:41:26,565 - root - INFO - Checking for process beam.smp on linux 2024-01-29 11:41:26,649 - root - INFO - process beam.smp is running on 172.23.106.112: with pid 879267 2024-01-29 11:41:26,679 - root - INFO - running command.raw on 172.23.104.102: systemctl status couchbase-server.service | grep ExecStop=/opt/couchbase/bin/couchbase-server 2024-01-29 11:41:26,703 - root - INFO - wait 30 seconds for Couchbase server to start 2024-01-29 11:41:26,725 - root - INFO - command executed successfully with root 2024-01-29 11:41:26,725 - root - INFO - Couchbase server status: [] 2024-01-29 11:41:26,725 - root - INFO - Checking for process beam.smp on linux 2024-01-29 11:41:26,790 - root - INFO - process beam.smp is running on 172.23.104.102: with pid 1270509 2024-01-29 11:41:26,848 - root - INFO - wait 30 seconds for Couchbase server to start 2024-01-29 11:41:27,558 - root - INFO - If couchbase server is running with this error. Go to log_command_output to add error mesg to bypass it. 2024-01-29 11:41:27,558 - root - INFO - Created symlink /etc/systemd/system/multi-user.target.wants/couchbase-server.service → /lib/systemd/system/couchbase-server.service. 2024-01-29 11:41:27,558 - root - INFO - This message comes only in debian8 and debian9 during installation. This can be ignored. 2024-01-29 11:41:27,606 - root - INFO - Checking if the directory /tmp exists or not. 2024-01-29 11:41:27,607 - root - ERROR - Directory at /tmp DOES exist. Fx returns True 2024-01-29 11:41:27,670 - root - INFO - running command.raw on 172.23.105.33: sed -i 's/export PATH/export PATH\nexport CBFT_ENV_OPTIONS=bleveMaxResultWindow=10000000/' /opt/couchbase/bin/couchbase-server 2024-01-29 11:41:27,719 - root - INFO - command executed successfully with root 2024-01-29 11:41:27,719 - root - INFO - Checking for process beam.smp on linux 2024-01-29 11:41:27,772 - root - INFO - Starting couchbase server 2024-01-29 11:41:27,840 - root - INFO - Running systemd command on this server 2024-01-29 11:41:27,840 - root - INFO - running command.raw on 172.23.105.33: systemctl start couchbase-server.service 2024-01-29 11:41:27,851 - root - INFO - command executed successfully with root 2024-01-29 11:41:27,852 - root - INFO - 172.23.105.33:sleep for 5 secs. waiting for couchbase server to come up ... 2024-01-29 11:41:32,856 - root - INFO - running command.raw on 172.23.105.33: systemctl status couchbase-server.service | grep ExecStop=/opt/couchbase/bin/couchbase-server 2024-01-29 11:41:32,871 - root - INFO - command executed successfully with root 2024-01-29 11:41:32,871 - root - INFO - Couchbase server status: [] 2024-01-29 11:41:32,871 - root - INFO - Checking for process beam.smp on linux 2024-01-29 11:41:32,927 - root - INFO - process beam.smp is running on 172.23.105.33: with pid 31724 2024-01-29 11:41:32,976 - root - INFO - wait 30 seconds for Couchbase server to start thread installer-thread-172.23.106.112 finished thread installer-thread-172.23.107.26 finished thread installer-thread-172.23.104.102 finished thread installer-thread-172.23.105.33 finished 2024-01-29 11:42:03,008 - root - INFO - SSH Connecting to 172.23.106.112 with username:root, attempt#1 of 5 2024-01-29 11:42:03,011 - root - INFO - SSH Connecting to 172.23.104.102 with username:root, attempt#1 of 5 2024-01-29 11:42:03,012 - root - INFO - SSH Connecting to 172.23.107.26 with username:root, attempt#1 of 5 2024-01-29 11:42:03,013 - root - INFO - SSH Connecting to 172.23.105.33 with username:root, attempt#1 of 5 2024-01-29 11:42:03,087 - root - INFO - SSH Connected to 172.23.105.33 as root 2024-01-29 11:42:03,191 - root - INFO - SSH Connected to 172.23.107.26 as root 2024-01-29 11:42:03,194 - root - INFO - SSH Connected to 172.23.104.102 as root 2024-01-29 11:42:03,195 - root - INFO - SSH Connected to 172.23.106.112 as root 2024-01-29 11:42:03,210 - root - INFO - os_distro: Ubuntu, os_version: debian 10, is_linux_distro: True 2024-01-29 11:42:03,351 - root - INFO - os_distro: Ubuntu, os_version: debian 11, is_linux_distro: True 2024-01-29 11:42:03,356 - root - INFO - os_distro: Ubuntu, os_version: debian 11, is_linux_distro: True 2024-01-29 11:42:03,361 - root - INFO - os_distro: Ubuntu, os_version: debian 11, is_linux_distro: True 2024-01-29 11:42:03,467 - root - INFO - extract_remote_info-->distribution_type: Ubuntu, distribution_version: debian 10 2024-01-29 11:42:03,521 - root - INFO - 172.23.105.33 **** The linux version file /opt/couchbase/ VERSION.txt exists 2024-01-29 11:42:03,521 - root - INFO - creating RestConnection(ip:172.23.105.33 port:8091 ssh_username:root) and the server type is 2024-01-29 11:42:03,524 - root - ERROR - GET http://172.23.105.33:8091/pools/default body: headers: {'Content-Type': 'application/json', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==', 'Accept': '*/*'} error: 404 reason: unknown b'"unknown pool"' auth: Administrator:password http://172.23.105.33:8091/pools/default with status False: unknown pool 2024-01-29 11:42:03,524 - root - INFO - Done creating rest connection 2024-01-29 11:42:03,653 - root - INFO - extract_remote_info-->distribution_type: Ubuntu, distribution_version: debian 11 2024-01-29 11:42:03,654 - root - INFO - extract_remote_info-->distribution_type: Ubuntu, distribution_version: debian 11 2024-01-29 11:42:03,662 - root - INFO - extract_remote_info-->distribution_type: Ubuntu, distribution_version: debian 11 2024-01-29 11:42:03,714 - root - INFO - 172.23.107.26 **** The linux version file /opt/couchbase/ VERSION.txt exists 2024-01-29 11:42:03,714 - root - INFO - creating RestConnection(ip:172.23.107.26 port:8091 ssh_username:root) and the server type is 2024-01-29 11:42:03,719 - root - INFO - 172.23.104.102 **** The linux version file /opt/couchbase/ VERSION.txt exists 2024-01-29 11:42:03,720 - root - ERROR - GET http://172.23.107.26:8091/pools/default body: headers: {'Content-Type': 'application/json', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==', 'Accept': '*/*'} error: 404 reason: unknown b'"unknown pool"' auth: Administrator:password 2024-01-29 11:42:03,720 - root - INFO - creating RestConnection(ip:172.23.104.102 port:8091 ssh_username:root) and the server type is http://172.23.107.26:8091/pools/default with status False: unknown pool 2024-01-29 11:42:03,722 - root - INFO - Done creating rest connection 2024-01-29 11:42:03,725 - root - INFO - 172.23.106.112 **** The linux version file /opt/couchbase/ VERSION.txt exists 2024-01-29 11:42:03,725 - root - INFO - creating RestConnection(ip:172.23.106.112 port:8091 ssh_username:root) and the server type is 2024-01-29 11:42:03,726 - root - ERROR - GET http://172.23.104.102:8091/pools/default body: headers: {'Content-Type': 'application/json', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==', 'Accept': '*/*'} error: 404 reason: unknown b'"unknown pool"' auth: Administrator:password http://172.23.104.102:8091/pools/default with status False: unknown pool 2024-01-29 11:42:03,727 - root - INFO - Done creating rest connection 2024-01-29 11:42:03,728 - root - ERROR - GET http://172.23.106.112:8091/pools/default body: headers: {'Content-Type': 'application/json', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==', 'Accept': '*/*'} error: 404 reason: unknown b'"unknown pool"' auth: Administrator:password http://172.23.106.112:8091/pools/default with status False: unknown pool 2024-01-29 11:42:03,729 - root - INFO - Done creating rest connection 2024-01-29 11:42:07,539 - root - INFO - --> rest.get_nodes_self().mcdMemoryReserved 2024-01-29 11:42:07,539 - root - INFO - --> info.mcdMemoryReserved=3300,CLUSTER_QUOTA_RATIO=0.67 2024-01-29 11:42:07,539 - root - INFO - quota for index service will be 256 MB 2024-01-29 11:42:07,539 - root - INFO - set index quota to node 172.23.105.33 2024-01-29 11:42:07,540 - root - INFO - pools/default params : indexMemoryQuota=256 2024-01-29 11:42:07,546 - root - INFO - quota for fts service will be 512 MB 2024-01-29 11:42:07,546 - root - INFO - set both index and fts quota at node 172.23.105.33 2024-01-29 11:42:07,546 - root - INFO - pools/default params : ftsMemoryQuota=512 2024-01-29 11:42:07,550 - root - INFO - quota for kv: 1442 MB 2024-01-29 11:42:07,550 - root - INFO - pools/default params : memoryQuota=1442 2024-01-29 11:42:07,553 - root - INFO - --> init_node_services(Administrator,password,127.0.0.1,8091,['kv', 'index', 'n1ql', 'fts']) 2024-01-29 11:42:07,553 - root - INFO - node/controller/setupServices params on 172.23.105.33: 8091:hostname=127.0.0.1%3A8091&user=Administrator&password=password&services=kv%2Cindex%2Cn1ql%2Cfts 2024-01-29 11:42:07,582 - root - INFO - settings/indexes params : storageMode=plasma 2024-01-29 11:42:07,588 - root - INFO - --> in init_cluster...Administrator,password,8091 2024-01-29 11:42:07,588 - root - INFO - settings/web params on 172.23.105.33:8091:port=8091&username=Administrator&password=password 2024-01-29 11:42:07,644 - root - INFO - --> status:True 2024-01-29 11:42:07,682 - root - INFO - -->is_ns_server_running? 2024-01-29 11:42:07,691 - root - INFO - Node version in cluster 7.1.6-3955-enterprise 2024-01-29 11:42:07,691 - root - INFO - ***** set NS_SERVER_CBAUTH env in linux ***** 2024-01-29 11:42:07,747 - root - INFO - --> rest.get_nodes_self().mcdMemoryReserved 2024-01-29 11:42:07,748 - root - INFO - --> info.mcdMemoryReserved=3115,CLUSTER_QUOTA_RATIO=0.67 2024-01-29 11:42:07,748 - root - INFO - quota for index service will be 256 MB 2024-01-29 11:42:07,748 - root - INFO - set index quota to node 172.23.107.26 2024-01-29 11:42:07,748 - root - INFO - pools/default params : indexMemoryQuota=256 2024-01-29 11:42:07,752 - root - INFO - --> rest.get_nodes_self().mcdMemoryReserved 2024-01-29 11:42:07,752 - root - INFO - --> info.mcdMemoryReserved=3115,CLUSTER_QUOTA_RATIO=0.67 2024-01-29 11:42:07,753 - root - INFO - quota for index service will be 256 MB 2024-01-29 11:42:07,753 - root - INFO - set index quota to node 172.23.104.102 2024-01-29 11:42:07,753 - root - INFO - pools/default params : indexMemoryQuota=256 2024-01-29 11:42:07,756 - root - INFO - --> rest.get_nodes_self().mcdMemoryReserved 2024-01-29 11:42:07,756 - root - INFO - --> info.mcdMemoryReserved=3115,CLUSTER_QUOTA_RATIO=0.67 2024-01-29 11:42:07,756 - root - INFO - quota for index service will be 256 MB 2024-01-29 11:42:07,756 - root - INFO - set index quota to node 172.23.106.112 2024-01-29 11:42:07,756 - root - INFO - pools/default params : indexMemoryQuota=256 2024-01-29 11:42:07,757 - root - INFO - quota for fts service will be 512 MB 2024-01-29 11:42:07,758 - root - INFO - set both index and fts quota at node 172.23.107.26 2024-01-29 11:42:07,758 - root - INFO - pools/default params : ftsMemoryQuota=512 2024-01-29 11:42:07,763 - root - INFO - quota for kv: 1318 MB 2024-01-29 11:42:07,763 - root - INFO - pools/default params : memoryQuota=1318 2024-01-29 11:42:07,764 - root - INFO - quota for fts service will be 512 MB 2024-01-29 11:42:07,764 - root - INFO - set both index and fts quota at node 172.23.106.112 2024-01-29 11:42:07,764 - root - INFO - pools/default params : ftsMemoryQuota=512 2024-01-29 11:42:07,765 - root - INFO - quota for fts service will be 512 MB 2024-01-29 11:42:07,765 - root - INFO - set both index and fts quota at node 172.23.104.102 2024-01-29 11:42:07,765 - root - INFO - pools/default params : ftsMemoryQuota=512 2024-01-29 11:42:07,769 - root - INFO - quota for kv: 1318 MB 2024-01-29 11:42:07,769 - root - INFO - pools/default params : memoryQuota=1318 2024-01-29 11:42:07,770 - root - INFO - --> init_node_services(Administrator,password,127.0.0.1,8091,['kv', 'index', 'n1ql', 'fts']) 2024-01-29 11:42:07,770 - root - INFO - node/controller/setupServices params on 172.23.107.26: 8091:hostname=127.0.0.1%3A8091&user=Administrator&password=password&services=kv%2Cindex%2Cn1ql%2Cfts 2024-01-29 11:42:07,771 - root - INFO - quota for kv: 1318 MB 2024-01-29 11:42:07,771 - root - INFO - pools/default params : memoryQuota=1318 2024-01-29 11:42:07,776 - root - INFO - --> init_node_services(Administrator,password,127.0.0.1,8091,['kv', 'index', 'n1ql', 'fts']) 2024-01-29 11:42:07,776 - root - INFO - node/controller/setupServices params on 172.23.104.102: 8091:hostname=127.0.0.1%3A8091&user=Administrator&password=password&services=kv%2Cindex%2Cn1ql%2Cfts 2024-01-29 11:42:07,777 - root - INFO - --> init_node_services(Administrator,password,127.0.0.1,8091,['kv', 'index', 'n1ql', 'fts']) 2024-01-29 11:42:07,777 - root - INFO - node/controller/setupServices params on 172.23.106.112: 8091:hostname=127.0.0.1%3A8091&user=Administrator&password=password&services=kv%2Cindex%2Cn1ql%2Cfts 2024-01-29 11:42:07,805 - root - INFO - settings/indexes params : storageMode=plasma 2024-01-29 11:42:07,812 - root - INFO - settings/indexes params : storageMode=plasma 2024-01-29 11:42:07,813 - root - INFO - settings/indexes params : storageMode=plasma 2024-01-29 11:42:07,816 - root - INFO - --> in init_cluster...Administrator,password,8091 2024-01-29 11:42:07,816 - root - INFO - settings/web params on 172.23.107.26:8091:port=8091&username=Administrator&password=password 2024-01-29 11:42:07,822 - root - INFO - --> in init_cluster...Administrator,password,8091 2024-01-29 11:42:07,822 - root - INFO - settings/web params on 172.23.104.102:8091:port=8091&username=Administrator&password=password 2024-01-29 11:42:07,823 - root - INFO - --> in init_cluster...Administrator,password,8091 2024-01-29 11:42:07,823 - root - INFO - settings/web params on 172.23.106.112:8091:port=8091&username=Administrator&password=password 2024-01-29 11:42:07,866 - root - INFO - --> status:True 2024-01-29 11:42:07,877 - root - INFO - --> status:True 2024-01-29 11:42:07,884 - root - INFO - -->is_ns_server_running? 2024-01-29 11:42:07,888 - root - INFO - --> status:True 2024-01-29 11:42:07,896 - root - INFO - -->is_ns_server_running? 2024-01-29 11:42:07,897 - root - INFO - Node version in cluster 7.1.6-3955-enterprise 2024-01-29 11:42:07,897 - root - INFO - ***** set NS_SERVER_CBAUTH env in linux ***** 2024-01-29 11:42:07,902 - root - INFO - check if man installed on vm? 2024-01-29 11:42:07,903 - root - INFO - man page does not install man page on vm 172.23.105.33 Let do install it now 2024-01-29 11:42:07,903 - root - INFO - running command.raw on 172.23.105.33: yum install -y man 2024-01-29 11:42:07,906 - root - INFO - Node version in cluster 7.1.6-3955-enterprise 2024-01-29 11:42:07,906 - root - INFO - ***** set NS_SERVER_CBAUTH env in linux ***** 2024-01-29 11:42:07,912 - root - INFO - -->is_ns_server_running? 2024-01-29 11:42:07,922 - root - INFO - Node version in cluster 7.1.6-3955-enterprise 2024-01-29 11:42:07,922 - root - INFO - ***** set NS_SERVER_CBAUTH env in linux ***** 2024-01-29 11:42:07,952 - root - INFO - command executed with root but got an error ['bash: yum: command not found'] ... 2024-01-29 11:42:07,952 - root - INFO - ---> Run command unzip to check if it is ready on server 172.23.105.33 2024-01-29 11:42:07,952 - root - INFO - running command.raw on 172.23.105.33: unzip 2024-01-29 11:42:08,015 - root - INFO - command executed successfully with root 2024-01-29 11:42:08,015 - root - INFO - Check if ntp is installed 2024-01-29 11:42:08,016 - root - INFO - This OS version debian 10 2024-01-29 11:42:08,016 - root - INFO - running command.raw on 172.23.105.33: systemctl status ntp 2024-01-29 11:42:08,072 - root - INFO - command executed successfully with root 2024-01-29 11:42:08,072 - root - INFO - running command.raw on 172.23.105.33: date 2024-01-29 11:42:08,104 - root - INFO - check if man installed on vm? 2024-01-29 11:42:08,105 - root - INFO - man page does not install man page on vm 172.23.107.26 Let do install it now 2024-01-29 11:42:08,105 - root - INFO - running command.raw on 172.23.107.26: yum install -y man 2024-01-29 11:42:08,123 - root - INFO - command executed successfully with root 2024-01-29 11:42:08,125 - root - INFO - check if man installed on vm? 2024-01-29 11:42:08,125 - root - INFO - man page does not install man page on vm 172.23.104.102 Let do install it now 2024-01-29 11:42:08,125 - root - INFO - running command.raw on 172.23.104.102: yum install -y man 2024-01-29 11:42:08,155 - root - INFO - check if man installed on vm? 2024-01-29 11:42:08,156 - root - INFO - man page does not install man page on vm 172.23.106.112 Let do install it now 2024-01-29 11:42:08,156 - root - INFO - running command.raw on 172.23.106.112: yum install -y man 2024-01-29 11:42:08,162 - root - INFO - command executed with root but got an error ['bash: line 1: yum: command not found'] ... 2024-01-29 11:42:08,163 - root - INFO - ---> Run command unzip to check if it is ready on server 172.23.107.26 2024-01-29 11:42:08,163 - root - INFO - running command.raw on 172.23.107.26: unzip 2024-01-29 11:42:08,173 - root - INFO - ['Mon 29 Jan 2024 11:42:08 AM PST'] IP: 172.23.105.33 2024-01-29 11:42:08,175 - root - INFO - command executed with root but got an error ['bash: line 1: yum: command not found'] ... 2024-01-29 11:42:08,175 - root - INFO - ---> Run command unzip to check if it is ready on server 172.23.104.102 2024-01-29 11:42:08,175 - root - INFO - running command.raw on 172.23.104.102: unzip 2024-01-29 11:42:08,212 - root - INFO - command executed with root but got an error ['bash: line 1: yum: command not found'] ... 2024-01-29 11:42:08,212 - root - INFO - ---> Run command unzip to check if it is ready on server 172.23.106.112 2024-01-29 11:42:08,212 - root - INFO - running command.raw on 172.23.106.112: unzip 2024-01-29 11:42:08,220 - root - INFO - command executed with root but got an error ['bash: line 1: unzip: command not found'] ... 2024-01-29 11:42:08,221 - root - INFO - Check if ntp is installed 2024-01-29 11:42:08,221 - root - INFO - This OS version debian 11 2024-01-29 11:42:08,221 - root - INFO - will add install in other os later, no set do install 2024-01-29 11:42:08,230 - root - INFO - command executed with root but got an error ['bash: line 1: unzip: command not found'] ... 2024-01-29 11:42:08,230 - root - INFO - Check if ntp is installed 2024-01-29 11:42:08,230 - root - INFO - This OS version debian 11 2024-01-29 11:42:08,231 - root - INFO - will add install in other os later, no set do install 2024-01-29 11:42:08,265 - root - INFO - command executed with root but got an error ['bash: line 1: unzip: command not found'] ... 2024-01-29 11:42:08,265 - root - INFO - Check if ntp is installed 2024-01-29 11:42:08,265 - root - INFO - This OS version debian 11 2024-01-29 11:42:08,266 - root - INFO - will add install in other os later, no set do install 2024-01-29 11:42:08,282 - root - INFO - ['Mon 29 Jan 2024 11:42:08 AM PST'] IP: 172.23.104.102 2024-01-29 11:42:08,287 - root - INFO - ['Mon 29 Jan 2024 11:42:08 AM PST'] IP: 172.23.107.26 2024-01-29 11:42:08,314 - root - INFO - ['Tue 30 Jan 2024 08:42:08 AM +13'] IP: 172.23.106.112 thread initializer-thread-172.23.106.112 finished thread initializer-thread-172.23.107.26 finished thread initializer-thread-172.23.104.102 finished thread initializer-thread-172.23.105.33 finished 2024-01-29 11:42:08,315 - root - INFO - SSH Connecting to 172.23.106.112 with username:root, attempt#1 of 5 2024-01-29 11:42:08,482 - root - INFO - SSH Connected to 172.23.106.112 as root 2024-01-29 11:42:08,626 - root - INFO - os_distro: Ubuntu, os_version: debian 11, is_linux_distro: True 2024-01-29 11:42:08,914 - root - INFO - extract_remote_info-->distribution_type: Ubuntu, distribution_version: debian 11 2024-01-29 11:42:08,916 - root - INFO - SSH Connecting to 172.23.106.112 with username:root, attempt#1 of 5 2024-01-29 11:42:09,093 - root - INFO - SSH Connected to 172.23.106.112 as root 2024-01-29 11:42:09,246 - root - INFO - os_distro: Ubuntu, os_version: debian 11, is_linux_distro: True 2024-01-29 11:42:09,547 - root - INFO - extract_remote_info-->distribution_type: Ubuntu, distribution_version: debian 11 2024-01-29 11:42:09,610 - root - INFO - 172.23.106.112 **** The linux version file /opt/couchbase/ VERSION.txt exists 2024-01-29 11:42:09,610 - root - INFO - sleep for 5 secs. sleep 5 seconds to let cb up completely ... 2024-01-29 11:42:14,624 - root - INFO - -->is_ns_server_running? 2024-01-29 11:42:14,630 - root - INFO - SSH Connecting to 172.23.107.26 with username:root, attempt#1 of 5 2024-01-29 11:42:14,816 - root - INFO - SSH Connected to 172.23.107.26 as root 2024-01-29 11:42:14,972 - root - INFO - os_distro: Ubuntu, os_version: debian 11, is_linux_distro: True 2024-01-29 11:42:15,265 - root - INFO - extract_remote_info-->distribution_type: Ubuntu, distribution_version: debian 11 2024-01-29 11:42:15,281 - root - INFO - 172.23.107.26 **** The linux version file /opt/couchbase/ VERSION.txt exists 2024-01-29 11:42:15,281 - root - INFO - sleep for 5 secs. sleep 5 seconds to let cb up completely ... 2024-01-29 11:42:20,291 - root - INFO - -->is_ns_server_running? 2024-01-29 11:42:20,295 - root - INFO - SSH Connecting to 172.23.104.102 with username:root, attempt#1 of 5 2024-01-29 11:42:20,480 - root - INFO - SSH Connected to 172.23.104.102 as root 2024-01-29 11:42:20,632 - root - INFO - os_distro: Ubuntu, os_version: debian 11, is_linux_distro: True 2024-01-29 11:42:20,919 - root - INFO - extract_remote_info-->distribution_type: Ubuntu, distribution_version: debian 11 2024-01-29 11:42:20,940 - root - INFO - 172.23.104.102 **** The linux version file /opt/couchbase/ VERSION.txt exists 2024-01-29 11:42:20,940 - root - INFO - sleep for 5 secs. sleep 5 seconds to let cb up completely ... 2024-01-29 11:42:25,953 - root - INFO - -->is_ns_server_running? 2024-01-29 11:42:25,960 - root - INFO - SSH Connecting to 172.23.105.33 with username:root, attempt#1 of 5 2024-01-29 11:42:26,035 - root - INFO - SSH Connected to 172.23.105.33 as root 2024-01-29 11:42:26,158 - root - INFO - os_distro: Ubuntu, os_version: debian 10, is_linux_distro: True 2024-01-29 11:42:26,414 - root - INFO - extract_remote_info-->distribution_type: Ubuntu, distribution_version: debian 10 2024-01-29 11:42:26,464 - root - INFO - 172.23.105.33 **** The linux version file /opt/couchbase/ VERSION.txt exists 2024-01-29 11:42:26,465 - root - INFO - sleep for 5 secs. sleep 5 seconds to let cb up completely ... 2024-01-29 11:42:31,474 - root - INFO - -->is_ns_server_running? 2024-01-29 11:42:31,477 - root - INFO - sleep for 20 secs. sleep 20 seconds before run next test ... 2024-01-29 11:42:51,496 - root - INFO - ============== NewUpgradeBaseTest tearDown has completed ============== 2024-01-29 11:42:51,498 - root - INFO - initializing cluster 2024-01-29 11:42:52,509 - root - INFO - server: ip:172.23.106.112 port:8091 ssh_username:root, nodes/self 2024-01-29 11:42:52,514 - root - INFO - {'uptime': '86', 'memoryTotal': 4083224576, 'memoryFree': 3441524736, 'mcdMemoryReserved': 3115, 'mcdMemoryAllocated': 3115, 'status': 'healthy', 'hostname': '172.23.106.112:8091', 'clusterCompatibility': 458753, 'clusterMembership': 'active', 'recoveryType': 'none', 'version': '7.1.6-3955-enterprise', 'os': 'x86_64-pc-linux-gnu', 'ports': [], 'availableStorage': [], 'storage': [], 'memoryQuota': 1318, 'memcached': 11210, 'id': 'ns_1@cb.local', 'ip': '172.23.106.112', 'internal_ip': '', 'rest_username': '', 'rest_password': '', 'port': '8091', 'services': ['fts', 'index', 'kv', 'n1ql'], 'storageTotalRam': 3894, 'curr_items': 0} 2024-01-29 11:42:52,517 - root - INFO - quota for index service will be 256 MB 2024-01-29 11:42:52,517 - root - INFO - set index quota to node 172.23.106.112 2024-01-29 11:42:52,517 - root - INFO - pools/default params : indexMemoryQuota=256 2024-01-29 11:42:52,531 - root - INFO - pools/default params : memoryQuota=1831 2024-01-29 11:42:52,536 - root - INFO - --> init_node_services(Administrator,password,172.23.106.112,8091,['kv', 'n1ql', 'index']) 2024-01-29 11:42:52,536 - root - INFO - node/controller/setupServices params on 172.23.106.112: 8091:hostname=172.23.106.112&user=Administrator&password=password&services=kv%2Cn1ql%2Cindex 2024-01-29 11:42:52,539 - root - ERROR - POST http://172.23.106.112:8091/node/controller/setupServices body: hostname=172.23.106.112&user=Administrator&password=password&services=kv%2Cn1ql%2Cindex headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==', 'Accept': '*/*'} error: 400 reason: unknown b'{"errors":{"services":"cannot change node services after cluster is provisioned"}}' auth: Administrator:password 2024-01-29 11:42:52,541 - root - INFO - This node is already provisioned with services, we do not consider this as failure for test case 2024-01-29 11:42:52,541 - root - INFO - --> in init_cluster...Administrator,password,8091 2024-01-29 11:42:52,541 - root - INFO - settings/web params on 172.23.106.112:8091:port=8091&username=Administrator&password=password 2024-01-29 11:42:52,578 - root - INFO - --> status:True 2024-01-29 11:42:52,579 - root - INFO - SSH Connecting to 172.23.106.112 with username:root, attempt#1 of 5 2024-01-29 11:42:52,766 - root - INFO - SSH Connected to 172.23.106.112 as root 2024-01-29 11:42:52,933 - root - INFO - os_distro: Ubuntu, os_version: debian 11, is_linux_distro: True 2024-01-29 11:42:53,239 - root - INFO - extract_remote_info-->distribution_type: Ubuntu, distribution_version: debian 11 2024-01-29 11:42:53,240 - root - INFO - running command.raw on 172.23.106.112: curl --silent --show-error http://Administrator:password@localhost:8091/diag/eval -X POST -d 'ns_config:set(allow_nonlocal_eval, true).' 2024-01-29 11:42:53,262 - root - INFO - command executed successfully with root 2024-01-29 11:42:53,263 - root - INFO - ['ok'] 2024-01-29 11:42:53,266 - root - INFO - diag/eval status on 172.23.106.112:8091: True content: [7,1] command: cluster_compat_mode:get_compat_version(). 2024-01-29 11:42:53,269 - root - INFO - diag/eval status on 172.23.106.112:8091: True content: [7,1] command: cluster_compat_mode:get_compat_version(). 2024-01-29 11:42:53,274 - root - INFO - settings/indexes params : storageMode=plasma 2024-01-29 11:42:53,302 - root - INFO - server: ip:172.23.107.26 port:8091 ssh_username:root, nodes/self 2024-01-29 11:42:53,307 - root - INFO - {'uptime': '86', 'memoryTotal': 4083224576, 'memoryFree': 3442819072, 'mcdMemoryReserved': 3115, 'mcdMemoryAllocated': 3115, 'status': 'healthy', 'hostname': '172.23.107.26:8091', 'clusterCompatibility': 458753, 'clusterMembership': 'active', 'recoveryType': 'none', 'version': '7.1.6-3955-enterprise', 'os': 'x86_64-pc-linux-gnu', 'ports': [], 'availableStorage': [], 'storage': [], 'memoryQuota': 1318, 'memcached': 11210, 'id': 'ns_1@cb.local', 'ip': '172.23.107.26', 'internal_ip': '', 'rest_username': '', 'rest_password': '', 'port': '8091', 'services': ['fts', 'index', 'kv', 'n1ql'], 'storageTotalRam': 3894, 'curr_items': 0} 2024-01-29 11:42:53,310 - root - INFO - pools/default params : memoryQuota=2087 2024-01-29 11:42:53,316 - root - INFO - --> in init_cluster...Administrator,password,8091 2024-01-29 11:42:53,316 - root - INFO - settings/web params on 172.23.107.26:8091:port=8091&username=Administrator&password=password 2024-01-29 11:42:53,357 - root - INFO - --> status:True 2024-01-29 11:42:53,358 - root - INFO - SSH Connecting to 172.23.107.26 with username:root, attempt#1 of 5 2024-01-29 11:42:53,532 - root - INFO - SSH Connected to 172.23.107.26 as root 2024-01-29 11:42:53,688 - root - INFO - os_distro: Ubuntu, os_version: debian 11, is_linux_distro: True 2024-01-29 11:42:53,996 - root - INFO - extract_remote_info-->distribution_type: Ubuntu, distribution_version: debian 11 2024-01-29 11:42:53,997 - root - INFO - running command.raw on 172.23.107.26: curl --silent --show-error http://Administrator:password@localhost:8091/diag/eval -X POST -d 'ns_config:set(allow_nonlocal_eval, true).' 2024-01-29 11:42:54,069 - root - INFO - command executed successfully with root 2024-01-29 11:42:54,070 - root - INFO - ['ok'] 2024-01-29 11:42:54,074 - root - INFO - diag/eval status on 172.23.107.26:8091: True content: [7,1] command: cluster_compat_mode:get_compat_version(). 2024-01-29 11:42:54,076 - root - INFO - diag/eval status on 172.23.107.26:8091: True content: [7,1] command: cluster_compat_mode:get_compat_version(). 2024-01-29 11:42:54,081 - root - INFO - settings/indexes params : storageMode=plasma 2024-01-29 11:42:54,105 - root - INFO - server: ip:172.23.104.102 port:8091 ssh_username:root, nodes/self 2024-01-29 11:42:54,111 - root - INFO - {'uptime': '86', 'memoryTotal': 4083224576, 'memoryFree': 3436544000, 'mcdMemoryReserved': 3115, 'mcdMemoryAllocated': 3115, 'status': 'healthy', 'hostname': '172.23.104.102:8091', 'clusterCompatibility': 458753, 'clusterMembership': 'active', 'recoveryType': 'none', 'version': '7.1.6-3955-enterprise', 'os': 'x86_64-pc-linux-gnu', 'ports': [], 'availableStorage': [], 'storage': [], 'memoryQuota': 1318, 'memcached': 11210, 'id': 'ns_1@cb.local', 'ip': '172.23.104.102', 'internal_ip': '', 'rest_username': '', 'rest_password': '', 'port': '8091', 'services': ['fts', 'index', 'kv', 'n1ql'], 'storageTotalRam': 3894, 'curr_items': 0} 2024-01-29 11:42:54,114 - root - INFO - pools/default params : memoryQuota=2087 2024-01-29 11:42:54,121 - root - INFO - --> in init_cluster...Administrator,password,8091 2024-01-29 11:42:54,121 - root - INFO - settings/web params on 172.23.104.102:8091:port=8091&username=Administrator&password=password 2024-01-29 11:42:54,156 - root - INFO - --> status:True 2024-01-29 11:42:54,157 - root - INFO - SSH Connecting to 172.23.104.102 with username:root, attempt#1 of 5 2024-01-29 11:42:54,327 - root - INFO - SSH Connected to 172.23.104.102 as root 2024-01-29 11:42:54,475 - root - INFO - os_distro: Ubuntu, os_version: debian 11, is_linux_distro: True 2024-01-29 11:42:54,773 - root - INFO - extract_remote_info-->distribution_type: Ubuntu, distribution_version: debian 11 2024-01-29 11:42:54,774 - root - INFO - running command.raw on 172.23.104.102: curl --silent --show-error http://Administrator:password@localhost:8091/diag/eval -X POST -d 'ns_config:set(allow_nonlocal_eval, true).' 2024-01-29 11:42:54,799 - root - INFO - command executed successfully with root 2024-01-29 11:42:54,799 - root - INFO - ['ok'] 2024-01-29 11:42:54,803 - root - INFO - diag/eval status on 172.23.104.102:8091: True content: [7,1] command: cluster_compat_mode:get_compat_version(). 2024-01-29 11:42:54,805 - root - INFO - diag/eval status on 172.23.104.102:8091: True content: [7,1] command: cluster_compat_mode:get_compat_version(). 2024-01-29 11:42:54,809 - root - INFO - settings/indexes params : storageMode=plasma 2024-01-29 11:42:54,830 - root - INFO - server: ip:172.23.105.33 port:8091 ssh_username:root, nodes/self 2024-01-29 11:42:54,833 - root - INFO - {'uptime': '80', 'memoryTotal': 4326473728, 'memoryFree': 3609522176, 'mcdMemoryReserved': 3300, 'mcdMemoryAllocated': 3300, 'status': 'healthy', 'hostname': '172.23.105.33:8091', 'clusterCompatibility': 458753, 'clusterMembership': 'active', 'recoveryType': 'none', 'version': '7.1.6-3955-enterprise', 'os': 'x86_64-pc-linux-gnu', 'ports': [], 'availableStorage': [], 'storage': [], 'memoryQuota': 1442, 'memcached': 11210, 'id': 'ns_1@cb.local', 'ip': '172.23.105.33', 'internal_ip': '', 'rest_username': '', 'rest_password': '', 'port': '8091', 'services': ['fts', 'index', 'kv', 'n1ql'], 'storageTotalRam': 4126, 'curr_items': 0} 2024-01-29 11:42:54,835 - root - INFO - pools/default params : memoryQuota=2211 2024-01-29 11:42:54,839 - root - INFO - --> in init_cluster...Administrator,password,8091 2024-01-29 11:42:54,839 - root - INFO - settings/web params on 172.23.105.33:8091:port=8091&username=Administrator&password=password 2024-01-29 11:42:54,887 - root - INFO - --> status:True 2024-01-29 11:42:54,889 - root - INFO - SSH Connecting to 172.23.105.33 with username:root, attempt#1 of 5 2024-01-29 11:42:54,981 - root - INFO - SSH Connected to 172.23.105.33 as root 2024-01-29 11:42:55,104 - root - INFO - os_distro: Ubuntu, os_version: debian 10, is_linux_distro: True 2024-01-29 11:42:55,358 - root - INFO - extract_remote_info-->distribution_type: Ubuntu, distribution_version: debian 10 2024-01-29 11:42:55,359 - root - INFO - running command.raw on 172.23.105.33: curl --silent --show-error http://Administrator:password@localhost:8091/diag/eval -X POST -d 'ns_config:set(allow_nonlocal_eval, true).' 2024-01-29 11:42:55,416 - root - INFO - command executed successfully with root 2024-01-29 11:42:55,416 - root - INFO - ['ok'] 2024-01-29 11:42:55,419 - root - INFO - diag/eval status on 172.23.105.33:8091: True content: [7,1] command: cluster_compat_mode:get_compat_version(). 2024-01-29 11:42:55,421 - root - INFO - diag/eval status on 172.23.105.33:8091: True content: [7,1] command: cluster_compat_mode:get_compat_version(). 2024-01-29 11:42:55,427 - root - INFO - settings/indexes params : storageMode=plasma 2024-01-29 11:42:55,439 - root - INFO - **** add built-in 'cbadminbucket' user to node 172.23.106.112 **** 2024-01-29 11:42:55,454 - root - ERROR - DELETE http://172.23.106.112:8091/settings/rbac/users/local/cbadminbucket body: headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==', 'Accept': '*/*'} error: 404 reason: unknown b'"User was not found."' auth: Administrator:password 2024-01-29 11:42:55,455 - root - INFO - Exception while deleting user. Exception is -b'"User was not found."' 2024-01-29 11:42:55,500 - root - INFO - sleep for 5 secs. ... 2024-01-29 11:43:00,505 - root - INFO - **** add 'admin' role to 'cbadminbucket' user **** 2024-01-29 11:43:00,521 - root - INFO - done initializing cluster 2024-01-29 11:43:00,530 - root - INFO - Node version in cluster 7.1.6-3955-enterprise 2024-01-29 11:43:01,444 - root - INFO - adding node 172.23.107.26:8091 to cluster 2024-01-29 11:43:01,461 - root - INFO - adding remote node @172.23.107.26:18091 to this cluster @172.23.106.112:8091 2024-01-29 11:43:11,477 - root - INFO - rebalance progress took 10.02 seconds 2024-01-29 11:43:11,477 - root - INFO - sleep for 10 seconds after rebalance... 2024-01-29 11:43:25,154 - root - INFO - adding node 172.23.104.102:8091 to cluster 2024-01-29 11:43:25,168 - root - INFO - adding remote node @172.23.104.102:18091 to this cluster @172.23.106.112:8091 2024-01-29 11:43:35,188 - root - INFO - rebalance progress took 10.02 seconds 2024-01-29 11:43:35,188 - root - INFO - sleep for 10 seconds after rebalance... 2024-01-29 11:43:48,384 - root - INFO - adding node 172.23.105.33:8091 to cluster 2024-01-29 11:43:48,403 - root - INFO - adding remote node @172.23.105.33:18091 to this cluster @172.23.106.112:8091 2024-01-29 11:43:58,420 - root - INFO - rebalance progress took 10.02 seconds 2024-01-29 11:43:58,420 - root - INFO - sleep for 10 seconds after rebalance... 2024-01-29 11:44:12,008 - root - INFO - Node 172.23.104.102 not part of cluster inactiveAdded 2024-01-29 11:44:12,009 - root - INFO - Node 172.23.105.33 not part of cluster inactiveAdded 2024-01-29 11:44:12,009 - root - INFO - Node 172.23.107.26 not part of cluster inactiveAdded 2024-01-29 11:44:12,027 - root - INFO - rebalance params : {'knownNodes': 'ns_1@172.23.104.102,ns_1@172.23.105.33,ns_1@172.23.106.112,ns_1@172.23.107.26', 'ejectedNodes': '', 'user': 'Administrator', 'password': 'password'} 2024-01-29 11:44:12,089 - root - INFO - rebalance operation started 2024-01-29 11:44:22,102 - root - INFO - Rebalance - status: none, progress: 100.00% 2024-01-29 11:44:22,110 - root - INFO - rebalancing was completed with progress: 100% in 10.02170729637146 sec 2024-01-29 11:44:22,112 - root - INFO - SSH Connecting to 172.23.106.112 with username:root, attempt#1 of 5 2024-01-29 11:44:22,295 - root - INFO - SSH Connected to 172.23.106.112 as root 2024-01-29 11:44:22,454 - root - INFO - os_distro: Ubuntu, os_version: debian 11, is_linux_distro: True 2024-01-29 11:44:22,758 - root - INFO - extract_remote_info-->distribution_type: Ubuntu, distribution_version: debian 11 2024-01-29 11:44:22,758 - root - INFO - running command.raw on 172.23.106.112: curl --silent --show-error http://Administrator:password@localhost:8091/diag/eval -X POST -d 'ns_config:set(allow_nonlocal_eval, true).' 2024-01-29 11:44:22,830 - root - INFO - command executed successfully with root 2024-01-29 11:44:22,830 - root - INFO - ['ok'] 2024-01-29 11:44:22,830 - root - INFO - Enabled diag/eval for non-local hosts from 172.23.106.112 2024-01-29 11:44:23,760 - root - INFO - http://172.23.106.112:8091/pools/default/buckets with param: name=default&ramQuotaMB=256&replicaNumber=1&bucketType=membase&replicaIndex=1&threadsNumber=3&flushEnabled=1&evictionPolicy=valueOnly&compressionMode=passive&storageBackend=couchstore 2024-01-29 11:44:23,800 - root - INFO - 0.04 seconds to create bucket default 2024-01-29 11:44:23,821 - root - INFO - http://172.23.106.112:8091/pools/default/buckets/default/stats?zoom=minute 2024-01-29 11:44:23,867 - root - INFO - Bucket statistics +---------+---------+------------+----------+------------+------+-------+----------+-----------+----------+-----------+-----+ | Bucket | Type | Storage | Replicas | Durability | TTL | Items | Vbuckets | RAM Quota | RAM Used | Disk Used | ARR | +---------+---------+------------+----------+------------+------+-------+----------+-----------+----------+-----------+-----+ | default | membase | couchstore | 1 | None | None | 0 | None | 0.0 Byte | 0.0 Byte | 0.0 Byte | 100 | +---------+---------+------------+----------+------------+------+-------+----------+-----------+----------+-----------+-----+ 2024-01-29 11:44:23,867 - root - INFO - waiting for memcached bucket : default in 172.23.106.112 to accept set ops 2024-01-29 11:44:25,661 - root - INFO - creating direct client 172.23.104.102:11210 default 2024-01-29 11:44:25,801 - root - INFO - creating direct client 172.23.105.33:11210 default 2024-01-29 11:44:25,954 - root - INFO - creating direct client 172.23.106.112:11210 default 2024-01-29 11:44:26,068 - root - INFO - creating direct client 172.23.107.26:11210 default 2024-01-29 11:44:26,257 - root - INFO - creating direct client 172.23.104.102:11210 default 2024-01-29 11:44:26,350 - root - INFO - creating direct client 172.23.105.33:11210 default 2024-01-29 11:44:26,452 - root - INFO - creating direct client 172.23.106.112:11210 default 2024-01-29 11:44:26,607 - root - INFO - creating direct client 172.23.107.26:11210 default 2024-01-29 11:44:27,272 - root - INFO - bucket 'default' was created with per node RAM quota: 256 2024-01-29 11:44:28,294 - root - INFO - http://172.23.106.112:8091/pools/default/buckets with param: name=standard_bucket0&ramQuotaMB=256&replicaNumber=1&bucketType=membase&replicaIndex=1&threadsNumber=3&flushEnabled=1&evictionPolicy=valueOnly&compressionMode=passive&storageBackend=couchstore 2024-01-29 11:44:28,332 - root - INFO - 0.04 seconds to create bucket standard_bucket0 2024-01-29 11:44:28,369 - root - INFO - http://172.23.106.112:8091/pools/default/buckets/default/stats?zoom=minute 2024-01-29 11:44:28,542 - root - INFO - http://172.23.106.112:8091/pools/default/buckets/standard_bucket0/stats?zoom=minute 2024-01-29 11:44:28,695 - root - INFO - Bucket statistics +------------------+---------+------------+----------+------------+------+-------+----------+-----------+----------+-----------+-----+ | Bucket | Type | Storage | Replicas | Durability | TTL | Items | Vbuckets | RAM Quota | RAM Used | Disk Used | ARR | +------------------+---------+------------+----------+------------+------+-------+----------+-----------+----------+-----------+-----+ | default | membase | couchstore | 1 | None | None | 0 | None | 1.00 GiB | 0.0 Byte | 34.0 Byte | 100 | | standard_bucket0 | membase | couchstore | 1 | None | None | 0 | None | 0.0 Byte | 0.0 Byte | 0.0 Byte | 100 | +------------------+---------+------------+----------+------------+------+-------+----------+-----------+----------+-----------+-----+ 2024-01-29 11:44:28,697 - root - INFO - waiting for memcached bucket : standard_bucket0 in 172.23.106.112 to accept set ops 2024-01-29 11:44:30,502 - root - INFO - creating direct client 172.23.104.102:11210 standard_bucket0 2024-01-29 11:44:30,629 - root - INFO - creating direct client 172.23.105.33:11210 standard_bucket0 2024-01-29 11:44:30,724 - root - INFO - creating direct client 172.23.106.112:11210 standard_bucket0 2024-01-29 11:44:30,908 - root - INFO - creating direct client 172.23.107.26:11210 standard_bucket0 2024-01-29 11:44:31,097 - root - INFO - creating direct client 172.23.104.102:11210 standard_bucket0 2024-01-29 11:44:31,273 - root - INFO - creating direct client 172.23.105.33:11210 standard_bucket0 2024-01-29 11:44:31,387 - root - INFO - creating direct client 172.23.106.112:11210 standard_bucket0 2024-01-29 11:44:31,512 - root - INFO - creating direct client 172.23.107.26:11210 standard_bucket0 2024-01-29 11:44:32,220 - root - INFO - bucket 'standard_bucket0' was created with per node RAM quota: 256 2024-01-29 11:44:32,221 - root - INFO - ============== basetestcase setup was finished for test #1 test_upgrade ============== 2024-01-29 11:44:32,230 - root - INFO - SSH Connecting to 172.23.106.112 with username:root, attempt#1 of 5 2024-01-29 11:44:32,415 - root - INFO - SSH Connected to 172.23.106.112 as root 2024-01-29 11:44:32,566 - root - INFO - os_distro: Ubuntu, os_version: debian 11, is_linux_distro: True 2024-01-29 11:44:32,863 - root - INFO - extract_remote_info-->distribution_type: Ubuntu, distribution_version: debian 11 2024-01-29 11:44:32,865 - root - INFO - SSH Connecting to 172.23.107.26 with username:root, attempt#1 of 5 2024-01-29 11:44:33,047 - root - INFO - SSH Connected to 172.23.107.26 as root 2024-01-29 11:44:33,207 - root - INFO - os_distro: Ubuntu, os_version: debian 11, is_linux_distro: True 2024-01-29 11:44:33,494 - root - INFO - extract_remote_info-->distribution_type: Ubuntu, distribution_version: debian 11 2024-01-29 11:44:33,496 - root - INFO - SSH Connecting to 172.23.104.102 with username:root, attempt#1 of 5 2024-01-29 11:44:33,632 - root - INFO - SSH Connected to 172.23.104.102 as root 2024-01-29 11:44:33,784 - root - INFO - os_distro: Ubuntu, os_version: debian 11, is_linux_distro: True 2024-01-29 11:44:34,079 - root - INFO - extract_remote_info-->distribution_type: Ubuntu, distribution_version: debian 11 2024-01-29 11:44:34,081 - root - INFO - SSH Connecting to 172.23.105.33 with username:root, attempt#1 of 5 2024-01-29 11:44:34,151 - root - INFO - SSH Connected to 172.23.105.33 as root 2024-01-29 11:44:34,271 - root - INFO - os_distro: Ubuntu, os_version: debian 10, is_linux_distro: True 2024-01-29 11:44:34,526 - root - INFO - extract_remote_info-->distribution_type: Ubuntu, distribution_version: debian 10 2024-01-29 11:44:41,853 - root - INFO - ------- Cluster statistics ------- 2024-01-29 11:44:41,853 - root - INFO - 172.23.104.102:8091 => {'services': ['index', 'kv', 'n1ql'], 'cpu_utilization': 14.52442033409046, 'mem_free': 3334283264, 'mem_total': 4083224576, 'swap_mem_used': 16777216, 'swap_mem_total': 1027600384} 2024-01-29 11:44:41,854 - root - INFO - 172.23.105.33:8091 => {'services': ['index', 'kv', 'n1ql'], 'cpu_utilization': 9.136211187806923, 'mem_free': 3486658560, 'mem_total': 4326473728, 'swap_mem_used': 25690112, 'swap_mem_total': 4550815744} 2024-01-29 11:44:41,854 - root - INFO - 172.23.106.112:8091 => {'services': ['fts', 'index', 'kv', 'n1ql'], 'cpu_utilization': 24.78421364257335, 'mem_free': 3245248512, 'mem_total': 4083224576, 'swap_mem_used': 9175040, 'swap_mem_total': 1027600384} 2024-01-29 11:44:41,854 - root - INFO - 172.23.107.26:8091 => {'services': ['index', 'kv', 'n1ql'], 'cpu_utilization': 9.33095943891259, 'mem_free': 3329667072, 'mem_total': 4083224576, 'swap_mem_used': 4456448, 'swap_mem_total': 1027600384} 2024-01-29 11:44:41,854 - root - INFO - --- End of cluster statistics --- 2024-01-29 11:44:41,854 - root - INFO - Time to execute basesetup : 256.19361090660095 2024-01-29 11:44:41,854 - root - INFO - ============== NewUpgradeBaseTest setup has started ============== 2024-01-29 11:44:41,855 - root - INFO - SSH Connecting to 172.23.106.112 with username:root, attempt#1 of 5 2024-01-29 11:44:42,035 - root - INFO - SSH Connected to 172.23.106.112 as root 2024-01-29 11:44:42,186 - root - INFO - os_distro: Ubuntu, os_version: debian 11, is_linux_distro: True 2024-01-29 11:44:42,492 - root - INFO - extract_remote_info-->distribution_type: Ubuntu, distribution_version: debian 11 2024-01-29 11:44:42,493 - root - INFO - ============== NewUpgradeBaseTest setup has completed ============== 2024-01-29 11:44:42,493 - root - INFO - ============== QueryTests setup has started ============== 2024-01-29 11:44:42,494 - root - INFO - SSH Connecting to 172.23.106.112 with username:root, attempt#1 of 5 2024-01-29 11:44:42,666 - root - INFO - SSH Connected to 172.23.106.112 as root 2024-01-29 11:44:42,814 - root - INFO - os_distro: Ubuntu, os_version: debian 11, is_linux_distro: True 2024-01-29 11:44:43,115 - root - INFO - extract_remote_info-->distribution_type: Ubuntu, distribution_version: debian 11 2024-01-29 11:44:43,153 - root - INFO - SSH Connecting to 172.23.106.112 with username:root, attempt#1 of 5 2024-01-29 11:44:43,330 - root - INFO - SSH Connected to 172.23.106.112 as root 2024-01-29 11:44:43,492 - root - INFO - os_distro: Ubuntu, os_version: debian 11, is_linux_distro: True 2024-01-29 11:44:43,787 - root - INFO - extract_remote_info-->distribution_type: Ubuntu, distribution_version: debian 11 2024-01-29 11:44:43,788 - root - INFO - Generating default:default data... 2024-01-29 11:44:43,788 - root - INFO - -->start:generate_docs_employee for default... 2024-01-29 11:44:43,881 - root - INFO - -->end:generate_docs_employee for default... 2024-01-29 11:44:43,881 - root - INFO - Completed Generating default:default data... 2024-01-29 11:44:43,881 - root - INFO - --->gens_load is done... 2024-01-29 11:44:43,882 - root - INFO - -->starting rest connection... 2024-01-29 11:44:43,887 - root - INFO - server groups : ['0-1', '2-3'] 2024-01-29 11:44:43,907 - root - ERROR - There is not zone with name: ServerGroup_1 in cluster. 2024-01-29 11:44:43,939 - root - INFO - zone ServerGroup_1 is added 2024-01-29 11:44:43,944 - root - ERROR - There is not zone with name: ServerGroup_2 in cluster. 2024-01-29 11:44:43,972 - root - INFO - zone ServerGroup_2 is added 2024-01-29 11:44:44,404 - root - INFO - --> docs_per_day>0..generating TuqGenerators... 2024-01-29 11:44:44,681 - root - INFO - --> End: docs_per_day>0..generating TuqGenerators... 2024-01-29 11:44:44,682 - root - INFO - ============== QueryTests setup has completed ============== 2024-01-29 11:44:44,682 - root - INFO - ============== QueriesUpgradeTests setup has started ============== 2024-01-29 11:44:46,855 - root - INFO - Flush for bucket 'default' was triggered 2024-01-29 11:44:48,901 - root - INFO - Flush for bucket 'standard_bucket0' was triggered 2024-01-29 11:44:49,088 - root - INFO - creating direct client 172.23.104.102:11210 default 2024-01-29 11:44:49,154 - root - INFO - creating direct client 172.23.105.33:11210 default 2024-01-29 11:44:49,216 - root - INFO - creating direct client 172.23.106.112:11210 default 2024-01-29 11:44:49,359 - root - INFO - creating direct client 172.23.107.26:11210 default 2024-01-29 11:44:49,456 - root - INFO - creating direct client 172.23.104.102:11210 default 2024-01-29 11:44:49,526 - root - INFO - creating direct client 172.23.105.33:11210 default 2024-01-29 11:44:49,659 - root - INFO - creating direct client 172.23.106.112:11210 default 2024-01-29 11:44:49,732 - root - INFO - creating direct client 172.23.107.26:11210 default 2024-01-29 11:44:50,489 - root - INFO - creating direct client 172.23.104.102:11210 standard_bucket0 2024-01-29 11:44:50,645 - root - INFO - creating direct client 172.23.105.33:11210 standard_bucket0 2024-01-29 11:44:50,705 - root - INFO - creating direct client 172.23.106.112:11210 standard_bucket0 2024-01-29 11:44:50,791 - root - INFO - creating direct client 172.23.107.26:11210 standard_bucket0 2024-01-29 11:44:50,957 - root - INFO - creating direct client 172.23.104.102:11210 standard_bucket0 2024-01-29 11:44:51,017 - root - INFO - creating direct client 172.23.105.33:11210 standard_bucket0 2024-01-29 11:44:51,133 - root - INFO - creating direct client 172.23.106.112:11210 standard_bucket0 2024-01-29 11:44:51,196 - root - INFO - creating direct client 172.23.107.26:11210 standard_bucket0 2024-01-29 11:44:52,667 - root - INFO - create 2016.0 to default documents... 2024-01-29 11:44:52,819 - root - INFO - creating direct client 172.23.104.102:11210 default 2024-01-29 11:44:52,889 - root - INFO - creating direct client 172.23.105.33:11210 default 2024-01-29 11:44:52,945 - root - INFO - creating direct client 172.23.106.112:11210 default 2024-01-29 11:44:53,015 - root - INFO - creating direct client 172.23.107.26:11210 default ./lib/log_scanner.py:15: DeprecationWarning: invalid escape sequence \s "Basic\s[a-zA-Z]\{10,\}==", ./lib/log_scanner.py:16: DeprecationWarning: invalid escape sequence \[ "Menelaus-Auth-User:\[", 2024-01-29 11:44:53,176 - root - INFO - create 2016.0 to standard_bucket0 documents... 2024-01-29 11:44:53,246 - root - INFO - creating direct client 172.23.104.102:11210 standard_bucket0 2024-01-29 11:44:53,313 - root - INFO - creating direct client 172.23.105.33:11210 standard_bucket0 2024-01-29 11:44:53,371 - root - INFO - creating direct client 172.23.106.112:11210 standard_bucket0 2024-01-29 11:44:53,579 - root - INFO - creating direct client 172.23.107.26:11210 standard_bucket0 2024-01-29 11:44:55,853 - root - INFO - Node versions in cluster ['7.1.6-3955-enterprise', '7.1.6-3955-enterprise', '7.1.6-3955-enterprise', '7.1.6-3955-enterprise'] 2024-01-29 11:44:55,859 - root - INFO - list of kv nodes in cluster: [ip:172.23.104.102 port:8091 ssh_username:root, ip:172.23.105.33 port:8091 ssh_username:root, ip:172.23.106.112 port:8091 ssh_username:root, ip:172.23.107.26 port:8091 ssh_username:root] 2024-01-29 11:44:55,864 - root - INFO - Node versions in cluster ['7.1.6-3955-enterprise', '7.1.6-3955-enterprise', '7.1.6-3955-enterprise', '7.1.6-3955-enterprise'] 2024-01-29 11:44:55,869 - root - INFO - list of kv nodes in cluster: [ip:172.23.104.102 port:8091 ssh_username:root, ip:172.23.105.33 port:8091 ssh_username:root, ip:172.23.106.112 port:8091 ssh_username:root, ip:172.23.107.26 port:8091 ssh_username:root] 2024-01-29 11:44:55,999 - root - INFO - creating direct client 172.23.106.112:11210 default 2024-01-29 11:44:56,068 - root - INFO - Saw ep_queue_size 0 == 0 expected on '172.23.106.112:8091',default bucket 2024-01-29 11:44:56,077 - root - INFO - creating direct client 172.23.106.112:11210 standard_bucket0 2024-01-29 11:44:56,142 - root - INFO - Saw ep_queue_size 0 == 0 expected on '172.23.106.112:8091',standard_bucket0 bucket 2024-01-29 11:44:56,159 - root - INFO - creating direct client 172.23.107.26:11210 default 2024-01-29 11:44:56,337 - root - INFO - Saw ep_queue_size 0 == 0 expected on '172.23.107.26:8091',default bucket 2024-01-29 11:44:56,345 - root - INFO - creating direct client 172.23.107.26:11210 standard_bucket0 2024-01-29 11:44:56,411 - root - INFO - Saw ep_queue_size 0 == 0 expected on '172.23.107.26:8091',standard_bucket0 bucket 2024-01-29 11:44:56,429 - root - INFO - creating direct client 172.23.104.102:11210 default 2024-01-29 11:44:56,493 - root - INFO - Saw ep_queue_size 0 == 0 expected on '172.23.104.102:8091',default bucket 2024-01-29 11:44:56,502 - root - INFO - creating direct client 172.23.104.102:11210 standard_bucket0 2024-01-29 11:44:56,563 - root - INFO - Saw ep_queue_size 0 == 0 expected on '172.23.104.102:8091',standard_bucket0 bucket 2024-01-29 11:44:56,574 - root - INFO - creating direct client 172.23.105.33:11210 default 2024-01-29 11:44:56,707 - root - INFO - Saw ep_queue_size 0 == 0 expected on '172.23.105.33:8091',default bucket 2024-01-29 11:44:56,713 - root - INFO - creating direct client 172.23.105.33:11210 standard_bucket0 2024-01-29 11:44:56,759 - root - INFO - Saw ep_queue_size 0 == 0 expected on '172.23.105.33:8091',standard_bucket0 bucket 2024-01-29 11:44:56,812 - root - INFO - creating direct client 172.23.104.102:11210 default 2024-01-29 11:44:56,882 - root - INFO - creating direct client 172.23.105.33:11210 default 2024-01-29 11:44:56,937 - root - INFO - creating direct client 172.23.106.112:11210 default 2024-01-29 11:44:57,097 - root - INFO - creating direct client 172.23.107.26:11210 default 2024-01-29 11:44:57,194 - root - INFO - 2016 items will be verified on default bucket 2024-01-29 11:44:57,760 - root - INFO - 0 items were verified 2024-01-29 11:44:58,239 - root - INFO - 2016 items were verified in 1.0437252521514893 sec.the average number of ops - 1931.0 per second 2024-01-29 11:44:58,297 - root - INFO - creating direct client 172.23.104.102:11210 standard_bucket0 2024-01-29 11:44:58,364 - root - INFO - creating direct client 172.23.105.33:11210 standard_bucket0 2024-01-29 11:44:58,552 - root - INFO - creating direct client 172.23.106.112:11210 standard_bucket0 2024-01-29 11:44:58,642 - root - INFO - creating direct client 172.23.107.26:11210 standard_bucket0 2024-01-29 11:44:58,740 - root - INFO - 2016 items will be verified on standard_bucket0 bucket 2024-01-29 11:44:58,790 - root - INFO - 0 items were verified 2024-01-29 11:44:59,250 - root - INFO - 2016 items were verified in 0.5087862014770508 sec.the average number of ops - 3962.0 per second 2024-01-29 11:44:59,250 - root - INFO - waiting for sum_of_curr_items == total_items.... 2024-01-29 11:45:00,009 - root - INFO - curr_items from 172.23.104.102:8091 : 505 2024-01-29 11:45:00,009 - root - INFO - vb_pending_num from 172.23.104.102:8091 : 0 2024-01-29 11:45:00,009 - root - INFO - vb_active_num from 172.23.104.102:8091 : 256 2024-01-29 11:45:00,009 - root - INFO - vb_replica_num from 172.23.104.102:8091 : 256 2024-01-29 11:45:00,010 - root - INFO - curr_items from 172.23.105.33:8091 : 0 2024-01-29 11:45:00,010 - root - INFO - vb_pending_num from 172.23.105.33:8091 : 0 2024-01-29 11:45:00,010 - root - INFO - vb_active_num from 172.23.105.33:8091 : 256 2024-01-29 11:45:00,010 - root - INFO - vb_replica_num from 172.23.105.33:8091 : 256 2024-01-29 11:45:00,010 - root - INFO - curr_items from 172.23.106.112:8091 : 511 2024-01-29 11:45:00,010 - root - INFO - vb_pending_num from 172.23.106.112:8091 : 0 2024-01-29 11:45:00,010 - root - INFO - vb_active_num from 172.23.106.112:8091 : 256 2024-01-29 11:45:00,010 - root - INFO - vb_replica_num from 172.23.106.112:8091 : 255 2024-01-29 11:45:00,010 - root - INFO - curr_items from 172.23.107.26:8091 : 0 2024-01-29 11:45:00,010 - root - INFO - vb_pending_num from 172.23.107.26:8091 : 0 2024-01-29 11:45:00,010 - root - INFO - vb_active_num from 172.23.107.26:8091 : 256 2024-01-29 11:45:00,010 - root - INFO - vb_replica_num from 172.23.107.26:8091 : 257 2024-01-29 11:45:00,010 - root - INFO - summation of vb_active_num : 1024 vb_pending_num : 0 vb_replica_num : 1024 2024-01-29 11:45:00,010 - root - INFO - sum : 1016 and sum * (replica_factor + 1) (2) : 2032 2024-01-29 11:45:00,222 - root - INFO - curr_items_tot from master: 2004 2024-01-29 11:45:00,222 - root - INFO - Nodes stats are: ['172.23.104.102', '172.23.105.33', '172.23.106.112', '172.23.107.26'] 2024-01-29 11:45:00,222 - root - INFO - delta : 28 missing_percentage : 0.013779527559055118 replica_factor : 1 2024-01-29 11:45:02,963 - root - INFO - curr_items from 172.23.104.102:8091 : 505 2024-01-29 11:45:02,963 - root - INFO - vb_pending_num from 172.23.104.102:8091 : 0 2024-01-29 11:45:02,963 - root - INFO - vb_active_num from 172.23.104.102:8091 : 256 2024-01-29 11:45:02,963 - root - INFO - vb_replica_num from 172.23.104.102:8091 : 256 2024-01-29 11:45:02,963 - root - INFO - curr_items from 172.23.105.33:8091 : 0 2024-01-29 11:45:02,963 - root - INFO - vb_pending_num from 172.23.105.33:8091 : 0 2024-01-29 11:45:02,963 - root - INFO - vb_active_num from 172.23.105.33:8091 : 256 2024-01-29 11:45:02,963 - root - INFO - vb_replica_num from 172.23.105.33:8091 : 256 2024-01-29 11:45:02,964 - root - INFO - curr_items from 172.23.106.112:8091 : 511 2024-01-29 11:45:02,964 - root - INFO - vb_pending_num from 172.23.106.112:8091 : 0 2024-01-29 11:45:02,964 - root - INFO - vb_active_num from 172.23.106.112:8091 : 256 2024-01-29 11:45:02,964 - root - INFO - vb_replica_num from 172.23.106.112:8091 : 255 2024-01-29 11:45:02,964 - root - INFO - curr_items from 172.23.107.26:8091 : 483 2024-01-29 11:45:02,964 - root - INFO - vb_pending_num from 172.23.107.26:8091 : 0 2024-01-29 11:45:02,964 - root - INFO - vb_active_num from 172.23.107.26:8091 : 256 2024-01-29 11:45:02,964 - root - INFO - vb_replica_num from 172.23.107.26:8091 : 257 2024-01-29 11:45:02,964 - root - INFO - summation of vb_active_num : 1024 vb_pending_num : 0 vb_replica_num : 1024 2024-01-29 11:45:02,964 - root - INFO - sum : 1499 and sum * (replica_factor + 1) (2) : 2998 2024-01-29 11:45:03,162 - root - INFO - curr_items_tot from master: 3016 2024-01-29 11:45:03,162 - root - INFO - Nodes stats are: ['172.23.104.102', '172.23.105.33', '172.23.106.112', '172.23.107.26'] 2024-01-29 11:45:03,162 - root - INFO - delta : 18 missing_percentage : 0.00600400266844563 replica_factor : 1 2024-01-29 11:45:05,901 - root - INFO - curr_items from 172.23.104.102:8091 : 505 2024-01-29 11:45:05,901 - root - INFO - vb_pending_num from 172.23.104.102:8091 : 0 2024-01-29 11:45:05,901 - root - INFO - vb_active_num from 172.23.104.102:8091 : 256 2024-01-29 11:45:05,901 - root - INFO - vb_replica_num from 172.23.104.102:8091 : 256 2024-01-29 11:45:05,901 - root - INFO - curr_items from 172.23.105.33:8091 : 517 2024-01-29 11:45:05,901 - root - INFO - vb_pending_num from 172.23.105.33:8091 : 0 2024-01-29 11:45:05,901 - root - INFO - vb_active_num from 172.23.105.33:8091 : 256 2024-01-29 11:45:05,901 - root - INFO - vb_replica_num from 172.23.105.33:8091 : 256 2024-01-29 11:45:05,901 - root - INFO - curr_items from 172.23.106.112:8091 : 511 2024-01-29 11:45:05,901 - root - INFO - vb_pending_num from 172.23.106.112:8091 : 0 2024-01-29 11:45:05,902 - root - INFO - vb_active_num from 172.23.106.112:8091 : 256 2024-01-29 11:45:05,902 - root - INFO - vb_replica_num from 172.23.106.112:8091 : 255 2024-01-29 11:45:05,902 - root - INFO - curr_items from 172.23.107.26:8091 : 483 2024-01-29 11:45:05,902 - root - INFO - vb_pending_num from 172.23.107.26:8091 : 0 2024-01-29 11:45:05,902 - root - INFO - vb_active_num from 172.23.107.26:8091 : 256 2024-01-29 11:45:05,902 - root - INFO - vb_replica_num from 172.23.107.26:8091 : 257 2024-01-29 11:45:05,902 - root - INFO - summation of vb_active_num : 1024 vb_pending_num : 0 vb_replica_num : 1024 2024-01-29 11:45:05,902 - root - INFO - sum : 2016 and sum * (replica_factor + 1) (2) : 4032 2024-01-29 11:45:06,107 - root - INFO - curr_items_tot from master: 4032 2024-01-29 11:45:06,107 - root - INFO - delta : 0 missing_percentage : 1 replica_factor : 1 2024-01-29 11:45:06,108 - root - INFO - waiting for sum_of_curr_items == total_items.... 2024-01-29 11:45:06,823 - root - INFO - curr_items from 172.23.104.102:8091 : 505 2024-01-29 11:45:06,824 - root - INFO - vb_pending_num from 172.23.104.102:8091 : 0 2024-01-29 11:45:06,824 - root - INFO - vb_active_num from 172.23.104.102:8091 : 256 2024-01-29 11:45:06,824 - root - INFO - vb_replica_num from 172.23.104.102:8091 : 256 2024-01-29 11:45:06,824 - root - INFO - curr_items from 172.23.105.33:8091 : 517 2024-01-29 11:45:06,824 - root - INFO - vb_pending_num from 172.23.105.33:8091 : 0 2024-01-29 11:45:06,824 - root - INFO - vb_active_num from 172.23.105.33:8091 : 256 2024-01-29 11:45:06,824 - root - INFO - vb_replica_num from 172.23.105.33:8091 : 256 2024-01-29 11:45:06,824 - root - INFO - curr_items from 172.23.106.112:8091 : 511 2024-01-29 11:45:06,824 - root - INFO - vb_pending_num from 172.23.106.112:8091 : 0 2024-01-29 11:45:06,824 - root - INFO - vb_active_num from 172.23.106.112:8091 : 256 2024-01-29 11:45:06,824 - root - INFO - vb_replica_num from 172.23.106.112:8091 : 255 2024-01-29 11:45:06,824 - root - INFO - curr_items from 172.23.107.26:8091 : 483 2024-01-29 11:45:06,824 - root - INFO - vb_pending_num from 172.23.107.26:8091 : 0 2024-01-29 11:45:06,825 - root - INFO - vb_active_num from 172.23.107.26:8091 : 256 2024-01-29 11:45:06,825 - root - INFO - vb_replica_num from 172.23.107.26:8091 : 257 2024-01-29 11:45:06,825 - root - INFO - summation of vb_active_num : 1024 vb_pending_num : 0 vb_replica_num : 1024 2024-01-29 11:45:06,825 - root - INFO - sum : 2016 and sum * (replica_factor + 1) (2) : 4032 2024-01-29 11:45:07,040 - root - INFO - curr_items_tot from master: 4032 2024-01-29 11:45:07,041 - root - INFO - delta : 0 missing_percentage : 1 replica_factor : 1 2024-01-29 11:45:07,042 - root - INFO - LOAD IS FINISHED 2024-01-29 11:45:10,169 - root - INFO - ============== QueriesUpgradeTests setup has completed ============== 2024-01-29 11:45:10,169 - root - INFO - Begin n1ql upgrade test: offline 2024-01-29 11:45:10,179 - root - INFO - RUN QUERY SELECT * FROM system:indexes WHERE indexes.bucket_id is missing 2024-01-29 11:45:10,179 - root - INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+WHERE+indexes.bucket_id+is+missing&scan_consistency=REQUEST_PLUS 2024-01-29 11:45:10,672 - root - INFO - TOTAL ELAPSED TIME: 490.411451ms 2024-01-29 11:45:10,680 - root - INFO - RUN QUERY SELECT * FROM system:keyspaces WHERE keyspaces.`bucket` is missing 2024-01-29 11:45:10,680 - root - INFO - query params : statement=SELECT+%2A+FROM+system%3Akeyspaces+WHERE+keyspaces.%60bucket%60+is+missing&scan_consistency=REQUEST_PLUS 2024-01-29 11:45:10,733 - root - INFO - TOTAL ELAPSED TIME: 49.793079ms 2024-01-29 11:45:10,733 - root - INFO - ============== System Config: ============== 2024-01-29 11:45:10,736 - root - INFO - RUN QUERY SELECT COUNT(*) FROM `default` 2024-01-29 11:45:10,736 - root - INFO - query params : statement=SELECT+COUNT%28%2A%29+FROM+%60default%60&scan_consistency=REQUEST_PLUS 2024-01-29 11:45:10,750 - root - INFO - TOTAL ELAPSED TIME: 12.409307ms 2024-01-29 11:45:10,751 - root - INFO - Bucket: default 2024-01-29 11:45:10,751 - root - INFO - Indexes: [] 2024-01-29 11:45:10,751 - root - INFO - Docs: 2016 2024-01-29 11:45:10,753 - root - INFO - RUN QUERY SELECT COUNT(*) FROM `standard_bucket0` 2024-01-29 11:45:10,754 - root - INFO - query params : statement=SELECT+COUNT%28%2A%29+FROM+%60standard_bucket0%60&scan_consistency=REQUEST_PLUS 2024-01-29 11:45:10,769 - root - INFO - TOTAL ELAPSED TIME: 13.695292ms 2024-01-29 11:45:10,769 - root - INFO - Bucket: standard_bucket0 2024-01-29 11:45:10,769 - root - INFO - Indexes: [] 2024-01-29 11:45:10,769 - root - INFO - Docs: 2016 2024-01-29 11:45:10,774 - root - INFO - RUN QUERY SELECT COUNT(*) FROM `travel-sample` 2024-01-29 11:45:10,774 - root - INFO - query params : statement=SELECT+COUNT%28%2A%29+FROM+%60travel-sample%60&scan_consistency=REQUEST_PLUS 2024-01-29 11:45:10,806 - root - INFO - TOTAL ELAPSED TIME: 30.337529ms 2024-01-29 11:45:10,806 - root - INFO - Bucket: travel-sample 2024-01-29 11:45:10,806 - root - INFO - Indexes: [] 2024-01-29 11:45:10,806 - root - INFO - Docs: 0 2024-01-29 11:45:10,806 - root - INFO - ============================================= 2024-01-29 11:45:11,103 - root - INFO - RUN QUERY SELECT COUNT(*) FROM `default` 2024-01-29 11:45:11,103 - root - INFO - query params : statement=SELECT+COUNT%28%2A%29+FROM+%60default%60&scan_consistency=REQUEST_PLUS 2024-01-29 11:45:11,111 - root - INFO - TOTAL ELAPSED TIME: 5.555676ms 2024-01-29 11:45:11,116 - root - INFO - RUN QUERY SELECT COUNT(*) FROM `standard_bucket0` 2024-01-29 11:45:11,116 - root - INFO - query params : statement=SELECT+COUNT%28%2A%29+FROM+%60standard_bucket0%60&scan_consistency=REQUEST_PLUS 2024-01-29 11:45:11,125 - root - INFO - TOTAL ELAPSED TIME: 7.636009ms 2024-01-29 11:45:11,130 - root - INFO - RUN QUERY SELECT COUNT(*) FROM `travel-sample` 2024-01-29 11:45:11,130 - root - INFO - query params : statement=SELECT+COUNT%28%2A%29+FROM+%60travel-sample%60&scan_consistency=REQUEST_PLUS 2024-01-29 11:45:11,135 - root - INFO - TOTAL ELAPSED TIME: 2.631762ms 2024-01-29 11:45:11,135 - root - INFO - Bucket Docs Not Ready For Bucket: travel-sample... Expected: 31591 Query: 0 Rest: 0 2024-01-29 11:45:11,135 - root - INFO - sleep for 5 secs. incorrect results, sleeping for 5 ... 2024-01-29 11:45:16,213 - root - INFO - RUN QUERY SELECT COUNT(*) FROM `default` 2024-01-29 11:45:16,214 - root - INFO - query params : statement=SELECT+COUNT%28%2A%29+FROM+%60default%60&scan_consistency=REQUEST_PLUS 2024-01-29 11:45:16,233 - root - INFO - TOTAL ELAPSED TIME: 17.457971ms 2024-01-29 11:45:16,244 - root - INFO - RUN QUERY SELECT COUNT(*) FROM `standard_bucket0` 2024-01-29 11:45:16,244 - root - INFO - query params : statement=SELECT+COUNT%28%2A%29+FROM+%60standard_bucket0%60&scan_consistency=REQUEST_PLUS 2024-01-29 11:45:16,262 - root - INFO - TOTAL ELAPSED TIME: 16.030674ms 2024-01-29 11:45:16,270 - root - INFO - RUN QUERY SELECT COUNT(*) FROM `travel-sample` 2024-01-29 11:45:16,270 - root - INFO - query params : statement=SELECT+COUNT%28%2A%29+FROM+%60travel-sample%60&scan_consistency=REQUEST_PLUS 2024-01-29 11:45:16,293 - root - INFO - TOTAL ELAPSED TIME: 21.297895ms 2024-01-29 11:45:16,293 - root - INFO - Bucket Docs Not Ready For Bucket: travel-sample... Expected: 31591 Query: 0 Rest: 0 2024-01-29 11:45:16,293 - root - INFO - sleep for 5 secs. incorrect results, sleeping for 5 ... 2024-01-29 11:45:21,346 - root - INFO - RUN QUERY SELECT COUNT(*) FROM `default` 2024-01-29 11:45:21,346 - root - INFO - query params : statement=SELECT+COUNT%28%2A%29+FROM+%60default%60&scan_consistency=REQUEST_PLUS 2024-01-29 11:45:21,361 - root - INFO - TOTAL ELAPSED TIME: 12.496826ms 2024-01-29 11:45:21,364 - root - INFO - RUN QUERY SELECT COUNT(*) FROM `standard_bucket0` 2024-01-29 11:45:21,364 - root - INFO - query params : statement=SELECT+COUNT%28%2A%29+FROM+%60standard_bucket0%60&scan_consistency=REQUEST_PLUS 2024-01-29 11:45:21,515 - root - INFO - TOTAL ELAPSED TIME: 149.357743ms 2024-01-29 11:45:21,519 - root - INFO - RUN QUERY SELECT COUNT(*) FROM `travel-sample` 2024-01-29 11:45:21,519 - root - INFO - query params : statement=SELECT+COUNT%28%2A%29+FROM+%60travel-sample%60&scan_consistency=REQUEST_PLUS 2024-01-29 11:45:21,533 - root - INFO - TOTAL ELAPSED TIME: 11.876827ms 2024-01-29 11:45:21,533 - root - INFO - Bucket Docs Not Ready For Bucket: travel-sample... Expected: 31591 Query: 4075 Rest: 0 2024-01-29 11:45:21,533 - root - INFO - sleep for 5 secs. incorrect results, sleeping for 5 ... 2024-01-29 11:45:26,682 - root - INFO - RUN QUERY SELECT COUNT(*) FROM `default` 2024-01-29 11:45:26,682 - root - INFO - query params : statement=SELECT+COUNT%28%2A%29+FROM+%60default%60&scan_consistency=REQUEST_PLUS 2024-01-29 11:45:26,701 - root - INFO - TOTAL ELAPSED TIME: 17.058085ms 2024-01-29 11:45:26,704 - root - INFO - RUN QUERY SELECT COUNT(*) FROM `standard_bucket0` 2024-01-29 11:45:26,705 - root - INFO - query params : statement=SELECT+COUNT%28%2A%29+FROM+%60standard_bucket0%60&scan_consistency=REQUEST_PLUS 2024-01-29 11:45:26,732 - root - INFO - TOTAL ELAPSED TIME: 25.828901ms 2024-01-29 11:45:26,735 - root - INFO - RUN QUERY SELECT COUNT(*) FROM `travel-sample` 2024-01-29 11:45:26,736 - root - INFO - query params : statement=SELECT+COUNT%28%2A%29+FROM+%60travel-sample%60&scan_consistency=REQUEST_PLUS 2024-01-29 11:45:26,772 - root - INFO - TOTAL ELAPSED TIME: 34.333306ms 2024-01-29 11:45:26,772 - root - INFO - Bucket Docs Not Ready For Bucket: travel-sample... Expected: 31591 Query: 15321 Rest: 10852 2024-01-29 11:45:26,772 - root - INFO - sleep for 5 secs. incorrect results, sleeping for 5 ... 2024-01-29 11:45:31,853 - root - INFO - RUN QUERY SELECT COUNT(*) FROM `default` 2024-01-29 11:45:31,853 - root - INFO - query params : statement=SELECT+COUNT%28%2A%29+FROM+%60default%60&scan_consistency=REQUEST_PLUS 2024-01-29 11:45:31,866 - root - INFO - TOTAL ELAPSED TIME: 10.99548ms 2024-01-29 11:45:31,869 - root - INFO - RUN QUERY SELECT COUNT(*) FROM `standard_bucket0` 2024-01-29 11:45:31,870 - root - INFO - query params : statement=SELECT+COUNT%28%2A%29+FROM+%60standard_bucket0%60&scan_consistency=REQUEST_PLUS 2024-01-29 11:45:31,882 - root - INFO - TOTAL ELAPSED TIME: 10.280113ms 2024-01-29 11:45:31,885 - root - INFO - RUN QUERY SELECT COUNT(*) FROM `travel-sample` 2024-01-29 11:45:31,886 - root - INFO - query params : statement=SELECT+COUNT%28%2A%29+FROM+%60travel-sample%60&scan_consistency=REQUEST_PLUS 2024-01-29 11:45:31,897 - root - INFO - TOTAL ELAPSED TIME: 9.301839ms 2024-01-29 11:45:31,897 - root - INFO - Bucket Docs Not Ready For Bucket: travel-sample... Expected: 31591 Query: 29923 Rest: 35881 2024-01-29 11:45:31,897 - root - INFO - sleep for 5 secs. incorrect results, sleeping for 5 ... 2024-01-29 11:45:36,957 - root - INFO - RUN QUERY SELECT COUNT(*) FROM `default` 2024-01-29 11:45:36,957 - root - INFO - query params : statement=SELECT+COUNT%28%2A%29+FROM+%60default%60&scan_consistency=REQUEST_PLUS 2024-01-29 11:45:36,979 - root - INFO - TOTAL ELAPSED TIME: 19.503131ms 2024-01-29 11:45:36,983 - root - INFO - RUN QUERY SELECT COUNT(*) FROM `standard_bucket0` 2024-01-29 11:45:36,983 - root - INFO - query params : statement=SELECT+COUNT%28%2A%29+FROM+%60standard_bucket0%60&scan_consistency=REQUEST_PLUS 2024-01-29 11:45:36,999 - root - INFO - TOTAL ELAPSED TIME: 13.441221ms 2024-01-29 11:45:37,002 - root - INFO - RUN QUERY SELECT COUNT(*) FROM `travel-sample` 2024-01-29 11:45:37,002 - root - INFO - query params : statement=SELECT+COUNT%28%2A%29+FROM+%60travel-sample%60&scan_consistency=REQUEST_PLUS 2024-01-29 11:45:37,017 - root - INFO - TOTAL ELAPSED TIME: 12.877968ms 2024-01-29 11:45:37,020 - root - INFO - RUN QUERY SELECT * FROM system:indexes WHERE indexes.bucket_id is missing 2024-01-29 11:45:37,020 - root - INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+WHERE+indexes.bucket_id+is+missing&scan_consistency=REQUEST_PLUS 2024-01-29 11:45:37,065 - root - INFO - TOTAL ELAPSED TIME: 42.883945ms 2024-01-29 11:45:37,068 - root - INFO - RUN QUERY SELECT * FROM system:keyspaces WHERE keyspaces.`bucket` is missing 2024-01-29 11:45:37,068 - root - INFO - query params : statement=SELECT+%2A+FROM+system%3Akeyspaces+WHERE+keyspaces.%60bucket%60+is+missing&scan_consistency=REQUEST_PLUS 2024-01-29 11:45:37,072 - root - INFO - TOTAL ELAPSED TIME: 1.713522ms 2024-01-29 11:45:37,072 - root - INFO - ============== System Config: ============== 2024-01-29 11:45:37,075 - root - INFO - RUN QUERY SELECT COUNT(*) FROM `travel-sample` 2024-01-29 11:45:37,076 - root - INFO - query params : statement=SELECT+COUNT%28%2A%29+FROM+%60travel-sample%60&scan_consistency=REQUEST_PLUS 2024-01-29 11:45:37,080 - root - INFO - TOTAL ELAPSED TIME: 2.579231ms 2024-01-29 11:45:37,080 - root - INFO - Bucket: travel-sample 2024-01-29 11:45:37,080 - root - INFO - Indexes: ['def_name_type', 'def_airportname', 'def_city', 'def_icao', 'def_faa', 'def_type', 'def_sourceairport', 'def_schedule_utc', 'def_route_src_dst_day', 'def_primary'] 2024-01-29 11:45:37,080 - root - INFO - Docs: 31591 2024-01-29 11:45:37,083 - root - INFO - RUN QUERY SELECT COUNT(*) FROM `default` 2024-01-29 11:45:37,083 - root - INFO - query params : statement=SELECT+COUNT%28%2A%29+FROM+%60default%60&scan_consistency=REQUEST_PLUS 2024-01-29 11:45:37,086 - root - INFO - TOTAL ELAPSED TIME: 2.094648ms 2024-01-29 11:45:37,086 - root - INFO - Bucket: default 2024-01-29 11:45:37,086 - root - INFO - Indexes: [] 2024-01-29 11:45:37,086 - root - INFO - Docs: 2016 2024-01-29 11:45:37,089 - root - INFO - RUN QUERY SELECT COUNT(*) FROM `standard_bucket0` 2024-01-29 11:45:37,089 - root - INFO - query params : statement=SELECT+COUNT%28%2A%29+FROM+%60standard_bucket0%60&scan_consistency=REQUEST_PLUS 2024-01-29 11:45:37,094 - root - INFO - TOTAL ELAPSED TIME: 3.28339ms 2024-01-29 11:45:37,094 - root - INFO - Bucket: standard_bucket0 2024-01-29 11:45:37,095 - root - INFO - Indexes: [] 2024-01-29 11:45:37,095 - root - INFO - Docs: 2016 2024-01-29 11:45:37,095 - root - INFO - ============================================= 2024-01-29 11:45:37,098 - root - INFO - RUN QUERY SELECT * FROM system:indexes WHERE indexes.bucket_id is missing 2024-01-29 11:45:37,098 - root - INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+WHERE+indexes.bucket_id+is+missing&scan_consistency=REQUEST_PLUS 2024-01-29 11:45:37,102 - root - INFO - TOTAL ELAPSED TIME: 2.342512ms 2024-01-29 11:45:37,105 - root - INFO - RUN QUERY SELECT * FROM system:indexes where name='def_sourceairport' 2024-01-29 11:45:37,105 - root - INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name%3D%27def_sourceairport%27&scan_consistency=REQUEST_PLUS 2024-01-29 11:45:37,109 - root - INFO - TOTAL ELAPSED TIME: 2.030754ms 2024-01-29 11:45:37,112 - root - INFO - RUN QUERY SELECT * FROM system:indexes where name='def_schedule_utc' 2024-01-29 11:45:37,112 - root - INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name%3D%27def_schedule_utc%27&scan_consistency=REQUEST_PLUS 2024-01-29 11:45:37,116 - root - INFO - TOTAL ELAPSED TIME: 2.384226ms 2024-01-29 11:45:37,119 - root - INFO - RUN QUERY SELECT * FROM system:indexes where name='def_route_src_dst_day' 2024-01-29 11:45:37,119 - root - INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name%3D%27def_route_src_dst_day%27&scan_consistency=REQUEST_PLUS 2024-01-29 11:45:37,125 - root - INFO - TOTAL ELAPSED TIME: 3.833961ms 2024-01-29 11:45:37,129 - root - INFO - RUN QUERY SELECT * FROM system:indexes where name='def_type' 2024-01-29 11:45:37,129 - root - INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name%3D%27def_type%27&scan_consistency=REQUEST_PLUS 2024-01-29 11:45:37,133 - root - INFO - TOTAL ELAPSED TIME: 1.873845ms 2024-01-29 11:45:37,135 - root - INFO - RUN QUERY SELECT * FROM system:indexes where name='def_airportname' 2024-01-29 11:45:37,136 - root - INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name%3D%27def_airportname%27&scan_consistency=REQUEST_PLUS 2024-01-29 11:45:37,139 - root - INFO - TOTAL ELAPSED TIME: 1.992952ms 2024-01-29 11:45:37,142 - root - INFO - RUN QUERY SELECT * FROM system:indexes where name='def_city' 2024-01-29 11:45:37,142 - root - INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name%3D%27def_city%27&scan_consistency=REQUEST_PLUS 2024-01-29 11:45:37,146 - root - INFO - TOTAL ELAPSED TIME: 2.270957ms 2024-01-29 11:45:37,148 - root - INFO - RUN QUERY SELECT * FROM system:indexes where name='def_icao' 2024-01-29 11:45:37,149 - root - INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name%3D%27def_icao%27&scan_consistency=REQUEST_PLUS 2024-01-29 11:45:37,152 - root - INFO - TOTAL ELAPSED TIME: 1.448402ms 2024-01-29 11:45:37,154 - root - INFO - RUN QUERY SELECT * FROM system:indexes where name='def_faa' 2024-01-29 11:45:37,154 - root - INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name%3D%27def_faa%27&scan_consistency=REQUEST_PLUS 2024-01-29 11:45:37,194 - root - INFO - TOTAL ELAPSED TIME: 38.115321ms 2024-01-29 11:45:37,198 - root - INFO - RUN QUERY SELECT * FROM system:indexes where name='def_name_type' 2024-01-29 11:45:37,198 - root - INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name%3D%27def_name_type%27&scan_consistency=REQUEST_PLUS 2024-01-29 11:45:37,201 - root - INFO - TOTAL ELAPSED TIME: 1.610601ms 2024-01-29 11:45:37,204 - root - INFO - RUN QUERY SELECT * FROM system:indexes where name='def_primary' 2024-01-29 11:45:37,204 - root - INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name%3D%27def_primary%27&scan_consistency=REQUEST_PLUS 2024-01-29 11:45:37,208 - root - INFO - TOTAL ELAPSED TIME: 2.036381ms 2024-01-29 11:45:37,208 - root - INFO - --> start: ensure_primary_indexes_exist.. 2024-01-29 11:45:37,210 - root - INFO - RUN QUERY SELECT * FROM system:keyspaces WHERE keyspaces.`bucket` is missing 2024-01-29 11:45:37,210 - root - INFO - query params : statement=SELECT+%2A+FROM+system%3Akeyspaces+WHERE+keyspaces.%60bucket%60+is+missing&scan_consistency=REQUEST_PLUS 2024-01-29 11:45:37,214 - root - INFO - TOTAL ELAPSED TIME: 1.739893ms 2024-01-29 11:45:37,214 - root - INFO - -->query_response:{'requestID': '0c290208-313f-4dda-b30d-46783320a028', 'signature': {'*': '*'}, 'results': [{'keyspaces': {'datastore_id': 'http://127.0.0.1:8091', 'id': 'default', 'name': 'default', 'namespace': 'default', 'namespace_id': 'default', 'path': 'default:default'}}, {'keyspaces': {'datastore_id': 'http://127.0.0.1:8091', 'id': 'standard_bucket0', 'name': 'standard_bucket0', 'namespace': 'default', 'namespace_id': 'default', 'path': 'default:standard_bucket0'}}, {'keyspaces': {'datastore_id': 'http://127.0.0.1:8091', 'id': 'travel-sample', 'name': 'travel-sample', 'namespace': 'default', 'namespace_id': 'default', 'path': 'default:travel-sample'}}], 'status': 'success', 'metrics': {'elapsedTime': '1.739893ms', 'executionTime': '1.676685ms', 'resultCount': 3, 'resultSize': 519, 'serviceLoad': 1}} 2024-01-29 11:45:37,216 - root - INFO - RUN QUERY SELECT * FROM system:indexes WHERE indexes.bucket_id is missing 2024-01-29 11:45:37,217 - root - INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+WHERE+indexes.bucket_id+is+missing&scan_consistency=REQUEST_PLUS 2024-01-29 11:45:37,220 - root - INFO - TOTAL ELAPSED TIME: 2.053903ms 2024-01-29 11:45:37,220 - root - INFO - -->before create indexes: [{'name': '#primary', 'bucket': 'default', 'fields': [], 'state': 'online', 'using': 'gsi', 'is_primary': True}, {'name': '#primary', 'bucket': 'standard_bucket0', 'fields': [], 'state': 'online', 'using': 'gsi', 'is_primary': True}, {'name': '#primary', 'bucket': 'travel-sample', 'fields': [], 'state': 'online', 'using': 'gsi', 'is_primary': True}],[{'name': '#primary', 'bucket': 'default', 'fields': frozenset(), 'state': 'online', 'using': 'gsi', 'where': '', 'is_primary': True}, {'name': '#primary', 'bucket': 'standard_bucket0', 'fields': frozenset(), 'state': 'online', 'using': 'gsi', 'where': '', 'is_primary': True}, {'name': '#primary', 'bucket': 'travel-sample', 'fields': frozenset(), 'state': 'online', 'using': 'gsi', 'where': '', 'is_primary': True}],[{'name': 'def_type', 'bucket': 'travel-sample', 'fields': frozenset({('type', 0)}), 'state': 'online', 'using': 'gsi', 'where': '', 'is_primary': False}, {'name': 'def_sourceairport', 'bucket': 'travel-sample', 'fields': frozenset({('sourceairport', 0)}), 'state': 'online', 'using': 'gsi', 'where': '', 'is_primary': False}, {'name': 'def_schedule_utc', 'bucket': 'travel-sample', 'fields': frozenset({('array s.utc for s in schedule end', 0)}), 'state': 'online', 'using': 'gsi', 'where': '', 'is_primary': False}, {'name': 'def_route_src_dst_day', 'bucket': 'travel-sample', 'fields': frozenset({('destinationairport', 1), ('distinct array v.day for v in schedule end', 2), ('sourceairport', 0)}), 'state': 'online', 'using': 'gsi', 'where': '(`type` = "route")', 'is_primary': False}, {'name': 'def_name_type', 'bucket': 'travel-sample', 'fields': frozenset({('name', 0)}), 'state': 'online', 'using': 'gsi', 'where': '(`_type` = "User")', 'is_primary': False}, {'name': 'def_airportname', 'bucket': 'travel-sample', 'fields': frozenset({('airportname', 0)}), 'state': 'online', 'using': 'gsi', 'where': '', 'is_primary': False}, {'name': 'def_city', 'bucket': 'travel-sample', 'fields': frozenset({('city', 0)}), 'state': 'online', 'using': 'gsi', 'where': '', 'is_primary': False}, {'name': 'def_icao', 'bucket': 'travel-sample', 'fields': frozenset({('icao', 0)}), 'state': 'online', 'using': 'gsi', 'where': '', 'is_primary': False}, {'name': 'def_faa', 'bucket': 'travel-sample', 'fields': frozenset({('faa', 0)}), 'state': 'online', 'using': 'gsi', 'where': '', 'is_primary': False}, {'name': 'def_primary', 'bucket': 'travel-sample', 'fields': frozenset(), 'state': 'online', 'using': 'gsi', 'where': '', 'is_primary': True}] 2024-01-29 11:45:37,221 - root - INFO - -->Create desired indexes..desiredset=frozenset({frozenset({('fields', frozenset()), ('bucket', 'default'), ('state', 'online'), ('where', ''), ('name', '#primary'), ('is_primary', True), ('using', 'gsi')}), frozenset({('fields', frozenset()), ('state', 'online'), ('bucket', 'travel-sample'), ('where', ''), ('name', '#primary'), ('is_primary', True), ('using', 'gsi')}), frozenset({('fields', frozenset()), ('state', 'online'), ('where', ''), ('bucket', 'standard_bucket0'), ('name', '#primary'), ('is_primary', True), ('using', 'gsi')})}),currentset=frozenset({frozenset({('name', 'def_route_src_dst_day'), ('where', '(`type` = "route")'), ('state', 'online'), ('bucket', 'travel-sample'), ('fields', frozenset({('destinationairport', 1), ('distinct array v.day for v in schedule end', 2), ('sourceairport', 0)})), ('is_primary', False), ('using', 'gsi')}), frozenset({('name', 'def_sourceairport'), ('state', 'online'), ('fields', frozenset({('sourceairport', 0)})), ('bucket', 'travel-sample'), ('where', ''), ('is_primary', False), ('using', 'gsi')}), frozenset({('fields', frozenset({('array s.utc for s in schedule end', 0)})), ('state', 'online'), ('bucket', 'travel-sample'), ('where', ''), ('name', 'def_schedule_utc'), ('is_primary', False), ('using', 'gsi')}), frozenset({('name', 'def_faa'), ('state', 'online'), ('bucket', 'travel-sample'), ('where', ''), ('fields', frozenset({('faa', 0)})), ('is_primary', False), ('using', 'gsi')}), frozenset({('fields', frozenset({('city', 0)})), ('state', 'online'), ('bucket', 'travel-sample'), ('name', 'def_city'), ('where', ''), ('is_primary', False), ('using', 'gsi')}), frozenset({('fields', frozenset({('icao', 0)})), ('state', 'online'), ('bucket', 'travel-sample'), ('where', ''), ('name', 'def_icao'), ('is_primary', False), ('using', 'gsi')}), frozenset({('name', 'def_airportname'), ('fields', frozenset({('airportname', 0)})), ('state', 'online'), ('bucket', 'travel-sample'), ('where', ''), ('is_primary', False), ('using', 'gsi')}), frozenset({('name', 'def_name_type'), ('where', '(`_type` = "User")'), ('is_primary', False), ('state', 'online'), ('bucket', 'travel-sample'), ('fields', frozenset({('name', 0)})), ('using', 'gsi')}), frozenset({('fields', frozenset({('type', 0)})), ('state', 'online'), ('bucket', 'travel-sample'), ('name', 'def_type'), ('where', ''), ('is_primary', False), ('using', 'gsi')}), frozenset({('fields', frozenset()), ('state', 'online'), ('bucket', 'travel-sample'), ('where', ''), ('name', 'def_primary'), ('is_primary', True), ('using', 'gsi')})}),desiredindexes=[{'name': '#primary', 'bucket': 'default', 'fields': frozenset(), 'state': 'online', 'using': 'gsi', 'where': '', 'is_primary': True}, {'name': '#primary', 'bucket': 'standard_bucket0', 'fields': frozenset(), 'state': 'online', 'using': 'gsi', 'where': '', 'is_primary': True}, {'name': '#primary', 'bucket': 'travel-sample', 'fields': frozenset(), 'state': 'online', 'using': 'gsi', 'where': '', 'is_primary': True}] 2024-01-29 11:45:37,221 - root - INFO - creating index: default #primary gsi 2024-01-29 11:45:37,223 - root - INFO - RUN QUERY CREATE PRIMARY INDEX ON `default` USING gsi 2024-01-29 11:45:37,224 - root - INFO - query params : statement=CREATE+PRIMARY+INDEX+ON+%60default%60+USING+gsi&scan_consistency=REQUEST_PLUS 2024-01-29 11:45:39,678 - root - INFO - TOTAL ELAPSED TIME: 2.452251658s 2024-01-29 11:45:39,698 - root - INFO - RUN QUERY SELECT * FROM system:indexes WHERE indexes.bucket_id is missing 2024-01-29 11:45:39,698 - root - INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+WHERE+indexes.bucket_id+is+missing&scan_consistency=REQUEST_PLUS 2024-01-29 11:45:39,739 - root - INFO - TOTAL ELAPSED TIME: 38.303889ms 2024-01-29 11:45:39,739 - root - INFO - creating index: standard_bucket0 #primary gsi 2024-01-29 11:45:39,745 - root - INFO - RUN QUERY CREATE PRIMARY INDEX ON `standard_bucket0` USING gsi 2024-01-29 11:45:39,746 - root - INFO - query params : statement=CREATE+PRIMARY+INDEX+ON+%60standard_bucket0%60+USING+gsi&scan_consistency=REQUEST_PLUS 2024-01-29 11:45:42,241 - root - INFO - TOTAL ELAPSED TIME: 2.492691153s 2024-01-29 11:45:42,266 - root - INFO - RUN QUERY SELECT * FROM system:indexes WHERE indexes.bucket_id is missing 2024-01-29 11:45:42,266 - root - INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+WHERE+indexes.bucket_id+is+missing&scan_consistency=REQUEST_PLUS 2024-01-29 11:45:42,330 - root - INFO - TOTAL ELAPSED TIME: 61.671073ms 2024-01-29 11:45:42,331 - root - INFO - creating index: travel-sample #primary gsi 2024-01-29 11:45:42,334 - root - INFO - RUN QUERY CREATE PRIMARY INDEX ON `travel-sample` USING gsi 2024-01-29 11:45:42,334 - root - INFO - query params : statement=CREATE+PRIMARY+INDEX+ON+%60travel-sample%60+USING+gsi&scan_consistency=REQUEST_PLUS 2024-01-29 11:45:45,553 - root - INFO - TOTAL ELAPSED TIME: 3.21668144s 2024-01-29 11:45:45,566 - root - INFO - RUN QUERY SELECT * FROM system:indexes WHERE indexes.bucket_id is missing 2024-01-29 11:45:45,566 - root - INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+WHERE+indexes.bucket_id+is+missing&scan_consistency=REQUEST_PLUS 2024-01-29 11:45:45,614 - root - INFO - TOTAL ELAPSED TIME: 44.101651ms 2024-01-29 11:45:45,615 - root - INFO - --> end: ensure_primary_indexes_exist.. 2024-01-29 11:45:45,618 - root - INFO - RUN QUERY SELECT * FROM system:indexes WHERE indexes.bucket_id is missing 2024-01-29 11:45:45,618 - root - INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+WHERE+indexes.bucket_id+is+missing&scan_consistency=REQUEST_PLUS 2024-01-29 11:45:45,623 - root - INFO - TOTAL ELAPSED TIME: 2.896676ms 2024-01-29 11:45:45,626 - root - INFO - RUN QUERY SELECT * FROM system:indexes where name='#primary' 2024-01-29 11:45:45,626 - root - INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name%3D%27%23primary%27&scan_consistency=REQUEST_PLUS 2024-01-29 11:45:45,630 - root - INFO - TOTAL ELAPSED TIME: 2.054074ms 2024-01-29 11:45:45,633 - root - INFO - RUN QUERY SELECT * FROM system:indexes where name='#primary' 2024-01-29 11:45:45,633 - root - INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name%3D%27%23primary%27&scan_consistency=REQUEST_PLUS 2024-01-29 11:45:45,636 - root - INFO - TOTAL ELAPSED TIME: 1.822262ms 2024-01-29 11:45:45,639 - root - INFO - RUN QUERY SELECT * FROM system:indexes where name='def_route_src_dst_day' 2024-01-29 11:45:45,639 - root - INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name%3D%27def_route_src_dst_day%27&scan_consistency=REQUEST_PLUS 2024-01-29 11:45:45,642 - root - INFO - TOTAL ELAPSED TIME: 1.866267ms 2024-01-29 11:45:45,645 - root - INFO - RUN QUERY SELECT * FROM system:indexes where name='def_sourceairport' 2024-01-29 11:45:45,645 - root - INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name%3D%27def_sourceairport%27&scan_consistency=REQUEST_PLUS 2024-01-29 11:45:45,649 - root - INFO - TOTAL ELAPSED TIME: 2.065091ms 2024-01-29 11:45:45,651 - root - INFO - RUN QUERY SELECT * FROM system:indexes where name='def_schedule_utc' 2024-01-29 11:45:45,651 - root - INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name%3D%27def_schedule_utc%27&scan_consistency=REQUEST_PLUS 2024-01-29 11:45:45,654 - root - INFO - TOTAL ELAPSED TIME: 1.688914ms 2024-01-29 11:45:45,657 - root - INFO - RUN QUERY SELECT * FROM system:indexes where name='def_city' 2024-01-29 11:45:45,657 - root - INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name%3D%27def_city%27&scan_consistency=REQUEST_PLUS 2024-01-29 11:45:45,660 - root - INFO - TOTAL ELAPSED TIME: 1.517892ms 2024-01-29 11:45:45,663 - root - INFO - RUN QUERY SELECT * FROM system:indexes where name='def_airportname' 2024-01-29 11:45:45,663 - root - INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name%3D%27def_airportname%27&scan_consistency=REQUEST_PLUS 2024-01-29 11:45:45,668 - root - INFO - TOTAL ELAPSED TIME: 2.372163ms 2024-01-29 11:45:45,672 - root - INFO - RUN QUERY SELECT * FROM system:indexes where name='def_type' 2024-01-29 11:45:45,672 - root - INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name%3D%27def_type%27&scan_consistency=REQUEST_PLUS 2024-01-29 11:45:45,675 - root - INFO - TOTAL ELAPSED TIME: 1.894497ms 2024-01-29 11:45:45,678 - root - INFO - RUN QUERY SELECT * FROM system:indexes where name='def_faa' 2024-01-29 11:45:45,678 - root - INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name%3D%27def_faa%27&scan_consistency=REQUEST_PLUS 2024-01-29 11:45:45,681 - root - INFO - TOTAL ELAPSED TIME: 1.771268ms 2024-01-29 11:45:45,684 - root - INFO - RUN QUERY SELECT * FROM system:indexes where name='def_name_type' 2024-01-29 11:45:45,684 - root - INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name%3D%27def_name_type%27&scan_consistency=REQUEST_PLUS 2024-01-29 11:45:45,690 - root - INFO - TOTAL ELAPSED TIME: 4.185724ms 2024-01-29 11:45:45,696 - root - INFO - RUN QUERY SELECT * FROM system:indexes where name='def_icao' 2024-01-29 11:45:45,696 - root - INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name%3D%27def_icao%27&scan_consistency=REQUEST_PLUS 2024-01-29 11:45:45,700 - root - INFO - TOTAL ELAPSED TIME: 2.081057ms 2024-01-29 11:45:45,705 - root - INFO - RUN QUERY SELECT * FROM system:indexes where name='def_primary' 2024-01-29 11:45:45,706 - root - INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name%3D%27def_primary%27&scan_consistency=REQUEST_PLUS 2024-01-29 11:45:45,710 - root - INFO - TOTAL ELAPSED TIME: 2.863855ms 2024-01-29 11:45:45,713 - root - INFO - RUN QUERY SELECT * FROM system:indexes where name='#primary' 2024-01-29 11:45:45,713 - root - INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name%3D%27%23primary%27&scan_consistency=REQUEST_PLUS 2024-01-29 11:45:45,739 - root - INFO - TOTAL ELAPSED TIME: 24.979297ms 2024-01-29 11:45:45,740 - root - INFO - UPGRADE_VERSIONS = ['7.6.0-2054'] 2024-01-29 11:45:45,740 - root - INFO - running pre upgrade test 2024-01-29 11:45:45,740 - root - INFO - running pre-upgrade test for N1QL 2024-01-29 11:45:45,743 - root - INFO - RUN QUERY prepare lyon_airport as select array_agg(airportname) from `travel-sample` where type = 'airport' and city = 'Lyon' 2024-01-29 11:45:45,743 - root - INFO - query params : statement=prepare+lyon_airport+as+select+array_agg%28airportname%29+from+%60travel-sample%60+where+type+%3D+%27airport%27+and+city+%3D+%27Lyon%27&scan_consistency=REQUEST_PLUS 2024-01-29 11:45:45,752 - root - INFO - TOTAL ELAPSED TIME: 6.41022ms 2024-01-29 11:45:45,755 - root - INFO - RUN QUERY CREATE OR REPLACE FUNCTION func1(a,b,c) { (SELECT RAW SUM((a+b+c-40))) } 2024-01-29 11:45:45,755 - root - INFO - query params : statement=CREATE+OR+REPLACE+FUNCTION+func1%28a%2Cb%2Cc%29+%7B+%28SELECT+RAW+SUM%28%28a%2Bb%2Bc-40%29%29%29+%7D&scan_consistency=REQUEST_PLUS 2024-01-29 11:45:45,768 - root - INFO - TOTAL ELAPSED TIME: 11.074972ms 2024-01-29 11:45:45,783 - root - INFO - RUN QUERY UPDATE STATISTICS FOR `travel-sample`(`type`) WITH {'update_statistics_timeout':600} 2024-01-29 11:45:45,783 - root - INFO - query params : statement=UPDATE+STATISTICS+FOR+%60travel-sample%60%28%60type%60%29+WITH+%7B%27update_statistics_timeout%27%3A600%7D&scan_consistency=REQUEST_PLUS 2024-01-29 11:47:08,955 - root - INFO - TOTAL ELAPSED TIME: 1m23.168515717s 2024-01-29 11:47:08,970 - root - INFO - RUN QUERY EXPLAIN select type from `travel-sample` where type = 'hotel' 2024-01-29 11:47:08,970 - root - INFO - query params : statement=EXPLAIN+select+type+from+%60travel-sample%60+where+type+%3D+%27hotel%27&scan_consistency=REQUEST_PLUS 2024-01-29 11:47:08,984 - root - INFO - TOTAL ELAPSED TIME: 11.483496ms 2024-01-29 11:47:08,985 - root - INFO - Create math library 2024-01-29 11:47:08,985 - root - INFO - running command.raw on 172.23.106.112: curl -s -k -X DELETE http://172.23.106.112:8093/evaluator/v1/libraries/math -u Administrator:password 2024-01-29 11:47:09,028 - root - INFO - command executed successfully with root 2024-01-29 11:47:09,028 - root - INFO - running command.raw on 172.23.106.112: curl -s -k -X POST http://172.23.106.112:8093/evaluator/v1/libraries/math -u Administrator:password -H 'content-type: application/json' -d 'function adder(a, b, c) { for (i=0; i< b; i++){a = a + c;} return a; }' 2024-01-29 11:47:09,124 - root - INFO - command executed successfully with root 2024-01-29 11:47:09,124 - root - INFO - (['{"status": "OK"}'], []) 2024-01-29 11:47:09,124 - root - INFO - running command.raw on 172.23.106.112: curl -s -k http://172.23.106.112:8093/evaluator/v1/libraries/math -u Administrator:password 2024-01-29 11:47:09,181 - root - INFO - command executed successfully with root 2024-01-29 11:47:09,191 - root - INFO - RUN QUERY CREATE FUNCTION func2(a,b,c) LANGUAGE JAVASCRIPT AS "adder" AT "math" 2024-01-29 11:47:09,191 - root - INFO - query params : statement=CREATE+FUNCTION+func2%28a%2Cb%2Cc%29+LANGUAGE+JAVASCRIPT+AS+%22adder%22+AT+%22math%22&scan_consistency=REQUEST_PLUS 2024-01-29 11:47:09,207 - root - INFO - TOTAL ELAPSED TIME: 13.387987ms 2024-01-29 11:47:09,215 - root - INFO - RUN QUERY EXECUTE FUNCTION func2(1,3,5) 2024-01-29 11:47:09,215 - root - INFO - query params : statement=EXECUTE+FUNCTION+func2%281%2C3%2C5%29&scan_consistency=REQUEST_PLUS 2024-01-29 11:47:09,230 - root - INFO - TOTAL ELAPSED TIME: 12.807386ms 2024-01-29 11:47:09,231 - root - INFO - completed pre upgrade test 2024-01-29 11:47:09,231 - root - INFO - upgrading servers for mixed mode 2024-01-29 11:47:09,232 - root - INFO - SSH Connecting to 172.23.106.112 with username:root, attempt#1 of 5 2024-01-29 11:47:09,417 - root - INFO - SSH Connected to 172.23.106.112 as root 2024-01-29 11:47:09,581 - root - INFO - os_distro: Ubuntu, os_version: debian 11, is_linux_distro: True 2024-01-29 11:47:09,882 - root - INFO - extract_remote_info-->distribution_type: Ubuntu, distribution_version: debian 11 2024-01-29 11:47:09,943 - root - INFO - 172.23.106.112 **** The linux version file /opt/couchbase/ VERSION.txt exists 2024-01-29 11:47:09,991 - root - INFO - Running systemd stop command on this server 2024-01-29 11:47:09,991 - root - INFO - running command.raw on 172.23.106.112: systemctl stop couchbase-server.service 2024-01-29 11:47:17,220 - root - INFO - command executed successfully with root 2024-01-29 11:47:17,221 - root - INFO - SSH Connecting to 172.23.107.26 with username:root, attempt#1 of 5 2024-01-29 11:47:17,404 - root - INFO - SSH Connected to 172.23.107.26 as root 2024-01-29 11:47:17,586 - root - INFO - os_distro: Ubuntu, os_version: debian 11, is_linux_distro: True 2024-01-29 11:47:17,889 - root - INFO - extract_remote_info-->distribution_type: Ubuntu, distribution_version: debian 11 2024-01-29 11:47:17,904 - root - INFO - 172.23.107.26 **** The linux version file /opt/couchbase/ VERSION.txt exists 2024-01-29 11:47:17,944 - root - INFO - Running systemd stop command on this server 2024-01-29 11:47:17,944 - root - INFO - running command.raw on 172.23.107.26: systemctl stop couchbase-server.service 2024-01-29 11:47:20,836 - root - INFO - command executed successfully with root 2024-01-29 11:47:20,838 - root - INFO - SSH Connecting to 172.23.104.102 with username:root, attempt#1 of 5 2024-01-29 11:47:21,030 - root - INFO - SSH Connected to 172.23.104.102 as root 2024-01-29 11:47:21,194 - root - INFO - os_distro: Ubuntu, os_version: debian 11, is_linux_distro: True 2024-01-29 11:47:21,465 - root - INFO - extract_remote_info-->distribution_type: Ubuntu, distribution_version: debian 11 2024-01-29 11:47:21,477 - root - INFO - 172.23.104.102 **** The linux version file /opt/couchbase/ VERSION.txt exists 2024-01-29 11:47:21,520 - root - INFO - Running systemd stop command on this server 2024-01-29 11:47:21,520 - root - INFO - running command.raw on 172.23.104.102: systemctl stop couchbase-server.service 2024-01-29 11:47:24,617 - root - INFO - command executed successfully with root 2024-01-29 11:47:24,619 - root - INFO - SSH Connecting to 172.23.105.33 with username:root, attempt#1 of 5 2024-01-29 11:47:24,690 - root - INFO - SSH Connected to 172.23.105.33 as root 2024-01-29 11:47:24,812 - root - INFO - os_distro: Ubuntu, os_version: debian 10, is_linux_distro: True 2024-01-29 11:47:25,096 - root - INFO - extract_remote_info-->distribution_type: Ubuntu, distribution_version: debian 10 2024-01-29 11:47:25,152 - root - INFO - 172.23.105.33 **** The linux version file /opt/couchbase/ VERSION.txt exists 2024-01-29 11:47:25,232 - root - INFO - Running systemd stop command on this server 2024-01-29 11:47:25,233 - root - INFO - running command.raw on 172.23.105.33: systemctl stop couchbase-server.service 2024-01-29 11:47:27,685 - root - INFO - command executed successfully with root 2024-01-29 11:47:27,686 - root - INFO - *** servers ['172.23.106.112', '172.23.107.26', '172.23.104.102', '172.23.105.33'] will be upgraded to 7.6.0-2054 version *** 2024-01-29 11:47:27,687 - root - INFO - SSH Connecting to 172.23.106.112 with username:root, attempt#1 of 5 2024-01-29 11:47:27,689 - root - INFO - SSH Connecting to 172.23.107.26 with username:root, attempt#1 of 5 2024-01-29 11:47:27,690 - root - INFO - SSH Connecting to 172.23.104.102 with username:root, attempt#1 of 5 2024-01-29 11:47:27,692 - root - INFO - SSH Connecting to 172.23.105.33 with username:root, attempt#1 of 5 2024-01-29 11:47:27,766 - root - INFO - SSH Connected to 172.23.105.33 as root 2024-01-29 11:47:27,829 - root - INFO - SSH Connected to 172.23.104.102 as root 2024-01-29 11:47:27,872 - root - INFO - SSH Connected to 172.23.106.112 as root 2024-01-29 11:47:27,876 - root - INFO - SSH Connected to 172.23.107.26 as root 2024-01-29 11:47:27,888 - root - INFO - os_distro: Ubuntu, os_version: debian 10, is_linux_distro: True 2024-01-29 11:47:27,981 - root - INFO - os_distro: Ubuntu, os_version: debian 11, is_linux_distro: True 2024-01-29 11:47:28,110 - root - INFO - os_distro: Ubuntu, os_version: debian 11, is_linux_distro: True 2024-01-29 11:47:28,112 - root - INFO - os_distro: Ubuntu, os_version: debian 11, is_linux_distro: True 2024-01-29 11:47:28,202 - root - INFO - extract_remote_info-->distribution_type: Ubuntu, distribution_version: debian 10 2024-01-29 11:47:28,203 - root - INFO - finding build 7.6.0-2054 for machine ip:172.23.105.33 port:8091 ssh_username:root 2024-01-29 11:47:28,204 - root - INFO - running command.raw on 172.23.105.33: /sbin/iptables -F 2024-01-29 11:47:28,253 - root - INFO - command executed successfully with root 2024-01-29 11:47:28,254 - root - INFO - running command.raw on 172.23.105.33: /sbin/iptables -t nat -F 2024-01-29 11:47:28,301 - root - INFO - command executed successfully with root 2024-01-29 11:47:28,302 - root - INFO - running command.raw on 172.23.105.33: nft flush ruleset 2024-01-29 11:47:28,323 - root - INFO - extract_remote_info-->distribution_type: Ubuntu, distribution_version: debian 11 2024-01-29 11:47:28,324 - root - INFO - finding build 7.6.0-2054 for machine ip:172.23.104.102 port:8091 ssh_username:root 2024-01-29 11:47:28,324 - root - INFO - running command.raw on 172.23.104.102: /sbin/iptables -F 2024-01-29 11:47:28,344 - root - INFO - command executed successfully with root 2024-01-29 11:47:28,344 - root - INFO - running command.raw on 172.23.104.102: /sbin/iptables -t nat -F 2024-01-29 11:47:28,353 - root - INFO - command executed with root but got an error ['bash: nft: command not found'] ... 2024-01-29 11:47:28,353 - root - INFO - SSH Connecting to 172.23.105.33 with username:root, attempt#1 of 5 2024-01-29 11:47:28,400 - root - INFO - command executed successfully with root 2024-01-29 11:47:28,400 - root - INFO - running command.raw on 172.23.104.102: nft flush ruleset 2024-01-29 11:47:28,407 - root - INFO - extract_remote_info-->distribution_type: Ubuntu, distribution_version: debian 11 2024-01-29 11:47:28,408 - root - INFO - finding build 7.6.0-2054 for machine ip:172.23.106.112 port:8091 ssh_username:root 2024-01-29 11:47:28,408 - root - INFO - running command.raw on 172.23.106.112: /sbin/iptables -F 2024-01-29 11:47:28,420 - root - INFO - extract_remote_info-->distribution_type: Ubuntu, distribution_version: debian 11 2024-01-29 11:47:28,421 - root - INFO - finding build 7.6.0-2054 for machine ip:172.23.107.26 port:8091 ssh_username:root 2024-01-29 11:47:28,421 - root - INFO - running command.raw on 172.23.107.26: /sbin/iptables -F 2024-01-29 11:47:28,422 - root - INFO - SSH Connected to 172.23.105.33 as root 2024-01-29 11:47:28,440 - root - INFO - command executed successfully with root 2024-01-29 11:47:28,440 - root - INFO - running command.raw on 172.23.107.26: /sbin/iptables -t nat -F 2024-01-29 11:47:28,459 - root - INFO - command executed successfully with root 2024-01-29 11:47:28,459 - root - INFO - SSH Connecting to 172.23.104.102 with username:root, attempt#1 of 5 2024-01-29 11:47:28,463 - root - INFO - command executed successfully with root 2024-01-29 11:47:28,463 - root - INFO - running command.raw on 172.23.106.112: /sbin/iptables -t nat -F 2024-01-29 11:47:28,492 - root - INFO - command executed successfully with root 2024-01-29 11:47:28,492 - root - INFO - running command.raw on 172.23.107.26: nft flush ruleset 2024-01-29 11:47:28,520 - root - INFO - command executed successfully with root 2024-01-29 11:47:28,520 - root - INFO - running command.raw on 172.23.106.112: nft flush ruleset 2024-01-29 11:47:28,552 - root - INFO - command executed successfully with root 2024-01-29 11:47:28,552 - root - INFO - SSH Connecting to 172.23.107.26 with username:root, attempt#1 of 5 2024-01-29 11:47:28,568 - root - INFO - running command.raw on 172.23.105.33: cd /tmp;wget -nc -q -O couchbase-server-enterprise_7.6.0-2054-linux_amd64.deb http://172.23.126.166/builds/latestbuilds/couchbase-server/trinity/2054/couchbase-server-enterprise_7.6.0-2054-linux_amd64.deb;cd /tmp;ls -lh 2024-01-29 11:47:28,585 - root - INFO - command executed successfully with root 2024-01-29 11:47:28,585 - root - INFO - SSH Connecting to 172.23.106.112 with username:root, attempt#1 of 5 2024-01-29 11:47:28,639 - root - INFO - SSH Connected to 172.23.104.102 as root 2024-01-29 11:47:28,736 - root - INFO - SSH Connected to 172.23.107.26 as root 2024-01-29 11:47:28,775 - root - INFO - SSH Connected to 172.23.106.112 as root 2024-01-29 11:47:28,839 - root - INFO - running command.raw on 172.23.104.102: cd /tmp;wget -nc -q -O couchbase-server-enterprise_7.6.0-2054-linux_amd64.deb http://172.23.126.166/builds/latestbuilds/couchbase-server/trinity/2054/couchbase-server-enterprise_7.6.0-2054-linux_amd64.deb;cd /tmp;ls -lh 2024-01-29 11:47:28,926 - root - INFO - running command.raw on 172.23.107.26: cd /tmp;wget -nc -q -O couchbase-server-enterprise_7.6.0-2054-linux_amd64.deb http://172.23.126.166/builds/latestbuilds/couchbase-server/trinity/2054/couchbase-server-enterprise_7.6.0-2054-linux_amd64.deb;cd /tmp;ls -lh 2024-01-29 11:47:28,986 - root - INFO - running command.raw on 172.23.106.112: cd /tmp;wget -nc -q -O couchbase-server-enterprise_7.6.0-2054-linux_amd64.deb http://172.23.126.166/builds/latestbuilds/couchbase-server/trinity/2054/couchbase-server-enterprise_7.6.0-2054-linux_amd64.deb;cd /tmp;ls -lh 2024-01-29 11:47:30,077 - root - INFO - command executed successfully with root 2024-01-29 11:47:30,088 - root - INFO - deliverable_type : deb 2024-01-29 11:47:30,088 - root - INFO - /tmp/couchbase-server-enterprise_7.6.0-2054-linux_amd64.deb or /tmp/ 2024-01-29 11:47:30,088 - root - INFO - running command.raw on 172.23.105.33: dpkg -i /tmp/couchbase-server-enterprise_7.6.0-2054-linux_amd64.deb 2024-01-29 11:47:30,750 - root - INFO - command executed successfully with root 2024-01-29 11:47:30,770 - root - INFO - deliverable_type : deb 2024-01-29 11:47:30,770 - root - INFO - /tmp/couchbase-server-enterprise_7.6.0-2054-linux_amd64.deb or /tmp/ 2024-01-29 11:47:30,770 - root - INFO - running command.raw on 172.23.107.26: dpkg -i /tmp/couchbase-server-enterprise_7.6.0-2054-linux_amd64.deb 2024-01-29 11:47:30,852 - root - INFO - command executed successfully with root 2024-01-29 11:47:30,918 - root - INFO - deliverable_type : deb 2024-01-29 11:47:30,918 - root - INFO - /tmp/couchbase-server-enterprise_7.6.0-2054-linux_amd64.deb or /tmp/ 2024-01-29 11:47:30,918 - root - INFO - running command.raw on 172.23.106.112: dpkg -i /tmp/couchbase-server-enterprise_7.6.0-2054-linux_amd64.deb 2024-01-29 11:47:30,941 - root - INFO - command executed successfully with root 2024-01-29 11:47:31,003 - root - INFO - deliverable_type : deb 2024-01-29 11:47:31,003 - root - INFO - /tmp/couchbase-server-enterprise_7.6.0-2054-linux_amd64.deb or /tmp/ 2024-01-29 11:47:31,004 - root - INFO - running command.raw on 172.23.104.102: dpkg -i /tmp/couchbase-server-enterprise_7.6.0-2054-linux_amd64.deb 2024-01-29 11:48:17,989 - root - INFO - command executed successfully with root 2024-01-29 11:48:17,989 - root - INFO - Found 'You have successfully installed Couchbase Server.' in output 2024-01-29 11:48:17,989 - root - INFO - running command.raw on 172.23.107.26: sed -i 's/export PATH/export PATH\nexport CBFT_ENV_OPTIONS=bleveMaxResultWindow=10000000/' /opt/couchbase/bin/couchbase-server 2024-01-29 11:48:18,042 - root - INFO - command executed successfully with root 2024-01-29 11:48:18,043 - root - INFO - Checking for process beam.smp on linux 2024-01-29 11:48:18,105 - root - INFO - process beam.smp is running on 172.23.107.26: with pid 744205 2024-01-29 11:48:18,196 - root - INFO - Running systemd command on this server 2024-01-29 11:48:18,196 - root - INFO - running command.raw on 172.23.107.26: systemctl stop couchbase-server.service 2024-01-29 11:48:18,405 - root - INFO - command executed successfully with root 2024-01-29 11:48:18,405 - root - INFO - Found 'You have successfully installed Couchbase Server.' in output 2024-01-29 11:48:18,406 - root - INFO - running command.raw on 172.23.104.102: sed -i 's/export PATH/export PATH\nexport CBFT_ENV_OPTIONS=bleveMaxResultWindow=10000000/' /opt/couchbase/bin/couchbase-server 2024-01-29 11:48:18,437 - root - INFO - command executed successfully with root 2024-01-29 11:48:18,437 - root - INFO - Found 'You have successfully installed Couchbase Server.' in output 2024-01-29 11:48:18,437 - root - INFO - running command.raw on 172.23.106.112: sed -i 's/export PATH/export PATH\nexport CBFT_ENV_OPTIONS=bleveMaxResultWindow=10000000/' /opt/couchbase/bin/couchbase-server 2024-01-29 11:48:18,461 - root - INFO - command executed successfully with root 2024-01-29 11:48:18,462 - root - INFO - Checking for process beam.smp on linux 2024-01-29 11:48:18,493 - root - INFO - command executed successfully with root 2024-01-29 11:48:18,493 - root - INFO - Checking for process beam.smp on linux 2024-01-29 11:48:18,526 - root - INFO - process beam.smp is running on 172.23.104.102: with pid 1272800 2024-01-29 11:48:18,560 - root - INFO - process beam.smp is running on 172.23.106.112: with pid 881739 2024-01-29 11:48:18,621 - root - INFO - Running systemd command on this server 2024-01-29 11:48:18,621 - root - INFO - running command.raw on 172.23.104.102: systemctl stop couchbase-server.service 2024-01-29 11:48:18,663 - root - INFO - Running systemd command on this server 2024-01-29 11:48:18,663 - root - INFO - running command.raw on 172.23.106.112: systemctl stop couchbase-server.service 2024-01-29 11:48:19,670 - root - INFO - command executed successfully with root 2024-01-29 11:48:19,671 - root - INFO - Checking for process beam.smp on linux 2024-01-29 11:48:19,686 - root - INFO - Starting couchbase server 2024-01-29 11:48:19,717 - root - INFO - command executed successfully with root 2024-01-29 11:48:19,717 - root - INFO - Checking for process beam.smp on linux 2024-01-29 11:48:19,780 - root - INFO - Running systemd command on this server 2024-01-29 11:48:19,780 - root - INFO - running command.raw on 172.23.104.102: systemctl start couchbase-server.service 2024-01-29 11:48:19,783 - root - INFO - Starting couchbase server 2024-01-29 11:48:19,804 - root - INFO - command executed successfully with root 2024-01-29 11:48:19,804 - root - INFO - 172.23.104.102:sleep for 5 secs. waiting for couchbase server to come up ... 2024-01-29 11:48:19,869 - root - INFO - Running systemd command on this server 2024-01-29 11:48:19,869 - root - INFO - running command.raw on 172.23.106.112: systemctl start couchbase-server.service 2024-01-29 11:48:19,893 - root - INFO - command executed successfully with root 2024-01-29 11:48:19,893 - root - INFO - 172.23.106.112:sleep for 5 secs. waiting for couchbase server to come up ... 2024-01-29 11:48:24,809 - root - INFO - running command.raw on 172.23.104.102: systemctl status couchbase-server.service | grep ExecStop=/opt/couchbase/bin/couchbase-server 2024-01-29 11:48:24,839 - root - INFO - command executed successfully with root 2024-01-29 11:48:24,839 - root - INFO - Couchbase server status: [] 2024-01-29 11:48:24,840 - root - INFO - Checking for process beam.smp on linux 2024-01-29 11:48:24,845 - root - INFO - command executed successfully with root 2024-01-29 11:48:24,845 - root - INFO - Found 'You have successfully installed Couchbase Server.' in output 2024-01-29 11:48:24,845 - root - INFO - running command.raw on 172.23.105.33: sed -i 's/export PATH/export PATH\nexport CBFT_ENV_OPTIONS=bleveMaxResultWindow=10000000/' /opt/couchbase/bin/couchbase-server 2024-01-29 11:48:24,897 - root - INFO - command executed successfully with root 2024-01-29 11:48:24,897 - root - INFO - Checking for process beam.smp on linux 2024-01-29 11:48:24,898 - root - INFO - running command.raw on 172.23.106.112: systemctl status couchbase-server.service | grep ExecStop=/opt/couchbase/bin/couchbase-server 2024-01-29 11:48:24,902 - root - INFO - process beam.smp is running on 172.23.104.102: with pid 1272870 2024-01-29 11:48:24,903 - root - INFO - upgrade 172.23.104.102 to version 7.6.0-2054 is completed 2024-01-29 11:48:24,903 - root - INFO - sleep for 30 secs. ... 2024-01-29 11:48:24,920 - root - INFO - command executed successfully with root 2024-01-29 11:48:24,921 - root - INFO - Couchbase server status: [] 2024-01-29 11:48:24,921 - root - INFO - Checking for process beam.smp on linux 2024-01-29 11:48:24,954 - root - INFO - process beam.smp is running on 172.23.105.33: with pid 34026 2024-01-29 11:48:24,981 - root - INFO - process beam.smp is running on 172.23.106.112: with pid 881808 2024-01-29 11:48:24,982 - root - INFO - upgrade 172.23.106.112 to version 7.6.0-2054 is completed 2024-01-29 11:48:24,982 - root - INFO - sleep for 30 secs. ... 2024-01-29 11:48:25,018 - root - INFO - Running systemd command on this server 2024-01-29 11:48:25,018 - root - INFO - running command.raw on 172.23.105.33: systemctl stop couchbase-server.service 2024-01-29 11:48:54,950 - root - INFO - waiting for ns_server @ 172.23.104.102:8091 2024-01-29 11:48:54,951 - root - INFO - -->is_ns_server_running? 2024-01-29 11:48:54,956 - root - INFO - ns_server @ 172.23.104.102:8091 is running 2024-01-29 11:48:54,956 - root - INFO - --> in init_cluster...Administrator,password,8091 2024-01-29 11:48:54,956 - root - INFO - settings/web params on 172.23.104.102:8091:port=8091&username=Administrator&password=password 2024-01-29 11:48:54,959 - root - INFO - --> status:True 2024-01-29 11:48:54,959 - root - INFO - sleep for 15 secs. ... 2024-01-29 11:48:55,036 - root - INFO - waiting for ns_server @ 172.23.106.112:8091 2024-01-29 11:48:55,036 - root - INFO - -->is_ns_server_running? 2024-01-29 11:48:55,045 - root - INFO - ns_server @ 172.23.106.112:8091 is running 2024-01-29 11:48:55,045 - root - INFO - --> in init_cluster...Administrator,password,8091 2024-01-29 11:48:55,045 - root - INFO - settings/web params on 172.23.106.112:8091:port=8091&username=Administrator&password=password 2024-01-29 11:48:55,049 - root - INFO - --> status:True 2024-01-29 11:48:55,049 - root - INFO - sleep for 15 secs. ... 2024-01-29 11:49:09,971 - root - INFO - sleep for 30 secs. ... 2024-01-29 11:49:10,064 - root - INFO - sleep for 30 secs. ... 2024-01-29 11:49:48,385 - root - INFO - command executed successfully with root 2024-01-29 11:49:48,385 - root - INFO - Checking for process beam.smp on linux 2024-01-29 11:49:48,403 - root - INFO - Starting couchbase server 2024-01-29 11:49:48,490 - root - INFO - Running systemd command on this server 2024-01-29 11:49:48,490 - root - INFO - running command.raw on 172.23.107.26: systemctl start couchbase-server.service 2024-01-29 11:49:48,509 - root - INFO - command executed successfully with root 2024-01-29 11:49:48,510 - root - INFO - 172.23.107.26:sleep for 5 secs. waiting for couchbase server to come up ... 2024-01-29 11:49:53,514 - root - INFO - running command.raw on 172.23.107.26: systemctl status couchbase-server.service | grep ExecStop=/opt/couchbase/bin/couchbase-server 2024-01-29 11:49:53,565 - root - INFO - command executed successfully with root 2024-01-29 11:49:53,565 - root - INFO - Couchbase server status: [] 2024-01-29 11:49:53,565 - root - INFO - Checking for process beam.smp on linux 2024-01-29 11:49:53,583 - root - INFO - process beam.smp is running on 172.23.107.26: with pid 744708 2024-01-29 11:49:53,584 - root - INFO - upgrade 172.23.107.26 to version 7.6.0-2054 is completed 2024-01-29 11:49:53,584 - root - INFO - sleep for 30 secs. ... 2024-01-29 11:49:55,187 - root - INFO - command executed successfully with root 2024-01-29 11:49:55,187 - root - INFO - Checking for process beam.smp on linux 2024-01-29 11:49:55,245 - root - INFO - Starting couchbase server 2024-01-29 11:49:55,314 - root - INFO - Running systemd command on this server 2024-01-29 11:49:55,315 - root - INFO - running command.raw on 172.23.105.33: systemctl start couchbase-server.service 2024-01-29 11:49:55,328 - root - INFO - command executed successfully with root 2024-01-29 11:49:55,329 - root - INFO - 172.23.105.33:sleep for 5 secs. waiting for couchbase server to come up ... 2024-01-29 11:50:00,334 - root - INFO - running command.raw on 172.23.105.33: systemctl status couchbase-server.service | grep ExecStop=/opt/couchbase/bin/couchbase-server 2024-01-29 11:50:00,349 - root - INFO - command executed successfully with root 2024-01-29 11:50:00,349 - root - INFO - Couchbase server status: [] 2024-01-29 11:50:00,350 - root - INFO - Checking for process beam.smp on linux 2024-01-29 11:50:00,406 - root - INFO - process beam.smp is running on 172.23.105.33: with pid 34535 2024-01-29 11:50:00,406 - root - INFO - upgrade 172.23.105.33 to version 7.6.0-2054 is completed 2024-01-29 11:50:00,406 - root - INFO - sleep for 30 secs. ... 2024-01-29 11:50:23,635 - root - INFO - waiting for ns_server @ 172.23.107.26:8091 2024-01-29 11:50:23,635 - root - INFO - -->is_ns_server_running? 2024-01-29 11:50:23,640 - root - INFO - ns_server @ 172.23.107.26:8091 is running 2024-01-29 11:50:23,640 - root - INFO - --> in init_cluster...Administrator,password,8091 2024-01-29 11:50:23,640 - root - INFO - settings/web params on 172.23.107.26:8091:port=8091&username=Administrator&password=password 2024-01-29 11:50:23,643 - root - INFO - --> status:True 2024-01-29 11:50:23,643 - root - INFO - sleep for 15 secs. ... 2024-01-29 11:50:30,449 - root - INFO - waiting for ns_server @ 172.23.105.33:8091 2024-01-29 11:50:30,449 - root - INFO - -->is_ns_server_running? 2024-01-29 11:50:30,453 - root - INFO - ns_server @ 172.23.105.33:8091 is running 2024-01-29 11:50:30,454 - root - INFO - --> in init_cluster...Administrator,password,8091 2024-01-29 11:50:30,454 - root - INFO - settings/web params on 172.23.105.33:8091:port=8091&username=Administrator&password=password 2024-01-29 11:50:30,455 - root - INFO - --> status:True 2024-01-29 11:50:30,455 - root - INFO - sleep for 15 secs. ... 2024-01-29 11:50:38,658 - root - INFO - sleep for 30 secs. ... 2024-01-29 11:50:45,470 - root - INFO - sleep for 30 secs. ... 2024-01-29 11:51:15,500 - root - INFO - upgraded 1 servers: [ip:172.23.106.112 port:8091 ssh_username:root] 2024-01-29 11:51:15,500 - root - INFO - cluster is now in mixed mode 2024-01-29 11:51:15,518 - root - INFO - RUN QUERY SELECT * FROM system:indexes WHERE indexes.bucket_id is missing 2024-01-29 11:51:15,518 - root - INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+WHERE+indexes.bucket_id+is+missing&scan_consistency=REQUEST_PLUS 2024-01-29 11:51:15,629 - root - INFO - TOTAL ELAPSED TIME: 106.532747ms 2024-01-29 11:51:15,632 - root - INFO - RUN QUERY SELECT * FROM system:keyspaces WHERE keyspaces.`bucket` is missing 2024-01-29 11:51:15,632 - root - INFO - query params : statement=SELECT+%2A+FROM+system%3Akeyspaces+WHERE+keyspaces.%60bucket%60+is+missing&scan_consistency=REQUEST_PLUS 2024-01-29 11:51:15,681 - root - INFO - TOTAL ELAPSED TIME: 46.403236ms 2024-01-29 11:51:15,681 - root - INFO - ============== System Config: ============== 2024-01-29 11:51:15,687 - root - INFO - RUN QUERY SELECT COUNT(*) FROM `default` 2024-01-29 11:51:15,687 - root - INFO - query params : statement=SELECT+COUNT%28%2A%29+FROM+%60default%60&scan_consistency=REQUEST_PLUS 2024-01-29 11:51:15,739 - root - INFO - TOTAL ELAPSED TIME: 50.406694ms 2024-01-29 11:51:15,739 - root - INFO - Bucket: default 2024-01-29 11:51:15,739 - root - INFO - Indexes: ['#primary'] 2024-01-29 11:51:15,740 - root - INFO - Docs: 2016 2024-01-29 11:51:15,744 - root - INFO - RUN QUERY SELECT COUNT(*) FROM `standard_bucket0` 2024-01-29 11:51:15,744 - root - INFO - query params : statement=SELECT+COUNT%28%2A%29+FROM+%60standard_bucket0%60&scan_consistency=REQUEST_PLUS 2024-01-29 11:51:15,801 - root - INFO - TOTAL ELAPSED TIME: 54.705576ms 2024-01-29 11:51:15,801 - root - INFO - Bucket: standard_bucket0 2024-01-29 11:51:15,801 - root - INFO - Indexes: ['#primary'] 2024-01-29 11:51:15,801 - root - INFO - Docs: 2016 2024-01-29 11:51:15,806 - root - INFO - RUN QUERY SELECT COUNT(*) FROM `travel-sample` 2024-01-29 11:51:15,806 - root - INFO - query params : statement=SELECT+COUNT%28%2A%29+FROM+%60travel-sample%60&scan_consistency=REQUEST_PLUS 2024-01-29 11:51:15,866 - root - INFO - TOTAL ELAPSED TIME: 57.927555ms 2024-01-29 11:51:15,866 - root - INFO - Bucket: travel-sample 2024-01-29 11:51:15,866 - root - INFO - Indexes: ['def_faa', 'def_icao', 'def_sourceairport', 'def_schedule_utc', 'def_route_src_dst_day', 'def_airportname', 'def_city', 'def_name_type', 'def_type', '#primary', 'def_primary'] 2024-01-29 11:51:15,866 - root - INFO - Docs: 31591 2024-01-29 11:51:15,866 - root - INFO - ============================================= 2024-01-29 11:51:16,013 - root - INFO - RUN QUERY SELECT COUNT(*) FROM `default` 2024-01-29 11:51:16,014 - root - INFO - query params : statement=SELECT+COUNT%28%2A%29+FROM+%60default%60&scan_consistency=REQUEST_PLUS 2024-01-29 11:51:16,044 - root - INFO - TOTAL ELAPSED TIME: 28.148633ms 2024-01-29 11:51:16,047 - root - INFO - RUN QUERY SELECT COUNT(*) FROM `standard_bucket0` 2024-01-29 11:51:16,047 - root - INFO - query params : statement=SELECT+COUNT%28%2A%29+FROM+%60standard_bucket0%60&scan_consistency=REQUEST_PLUS 2024-01-29 11:51:16,075 - root - INFO - TOTAL ELAPSED TIME: 25.920955ms 2024-01-29 11:51:16,079 - root - INFO - RUN QUERY SELECT COUNT(*) FROM `travel-sample` 2024-01-29 11:51:16,079 - root - INFO - query params : statement=SELECT+COUNT%28%2A%29+FROM+%60travel-sample%60&scan_consistency=REQUEST_PLUS 2024-01-29 11:51:16,105 - root - INFO - TOTAL ELAPSED TIME: 24.960526ms 2024-01-29 11:51:16,108 - root - INFO - RUN QUERY SELECT * FROM system:indexes WHERE indexes.bucket_id is missing 2024-01-29 11:51:16,108 - root - INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+WHERE+indexes.bucket_id+is+missing&scan_consistency=REQUEST_PLUS 2024-01-29 11:51:16,248 - root - INFO - TOTAL ELAPSED TIME: 137.225107ms 2024-01-29 11:51:16,252 - root - INFO - RUN QUERY SELECT * FROM system:indexes where name='#primary' 2024-01-29 11:51:16,252 - root - INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name%3D%27%23primary%27&scan_consistency=REQUEST_PLUS 2024-01-29 11:51:16,908 - root - INFO - TOTAL ELAPSED TIME: 653.685234ms 2024-01-29 11:51:16,924 - root - INFO - RUN QUERY SELECT * FROM system:indexes where name='#primary' 2024-01-29 11:51:16,924 - root - INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name%3D%27%23primary%27&scan_consistency=REQUEST_PLUS 2024-01-29 11:51:17,471 - root - INFO - TOTAL ELAPSED TIME: 544.116747ms 2024-01-29 11:51:17,474 - root - INFO - RUN QUERY SELECT * FROM system:indexes where name='def_faa' 2024-01-29 11:51:17,474 - root - INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name%3D%27def_faa%27&scan_consistency=REQUEST_PLUS 2024-01-29 11:51:18,120 - root - INFO - TOTAL ELAPSED TIME: 644.535928ms 2024-01-29 11:51:18,128 - root - INFO - RUN QUERY SELECT * FROM system:indexes where name='def_icao' 2024-01-29 11:51:18,128 - root - INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name%3D%27def_icao%27&scan_consistency=REQUEST_PLUS 2024-01-29 11:51:18,653 - root - INFO - TOTAL ELAPSED TIME: 522.551234ms 2024-01-29 11:51:18,656 - root - INFO - RUN QUERY SELECT * FROM system:indexes where name='def_airportname' 2024-01-29 11:51:18,657 - root - INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name%3D%27def_airportname%27&scan_consistency=REQUEST_PLUS 2024-01-29 11:51:19,073 - root - INFO - TOTAL ELAPSED TIME: 414.460545ms 2024-01-29 11:51:19,077 - root - INFO - RUN QUERY SELECT * FROM system:indexes where name='def_city' 2024-01-29 11:51:19,077 - root - INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name%3D%27def_city%27&scan_consistency=REQUEST_PLUS 2024-01-29 11:51:19,643 - root - INFO - TOTAL ELAPSED TIME: 563.896071ms 2024-01-29 11:51:19,650 - root - INFO - RUN QUERY SELECT * FROM system:indexes where name='def_type' 2024-01-29 11:51:19,650 - root - INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name%3D%27def_type%27&scan_consistency=REQUEST_PLUS 2024-01-29 11:51:20,055 - root - INFO - TOTAL ELAPSED TIME: 402.370276ms 2024-01-29 11:51:20,059 - root - INFO - RUN QUERY SELECT * FROM system:indexes where name='def_sourceairport' 2024-01-29 11:51:20,059 - root - INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name%3D%27def_sourceairport%27&scan_consistency=REQUEST_PLUS 2024-01-29 11:51:20,718 - root - INFO - TOTAL ELAPSED TIME: 656.213333ms 2024-01-29 11:51:20,726 - root - INFO - RUN QUERY SELECT * FROM system:indexes where name='def_schedule_utc' 2024-01-29 11:51:20,726 - root - INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name%3D%27def_schedule_utc%27&scan_consistency=REQUEST_PLUS 2024-01-29 11:51:21,311 - root - INFO - TOTAL ELAPSED TIME: 583.296046ms 2024-01-29 11:51:21,315 - root - INFO - RUN QUERY SELECT * FROM system:indexes where name='def_name_type' 2024-01-29 11:51:21,315 - root - INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name%3D%27def_name_type%27&scan_consistency=REQUEST_PLUS 2024-01-29 11:51:21,871 - root - INFO - TOTAL ELAPSED TIME: 554.036819ms 2024-01-29 11:51:21,878 - root - INFO - RUN QUERY SELECT * FROM system:indexes where name='def_route_src_dst_day' 2024-01-29 11:51:21,879 - root - INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name%3D%27def_route_src_dst_day%27&scan_consistency=REQUEST_PLUS 2024-01-29 11:51:22,493 - root - INFO - TOTAL ELAPSED TIME: 612.619514ms 2024-01-29 11:51:22,496 - root - INFO - RUN QUERY SELECT * FROM system:indexes where name='#primary' 2024-01-29 11:51:22,496 - root - INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name%3D%27%23primary%27&scan_consistency=REQUEST_PLUS 2024-01-29 11:51:22,987 - root - INFO - TOTAL ELAPSED TIME: 488.263828ms 2024-01-29 11:51:22,995 - root - INFO - RUN QUERY SELECT * FROM system:indexes where name='def_primary' 2024-01-29 11:51:22,996 - root - INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name%3D%27def_primary%27&scan_consistency=REQUEST_PLUS 2024-01-29 11:51:23,609 - root - INFO - TOTAL ELAPSED TIME: 610.806843ms 2024-01-29 11:51:23,612 - root - INFO - RUN QUERY SELECT * FROM system:indexes WHERE indexes.bucket_id is missing 2024-01-29 11:51:23,612 - root - INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+WHERE+indexes.bucket_id+is+missing&scan_consistency=REQUEST_PLUS 2024-01-29 11:51:23,866 - root - INFO - TOTAL ELAPSED TIME: 251.838571ms 2024-01-29 11:51:23,870 - root - INFO - RUN QUERY SELECT * FROM system:keyspaces WHERE keyspaces.`bucket` is missing 2024-01-29 11:51:23,870 - root - INFO - query params : statement=SELECT+%2A+FROM+system%3Akeyspaces+WHERE+keyspaces.%60bucket%60+is+missing&scan_consistency=REQUEST_PLUS 2024-01-29 11:51:23,941 - root - INFO - TOTAL ELAPSED TIME: 68.541855ms 2024-01-29 11:51:23,941 - root - INFO - ============== System Config: ============== 2024-01-29 11:51:23,944 - root - INFO - RUN QUERY SELECT COUNT(*) FROM `default` 2024-01-29 11:51:23,945 - root - INFO - query params : statement=SELECT+COUNT%28%2A%29+FROM+%60default%60&scan_consistency=REQUEST_PLUS 2024-01-29 11:51:23,969 - root - INFO - TOTAL ELAPSED TIME: 22.67243ms 2024-01-29 11:51:23,969 - root - INFO - Bucket: default 2024-01-29 11:51:23,969 - root - INFO - Indexes: ['#primary'] 2024-01-29 11:51:23,969 - root - INFO - Docs: 2016 2024-01-29 11:51:23,973 - root - INFO - RUN QUERY SELECT COUNT(*) FROM `standard_bucket0` 2024-01-29 11:51:23,973 - root - INFO - query params : statement=SELECT+COUNT%28%2A%29+FROM+%60standard_bucket0%60&scan_consistency=REQUEST_PLUS 2024-01-29 11:51:24,000 - root - INFO - TOTAL ELAPSED TIME: 25.563209ms 2024-01-29 11:51:24,000 - root - INFO - Bucket: standard_bucket0 2024-01-29 11:51:24,000 - root - INFO - Indexes: ['#primary'] 2024-01-29 11:51:24,000 - root - INFO - Docs: 2016 2024-01-29 11:51:24,008 - root - INFO - RUN QUERY SELECT COUNT(*) FROM `travel-sample` 2024-01-29 11:51:24,008 - root - INFO - query params : statement=SELECT+COUNT%28%2A%29+FROM+%60travel-sample%60&scan_consistency=REQUEST_PLUS 2024-01-29 11:51:24,033 - root - INFO - TOTAL ELAPSED TIME: 23.320518ms 2024-01-29 11:51:24,033 - root - INFO - Bucket: travel-sample 2024-01-29 11:51:24,034 - root - INFO - Indexes: ['def_route_src_dst_day', 'def_airportname', 'def_icao', 'def_sourceairport', 'def_schedule_utc', 'def_name_type', 'def_faa', 'def_type', 'def_city', '#primary', 'def_primary'] 2024-01-29 11:51:24,034 - root - INFO - Docs: 31591 2024-01-29 11:51:24,034 - root - INFO - ============================================= 2024-01-29 11:51:24,034 - root - INFO - --> start: ensure_primary_indexes_exist.. 2024-01-29 11:51:24,038 - root - INFO - RUN QUERY SELECT * FROM system:keyspaces WHERE keyspaces.`bucket` is missing 2024-01-29 11:51:24,038 - root - INFO - query params : statement=SELECT+%2A+FROM+system%3Akeyspaces+WHERE+keyspaces.%60bucket%60+is+missing&scan_consistency=REQUEST_PLUS 2024-01-29 11:51:24,130 - root - INFO - TOTAL ELAPSED TIME: 89.670451ms 2024-01-29 11:51:24,130 - root - INFO - -->query_response:{'requestID': '8b1c52a5-f277-459b-b258-660c7d1b5c0a', 'signature': {'*': '*'}, 'results': [{'keyspaces': {'datastore_id': 'http://127.0.0.1:8091', 'id': 'default', 'name': 'default', 'namespace': 'default', 'namespace_id': 'default', 'path': 'default:default'}}, {'keyspaces': {'datastore_id': 'http://127.0.0.1:8091', 'id': 'standard_bucket0', 'name': 'standard_bucket0', 'namespace': 'default', 'namespace_id': 'default', 'path': 'default:standard_bucket0'}}, {'keyspaces': {'datastore_id': 'http://127.0.0.1:8091', 'id': 'travel-sample', 'name': 'travel-sample', 'namespace': 'default', 'namespace_id': 'default', 'path': 'default:travel-sample'}}], 'status': 'success', 'metrics': {'elapsedTime': '89.670451ms', 'executionTime': '89.540466ms', 'resultCount': 3, 'resultSize': 519, 'serviceLoad': 1}} 2024-01-29 11:51:24,136 - root - INFO - RUN QUERY SELECT * FROM system:indexes WHERE indexes.bucket_id is missing 2024-01-29 11:51:24,136 - root - INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+WHERE+indexes.bucket_id+is+missing&scan_consistency=REQUEST_PLUS 2024-01-29 11:51:24,487 - root - INFO - TOTAL ELAPSED TIME: 347.922795ms 2024-01-29 11:51:24,487 - root - INFO - -->before create indexes: [{'name': '#primary', 'bucket': 'default', 'fields': [], 'state': 'online', 'using': 'gsi', 'is_primary': True}, {'name': '#primary', 'bucket': 'standard_bucket0', 'fields': [], 'state': 'online', 'using': 'gsi', 'is_primary': True}, {'name': '#primary', 'bucket': 'travel-sample', 'fields': [], 'state': 'online', 'using': 'gsi', 'is_primary': True}],[{'name': '#primary', 'bucket': 'default', 'fields': frozenset(), 'state': 'online', 'using': 'gsi', 'where': '', 'is_primary': True}, {'name': '#primary', 'bucket': 'standard_bucket0', 'fields': frozenset(), 'state': 'online', 'using': 'gsi', 'where': '', 'is_primary': True}, {'name': '#primary', 'bucket': 'travel-sample', 'fields': frozenset(), 'state': 'online', 'using': 'gsi', 'where': '', 'is_primary': True}],[{'name': '#primary', 'bucket': 'default', 'fields': frozenset(), 'state': 'online', 'using': 'gsi', 'where': '', 'is_primary': True}, {'name': '#primary', 'bucket': 'standard_bucket0', 'fields': frozenset(), 'state': 'online', 'using': 'gsi', 'where': '', 'is_primary': True}, {'name': 'def_route_src_dst_day', 'bucket': 'travel-sample', 'fields': frozenset({('destinationairport', 1), ('distinct array v.day for v in schedule end', 2), ('sourceairport', 0)}), 'state': 'online', 'using': 'gsi', 'where': '(`type` = "route")', 'is_primary': False}, {'name': 'def_faa', 'bucket': 'travel-sample', 'fields': frozenset({('faa', 0)}), 'state': 'online', 'using': 'gsi', 'where': '', 'is_primary': False}, {'name': 'def_sourceairport', 'bucket': 'travel-sample', 'fields': frozenset({('sourceairport', 0)}), 'state': 'online', 'using': 'gsi', 'where': '', 'is_primary': False}, {'name': 'def_schedule_utc', 'bucket': 'travel-sample', 'fields': frozenset({('array s.utc for s in schedule end', 0)}), 'state': 'online', 'using': 'gsi', 'where': '', 'is_primary': False}, {'name': 'def_airportname', 'bucket': 'travel-sample', 'fields': frozenset({('airportname', 0)}), 'state': 'online', 'using': 'gsi', 'where': '', 'is_primary': False}, {'name': 'def_city', 'bucket': 'travel-sample', 'fields': frozenset({('city', 0)}), 'state': 'online', 'using': 'gsi', 'where': '', 'is_primary': False}, {'name': 'def_icao', 'bucket': 'travel-sample', 'fields': frozenset({('icao', 0)}), 'state': 'online', 'using': 'gsi', 'where': '', 'is_primary': False}, {'name': 'def_name_type', 'bucket': 'travel-sample', 'fields': frozenset({('name', 0)}), 'state': 'online', 'using': 'gsi', 'where': '(`_type` = "User")', 'is_primary': False}, {'name': 'def_type', 'bucket': 'travel-sample', 'fields': frozenset({('type', 0)}), 'state': 'online', 'using': 'gsi', 'where': '', 'is_primary': False}, {'name': 'def_primary', 'bucket': 'travel-sample', 'fields': frozenset(), 'state': 'online', 'using': 'gsi', 'where': '', 'is_primary': True}, {'name': '#primary', 'bucket': 'travel-sample', 'fields': frozenset(), 'state': 'online', 'using': 'gsi', 'where': '', 'is_primary': True}] 2024-01-29 11:51:24,488 - root - INFO - -->Create desired indexes..desiredset=frozenset({frozenset({('fields', frozenset()), ('bucket', 'default'), ('state', 'online'), ('where', ''), ('name', '#primary'), ('is_primary', True), ('using', 'gsi')}), frozenset({('fields', frozenset()), ('state', 'online'), ('bucket', 'travel-sample'), ('where', ''), ('name', '#primary'), ('is_primary', True), ('using', 'gsi')}), frozenset({('fields', frozenset()), ('state', 'online'), ('where', ''), ('bucket', 'standard_bucket0'), ('name', '#primary'), ('is_primary', True), ('using', 'gsi')})}),currentset=frozenset({frozenset({('fields', frozenset()), ('bucket', 'default'), ('state', 'online'), ('where', ''), ('name', '#primary'), ('is_primary', True), ('using', 'gsi')}), frozenset({('name', 'def_route_src_dst_day'), ('where', '(`type` = "route")'), ('state', 'online'), ('bucket', 'travel-sample'), ('fields', frozenset({('destinationairport', 1), ('distinct array v.day for v in schedule end', 2), ('sourceairport', 0)})), ('is_primary', False), ('using', 'gsi')}), frozenset({('name', 'def_sourceairport'), ('state', 'online'), ('fields', frozenset({('sourceairport', 0)})), ('bucket', 'travel-sample'), ('where', ''), ('is_primary', False), ('using', 'gsi')}), frozenset({('fields', frozenset()), ('state', 'online'), ('where', ''), ('bucket', 'standard_bucket0'), ('name', '#primary'), ('is_primary', True), ('using', 'gsi')}), frozenset({('fields', frozenset({('array s.utc for s in schedule end', 0)})), ('state', 'online'), ('bucket', 'travel-sample'), ('where', ''), ('name', 'def_schedule_utc'), ('is_primary', False), ('using', 'gsi')}), frozenset({('fields', frozenset()), ('state', 'online'), ('bucket', 'travel-sample'), ('where', ''), ('name', '#primary'), ('is_primary', True), ('using', 'gsi')}), frozenset({('name', 'def_faa'), ('state', 'online'), ('bucket', 'travel-sample'), ('where', ''), ('fields', frozenset({('faa', 0)})), ('is_primary', False), ('using', 'gsi')}), frozenset({('fields', frozenset({('city', 0)})), ('state', 'online'), ('bucket', 'travel-sample'), ('name', 'def_city'), ('where', ''), ('is_primary', False), ('using', 'gsi')}), frozenset({('fields', frozenset({('icao', 0)})), ('state', 'online'), ('bucket', 'travel-sample'), ('where', ''), ('name', 'def_icao'), ('is_primary', False), ('using', 'gsi')}), frozenset({('name', 'def_airportname'), ('fields', frozenset({('airportname', 0)})), ('state', 'online'), ('bucket', 'travel-sample'), ('where', ''), ('is_primary', False), ('using', 'gsi')}), frozenset({('name', 'def_name_type'), ('where', '(`_type` = "User")'), ('is_primary', False), ('state', 'online'), ('bucket', 'travel-sample'), ('fields', frozenset({('name', 0)})), ('using', 'gsi')}), frozenset({('fields', frozenset({('type', 0)})), ('state', 'online'), ('bucket', 'travel-sample'), ('name', 'def_type'), ('where', ''), ('is_primary', False), ('using', 'gsi')}), frozenset({('fields', frozenset()), ('state', 'online'), ('bucket', 'travel-sample'), ('where', ''), ('name', 'def_primary'), ('is_primary', True), ('using', 'gsi')})}),desiredindexes=[{'name': '#primary', 'bucket': 'default', 'fields': frozenset(), 'state': 'online', 'using': 'gsi', 'where': '', 'is_primary': True}, {'name': '#primary', 'bucket': 'standard_bucket0', 'fields': frozenset(), 'state': 'online', 'using': 'gsi', 'where': '', 'is_primary': True}, {'name': '#primary', 'bucket': 'travel-sample', 'fields': frozenset(), 'state': 'online', 'using': 'gsi', 'where': '', 'is_primary': True}] 2024-01-29 11:51:24,488 - root - INFO - --> end: ensure_primary_indexes_exist.. 2024-01-29 11:51:24,488 - root - INFO - running mixed mode test 2024-01-29 11:51:24,488 - root - INFO - running mixed-mode test for N1QL 2024-01-29 11:51:24,492 - root - INFO - RUN QUERY SELECT COUNT(*) FILTER (WHERE city = 'Paris') as count_paris FROM `travel-sample` WHERE type = 'airport' 2024-01-29 11:51:24,492 - root - INFO - query params : statement=SELECT+COUNT%28%2A%29+FILTER+%28WHERE+city+%3D+%27Paris%27%29+as+count_paris+FROM+%60travel-sample%60+WHERE+type+%3D+%27airport%27&scan_consistency=REQUEST_PLUS 2024-01-29 11:51:24,939 - root - INFO - TOTAL ELAPSED TIME: 445.034466ms 2024-01-29 11:51:24,942 - root - INFO - RUN QUERY SELECT COUNT(CASE WHEN city = 'Paris' THEN 1 ELSE NULL END) as count_paris FROM `travel-sample` WHERE type = 'airport' 2024-01-29 11:51:24,943 - root - INFO - query params : statement=SELECT+COUNT%28CASE+WHEN+city+%3D+%27Paris%27+THEN+1+ELSE+NULL+END%29+as+count_paris+FROM+%60travel-sample%60+WHERE+type+%3D+%27airport%27&scan_consistency=REQUEST_PLUS 2024-01-29 11:51:25,150 - root - INFO - TOTAL ELAPSED TIME: 205.730827ms 2024-01-29 11:51:25,159 - root - INFO - RUN QUERY SELECT d.id, d.destinationairport, AVG(d.distance) OVER ( window1 ) AS `rank` FROM `travel-sample` AS d WHERE d.type='route' WINDOW window1 AS (PARTITION BY d.destinationairport) ORDER BY 1 LIMIT 5 2024-01-29 11:51:25,159 - root - INFO - query params : statement=SELECT+d.id%2C+d.destinationairport%2C+++++++++++++AVG%28d.distance%29+OVER+%28+window1+%29+AS+%60rank%60+++++++++++++FROM+%60travel-sample%60+AS+d+WHERE+d.type%3D%27route%27+++++++++++++WINDOW+window1+AS+%28PARTITION+BY+d.destinationairport%29+++++++++++++ORDER+BY+1+++++++++++++LIMIT+5&scan_consistency=REQUEST_PLUS 2024-01-29 11:51:27,749 - root - INFO - TOTAL ELAPSED TIME: 2.587221216s 2024-01-29 11:51:27,757 - root - INFO - RUN QUERY SELECT d.id, d.destinationairport, AVG(d.distance) OVER ( PARTITION BY d.destinationairport ) AS `rank` FROM `travel-sample` AS d WHERE d.type='route' ORDER BY 1 LIMIT 5 2024-01-29 11:51:27,757 - root - INFO - query params : statement=SELECT+d.id%2C+d.destinationairport%2C+++++++++++++AVG%28d.distance%29+OVER+%28+PARTITION+BY+d.destinationairport+%29+AS+%60rank%60+++++++++++++FROM+%60travel-sample%60+AS+d+WHERE+d.type%3D%27route%27+++++++++++++ORDER+BY+1+++++++++++++LIMIT+5&scan_consistency=REQUEST_PLUS 2024-01-29 11:51:30,381 - root - INFO - TOTAL ELAPSED TIME: 2.620044764s 2024-01-29 11:51:30,391 - root - INFO - RUN QUERY EXECUTE FUNCTION func1(10,20,30) 2024-01-29 11:51:30,391 - root - INFO - query params : statement=EXECUTE+FUNCTION+func1%2810%2C20%2C30%29&scan_consistency=REQUEST_PLUS 2024-01-29 11:51:30,401 - root - INFO - TOTAL ELAPSED TIME: 8.12978ms 2024-01-29 11:51:30,404 - root - INFO - RUN QUERY SELECT func1(10,20,30) 2024-01-29 11:51:30,404 - root - INFO - query params : statement=SELECT+func1%2810%2C20%2C30%29&scan_consistency=REQUEST_PLUS 2024-01-29 11:51:30,407 - root - INFO - TOTAL ELAPSED TIME: 1.344545ms 2024-01-29 11:51:30,411 - root - INFO - RUN QUERY EXECUTE FUNCTION func2(1,3,5) 2024-01-29 11:51:30,411 - root - INFO - query params : statement=EXECUTE+FUNCTION+func2%281%2C3%2C5%29&scan_consistency=REQUEST_PLUS 2024-01-29 11:51:30,420 - root - INFO - TOTAL ELAPSED TIME: 8.14185ms 2024-01-29 11:51:30,421 - root - INFO - Create n1ql library 2024-01-29 11:51:30,421 - root - INFO - running command.raw on 172.23.106.112: curl -s -k -X DELETE http://172.23.106.112:8093/evaluator/v1/libraries/n1ql -u Administrator:password 2024-01-29 11:51:30,451 - root - INFO - command executed successfully with root 2024-01-29 11:51:30,451 - root - INFO - running command.raw on 172.23.106.112: curl -s -k -X POST http://172.23.106.112:8093/evaluator/v1/libraries/n1ql -u Administrator:password -H 'content-type: application/json' -d 'function select1() { var query = SELECT airportname FROM `travel-sample` WHERE type = "airport" AND city = "Lyon" ORDER BY airportname; var acc = []; for (const row of query) { acc.push(row); } return acc; }' 2024-01-29 11:51:30,546 - root - INFO - command executed successfully with root 2024-01-29 11:51:30,546 - root - INFO - (['{"status": "OK"}'], []) 2024-01-29 11:51:30,546 - root - INFO - running command.raw on 172.23.106.112: curl -s -k http://172.23.106.112:8093/evaluator/v1/libraries/n1ql -u Administrator:password 2024-01-29 11:51:30,612 - root - INFO - command executed successfully with root 2024-01-29 11:51:30,624 - root - INFO - RUN QUERY CREATE FUNCTION select1() LANGUAGE JAVASCRIPT AS "select1" AT "n1ql" 2024-01-29 11:51:30,624 - root - INFO - query params : statement=CREATE+FUNCTION+select1%28%29+LANGUAGE+JAVASCRIPT+AS+%22select1%22+AT+%22n1ql%22&scan_consistency=REQUEST_PLUS 2024-01-29 11:51:30,641 - root - INFO - TOTAL ELAPSED TIME: 14.758651ms 2024-01-29 11:51:30,656 - root - INFO - RUN QUERY EXECUTE FUNCTION select1() 2024-01-29 11:51:30,657 - root - INFO - query params : statement=EXECUTE+FUNCTION+select1%28%29&scan_consistency=REQUEST_PLUS 2024-01-29 11:51:30,720 - root - INFO - TOTAL ELAPSED TIME: 61.381375ms 2024-01-29 11:51:30,720 - root - INFO - Delete n1ql library 2024-01-29 11:51:30,720 - root - INFO - running command.raw on 172.23.106.112: curl -X DELETE http://172.23.106.112:8093/evaluator/v1/libraries/n1ql -u Administrator:password 2024-01-29 11:51:30,751 - root - INFO - command executed with root but got an error [' % Total % Received % Xferd Average Speed Time Time Time Current', ' Dload Upload Total Spent Left Speed', '', ' 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0', '100 16 100 16 0 0 1000 0 --:--:-- --:--:-- --:--:-- 1000'] ... 2024-01-29 11:51:30,751 - root - INFO - running command.raw on 172.23.106.112: curl http://172.23.106.112:8093/evaluator/v1/libraries/n1ql -u Administrator:password 2024-01-29 11:51:30,812 - root - INFO - command executed with root but got an error [' % Total % Received % Xferd Average Speed Time Time Time Current', ' Dload Upload Total Spent Left Speed', '', ' 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0', '100 26 100 26 0 0 26000 0 --:--:-- --:--:-- --:--:-- 26000'] ... 2024-01-29 11:51:30,821 - root - INFO - RUN QUERY DROP FUNCTION select1 2024-01-29 11:51:30,821 - root - INFO - query params : statement=DROP+FUNCTION+select1&scan_consistency=REQUEST_PLUS 2024-01-29 11:51:30,832 - root - INFO - TOTAL ELAPSED TIME: 8.856778ms 2024-01-29 11:51:30,845 - root - INFO - RUN QUERY SELECT ADVISOR([ "UPDATE `travel-sample` SET city = 'San Francisco' WHERE lower(city) = 'sanfrancisco'", "UPDATE `travel-sample` SET city = 'San Francisco' WHERE lower(city) = 'sanfrancisco'" ]) 2024-01-29 11:51:30,846 - root - INFO - query params : statement=SELECT+ADVISOR%28%5B+++++++++++++++++%22UPDATE+%60travel-sample%60+SET+city+%3D+%27San+Francisco%27+WHERE+lower%28city%29+%3D+%27sanfrancisco%27%22%2C+++++++++++++++++%22UPDATE+%60travel-sample%60+SET+city+%3D+%27San+Francisco%27+WHERE+lower%28city%29+%3D+%27sanfrancisco%27%22+++++++++++++%5D%29&scan_consistency=REQUEST_PLUS 2024-01-29 11:51:44,089 - root - INFO - TOTAL ELAPSED TIME: 13.239905347s 2024-01-29 11:51:44,099 - root - INFO - RUN QUERY SELECT ADVISOR({'action': 'start', 'duration': '1h', 'query_count': 2 }) 2024-01-29 11:51:44,099 - root - INFO - query params : statement=SELECT+ADVISOR%28%7B%27action%27%3A+%27start%27%2C+%27duration%27%3A+%271h%27%2C+%27query_count%27%3A+2+%7D%29&scan_consistency=REQUEST_PLUS 2024-01-29 11:51:44,104 - root - INFO - TOTAL ELAPSED TIME: 1.54042ms 2024-01-29 11:51:44,106 - root - INFO - RUN QUERY SELECT airportname FROM `travel-sample` WHERE lower(city) = 'lyon' AND country = 'France' 2024-01-29 11:51:44,106 - root - INFO - query params : statement=SELECT+airportname+FROM+%60travel-sample%60+WHERE+lower%28city%29+%3D+%27lyon%27+AND+country+%3D+%27France%27&scan_consistency=REQUEST_PLUS 2024-01-29 11:51:44,184 - root - INFO - TOTAL ELAPSED TIME: 75.264392ms 2024-01-29 11:51:44,189 - root - INFO - RUN QUERY SELECT airportname FROM `travel-sample` WHERE lower(city) = 'lyon' AND country = 'France' 2024-01-29 11:51:44,189 - root - INFO - query params : statement=SELECT+airportname+FROM+%60travel-sample%60+WHERE+lower%28city%29+%3D+%27lyon%27+AND+country+%3D+%27France%27&scan_consistency=REQUEST_PLUS 2024-01-29 11:51:44,260 - root - INFO - TOTAL ELAPSED TIME: 68.115349ms 2024-01-29 11:51:44,260 - root - INFO - sleep for 3 secs. ... 2024-01-29 11:51:47,278 - root - INFO - RUN QUERY SELECT ADVISOR({'action': 'stop', 'session': '15768663-03ce-45d7-be4f-d29640c7eb47'}) 2024-01-29 11:51:47,279 - root - INFO - query params : statement=SELECT+ADVISOR%28%7B%27action%27%3A+%27stop%27%2C+%27session%27%3A+%2715768663-03ce-45d7-be4f-d29640c7eb47%27%7D%29&scan_consistency=REQUEST_PLUS 2024-01-29 11:51:58,734 - root - INFO - TOTAL ELAPSED TIME: 11.451743315s 2024-01-29 11:51:58,743 - root - INFO - RUN QUERY SELECT ADVISOR({'action': 'get', 'session': '15768663-03ce-45d7-be4f-d29640c7eb47'}) 2024-01-29 11:51:58,744 - root - INFO - query params : statement=SELECT+ADVISOR%28%7B%27action%27%3A+%27get%27%2C+%27session%27%3A+%2715768663-03ce-45d7-be4f-d29640c7eb47%27%7D%29&scan_consistency=REQUEST_PLUS 2024-01-29 11:51:58,752 - root - INFO - TOTAL ELAPSED TIME: 6.486653ms 2024-01-29 11:51:58,755 - root - INFO - RUN QUERY SELECT ADVISOR({'action': 'purge', 'session': '15768663-03ce-45d7-be4f-d29640c7eb47'}) 2024-01-29 11:51:58,755 - root - INFO - query params : statement=SELECT+ADVISOR%28%7B%27action%27%3A+%27purge%27%2C+%27session%27%3A+%2715768663-03ce-45d7-be4f-d29640c7eb47%27%7D%29&scan_consistency=REQUEST_PLUS 2024-01-29 11:51:58,766 - root - INFO - TOTAL ELAPSED TIME: 9.358377ms 2024-01-29 11:51:58,768 - root - INFO - RUN QUERY select * from system:keyspaces_info where id = 'travel-sample' 2024-01-29 11:51:58,769 - root - INFO - query params : statement=select+%2A+from+system%3Akeyspaces_info+where+id+%3D+%27travel-sample%27&scan_consistency=REQUEST_PLUS 2024-01-29 11:51:59,190 - root - INFO - TOTAL ELAPSED TIME: 419.165917ms 2024-01-29 11:51:59,194 - root - INFO - RUN QUERY select array_agg(`path`) from system:all_keyspaces where datastore_id = 'system' 2024-01-29 11:51:59,194 - root - INFO - query params : statement=select+array_agg%28%60path%60%29+from+system%3Aall_keyspaces+where+datastore_id+%3D+%27system%27&scan_consistency=REQUEST_PLUS 2024-01-29 11:51:59,257 - root - INFO - TOTAL ELAPSED TIME: 60.724642ms 2024-01-29 11:51:59,260 - root - INFO - RUN QUERY select array_agg(distinct keyspace_id) from system:all_indexes where all_indexes.`namespace_id` = '#system' 2024-01-29 11:51:59,261 - root - INFO - query params : statement=select+array_agg%28distinct+keyspace_id%29+from+system%3Aall_indexes+where+all_indexes.%60namespace_id%60+%3D+%27%23system%27&scan_consistency=REQUEST_PLUS 2024-01-29 11:52:00,059 - root - INFO - TOTAL ELAPSED TIME: 796.61686ms 2024-01-29 11:52:00,068 - root - INFO - RUN QUERY select array_agg(id) from system:all_keyspaces_info where all_keyspaces_info.datastore_id = 'system' 2024-01-29 11:52:00,068 - root - INFO - query params : statement=select+array_agg%28id%29+from+system%3Aall_keyspaces_info+where+all_keyspaces_info.datastore_id+%3D+%27system%27&scan_consistency=REQUEST_PLUS 2024-01-29 11:52:02,045 - root - INFO - TOTAL ELAPSED TIME: 1.973201321s 2024-01-29 11:52:02,045 - root - INFO - completed mixed mode test 2024-01-29 11:52:02,045 - root - INFO - upgrading remaining servers 2024-01-29 11:52:02,045 - root - INFO - successfully upgraded 3 remaining servers: [ip:172.23.107.26 port:8091 ssh_username:root, ip:172.23.104.102 port:8091 ssh_username:root, ip:172.23.105.33 port:8091 ssh_username:root] 2024-01-29 11:52:02,055 - root - INFO - RUN QUERY SELECT * FROM system:indexes WHERE indexes.bucket_id is missing 2024-01-29 11:52:02,055 - root - INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+WHERE+indexes.bucket_id+is+missing&scan_consistency=REQUEST_PLUS 2024-01-29 11:52:02,300 - root - INFO - TOTAL ELAPSED TIME: 242.822213ms 2024-01-29 11:52:02,303 - root - INFO - RUN QUERY SELECT * FROM system:keyspaces WHERE keyspaces.`bucket` is missing 2024-01-29 11:52:02,303 - root - INFO - query params : statement=SELECT+%2A+FROM+system%3Akeyspaces+WHERE+keyspaces.%60bucket%60+is+missing&scan_consistency=REQUEST_PLUS 2024-01-29 11:52:02,361 - root - INFO - TOTAL ELAPSED TIME: 56.601256ms 2024-01-29 11:52:02,361 - root - INFO - ============== System Config: ============== 2024-01-29 11:52:02,367 - root - INFO - RUN QUERY SELECT COUNT(*) FROM `default` 2024-01-29 11:52:02,367 - root - INFO - query params : statement=SELECT+COUNT%28%2A%29+FROM+%60default%60&scan_consistency=REQUEST_PLUS 2024-01-29 11:52:02,395 - root - INFO - TOTAL ELAPSED TIME: 26.633661ms 2024-01-29 11:52:02,396 - root - INFO - Bucket: default 2024-01-29 11:52:02,396 - root - INFO - Indexes: ['#primary'] 2024-01-29 11:52:02,396 - root - INFO - Docs: 2016 2024-01-29 11:52:02,399 - root - INFO - RUN QUERY SELECT COUNT(*) FROM `standard_bucket0` 2024-01-29 11:52:02,399 - root - INFO - query params : statement=SELECT+COUNT%28%2A%29+FROM+%60standard_bucket0%60&scan_consistency=REQUEST_PLUS 2024-01-29 11:52:02,425 - root - INFO - TOTAL ELAPSED TIME: 24.715169ms 2024-01-29 11:52:02,425 - root - INFO - Bucket: standard_bucket0 2024-01-29 11:52:02,425 - root - INFO - Indexes: ['#primary'] 2024-01-29 11:52:02,426 - root - INFO - Docs: 2016 2024-01-29 11:52:02,429 - root - INFO - RUN QUERY SELECT COUNT(*) FROM `travel-sample` 2024-01-29 11:52:02,429 - root - INFO - query params : statement=SELECT+COUNT%28%2A%29+FROM+%60travel-sample%60&scan_consistency=REQUEST_PLUS 2024-01-29 11:52:02,455 - root - INFO - TOTAL ELAPSED TIME: 24.530405ms 2024-01-29 11:52:02,455 - root - INFO - Bucket: travel-sample 2024-01-29 11:52:02,456 - root - INFO - Indexes: ['def_faa', 'def_sourceairport', 'def_type', 'def_airportname', 'def_icao', 'def_schedule_utc', 'def_name_type', 'def_route_src_dst_day', 'def_city', '#primary', 'def_primary'] 2024-01-29 11:52:02,456 - root - INFO - Docs: 31591 2024-01-29 11:52:02,456 - root - INFO - ============================================= 2024-01-29 11:52:02,563 - root - INFO - RUN QUERY SELECT COUNT(*) FROM `default` 2024-01-29 11:52:02,563 - root - INFO - query params : statement=SELECT+COUNT%28%2A%29+FROM+%60default%60&scan_consistency=REQUEST_PLUS 2024-01-29 11:52:02,600 - root - INFO - TOTAL ELAPSED TIME: 34.673732ms 2024-01-29 11:52:02,603 - root - INFO - RUN QUERY SELECT COUNT(*) FROM `standard_bucket0` 2024-01-29 11:52:02,603 - root - INFO - query params : statement=SELECT+COUNT%28%2A%29+FROM+%60standard_bucket0%60&scan_consistency=REQUEST_PLUS 2024-01-29 11:52:02,627 - root - INFO - TOTAL ELAPSED TIME: 22.449058ms 2024-01-29 11:52:02,630 - root - INFO - RUN QUERY SELECT COUNT(*) FROM `travel-sample` 2024-01-29 11:52:02,630 - root - INFO - query params : statement=SELECT+COUNT%28%2A%29+FROM+%60travel-sample%60&scan_consistency=REQUEST_PLUS 2024-01-29 11:52:02,658 - root - INFO - TOTAL ELAPSED TIME: 25.941124ms 2024-01-29 11:52:02,661 - root - INFO - RUN QUERY SELECT * FROM system:indexes WHERE indexes.bucket_id is missing 2024-01-29 11:52:02,661 - root - INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+WHERE+indexes.bucket_id+is+missing&scan_consistency=REQUEST_PLUS 2024-01-29 11:52:03,006 - root - INFO - TOTAL ELAPSED TIME: 343.257794ms 2024-01-29 11:52:03,011 - root - INFO - RUN QUERY SELECT * FROM system:indexes where name='#primary' 2024-01-29 11:52:03,011 - root - INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name%3D%27%23primary%27&scan_consistency=REQUEST_PLUS 2024-01-29 11:52:03,643 - root - INFO - TOTAL ELAPSED TIME: 630.4894ms 2024-01-29 11:52:03,651 - root - INFO - RUN QUERY SELECT * FROM system:indexes where name='#primary' 2024-01-29 11:52:03,651 - root - INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name%3D%27%23primary%27&scan_consistency=REQUEST_PLUS 2024-01-29 11:52:04,167 - root - INFO - TOTAL ELAPSED TIME: 514.180381ms 2024-01-29 11:52:04,170 - root - INFO - RUN QUERY SELECT * FROM system:indexes where name='def_schedule_utc' 2024-01-29 11:52:04,171 - root - INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name%3D%27def_schedule_utc%27&scan_consistency=REQUEST_PLUS 2024-01-29 11:52:04,867 - root - INFO - TOTAL ELAPSED TIME: 695.007592ms 2024-01-29 11:52:04,878 - root - INFO - RUN QUERY SELECT * FROM system:indexes where name='def_name_type' 2024-01-29 11:52:04,878 - root - INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name%3D%27def_name_type%27&scan_consistency=REQUEST_PLUS 2024-01-29 11:52:05,618 - root - INFO - TOTAL ELAPSED TIME: 738.137313ms 2024-01-29 11:52:05,621 - root - INFO - RUN QUERY SELECT * FROM system:indexes where name='def_type' 2024-01-29 11:52:05,621 - root - INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name%3D%27def_type%27&scan_consistency=REQUEST_PLUS 2024-01-29 11:52:06,312 - root - INFO - TOTAL ELAPSED TIME: 688.966678ms 2024-01-29 11:52:06,319 - root - INFO - RUN QUERY SELECT * FROM system:indexes where name='def_airportname' 2024-01-29 11:52:06,320 - root - INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name%3D%27def_airportname%27&scan_consistency=REQUEST_PLUS 2024-01-29 11:52:06,964 - root - INFO - TOTAL ELAPSED TIME: 641.946906ms 2024-01-29 11:52:06,967 - root - INFO - RUN QUERY SELECT * FROM system:indexes where name='def_faa' 2024-01-29 11:52:06,967 - root - INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name%3D%27def_faa%27&scan_consistency=REQUEST_PLUS 2024-01-29 11:52:07,716 - root - INFO - TOTAL ELAPSED TIME: 747.002123ms 2024-01-29 11:52:07,725 - root - INFO - RUN QUERY SELECT * FROM system:indexes where name='def_sourceairport' 2024-01-29 11:52:07,725 - root - INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name%3D%27def_sourceairport%27&scan_consistency=REQUEST_PLUS 2024-01-29 11:52:08,159 - root - INFO - TOTAL ELAPSED TIME: 431.634564ms 2024-01-29 11:52:08,164 - root - INFO - RUN QUERY SELECT * FROM system:indexes where name='def_route_src_dst_day' 2024-01-29 11:52:08,164 - root - INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name%3D%27def_route_src_dst_day%27&scan_consistency=REQUEST_PLUS 2024-01-29 11:52:08,869 - root - INFO - TOTAL ELAPSED TIME: 703.176136ms 2024-01-29 11:52:08,880 - root - INFO - RUN QUERY SELECT * FROM system:indexes where name='def_city' 2024-01-29 11:52:08,880 - root - INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name%3D%27def_city%27&scan_consistency=REQUEST_PLUS 2024-01-29 11:52:09,443 - root - INFO - TOTAL ELAPSED TIME: 560.903456ms 2024-01-29 11:52:09,446 - root - INFO - RUN QUERY SELECT * FROM system:indexes where name='def_icao' 2024-01-29 11:52:09,447 - root - INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name%3D%27def_icao%27&scan_consistency=REQUEST_PLUS 2024-01-29 11:52:10,096 - root - INFO - TOTAL ELAPSED TIME: 647.601603ms 2024-01-29 11:52:10,104 - root - INFO - RUN QUERY SELECT * FROM system:indexes where name='#primary' 2024-01-29 11:52:10,104 - root - INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name%3D%27%23primary%27&scan_consistency=REQUEST_PLUS 2024-01-29 11:52:10,722 - root - INFO - TOTAL ELAPSED TIME: 615.010817ms 2024-01-29 11:52:10,725 - root - INFO - RUN QUERY SELECT * FROM system:indexes where name='def_primary' 2024-01-29 11:52:10,726 - root - INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name%3D%27def_primary%27&scan_consistency=REQUEST_PLUS 2024-01-29 11:52:11,361 - root - INFO - TOTAL ELAPSED TIME: 633.791378ms 2024-01-29 11:52:11,371 - root - INFO - RUN QUERY SELECT * FROM system:indexes WHERE indexes.bucket_id is missing 2024-01-29 11:52:11,371 - root - INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+WHERE+indexes.bucket_id+is+missing&scan_consistency=REQUEST_PLUS 2024-01-29 11:52:11,742 - root - INFO - TOTAL ELAPSED TIME: 368.875258ms 2024-01-29 11:52:11,745 - root - INFO - RUN QUERY SELECT * FROM system:keyspaces WHERE keyspaces.`bucket` is missing 2024-01-29 11:52:11,746 - root - INFO - query params : statement=SELECT+%2A+FROM+system%3Akeyspaces+WHERE+keyspaces.%60bucket%60+is+missing&scan_consistency=REQUEST_PLUS 2024-01-29 11:52:11,835 - root - INFO - TOTAL ELAPSED TIME: 86.414392ms 2024-01-29 11:52:11,835 - root - INFO - ============== System Config: ============== 2024-01-29 11:52:11,840 - root - INFO - RUN QUERY SELECT COUNT(*) FROM `default` 2024-01-29 11:52:11,841 - root - INFO - query params : statement=SELECT+COUNT%28%2A%29+FROM+%60default%60&scan_consistency=REQUEST_PLUS 2024-01-29 11:52:11,871 - root - INFO - TOTAL ELAPSED TIME: 27.788562ms 2024-01-29 11:52:11,871 - root - INFO - Bucket: default 2024-01-29 11:52:11,871 - root - INFO - Indexes: ['#primary'] 2024-01-29 11:52:11,871 - root - INFO - Docs: 2016 2024-01-29 11:52:11,875 - root - INFO - RUN QUERY SELECT COUNT(*) FROM `standard_bucket0` 2024-01-29 11:52:11,875 - root - INFO - query params : statement=SELECT+COUNT%28%2A%29+FROM+%60standard_bucket0%60&scan_consistency=REQUEST_PLUS 2024-01-29 11:52:11,907 - root - INFO - TOTAL ELAPSED TIME: 30.638616ms 2024-01-29 11:52:11,907 - root - INFO - Bucket: standard_bucket0 2024-01-29 11:52:11,907 - root - INFO - Indexes: ['#primary'] 2024-01-29 11:52:11,907 - root - INFO - Docs: 2016 2024-01-29 11:52:11,910 - root - INFO - RUN QUERY SELECT COUNT(*) FROM `travel-sample` 2024-01-29 11:52:11,911 - root - INFO - query params : statement=SELECT+COUNT%28%2A%29+FROM+%60travel-sample%60&scan_consistency=REQUEST_PLUS 2024-01-29 11:52:11,940 - root - INFO - TOTAL ELAPSED TIME: 27.650979ms 2024-01-29 11:52:11,940 - root - INFO - Bucket: travel-sample 2024-01-29 11:52:11,940 - root - INFO - Indexes: ['def_icao', 'def_sourceairport', 'def_schedule_utc', 'def_route_src_dst_day', 'def_faa', 'def_name_type', 'def_type', 'def_airportname', 'def_city', '#primary', 'def_primary'] 2024-01-29 11:52:11,940 - root - INFO - Docs: 31591 2024-01-29 11:52:11,941 - root - INFO - ============================================= 2024-01-29 11:52:11,941 - root - INFO - --> start: ensure_primary_indexes_exist.. 2024-01-29 11:52:11,943 - root - INFO - RUN QUERY SELECT * FROM system:keyspaces WHERE keyspaces.`bucket` is missing 2024-01-29 11:52:11,943 - root - INFO - query params : statement=SELECT+%2A+FROM+system%3Akeyspaces+WHERE+keyspaces.%60bucket%60+is+missing&scan_consistency=REQUEST_PLUS 2024-01-29 11:52:12,010 - root - INFO - TOTAL ELAPSED TIME: 64.502525ms 2024-01-29 11:52:12,010 - root - INFO - -->query_response:{'requestID': 'e264d402-385b-4691-84e0-3449ca3d3c86', 'signature': {'*': '*'}, 'results': [{'keyspaces': {'datastore_id': 'http://127.0.0.1:8091', 'id': 'default', 'name': 'default', 'namespace': 'default', 'namespace_id': 'default', 'path': 'default:default'}}, {'keyspaces': {'datastore_id': 'http://127.0.0.1:8091', 'id': 'standard_bucket0', 'name': 'standard_bucket0', 'namespace': 'default', 'namespace_id': 'default', 'path': 'default:standard_bucket0'}}, {'keyspaces': {'datastore_id': 'http://127.0.0.1:8091', 'id': 'travel-sample', 'name': 'travel-sample', 'namespace': 'default', 'namespace_id': 'default', 'path': 'default:travel-sample'}}], 'status': 'success', 'metrics': {'elapsedTime': '64.502525ms', 'executionTime': '64.430604ms', 'resultCount': 3, 'resultSize': 519, 'serviceLoad': 1}} 2024-01-29 11:52:12,014 - root - INFO - RUN QUERY SELECT * FROM system:indexes WHERE indexes.bucket_id is missing 2024-01-29 11:52:12,014 - root - INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+WHERE+indexes.bucket_id+is+missing&scan_consistency=REQUEST_PLUS 2024-01-29 11:52:12,330 - root - INFO - TOTAL ELAPSED TIME: 312.985864ms 2024-01-29 11:52:12,330 - root - INFO - -->before create indexes: [{'name': '#primary', 'bucket': 'default', 'fields': [], 'state': 'online', 'using': 'gsi', 'is_primary': True}, {'name': '#primary', 'bucket': 'standard_bucket0', 'fields': [], 'state': 'online', 'using': 'gsi', 'is_primary': True}, {'name': '#primary', 'bucket': 'travel-sample', 'fields': [], 'state': 'online', 'using': 'gsi', 'is_primary': True}],[{'name': '#primary', 'bucket': 'default', 'fields': frozenset(), 'state': 'online', 'using': 'gsi', 'where': '', 'is_primary': True}, {'name': '#primary', 'bucket': 'standard_bucket0', 'fields': frozenset(), 'state': 'online', 'using': 'gsi', 'where': '', 'is_primary': True}, {'name': '#primary', 'bucket': 'travel-sample', 'fields': frozenset(), 'state': 'online', 'using': 'gsi', 'where': '', 'is_primary': True}],[{'name': '#primary', 'bucket': 'default', 'fields': frozenset(), 'state': 'online', 'using': 'gsi', 'where': '', 'is_primary': True}, {'name': '#primary', 'bucket': 'standard_bucket0', 'fields': frozenset(), 'state': 'online', 'using': 'gsi', 'where': '', 'is_primary': True}, {'name': 'def_name_type', 'bucket': 'travel-sample', 'fields': frozenset({('name', 0)}), 'state': 'online', 'using': 'gsi', 'where': '(`_type` = "User")', 'is_primary': False}, {'name': 'def_type', 'bucket': 'travel-sample', 'fields': frozenset({('type', 0)}), 'state': 'online', 'using': 'gsi', 'where': '', 'is_primary': False}, {'name': 'def_airportname', 'bucket': 'travel-sample', 'fields': frozenset({('airportname', 0)}), 'state': 'online', 'using': 'gsi', 'where': '', 'is_primary': False}, {'name': 'def_city', 'bucket': 'travel-sample', 'fields': frozenset({('city', 0)}), 'state': 'online', 'using': 'gsi', 'where': '', 'is_primary': False}, {'name': 'def_faa', 'bucket': 'travel-sample', 'fields': frozenset({('faa', 0)}), 'state': 'online', 'using': 'gsi', 'where': '', 'is_primary': False}, {'name': 'def_icao', 'bucket': 'travel-sample', 'fields': frozenset({('icao', 0)}), 'state': 'online', 'using': 'gsi', 'where': '', 'is_primary': False}, {'name': 'def_sourceairport', 'bucket': 'travel-sample', 'fields': frozenset({('sourceairport', 0)}), 'state': 'online', 'using': 'gsi', 'where': '', 'is_primary': False}, {'name': 'def_schedule_utc', 'bucket': 'travel-sample', 'fields': frozenset({('array s.utc for s in schedule end', 0)}), 'state': 'online', 'using': 'gsi', 'where': '', 'is_primary': False}, {'name': 'def_route_src_dst_day', 'bucket': 'travel-sample', 'fields': frozenset({('destinationairport', 1), ('distinct array v.day for v in schedule end', 2), ('sourceairport', 0)}), 'state': 'online', 'using': 'gsi', 'where': '(`type` = "route")', 'is_primary': False}, {'name': '#primary', 'bucket': 'travel-sample', 'fields': frozenset(), 'state': 'online', 'using': 'gsi', 'where': '', 'is_primary': True}, {'name': 'def_primary', 'bucket': 'travel-sample', 'fields': frozenset(), 'state': 'online', 'using': 'gsi', 'where': '', 'is_primary': True}] 2024-01-29 11:52:12,331 - root - INFO - -->Create desired indexes..desiredset=frozenset({frozenset({('fields', frozenset()), ('bucket', 'default'), ('state', 'online'), ('where', ''), ('name', '#primary'), ('is_primary', True), ('using', 'gsi')}), frozenset({('fields', frozenset()), ('state', 'online'), ('bucket', 'travel-sample'), ('where', ''), ('name', '#primary'), ('is_primary', True), ('using', 'gsi')}), frozenset({('fields', frozenset()), ('state', 'online'), ('where', ''), ('bucket', 'standard_bucket0'), ('name', '#primary'), ('is_primary', True), ('using', 'gsi')})}),currentset=frozenset({frozenset({('fields', frozenset()), ('bucket', 'default'), ('state', 'online'), ('where', ''), ('name', '#primary'), ('is_primary', True), ('using', 'gsi')}), frozenset({('name', 'def_route_src_dst_day'), ('where', '(`type` = "route")'), ('state', 'online'), ('bucket', 'travel-sample'), ('fields', frozenset({('destinationairport', 1), ('distinct array v.day for v in schedule end', 2), ('sourceairport', 0)})), ('is_primary', False), ('using', 'gsi')}), frozenset({('name', 'def_sourceairport'), ('state', 'online'), ('fields', frozenset({('sourceairport', 0)})), ('bucket', 'travel-sample'), ('where', ''), ('is_primary', False), ('using', 'gsi')}), frozenset({('fields', frozenset()), ('state', 'online'), ('where', ''), ('bucket', 'standard_bucket0'), ('name', '#primary'), ('is_primary', True), ('using', 'gsi')}), frozenset({('fields', frozenset({('array s.utc for s in schedule end', 0)})), ('state', 'online'), ('bucket', 'travel-sample'), ('where', ''), ('name', 'def_schedule_utc'), ('is_primary', False), ('using', 'gsi')}), frozenset({('fields', frozenset()), ('state', 'online'), ('bucket', 'travel-sample'), ('where', ''), ('name', '#primary'), ('is_primary', True), ('using', 'gsi')}), frozenset({('name', 'def_faa'), ('state', 'online'), ('bucket', 'travel-sample'), ('where', ''), ('fields', frozenset({('faa', 0)})), ('is_primary', False), ('using', 'gsi')}), frozenset({('fields', frozenset({('city', 0)})), ('state', 'online'), ('bucket', 'travel-sample'), ('name', 'def_city'), ('where', ''), ('is_primary', False), ('using', 'gsi')}), frozenset({('fields', frozenset({('icao', 0)})), ('state', 'online'), ('bucket', 'travel-sample'), ('where', ''), ('name', 'def_icao'), ('is_primary', False), ('using', 'gsi')}), frozenset({('name', 'def_airportname'), ('fields', frozenset({('airportname', 0)})), ('state', 'online'), ('bucket', 'travel-sample'), ('where', ''), ('is_primary', False), ('using', 'gsi')}), frozenset({('name', 'def_name_type'), ('where', '(`_type` = "User")'), ('is_primary', False), ('state', 'online'), ('bucket', 'travel-sample'), ('fields', frozenset({('name', 0)})), ('using', 'gsi')}), frozenset({('fields', frozenset({('type', 0)})), ('state', 'online'), ('bucket', 'travel-sample'), ('name', 'def_type'), ('where', ''), ('is_primary', False), ('using', 'gsi')}), frozenset({('fields', frozenset()), ('state', 'online'), ('bucket', 'travel-sample'), ('where', ''), ('name', 'def_primary'), ('is_primary', True), ('using', 'gsi')})}),desiredindexes=[{'name': '#primary', 'bucket': 'default', 'fields': frozenset(), 'state': 'online', 'using': 'gsi', 'where': '', 'is_primary': True}, {'name': '#primary', 'bucket': 'standard_bucket0', 'fields': frozenset(), 'state': 'online', 'using': 'gsi', 'where': '', 'is_primary': True}, {'name': '#primary', 'bucket': 'travel-sample', 'fields': frozenset(), 'state': 'online', 'using': 'gsi', 'where': '', 'is_primary': True}] 2024-01-29 11:52:12,331 - root - INFO - --> end: ensure_primary_indexes_exist.. 2024-01-29 11:52:12,331 - root - INFO - running post upgrade test 2024-01-29 11:52:12,334 - root - INFO - RUN QUERY SELECT COUNT(*) FILTER (WHERE city = 'Paris') as count_paris FROM `travel-sample` WHERE type = 'airport' 2024-01-29 11:52:12,334 - root - INFO - query params : statement=SELECT+COUNT%28%2A%29+FILTER+%28WHERE+city+%3D+%27Paris%27%29+as+count_paris+FROM+%60travel-sample%60+WHERE+type+%3D+%27airport%27&scan_consistency=REQUEST_PLUS 2024-01-29 11:52:12,530 - root - INFO - TOTAL ELAPSED TIME: 193.928232ms 2024-01-29 11:52:12,539 - root - INFO - RUN QUERY SELECT COUNT(CASE WHEN city = 'Paris' THEN 1 ELSE NULL END) as count_paris FROM `travel-sample` WHERE type = 'airport' 2024-01-29 11:52:12,539 - root - INFO - query params : statement=SELECT+COUNT%28CASE+WHEN+city+%3D+%27Paris%27+THEN+1+ELSE+NULL+END%29+as+count_paris+FROM+%60travel-sample%60+WHERE+type+%3D+%27airport%27&scan_consistency=REQUEST_PLUS 2024-01-29 11:52:12,720 - root - INFO - TOTAL ELAPSED TIME: 179.035219ms 2024-01-29 11:52:12,725 - root - INFO - RUN QUERY SELECT d.id, d.destinationairport, AVG(d.distance) OVER ( window1 ) AS `rank` FROM `travel-sample` AS d WHERE d.type='route' WINDOW window1 AS (PARTITION BY d.destinationairport) ORDER BY 1 LIMIT 5 2024-01-29 11:52:12,725 - root - INFO - query params : statement=SELECT+d.id%2C+d.destinationairport%2C+++++++++++++AVG%28d.distance%29+OVER+%28+window1+%29+AS+%60rank%60+++++++++++++FROM+%60travel-sample%60+AS+d+WHERE+d.type%3D%27route%27+++++++++++++WINDOW+window1+AS+%28PARTITION+BY+d.destinationairport%29+++++++++++++ORDER+BY+1+++++++++++++LIMIT+5&scan_consistency=REQUEST_PLUS 2024-01-29 11:52:15,206 - root - INFO - TOTAL ELAPSED TIME: 2.477700809s 2024-01-29 11:52:15,215 - root - INFO - RUN QUERY SELECT d.id, d.destinationairport, AVG(d.distance) OVER ( PARTITION BY d.destinationairport ) AS `rank` FROM `travel-sample` AS d WHERE d.type='route' ORDER BY 1 LIMIT 5 2024-01-29 11:52:15,215 - root - INFO - query params : statement=SELECT+d.id%2C+d.destinationairport%2C+++++++++++++AVG%28d.distance%29+OVER+%28+PARTITION+BY+d.destinationairport+%29+AS+%60rank%60+++++++++++++FROM+%60travel-sample%60+AS+d+WHERE+d.type%3D%27route%27+++++++++++++ORDER+BY+1+++++++++++++LIMIT+5&scan_consistency=REQUEST_PLUS 2024-01-29 11:52:17,790 - root - INFO - TOTAL ELAPSED TIME: 2.572361146s 2024-01-29 11:52:17,799 - root - INFO - RUN QUERY EXECUTE FUNCTION func1(10,20,30) 2024-01-29 11:52:17,799 - root - INFO - query params : statement=EXECUTE+FUNCTION+func1%2810%2C20%2C30%29&scan_consistency=REQUEST_PLUS 2024-01-29 11:52:17,807 - root - INFO - TOTAL ELAPSED TIME: 4.52042ms 2024-01-29 11:52:17,810 - root - INFO - RUN QUERY SELECT func1(10,20,30) 2024-01-29 11:52:17,810 - root - INFO - query params : statement=SELECT+func1%2810%2C20%2C30%29&scan_consistency=REQUEST_PLUS 2024-01-29 11:52:17,812 - root - INFO - TOTAL ELAPSED TIME: 854.393µs 2024-01-29 11:52:17,815 - root - INFO - RUN QUERY EXECUTE FUNCTION func2(1,3,5) 2024-01-29 11:52:17,815 - root - INFO - query params : statement=EXECUTE+FUNCTION+func2%281%2C3%2C5%29&scan_consistency=REQUEST_PLUS 2024-01-29 11:52:17,821 - root - INFO - TOTAL ELAPSED TIME: 4.494102ms 2024-01-29 11:52:17,821 - root - INFO - Delete math library 2024-01-29 11:52:17,821 - root - INFO - running command.raw on 172.23.106.112: curl -X DELETE http://172.23.106.112:8093/evaluator/v1/libraries/math -u Administrator:password 2024-01-29 11:52:17,856 - root - INFO - command executed with root but got an error [' % Total % Received % Xferd Average Speed Time Time Time Current', ' Dload Upload Total Spent Left Speed', '', ' 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0', '100 16 100 16 0 0 941 0 --:--:-- --:--:-- --:--:-- 941'] ... 2024-01-29 11:52:17,856 - root - INFO - running command.raw on 172.23.106.112: curl http://172.23.106.112:8093/evaluator/v1/libraries/math -u Administrator:password 2024-01-29 11:52:17,917 - root - INFO - command executed with root but got an error [' % Total % Received % Xferd Average Speed Time Time Time Current', ' Dload Upload Total Spent Left Speed', '', ' 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0', '100 26 100 26 0 0 26000 0 --:--:-- --:--:-- --:--:-- 26000'] ... 2024-01-29 11:52:17,935 - root - INFO - RUN QUERY DROP FUNCTION func2 2024-01-29 11:52:17,935 - root - INFO - query params : statement=DROP+FUNCTION+func2&scan_consistency=REQUEST_PLUS 2024-01-29 11:52:17,951 - root - INFO - TOTAL ELAPSED TIME: 13.879596ms 2024-01-29 11:52:17,951 - root - INFO - Create math library 2024-01-29 11:52:17,951 - root - INFO - running command.raw on 172.23.106.112: curl -s -k -X DELETE http://172.23.106.112:8093/evaluator/v1/libraries/math -u Administrator:password 2024-01-29 11:52:17,973 - root - INFO - command executed successfully with root 2024-01-29 11:52:17,974 - root - INFO - running command.raw on 172.23.106.112: curl -s -k -X POST http://172.23.106.112:8093/evaluator/v1/libraries/math -u Administrator:password -H 'content-type: application/json' -d 'function adder(a, b, c) { for (i=0; i< b; i++){a = a + c;} return a; }' 2024-01-29 11:52:18,061 - root - INFO - command executed successfully with root 2024-01-29 11:52:18,061 - root - INFO - (['{"status": "OK"}'], []) 2024-01-29 11:52:18,061 - root - INFO - running command.raw on 172.23.106.112: curl -s -k http://172.23.106.112:8093/evaluator/v1/libraries/math -u Administrator:password 2024-01-29 11:52:18,121 - root - INFO - command executed successfully with root 2024-01-29 11:52:18,134 - root - INFO - RUN QUERY CREATE FUNCTION func2(a,b,c) LANGUAGE JAVASCRIPT AS "adder" AT "math" 2024-01-29 11:52:18,134 - root - INFO - query params : statement=CREATE+FUNCTION+func2%28a%2Cb%2Cc%29+LANGUAGE+JAVASCRIPT+AS+%22adder%22+AT+%22math%22&scan_consistency=REQUEST_PLUS 2024-01-29 11:52:18,147 - root - INFO - TOTAL ELAPSED TIME: 12.070649ms 2024-01-29 11:52:18,155 - root - INFO - RUN QUERY EXECUTE FUNCTION func2(1,3,5) 2024-01-29 11:52:18,155 - root - INFO - query params : statement=EXECUTE+FUNCTION+func2%281%2C3%2C5%29&scan_consistency=REQUEST_PLUS 2024-01-29 11:52:18,160 - root - INFO - TOTAL ELAPSED TIME: 3.028271ms 2024-01-29 11:52:18,160 - root - INFO - Delete math library 2024-01-29 11:52:18,160 - root - INFO - running command.raw on 172.23.106.112: curl -X DELETE http://172.23.106.112:8093/evaluator/v1/libraries/math -u Administrator:password 2024-01-29 11:52:18,186 - root - INFO - command executed with root but got an error [' % Total % Received % Xferd Average Speed Time Time Time Current', ' Dload Upload Total Spent Left Speed', '', ' 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0', '100 16 100 16 0 0 1777 0 --:--:-- --:--:-- --:--:-- 1777'] ... 2024-01-29 11:52:18,187 - root - INFO - running command.raw on 172.23.106.112: curl http://172.23.106.112:8093/evaluator/v1/libraries/math -u Administrator:password 2024-01-29 11:52:18,249 - root - INFO - command executed with root but got an error [' % Total % Received % Xferd Average Speed Time Time Time Current', ' Dload Upload Total Spent Left Speed', '', ' 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0', '100 26 100 26 0 0 26000 0 --:--:-- --:--:-- --:--:-- 26000'] ... 2024-01-29 11:52:18,262 - root - INFO - RUN QUERY DROP FUNCTION func2 2024-01-29 11:52:18,262 - root - INFO - query params : statement=DROP+FUNCTION+func2&scan_consistency=REQUEST_PLUS 2024-01-29 11:52:18,273 - root - INFO - TOTAL ELAPSED TIME: 7.803519ms 2024-01-29 11:52:18,273 - root - INFO - Create n1ql library 2024-01-29 11:52:18,273 - root - INFO - running command.raw on 172.23.106.112: curl -s -k -X DELETE http://172.23.106.112:8093/evaluator/v1/libraries/n1ql -u Administrator:password 2024-01-29 11:52:18,306 - root - INFO - command executed successfully with root 2024-01-29 11:52:18,306 - root - INFO - running command.raw on 172.23.106.112: curl -s -k -X POST http://172.23.106.112:8093/evaluator/v1/libraries/n1ql -u Administrator:password -H 'content-type: application/json' -d 'function select1() { var query = SELECT airportname FROM `travel-sample` WHERE type = "airport" AND city = "Lyon" ORDER BY airportname; var acc = []; for (const row of query) { acc.push(row); } return acc; }' 2024-01-29 11:52:18,404 - root - INFO - command executed successfully with root 2024-01-29 11:52:18,405 - root - INFO - (['{"status": "OK"}'], []) 2024-01-29 11:52:18,405 - root - INFO - running command.raw on 172.23.106.112: curl -s -k http://172.23.106.112:8093/evaluator/v1/libraries/n1ql -u Administrator:password 2024-01-29 11:52:18,421 - root - INFO - command executed successfully with root 2024-01-29 11:52:18,431 - root - INFO - RUN QUERY CREATE FUNCTION select1() LANGUAGE JAVASCRIPT AS "select1" AT "n1ql" 2024-01-29 11:52:18,431 - root - INFO - query params : statement=CREATE+FUNCTION+select1%28%29+LANGUAGE+JAVASCRIPT+AS+%22select1%22+AT+%22n1ql%22&scan_consistency=REQUEST_PLUS 2024-01-29 11:52:18,445 - root - INFO - TOTAL ELAPSED TIME: 12.50354ms 2024-01-29 11:52:18,455 - root - INFO - RUN QUERY EXECUTE FUNCTION select1() 2024-01-29 11:52:18,455 - root - INFO - query params : statement=EXECUTE+FUNCTION+select1%28%29&scan_consistency=REQUEST_PLUS 2024-01-29 11:52:18,515 - root - INFO - TOTAL ELAPSED TIME: 58.293581ms 2024-01-29 11:52:18,515 - root - INFO - Delete n1ql library 2024-01-29 11:52:18,516 - root - INFO - running command.raw on 172.23.106.112: curl -X DELETE http://172.23.106.112:8093/evaluator/v1/libraries/n1ql -u Administrator:password 2024-01-29 11:52:18,541 - root - INFO - command executed with root but got an error [' % Total % Received % Xferd Average Speed Time Time Time Current', ' Dload Upload Total Spent Left Speed', '', ' 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0', '100 16 100 16 0 0 1600 0 --:--:-- --:--:-- --:--:-- 1600'] ... 2024-01-29 11:52:18,541 - root - INFO - running command.raw on 172.23.106.112: curl http://172.23.106.112:8093/evaluator/v1/libraries/n1ql -u Administrator:password 2024-01-29 11:52:18,597 - root - INFO - command executed with root but got an error [' % Total % Received % Xferd Average Speed Time Time Time Current', ' Dload Upload Total Spent Left Speed', '', ' 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0', '100 26 100 26 0 0 26000 0 --:--:-- --:--:-- --:--:-- 26000'] ... 2024-01-29 11:52:18,606 - root - INFO - RUN QUERY DROP FUNCTION select1 2024-01-29 11:52:18,606 - root - INFO - query params : statement=DROP+FUNCTION+select1&scan_consistency=REQUEST_PLUS 2024-01-29 11:52:18,615 - root - INFO - TOTAL ELAPSED TIME: 7.815071ms 2024-01-29 11:52:18,623 - root - INFO - RUN QUERY DROP INDEX idx1 IF EXISTS ON `travel-sample` 2024-01-29 11:52:18,623 - root - INFO - query params : statement=DROP+INDEX+idx1+IF+EXISTS+ON+%60travel-sample%60&scan_consistency=REQUEST_PLUS 2024-01-29 11:52:18,654 - root - INFO - TOTAL ELAPSED TIME: 29.474767ms 2024-01-29 11:52:18,657 - root - INFO - RUN QUERY create index idx1 on `travel-sample`(DISTINCT ARRAY FLATTEN_KEYS(r.author,r.ratings.Cleanliness) FOR r IN reviews END, email, free_parking, type) 2024-01-29 11:52:18,658 - root - INFO - query params : statement=create+index+idx1+on+%60travel-sample%60%28DISTINCT+ARRAY+FLATTEN_KEYS%28r.author%2Cr.ratings.Cleanliness%29+FOR+r+IN+reviews+END%2C+email%2C+free_parking%2C+type%29&scan_consistency=REQUEST_PLUS 2024-01-29 11:52:22,428 - root - INFO - TOTAL ELAPSED TIME: 3.768215364s 2024-01-29 11:52:22,442 - root - INFO - RUN QUERY EXPLAIN SELECT MIN(r.ratings.Cleanliness), MAX(r.ratings.Cleanliness) FROM `travel-sample` AS d unnest reviews as r WHERE d.type = 'hotel' and r.author LIKE 'N%' and r.author is not missing and r.ratings.Cleanliness > 1 AND d.free_parking = False AND d.email is not missing GROUP BY r.ratings.Cleanliness 2024-01-29 11:52:22,443 - root - INFO - query params : statement=EXPLAIN+SELECT+MIN%28r.ratings.Cleanliness%29%2C+MAX%28r.ratings.Cleanliness%29+FROM+%60travel-sample%60+AS+d+unnest+reviews+as+r+WHERE+d.type+%3D+%27hotel%27+and+r.author+LIKE+%27N%25%27+and+r.author+is+not+missing+and+r.ratings.Cleanliness+%3E+1+AND+d.free_parking+%3D+False+AND+d.email+is+not+missing+GROUP+BY+r.ratings.Cleanliness&scan_consistency=REQUEST_PLUS 2024-01-29 11:52:22,473 - root - INFO - TOTAL ELAPSED TIME: 27.263326ms 2024-01-29 11:52:22,473 - root - INFO - running command.raw on 172.23.106.112: curl -X POST -u Administrator:password http://172.23.106.112:8093/query/service -d 'statement=select (select * from `default`)&memory_quota=1' 2024-01-29 11:52:22,714 - root - INFO - command executed with root but got an error [' % Total % Received % Xferd Average Speed Time Time Time Current', ' Dload Upload Total Spent Left Speed', '', ' 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0', '100 466 100 409 100 57 1884 262 --:--:-- --:--:-- --:--:-- 2147'] ... 2024-01-29 11:52:22,715 - root - INFO - (['{', '"requestID": "c180c85f-c04b-4cc8-a121-8adabe73ca80",', '"signature": {"$1":"array"},', '"results": [', '],', '"errors": [{"code":5500,"msg":"Request has exceeded memory quota"},{"code":5500,"msg":"Request has exceeded memory quota"}],', '"status": "fatal",', '"metrics": {"elapsedTime": "215.483859ms","executionTime": "215.372472ms","resultCount": 0,"resultSize": 0,"serviceLoad": 6,"usedMemory": 1818710,"errorCount": 2}', '}'], [' % Total % Received % Xferd Average Speed Time Time Time Current', ' Dload Upload Total Spent Left Speed', '', ' 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0', '100 466 100 409 100 57 1884 262 --:--:-- --:--:-- --:--:-- 2147']) 2024-01-29 11:52:22,753 - root - INFO - RUN QUERY SELECT ADVISOR([ "UPDATE `travel-sample` SET city = 'San Francisco' WHERE lower(city) = 'sanfrancisco'", "UPDATE `travel-sample` SET city = 'San Francisco' WHERE lower(city) = 'sanfrancisco'" ]) 2024-01-29 11:52:22,753 - root - INFO - query params : statement=SELECT+ADVISOR%28%5B+++++++++++++++++%22UPDATE+%60travel-sample%60+SET+city+%3D+%27San+Francisco%27+WHERE+lower%28city%29+%3D+%27sanfrancisco%27%22%2C+++++++++++++++++%22UPDATE+%60travel-sample%60+SET+city+%3D+%27San+Francisco%27+WHERE+lower%28city%29+%3D+%27sanfrancisco%27%22+++++++++++++%5D%29&scan_consistency=REQUEST_PLUS 2024-01-29 11:52:22,891 - root - INFO - TOTAL ELAPSED TIME: 135.464034ms 2024-01-29 11:52:22,894 - root - INFO - RUN QUERY SELECT ADVISOR({'action': 'start', 'duration': '1h', 'query_count': 2 }) 2024-01-29 11:52:22,895 - root - INFO - query params : statement=SELECT+ADVISOR%28%7B%27action%27%3A+%27start%27%2C+%27duration%27%3A+%271h%27%2C+%27query_count%27%3A+2+%7D%29&scan_consistency=REQUEST_PLUS 2024-01-29 11:52:22,898 - root - INFO - TOTAL ELAPSED TIME: 1.859063ms 2024-01-29 11:52:22,901 - root - INFO - RUN QUERY SELECT airportname FROM `travel-sample` WHERE lower(city) = 'lyon' AND country = 'France' 2024-01-29 11:52:22,902 - root - INFO - query params : statement=SELECT+airportname+FROM+%60travel-sample%60+WHERE+lower%28city%29+%3D+%27lyon%27+AND+country+%3D+%27France%27&scan_consistency=REQUEST_PLUS 2024-01-29 11:52:22,968 - root - INFO - TOTAL ELAPSED TIME: 64.565346ms 2024-01-29 11:52:22,971 - root - INFO - RUN QUERY SELECT airportname FROM `travel-sample` WHERE lower(city) = 'lyon' AND country = 'France' 2024-01-29 11:52:22,972 - root - INFO - query params : statement=SELECT+airportname+FROM+%60travel-sample%60+WHERE+lower%28city%29+%3D+%27lyon%27+AND+country+%3D+%27France%27&scan_consistency=REQUEST_PLUS 2024-01-29 11:52:23,056 - root - INFO - TOTAL ELAPSED TIME: 82.717685ms 2024-01-29 11:52:23,057 - root - INFO - sleep for 3 secs. ... 2024-01-29 11:52:26,069 - root - INFO - RUN QUERY SELECT ADVISOR({'action': 'stop', 'session': 'd63d39e5-7bac-4d75-acd3-b7e228a597f9'}) 2024-01-29 11:52:26,069 - root - INFO - query params : statement=SELECT+ADVISOR%28%7B%27action%27%3A+%27stop%27%2C+%27session%27%3A+%27d63d39e5-7bac-4d75-acd3-b7e228a597f9%27%7D%29&scan_consistency=REQUEST_PLUS 2024-01-29 11:52:26,238 - root - INFO - TOTAL ELAPSED TIME: 166.910674ms 2024-01-29 11:52:26,243 - root - INFO - RUN QUERY SELECT ADVISOR({'action': 'get', 'session': 'd63d39e5-7bac-4d75-acd3-b7e228a597f9'}) 2024-01-29 11:52:26,244 - root - INFO - query params : statement=SELECT+ADVISOR%28%7B%27action%27%3A+%27get%27%2C+%27session%27%3A+%27d63d39e5-7bac-4d75-acd3-b7e228a597f9%27%7D%29&scan_consistency=REQUEST_PLUS 2024-01-29 11:52:26,249 - root - INFO - TOTAL ELAPSED TIME: 4.35536ms 2024-01-29 11:52:26,253 - root - INFO - RUN QUERY SELECT ADVISOR({'action': 'purge', 'session': 'd63d39e5-7bac-4d75-acd3-b7e228a597f9'}) 2024-01-29 11:52:26,253 - root - INFO - query params : statement=SELECT+ADVISOR%28%7B%27action%27%3A+%27purge%27%2C+%27session%27%3A+%27d63d39e5-7bac-4d75-acd3-b7e228a597f9%27%7D%29&scan_consistency=REQUEST_PLUS 2024-01-29 11:52:26,262 - root - INFO - TOTAL ELAPSED TIME: 7.568913ms 2024-01-29 11:52:26,265 - root - INFO - RUN QUERY EXPLAIN select type from `travel-sample` where type = 'hotel' 2024-01-29 11:52:26,265 - root - INFO - query params : statement=EXPLAIN+select+type+from+%60travel-sample%60+where+type+%3D+%27hotel%27&scan_consistency=REQUEST_PLUS 2024-01-29 11:52:26,305 - root - INFO - TOTAL ELAPSED TIME: 38.302182ms 2024-01-29 11:52:26,308 - root - INFO - RUN QUERY UPDATE STATISTICS FOR `travel-sample`(city, country) WITH {'update_statistics_timeout':180} 2024-01-29 11:52:26,308 - root - INFO - query params : statement=UPDATE+STATISTICS+FOR+%60travel-sample%60%28city%2C+country%29+WITH+%7B%27update_statistics_timeout%27%3A180%7D&scan_consistency=REQUEST_PLUS 2024-01-29 11:52:37,583 - root - INFO - TOTAL ELAPSED TIME: 11.272172232s 2024-01-29 11:52:37,592 - root - INFO - RUN QUERY select `bucket`, `scope`, `collection`, `histogramKey` from `travel-sample`.`_system`.`_query` data WHERE type = 'histogram' and `scope` = '_default' and `collection` = '_default' 2024-01-29 11:52:37,592 - root - INFO - query params : statement=select+%60bucket%60%2C+%60scope%60%2C+%60collection%60%2C+%60histogramKey%60+from+%60travel-sample%60.%60_system%60.%60_query%60+data+WHERE+type+%3D+%27histogram%27+and+%60scope%60+%3D+%27_default%27+and+%60collection%60+%3D+%27_default%27&scan_consistency=REQUEST_PLUS 2024-01-29 11:52:37,654 - root - INFO - TOTAL ELAPSED TIME: 59.24766ms 2024-01-29 11:52:37,660 - root - INFO - RUN QUERY UPDATE STATISTICS FOR `travel-sample` DELETE ALL 2024-01-29 11:52:37,660 - root - INFO - query params : statement=UPDATE+STATISTICS+FOR+%60travel-sample%60+DELETE+ALL&scan_consistency=REQUEST_PLUS 2024-01-29 11:52:37,679 - root - INFO - TOTAL ELAPSED TIME: 16.050512ms 2024-01-29 11:52:37,683 - root - INFO - RUN QUERY UPDATE STATISTICS FOR `travel-sample`(city) WITH {'update_statistics_timeout':600} 2024-01-29 11:52:37,683 - root - INFO - query params : statement=UPDATE+STATISTICS+FOR+%60travel-sample%60%28city%29+WITH+%7B%27update_statistics_timeout%27%3A600%7D&scan_consistency=REQUEST_PLUS 2024-01-29 11:52:48,622 - root - INFO - TOTAL ELAPSED TIME: 10.933830826s 2024-01-29 11:52:48,630 - root - INFO - RUN QUERY EXPLAIN select airportname from `travel-sample` where city = 'Lyon' 2024-01-29 11:52:48,631 - root - INFO - query params : statement=EXPLAIN+select+airportname+from+%60travel-sample%60+where+city+%3D+%27Lyon%27&scan_consistency=REQUEST_PLUS 2024-01-29 11:52:48,697 - root - INFO - TOTAL ELAPSED TIME: 63.919634ms 2024-01-29 11:52:48,700 - root - INFO - RUN QUERY UPDATE STATISTICS FOR `travel-sample` DELETE ALL 2024-01-29 11:52:48,700 - root - INFO - query params : statement=UPDATE+STATISTICS+FOR+%60travel-sample%60+DELETE+ALL&scan_consistency=REQUEST_PLUS 2024-01-29 11:52:48,717 - root - INFO - TOTAL ELAPSED TIME: 15.582538ms 2024-01-29 11:52:48,721 - root - INFO - RUN QUERY select count(*) from `travel-sample`.`_default`.`_default` where type = "airport" 2024-01-29 11:52:48,721 - root - INFO - query params : statement=select+count%28%2A%29+from+%60travel-sample%60.%60_default%60.%60_default%60+where+type+%3D+%22airport%22&scan_consistency=REQUEST_PLUS 2024-01-29 11:52:48,789 - root - INFO - TOTAL ELAPSED TIME: 64.21878ms 2024-01-29 11:52:48,793 - root - INFO - RUN QUERY CREATE SCOPE `travel-sample`.`post-upgrade` 2024-01-29 11:52:48,793 - root - INFO - query params : statement=CREATE+SCOPE+%60travel-sample%60.%60post-upgrade%60&scan_consistency=REQUEST_PLUS 2024-01-29 11:52:48,853 - root - INFO - TOTAL ELAPSED TIME: 58.482329ms 2024-01-29 11:52:48,853 - root - INFO - sleep for 3 secs. ... 2024-01-29 11:52:51,872 - root - INFO - RUN QUERY CREATE COLLECTION `travel-sample`.`post-upgrade`.airport 2024-01-29 11:52:51,873 - root - INFO - query params : statement=CREATE+COLLECTION+%60travel-sample%60.%60post-upgrade%60.airport&scan_consistency=REQUEST_PLUS 2024-01-29 11:52:51,932 - root - INFO - TOTAL ELAPSED TIME: 56.519308ms 2024-01-29 11:52:51,932 - root - INFO - sleep for 3 secs. ... 2024-01-29 11:52:54,950 - root - INFO - RUN QUERY SELECT `path` FROM system:keyspaces WHERE `scope` = "post-upgrade" 2024-01-29 11:52:54,951 - root - INFO - query params : statement=SELECT+%60path%60+FROM+system%3Akeyspaces+WHERE+%60scope%60+%3D+%22post-upgrade%22&scan_consistency=REQUEST_PLUS 2024-01-29 11:52:55,031 - root - INFO - TOTAL ELAPSED TIME: 78.636591ms 2024-01-29 11:52:55,035 - root - INFO - RUN QUERY INSERT INTO `travel-sample`.`post-upgrade`.airport(KEY _k, VALUE _v) SELECT META().id _k, _v FROM `travel-sample` _v WHERE type = "airport" 2024-01-29 11:52:55,035 - root - INFO - query params : statement=INSERT+INTO+%60travel-sample%60.%60post-upgrade%60.airport%28KEY+_k%2C+VALUE+_v%29+SELECT+META%28%29.id+_k%2C+_v+FROM+%60travel-sample%60+_v+WHERE+type+%3D+%22airport%22&scan_consistency=REQUEST_PLUS 2024-01-29 11:52:56,687 - root - INFO - TOTAL ELAPSED TIME: 1.648999893s 2024-01-29 11:52:56,698 - root - INFO - RUN QUERY CREATE PRIMARY INDEX ON `travel-sample`.`post-upgrade`.`airport` 2024-01-29 11:52:56,698 - root - INFO - query params : statement=CREATE+PRIMARY+INDEX+ON+%60travel-sample%60.%60post-upgrade%60.%60airport%60&scan_consistency=REQUEST_PLUS 2024-01-29 11:53:00,319 - root - INFO - TOTAL ELAPSED TIME: 3.618853866s 2024-01-29 11:53:00,333 - root - INFO - RUN QUERY UPDATE `travel-sample`.`post-upgrade`.airport SET city = "lyon" WHERE city = "Lyon" 2024-01-29 11:53:00,333 - root - INFO - query params : statement=UPDATE+%60travel-sample%60.%60post-upgrade%60.airport+SET+city+%3D+%22lyon%22+WHERE+city+%3D+%22Lyon%22&scan_consistency=REQUEST_PLUS 2024-01-29 11:53:00,534 - root - INFO - TOTAL ELAPSED TIME: 198.544536ms 2024-01-29 11:53:00,537 - root - INFO - RUN QUERY DELETE FROM `travel-sample`.`post-upgrade`.airport WHERE airportname = "Bron" 2024-01-29 11:53:00,538 - root - INFO - query params : statement=DELETE+FROM+%60travel-sample%60.%60post-upgrade%60.airport+WHERE+airportname+%3D+%22Bron%22&scan_consistency=REQUEST_PLUS 2024-01-29 11:53:00,737 - root - INFO - TOTAL ELAPSED TIME: 197.617245ms 2024-01-29 11:53:00,741 - root - INFO - RUN QUERY SELECT array_agg(airportname) from `travel-sample`.`post-upgrade`.airport WHERE city = "lyon" 2024-01-29 11:53:00,741 - root - INFO - query params : statement=SELECT+array_agg%28airportname%29+from+%60travel-sample%60.%60post-upgrade%60.airport+WHERE+city+%3D+%22lyon%22&scan_consistency=REQUEST_PLUS 2024-01-29 11:53:00,905 - root - INFO - TOTAL ELAPSED TIME: 162.262389ms 2024-01-29 11:53:00,908 - root - INFO - RUN QUERY SELECT array_agg(airportname) FROM airport WHERE city = "lyon" 2024-01-29 11:53:00,908 - root - INFO - query params : statement=SELECT+array_agg%28airportname%29+FROM+airport+WHERE+city+%3D+%22lyon%22&scan_consistency=REQUEST_PLUS&query_context=travel-sample.post-upgrade 2024-01-29 11:53:00,909 - root - INFO - Running Query with query_context: travel-sample.post-upgrade 2024-01-29 11:53:01,060 - root - INFO - TOTAL ELAPSED TIME: 149.837146ms 2024-01-29 11:53:01,064 - root - INFO - RUN QUERY PREPARE `lyon_airport_post-upgrade` AS SELECT array_agg(airportname) FROM `travel-sample`.`post-upgrade`.airport WHERE city = "lyon" 2024-01-29 11:53:01,064 - root - INFO - query params : statement=PREPARE+%60lyon_airport_post-upgrade%60+AS+SELECT+array_agg%28airportname%29+FROM+%60travel-sample%60.%60post-upgrade%60.airport+WHERE+city+%3D+%22lyon%22&scan_consistency=REQUEST_PLUS 2024-01-29 11:53:01,092 - root - INFO - TOTAL ELAPSED TIME: 26.623056ms 2024-01-29 11:53:01,095 - root - INFO - RUN QUERY EXECUTE `lyon_airport_post-upgrade` 2024-01-29 11:53:01,095 - root - INFO - query params : statement=EXECUTE+%60lyon_airport_post-upgrade%60&scan_consistency=REQUEST_PLUS 2024-01-29 11:53:01,257 - root - INFO - TOTAL ELAPSED TIME: 160.320383ms 2024-01-29 11:53:01,261 - root - INFO - RUN QUERY DROP COLLECTION `travel-sample`.`post-upgrade`.airport 2024-01-29 11:53:01,261 - root - INFO - query params : statement=DROP+COLLECTION+%60travel-sample%60.%60post-upgrade%60.airport&scan_consistency=REQUEST_PLUS 2024-01-29 11:53:01,327 - root - INFO - TOTAL ELAPSED TIME: 64.008927ms 2024-01-29 11:53:01,398 - root - INFO - RUN QUERY DROP SCOPE `travel-sample`.`post-upgrade` 2024-01-29 11:53:01,398 - root - INFO - query params : statement=DROP+SCOPE+%60travel-sample%60.%60post-upgrade%60&scan_consistency=REQUEST_PLUS 2024-01-29 11:53:01,465 - root - INFO - TOTAL ELAPSED TIME: 64.542414ms 2024-01-29 11:53:01,508 - root - INFO - RUN QUERY CREATE INDEX faa_idx on `travel-sample`(faa) 2024-01-29 11:53:01,508 - root - INFO - query params : statement=CREATE+INDEX+faa_idx+on+%60travel-sample%60%28faa%29&scan_consistency=REQUEST_PLUS 2024-01-29 11:53:06,169 - root - INFO - TOTAL ELAPSED TIME: 4.658972644s 2024-01-29 11:53:06,182 - root - INFO - RUN QUERY BEGIN WORK 2024-01-29 11:53:06,182 - root - INFO - query params : statement=BEGIN+WORK&scan_consistency=REQUEST_PLUS&txtimeout=180s 2024-01-29 11:53:06,191 - root - INFO - TOTAL ELAPSED TIME: 6.363853ms 2024-01-29 11:53:06,194 - root - INFO - RUN QUERY select count(*) from `travel-sample` 2024-01-29 11:53:06,195 - root - INFO - query params : statement=select+count%28%2A%29+from+%60travel-sample%60&scan_consistency=REQUEST_PLUS&txid=7e36554d-b004-4925-8fff-9ed4d97369dc 2024-01-29 11:53:06,254 - root - INFO - TOTAL ELAPSED TIME: 55.183946ms 2024-01-29 11:53:06,259 - root - INFO - RUN QUERY SAVEPOINT S1 2024-01-29 11:53:06,259 - root - INFO - query params : statement=SAVEPOINT+S1&scan_consistency=REQUEST_PLUS&txid=7e36554d-b004-4925-8fff-9ed4d97369dc 2024-01-29 11:53:06,261 - root - INFO - TOTAL ELAPSED TIME: 424.173µs 2024-01-29 11:53:06,264 - root - INFO - RUN QUERY INSERT INTO `travel-sample`(KEY,VALUE) VALUES ('airport_9999', {'id':9999,'type':'airport','airportname':'Bron','city':'Lyon','country':'France'}) 2024-01-29 11:53:06,264 - root - INFO - query params : statement=INSERT+INTO+%60travel-sample%60%28KEY%2CVALUE%29+VALUES+%28%27airport_9999%27%2C+%7B%27id%27%3A9999%2C%27type%27%3A%27airport%27%2C%27airportname%27%3A%27Bron%27%2C%27city%27%3A%27Lyon%27%2C%27country%27%3A%27France%27%7D%29&scan_consistency=REQUEST_PLUS&txid=7e36554d-b004-4925-8fff-9ed4d97369dc 2024-01-29 11:53:06,285 - root - INFO - TOTAL ELAPSED TIME: 19.222527ms 2024-01-29 11:53:06,287 - root - INFO - RUN QUERY UPDATE `travel-sample` SET city = 'San Francisco' WHERE lower(city) = 'sanfrancisco' 2024-01-29 11:53:06,288 - root - INFO - query params : statement=UPDATE+%60travel-sample%60+SET+city+%3D+%27San+Francisco%27+WHERE+lower%28city%29+%3D+%27sanfrancisco%27&scan_consistency=REQUEST_PLUS&txid=7e36554d-b004-4925-8fff-9ed4d97369dc 2024-01-29 11:53:06,394 - root - INFO - TOTAL ELAPSED TIME: 103.385113ms 2024-01-29 11:53:06,399 - root - INFO - RUN QUERY MERGE INTO `travel-sample` AS target USING [{'iata':'DSA', 'name': 'Doncaster Sheffield Airport'}, {'iata':'VLY', 'name': 'Anglesey Airport / Maes Awyr Môn'}] AS source ON target.faa = source.iata WHEN MATCHED THEN UPDATE SET target.old_name = target.airportname, target.airportname = source.name, target.updated = true WHEN NOT MATCHED THEN INSERT (KEY UUID(), VALUE {'faa': source.iata, 'airportname': source.name, 'type': 'airport', 'inserted': true}) 2024-01-29 11:53:06,400 - root - INFO - query params : statement=MERGE+INTO+%60travel-sample%60+AS+target+USING++++++++++++++++%5B%7B%27iata%27%3A%27DSA%27%2C+%27name%27%3A+%27Doncaster+Sheffield+Airport%27%7D%2C+%7B%27iata%27%3A%27VLY%27%2C+%27name%27%3A+%27Anglesey+Airport+%2F+Maes+Awyr+M%C3%B4n%27%7D%5D+AS+source++++++++++++++++ON+target.faa+%3D+source.iata+++++++++++++++++WHEN+MATCHED+THEN+UPDATE+SET+target.old_name+%3D+target.airportname%2C+target.airportname+%3D+source.name%2C+target.updated+%3D+true++++++++++++++++WHEN+NOT+MATCHED+THEN+INSERT+%28KEY+UUID%28%29%2C+VALUE+%7B%27faa%27%3A+source.iata%2C+%27airportname%27%3A+source.name%2C+%27type%27%3A+%27airport%27%2C+%27inserted%27%3A+true%7D%29&scan_consistency=REQUEST_PLUS&txid=7e36554d-b004-4925-8fff-9ed4d97369dc 2024-01-29 11:53:06,473 - root - INFO - TOTAL ELAPSED TIME: 71.393012ms 2024-01-29 11:53:06,476 - root - INFO - RUN QUERY DELETE FROM `travel-sample` WHERE type = 'airport' AND airportname = 'Bron' 2024-01-29 11:53:06,476 - root - INFO - query params : statement=DELETE+FROM+%60travel-sample%60+WHERE+type+%3D+%27airport%27+AND+airportname+%3D+%27Bron%27&scan_consistency=REQUEST_PLUS&txid=7e36554d-b004-4925-8fff-9ed4d97369dc 2024-01-29 11:53:06,513 - root - INFO - TOTAL ELAPSED TIME: 34.593469ms 2024-01-29 11:53:06,516 - root - INFO - RUN QUERY ROLLBACK TO SAVEPOINT S1 2024-01-29 11:53:06,516 - root - INFO - query params : statement=ROLLBACK+TO+SAVEPOINT+S1&scan_consistency=REQUEST_PLUS&txid=7e36554d-b004-4925-8fff-9ed4d97369dc 2024-01-29 11:53:06,518 - root - INFO - TOTAL ELAPSED TIME: 497.339µs 2024-01-29 11:53:06,520 - root - INFO - RUN QUERY select count(*) from `travel-sample` 2024-01-29 11:53:06,521 - root - INFO - query params : statement=select+count%28%2A%29+from+%60travel-sample%60&scan_consistency=REQUEST_PLUS&txid=7e36554d-b004-4925-8fff-9ed4d97369dc 2024-01-29 11:53:06,546 - root - INFO - TOTAL ELAPSED TIME: 23.249225ms 2024-01-29 11:53:06,548 - root - INFO - RUN QUERY COMMIT 2024-01-29 11:53:06,549 - root - INFO - query params : statement=COMMIT&scan_consistency=REQUEST_PLUS&txid=7e36554d-b004-4925-8fff-9ed4d97369dc 2024-01-29 11:53:06,551 - root - INFO - TOTAL ELAPSED TIME: 566.095µs 2024-01-29 11:53:06,554 - root - INFO - RUN QUERY DROP INDEX faa_idx on `travel-sample` 2024-01-29 11:53:06,554 - root - INFO - query params : statement=DROP+INDEX+faa_idx+on+%60travel-sample%60&scan_consistency=REQUEST_PLUS 2024-01-29 11:53:06,613 - root - INFO - TOTAL ELAPSED TIME: 56.944853ms 2024-01-29 11:53:06,634 - root - INFO - RUN QUERY DROP SEQUENCE `default`.`_default`.seq_default_option IF EXISTS 2024-01-29 11:53:06,634 - root - INFO - query params : statement=DROP+SEQUENCE+%60default%60.%60_default%60.seq_default_option+IF+EXISTS&scan_consistency=REQUEST_PLUS 2024-01-29 11:53:06,652 - root - INFO - TOTAL ELAPSED TIME: 16.146287ms 2024-01-29 11:53:06,656 - root - INFO - RUN QUERY CREATE SEQUENCE `default`.`_default`.seq_default_option 2024-01-29 11:53:06,656 - root - INFO - query params : statement=CREATE+SEQUENCE+%60default%60.%60_default%60.seq_default_option&scan_consistency=REQUEST_PLUS 2024-01-29 11:53:06,679 - root - INFO - TOTAL ELAPSED TIME: 21.493291ms 2024-01-29 11:53:06,695 - root - INFO - RUN QUERY SELECT `cache`, `cycle`, `increment`, `max`, `min`, `path` FROM system:sequences WHERE name = 'seq_default_option' 2024-01-29 11:53:06,695 - root - INFO - query params : statement=SELECT+%60cache%60%2C+%60cycle%60%2C+%60increment%60%2C+%60max%60%2C+%60min%60%2C+%60path%60+FROM+system%3Asequences+WHERE+name+%3D+%27seq_default_option%27&scan_consistency=REQUEST_PLUS 2024-01-29 11:53:06,842 - root - INFO - TOTAL ELAPSED TIME: 144.656033ms 2024-01-29 11:53:06,846 - root - INFO - RUN QUERY SELECT NEXTVAL FOR `default`.`_default`.seq_default_option as val 2024-01-29 11:53:06,846 - root - INFO - query params : statement=SELECT+NEXTVAL+FOR+%60default%60.%60_default%60.seq_default_option+as+val&scan_consistency=REQUEST_PLUS 2024-01-29 11:53:06,875 - root - INFO - TOTAL ELAPSED TIME: 26.579161ms 2024-01-29 11:53:06,881 - root - INFO - RUN QUERY SELECT NEXTVAL FOR `default`.`_default`.seq_default_option as val 2024-01-29 11:53:06,882 - root - INFO - query params : statement=SELECT+NEXTVAL+FOR+%60default%60.%60_default%60.seq_default_option+as+val&scan_consistency=REQUEST_PLUS 2024-01-29 11:53:06,896 - root - INFO - TOTAL ELAPSED TIME: 12.406173ms 2024-01-29 11:53:06,900 - root - INFO - RUN QUERY SELECT PREVVAL FOR `default`.`_default`.seq_default_option as val 2024-01-29 11:53:06,900 - root - INFO - query params : statement=SELECT+PREVVAL+FOR+%60default%60.%60_default%60.seq_default_option+as+val&scan_consistency=REQUEST_PLUS 2024-01-29 11:53:06,916 - root - INFO - TOTAL ELAPSED TIME: 14.372537ms 2024-01-29 11:53:06,919 - root - INFO - RUN QUERY SELECT PREVVAL FOR `default`.`_default`.seq_default_option as val 2024-01-29 11:53:06,919 - root - INFO - query params : statement=SELECT+PREVVAL+FOR+%60default%60.%60_default%60.seq_default_option+as+val&scan_consistency=REQUEST_PLUS 2024-01-29 11:53:06,930 - root - INFO - TOTAL ELAPSED TIME: 9.542932ms 2024-01-29 11:53:06,933 - root - INFO - RUN QUERY SELECT NEXTVAL FOR `default`.`_default`.seq_default_option as val 2024-01-29 11:53:06,933 - root - INFO - query params : statement=SELECT+NEXTVAL+FOR+%60default%60.%60_default%60.seq_default_option+as+val&scan_consistency=REQUEST_PLUS 2024-01-29 11:53:06,945 - root - INFO - TOTAL ELAPSED TIME: 10.379703ms 2024-01-29 11:53:06,953 - root - INFO - RUN QUERY select * from system:keyspaces_info where id = 'travel-sample' 2024-01-29 11:53:06,953 - root - INFO - query params : statement=select+%2A+from+system%3Akeyspaces_info+where+id+%3D+%27travel-sample%27&scan_consistency=REQUEST_PLUS 2024-01-29 11:53:07,336 - root - INFO - TOTAL ELAPSED TIME: 381.078834ms 2024-01-29 11:53:07,340 - root - INFO - RUN QUERY select array_agg(`path`) from system:all_keyspaces where datastore_id = 'system' 2024-01-29 11:53:07,340 - root - INFO - query params : statement=select+array_agg%28%60path%60%29+from+system%3Aall_keyspaces+where+datastore_id+%3D+%27system%27&scan_consistency=REQUEST_PLUS 2024-01-29 11:53:07,402 - root - INFO - TOTAL ELAPSED TIME: 60.115089ms 2024-01-29 11:53:07,405 - root - INFO - RUN QUERY select array_agg(distinct keyspace_id) from system:all_indexes where all_indexes.`namespace_id` = '#system' 2024-01-29 11:53:07,405 - root - INFO - query params : statement=select+array_agg%28distinct+keyspace_id%29+from+system%3Aall_indexes+where+all_indexes.%60namespace_id%60+%3D+%27%23system%27&scan_consistency=REQUEST_PLUS 2024-01-29 11:53:08,252 - root - INFO - TOTAL ELAPSED TIME: 844.251894ms 2024-01-29 11:53:08,260 - root - INFO - RUN QUERY select array_agg(id) from system:all_keyspaces_info where all_keyspaces_info.datastore_id = 'system' 2024-01-29 11:53:08,260 - root - INFO - query params : statement=select+array_agg%28id%29+from+system%3Aall_keyspaces_info+where+all_keyspaces_info.datastore_id+%3D+%27system%27&scan_consistency=REQUEST_PLUS 2024-01-29 11:53:10,289 - root - INFO - TOTAL ELAPSED TIME: 2.026011327s 2024-01-29 11:53:10,289 - root - INFO - completed post upgrade test 2024-01-29 11:53:10,289 - root - INFO - ============== QueriesUpgradeTests tearDown has started ============== 2024-01-29 11:53:10,289 - root - INFO - ============== QueriesUpgradeTests tearDown has completed ============== 2024-01-29 11:53:10,290 - root - INFO - ============== QueryTests tearDown has started ============== 2024-01-29 11:53:10,290 - root - INFO - ============== QueryTests tearDown has completed ============== 2024-01-29 11:53:10,290 - root - INFO - ============== NewUpgradeBaseTest tearDown has started ============== 2024-01-29 11:53:10,309 - root - INFO - ------- Cluster statistics ------- 2024-01-29 11:53:10,309 - root - INFO - 172.23.104.102:8091 => {'services': ['index', 'kv', 'n1ql'], 'cpu_utilization': 29.47500000009313, 'mem_free': 2561110016, 'mem_total': 4083224576, 'swap_mem_used': 16777216, 'swap_mem_total': 1027600384} 2024-01-29 11:53:10,309 - root - INFO - 172.23.105.33:8091 => {'services': ['index', 'kv', 'n1ql'], 'cpu_utilization': 17.32499999983702, 'mem_free': 2901667840, 'mem_total': 4326473728, 'swap_mem_used': 26214400, 'swap_mem_total': 4550815744} 2024-01-29 11:53:10,309 - root - INFO - 172.23.106.112:8091 => {'services': ['fts', 'index', 'kv', 'n1ql'], 'cpu_utilization': 41.25, 'mem_free': 2369966080, 'mem_total': 4083224576, 'swap_mem_used': 9175040, 'swap_mem_total': 1027600384} 2024-01-29 11:53:10,309 - root - INFO - 172.23.107.26:8091 => {'services': ['index', 'kv', 'n1ql'], 'cpu_utilization': 28.40000000025611, 'mem_free': 2678104064, 'mem_total': 4083224576, 'swap_mem_used': 4456448, 'swap_mem_total': 1027600384} 2024-01-29 11:53:10,309 - root - INFO - --- End of cluster statistics --- 2024-01-29 11:53:10,311 - root - INFO - SSH Connecting to 172.23.106.112 with username:root, attempt#1 of 5 2024-01-29 11:53:10,494 - root - INFO - SSH Connected to 172.23.106.112 as root 2024-01-29 11:53:10,656 - root - INFO - os_distro: Ubuntu, os_version: debian 11, is_linux_distro: True 2024-01-29 11:53:10,953 - root - INFO - extract_remote_info-->distribution_type: Ubuntu, distribution_version: debian 11 2024-01-29 11:53:10,955 - root - INFO - SSH Connecting to 172.23.107.26 with username:root, attempt#1 of 5 2024-01-29 11:53:11,136 - root - INFO - SSH Connected to 172.23.107.26 as root 2024-01-29 11:53:11,307 - root - INFO - os_distro: Ubuntu, os_version: debian 11, is_linux_distro: True 2024-01-29 11:53:11,593 - root - INFO - extract_remote_info-->distribution_type: Ubuntu, distribution_version: debian 11 2024-01-29 11:53:11,594 - root - INFO - SSH Connecting to 172.23.104.102 with username:root, attempt#1 of 5 2024-01-29 11:53:11,778 - root - INFO - SSH Connected to 172.23.104.102 as root 2024-01-29 11:53:11,939 - root - INFO - os_distro: Ubuntu, os_version: debian 11, is_linux_distro: True 2024-01-29 11:53:12,239 - root - INFO - extract_remote_info-->distribution_type: Ubuntu, distribution_version: debian 11 2024-01-29 11:53:12,241 - root - INFO - SSH Connecting to 172.23.105.33 with username:root, attempt#1 of 5 2024-01-29 11:53:12,312 - root - INFO - SSH Connected to 172.23.105.33 as root 2024-01-29 11:53:12,437 - root - INFO - os_distro: Ubuntu, os_version: debian 10, is_linux_distro: True 2024-01-29 11:53:12,695 - root - INFO - extract_remote_info-->distribution_type: Ubuntu, distribution_version: debian 10 Number of matches : 2 matches : ['2024-01-30T08:48:40.359+13:00 [Fatal] metadataClient::updateTopology: index 2746386609218607170 on node 99745027e623695f1e5747bd5821f2ca which is not yet valid for tracker 8. Valid indexers in topology - map[172.23.104.102:9100:ec8c44e38694c88cf0ff83310aebaa38 172.23.105.33:9100:172.23.105.33:9100_Indexer_Id_1 172.23.106.112:9100:5863c6461bed95e0b661a717a2b0bf54 172.23.107.26:9100:167016690ba8dbdfaa069c72634dfbaa].', '2024-01-30T08:48:42.300+13:00 [Fatal] metadataClient::updateTopology: index 2746386609218607170 on node 99745027e623695f1e5747bd5821f2ca which is not yet valid for tracker 9. Valid indexers in topology - map[172.23.104.102:9100:ec8c44e38694c88cf0ff83310aebaa38 172.23.105.33:9100:172.23.105.33:9100_Indexer_Id_1 172.23.106.112:9100:5863c6461bed95e0b661a717a2b0bf54 172.23.107.26:9100:167016690ba8dbdfaa069c72634dfbaa].'] Number of matches : 15 matches : ['2024-01-29T11:48:38.751-08:00 [Fatal] metadataClient::updateTopology: index 2746386609218607170 on node 99745027e623695f1e5747bd5821f2ca which is not yet valid for tracker 1. Valid indexers in topology - map[172.23.104.102:9100:172.23.104.102:9100_Indexer_Id_1 172.23.105.33:9100:172.23.105.33:9100_Indexer_Id_2 172.23.106.112:9100:5863c6461bed95e0b661a717a2b0bf54 172.23.107.26:9100:167016690ba8dbdfaa069c72634dfbaa].', '2024-01-29T11:48:38.751-08:00 [Fatal] metadataClient::updateTopology: index 2746386609218607170 on node ec8c44e38694c88cf0ff83310aebaa38 which is not yet valid for tracker 1. Valid indexers in topology - map[172.23.104.102:9100:172.23.104.102:9100_Indexer_Id_1 172.23.105.33:9100:172.23.105.33:9100_Indexer_Id_2 172.23.106.112:9100:5863c6461bed95e0b661a717a2b0bf54 172.23.107.26:9100:167016690ba8dbdfaa069c72634dfbaa].', '2024-01-29T11:48:38.751-08:00 [Fatal] metadataClient::updateTopology: index 16759661820104705712 on node ec8c44e38694c88cf0ff83310aebaa38 which is not yet valid for tracker 1. Valid indexers in topology - map[172.23.104.102:9100:172.23.104.102:9100_Indexer_Id_1 172.23.105.33:9100:172.23.105.33:9100_Indexer_Id_2 172.23.106.112:9100:5863c6461bed95e0b661a717a2b0bf54 172.23.107.26:9100:167016690ba8dbdfaa069c72634dfbaa].', '2024-01-29T11:48:38.751-08:00 [Fatal] metadataClient::updateTopology: index 6193876885733591927 on node ec8c44e38694c88cf0ff83310aebaa38 which is not yet valid for tracker 1. Valid indexers in topology - map[172.23.104.102:9100:172.23.104.102:9100_Indexer_Id_1 172.23.105.33:9100:172.23.105.33:9100_Indexer_Id_2 172.23.106.112:9100:5863c6461bed95e0b661a717a2b0bf54 172.23.107.26:9100:167016690ba8dbdfaa069c72634dfbaa].', '2024-01-29T11:48:38.751-08:00 [Fatal] metadataClient::updateTopology: index 5146726825432286104 on node ec8c44e38694c88cf0ff83310aebaa38 which is not yet valid for tracker 1. Valid indexers in topology - map[172.23.104.102:9100:172.23.104.102:9100_Indexer_Id_1 172.23.105.33:9100:172.23.105.33:9100_Indexer_Id_2 172.23.106.112:9100:5863c6461bed95e0b661a717a2b0bf54 172.23.107.26:9100:167016690ba8dbdfaa069c72634dfbaa].', '2024-01-29T11:48:38.751-08:00 [Fatal] metadataClient::updateTopology: index 14583542362240008034 on node ec8c44e38694c88cf0ff83310aebaa38 which is not yet valid for tracker 1. Valid indexers in topology - map[172.23.104.102:9100:172.23.104.102:9100_Indexer_Id_1 172.23.105.33:9100:172.23.105.33:9100_Indexer_Id_2 172.23.106.112:9100:5863c6461bed95e0b661a717a2b0bf54 172.23.107.26:9100:167016690ba8dbdfaa069c72634dfbaa].', '2024-01-29T11:48:38.751-08:00 [Fatal] metadataClient::updateTopology: index 12030140075160937859 on node ec8c44e38694c88cf0ff83310aebaa38 which is not yet valid for tracker 1. Valid indexers in topology - map[172.23.104.102:9100:172.23.104.102:9100_Indexer_Id_1 172.23.105.33:9100:172.23.105.33:9100_Indexer_Id_2 172.23.106.112:9100:5863c6461bed95e0b661a717a2b0bf54 172.23.107.26:9100:167016690ba8dbdfaa069c72634dfbaa].', '2024-01-29T11:48:38.756-08:00 [Fatal] metadataClient::updateTopology: index 2746386609218607170 on node 99745027e623695f1e5747bd5821f2ca which is not yet valid for tracker 2. Valid indexers in topology - map[172.23.104.102:9100:ec8c44e38694c88cf0ff83310aebaa38 172.23.105.33:9100:172.23.105.33:9100_Indexer_Id_2 172.23.106.112:9100:5863c6461bed95e0b661a717a2b0bf54 172.23.107.26:9100:167016690ba8dbdfaa069c72634dfbaa].', '2024-01-29T11:48:38.766-08:00 [Fatal] metadataClient::updateTopology: index 2746386609218607170 on node 99745027e623695f1e5747bd5821f2ca which is not yet valid for tracker 3. Valid indexers in topology - map[172.23.104.102:9100:ec8c44e38694c88cf0ff83310aebaa38 172.23.105.33:9100:172.23.105.33:9100_Indexer_Id_2 172.23.106.112:9100:5863c6461bed95e0b661a717a2b0bf54 172.23.107.26:9100:167016690ba8dbdfaa069c72634dfbaa].', '2024-01-29T11:48:38.772-08:00 [Fatal] metadataClient::updateTopology: index 2746386609218607170 on node 99745027e623695f1e5747bd5821f2ca which is not yet valid for tracker 4. Valid indexers in topology - map[172.23.104.102:9100:ec8c44e38694c88cf0ff83310aebaa38 172.23.105.33:9100:172.23.105.33:9100_Indexer_Id_2 172.23.106.112:9100:5863c6461bed95e0b661a717a2b0bf54 172.23.107.26:9100:167016690ba8dbdfaa069c72634dfbaa].', '2024-01-29T11:48:38.782-08:00 [Fatal] metadataClient::updateTopology: index 2746386609218607170 on node 99745027e623695f1e5747bd5821f2ca which is not yet valid for tracker 6. Valid indexers in topology - map[172.23.104.102:9100:ec8c44e38694c88cf0ff83310aebaa38 172.23.105.33:9100:172.23.105.33:9100_Indexer_Id_2 172.23.106.112:9100:5863c6461bed95e0b661a717a2b0bf54 172.23.107.26:9100:167016690ba8dbdfaa069c72634dfbaa].', '2024-01-29T11:48:38.783-08:00 [Fatal] metadataClient::updateTopology: index 2746386609218607170 on node 99745027e623695f1e5747bd5821f2ca which is not yet valid for tracker 5. Valid indexers in topology - map[172.23.104.102:9100:ec8c44e38694c88cf0ff83310aebaa38 172.23.105.33:9100:172.23.105.33:9100_Indexer_Id_2 172.23.106.112:9100:5863c6461bed95e0b661a717a2b0bf54 172.23.107.26:9100:167016690ba8dbdfaa069c72634dfbaa].', '2024-01-29T11:48:38.785-08:00 [Fatal] metadataClient::updateTopology: index 2746386609218607170 on node 99745027e623695f1e5747bd5821f2ca which is not yet valid for tracker 5. Valid indexers in topology - map[172.23.104.102:9100:ec8c44e38694c88cf0ff83310aebaa38 172.23.105.33:9100:172.23.105.33:9100_Indexer_Id_2 172.23.106.112:9100:5863c6461bed95e0b661a717a2b0bf54 172.23.107.26:9100:167016690ba8dbdfaa069c72634dfbaa].', '2024-01-29T11:48:38.795-08:00 [Fatal] metadataClient::updateTopology: index 2746386609218607170 on node 99745027e623695f1e5747bd5821f2ca which is not yet valid for tracker 7. Valid indexers in topology - map[172.23.104.102:9100:ec8c44e38694c88cf0ff83310aebaa38 172.23.105.33:9100:172.23.105.33:9100_Indexer_Id_2 172.23.106.112:9100:5863c6461bed95e0b661a717a2b0bf54 172.23.107.26:9100:167016690ba8dbdfaa069c72634dfbaa].', '2024-01-29T11:48:38.808-08:00 [Fatal] metadataClient::updateTopology: index 2746386609218607170 on node 99745027e623695f1e5747bd5821f2ca which is not yet valid for tracker 8. Valid indexers in topology - map[172.23.104.102:9100:ec8c44e38694c88cf0ff83310aebaa38 172.23.105.33:9100:172.23.105.33:9100_Indexer_Id_2 172.23.106.112:9100:5863c6461bed95e0b661a717a2b0bf54 172.23.107.26:9100:167016690ba8dbdfaa069c72634dfbaa].'] Number of matches : 26 matches : ['2024-01-29T11:48:38.700-08:00 [Fatal] metadataClient::updateTopology: index 711131104148014940 on node 99745027e623695f1e5747bd5821f2ca which is not yet valid for tracker 1. Valid indexers in topology - map[172.23.104.102:9100:172.23.104.102:9100_Indexer_Id_1 172.23.105.33:9100:172.23.105.33:9100_Indexer_Id_2 172.23.106.112:9100:5863c6461bed95e0b661a717a2b0bf54 172.23.107.26:9100:167016690ba8dbdfaa069c72634dfbaa].', '2024-01-29T11:48:38.700-08:00 [Fatal] metadataClient::updateTopology: index 2746386609218607170 on node 99745027e623695f1e5747bd5821f2ca which is not yet valid for tracker 1. Valid indexers in topology - map[172.23.104.102:9100:172.23.104.102:9100_Indexer_Id_1 172.23.105.33:9100:172.23.105.33:9100_Indexer_Id_2 172.23.106.112:9100:5863c6461bed95e0b661a717a2b0bf54 172.23.107.26:9100:167016690ba8dbdfaa069c72634dfbaa].', '2024-01-29T11:48:38.700-08:00 [Fatal] metadataClient::updateTopology: index 6786070803911553049 on node 99745027e623695f1e5747bd5821f2ca which is not yet valid for tracker 1. Valid indexers in topology - map[172.23.104.102:9100:172.23.104.102:9100_Indexer_Id_1 172.23.105.33:9100:172.23.105.33:9100_Indexer_Id_2 172.23.106.112:9100:5863c6461bed95e0b661a717a2b0bf54 172.23.107.26:9100:167016690ba8dbdfaa069c72634dfbaa].', '2024-01-29T11:48:38.700-08:00 [Fatal] metadataClient::updateTopology: index 6896286666461423440 on node 99745027e623695f1e5747bd5821f2ca which is not yet valid for tracker 1. Valid indexers in topology - map[172.23.104.102:9100:172.23.104.102:9100_Indexer_Id_1 172.23.105.33:9100:172.23.105.33:9100_Indexer_Id_2 172.23.106.112:9100:5863c6461bed95e0b661a717a2b0bf54 172.23.107.26:9100:167016690ba8dbdfaa069c72634dfbaa].', '2024-01-29T11:48:38.700-08:00 [Fatal] metadataClient::updateTopology: index 12230768353147897350 on node 99745027e623695f1e5747bd5821f2ca which is not yet valid for tracker 1. Valid indexers in topology - map[172.23.104.102:9100:172.23.104.102:9100_Indexer_Id_1 172.23.105.33:9100:172.23.105.33:9100_Indexer_Id_2 172.23.106.112:9100:5863c6461bed95e0b661a717a2b0bf54 172.23.107.26:9100:167016690ba8dbdfaa069c72634dfbaa].', '2024-01-29T11:48:38.700-08:00 [Fatal] metadataClient::updateTopology: index 12719682417778041975 on node 99745027e623695f1e5747bd5821f2ca which is not yet valid for tracker 1. Valid indexers in topology - map[172.23.104.102:9100:172.23.104.102:9100_Indexer_Id_1 172.23.105.33:9100:172.23.105.33:9100_Indexer_Id_2 172.23.106.112:9100:5863c6461bed95e0b661a717a2b0bf54 172.23.107.26:9100:167016690ba8dbdfaa069c72634dfbaa].', '2024-01-29T11:48:38.700-08:00 [Fatal] metadataClient::updateTopology: index 5673763664465736853 on node 99745027e623695f1e5747bd5821f2ca which is not yet valid for tracker 1. Valid indexers in topology - map[172.23.104.102:9100:172.23.104.102:9100_Indexer_Id_1 172.23.105.33:9100:172.23.105.33:9100_Indexer_Id_2 172.23.106.112:9100:5863c6461bed95e0b661a717a2b0bf54 172.23.107.26:9100:167016690ba8dbdfaa069c72634dfbaa].', '2024-01-29T11:48:38.700-08:00 [Fatal] metadataClient::updateTopology: index 17637709398337244807 on node 99745027e623695f1e5747bd5821f2ca which is not yet valid for tracker 1. Valid indexers in topology - map[172.23.104.102:9100:172.23.104.102:9100_Indexer_Id_1 172.23.105.33:9100:172.23.105.33:9100_Indexer_Id_2 172.23.106.112:9100:5863c6461bed95e0b661a717a2b0bf54 172.23.107.26:9100:167016690ba8dbdfaa069c72634dfbaa].', '2024-01-29T11:48:38.700-08:00 [Fatal] metadataClient::updateTopology: index 9402338192706680353 on node 99745027e623695f1e5747bd5821f2ca which is not yet valid for tracker 1. Valid indexers in topology - map[172.23.104.102:9100:172.23.104.102:9100_Indexer_Id_1 172.23.105.33:9100:172.23.105.33:9100_Indexer_Id_2 172.23.106.112:9100:5863c6461bed95e0b661a717a2b0bf54 172.23.107.26:9100:167016690ba8dbdfaa069c72634dfbaa].', '2024-01-29T11:48:38.700-08:00 [Fatal] metadataClient::updateTopology: index 10303464563694372308 on node 99745027e623695f1e5747bd5821f2ca which is not yet valid for tracker 1. Valid indexers in topology - map[172.23.104.102:9100:172.23.104.102:9100_Indexer_Id_1 172.23.105.33:9100:172.23.105.33:9100_Indexer_Id_2 172.23.106.112:9100:5863c6461bed95e0b661a717a2b0bf54 172.23.107.26:9100:167016690ba8dbdfaa069c72634dfbaa].', '2024-01-29T11:48:38.700-08:00 [Fatal] metadataClient::updateTopology: index 6193876885733591927 on node ec8c44e38694c88cf0ff83310aebaa38 which is not yet valid for tracker 1. Valid indexers in topology - map[172.23.104.102:9100:172.23.104.102:9100_Indexer_Id_1 172.23.105.33:9100:172.23.105.33:9100_Indexer_Id_2 172.23.106.112:9100:5863c6461bed95e0b661a717a2b0bf54 172.23.107.26:9100:167016690ba8dbdfaa069c72634dfbaa].', '2024-01-29T11:48:38.700-08:00 [Fatal] metadataClient::updateTopology: index 16759661820104705712 on node ec8c44e38694c88cf0ff83310aebaa38 which is not yet valid for tracker 1. Valid indexers in topology - map[172.23.104.102:9100:172.23.104.102:9100_Indexer_Id_1 172.23.105.33:9100:172.23.105.33:9100_Indexer_Id_2 172.23.106.112:9100:5863c6461bed95e0b661a717a2b0bf54 172.23.107.26:9100:167016690ba8dbdfaa069c72634dfbaa].', '2024-01-29T11:48:38.700-08:00 [Fatal] metadataClient::updateTopology: index 14583542362240008034 on node ec8c44e38694c88cf0ff83310aebaa38 which is not yet valid for tracker 1. Valid indexers in topology - map[172.23.104.102:9100:172.23.104.102:9100_Indexer_Id_1 172.23.105.33:9100:172.23.105.33:9100_Indexer_Id_2 172.23.106.112:9100:5863c6461bed95e0b661a717a2b0bf54 172.23.107.26:9100:167016690ba8dbdfaa069c72634dfbaa].', '2024-01-29T11:48:38.701-08:00 [Fatal] metadataClient::updateTopology: index 12030140075160937859 on node ec8c44e38694c88cf0ff83310aebaa38 which is not yet valid for tracker 1. Valid indexers in topology - map[172.23.104.102:9100:172.23.104.102:9100_Indexer_Id_1 172.23.105.33:9100:172.23.105.33:9100_Indexer_Id_2 172.23.106.112:9100:5863c6461bed95e0b661a717a2b0bf54 172.23.107.26:9100:167016690ba8dbdfaa069c72634dfbaa].', '2024-01-29T11:48:38.701-08:00 [Fatal] metadataClient::updateTopology: index 2746386609218607170 on node ec8c44e38694c88cf0ff83310aebaa38 which is not yet valid for tracker 1. Valid indexers in topology - map[172.23.104.102:9100:172.23.104.102:9100_Indexer_Id_1 172.23.105.33:9100:172.23.105.33:9100_Indexer_Id_2 172.23.106.112:9100:5863c6461bed95e0b661a717a2b0bf54 172.23.107.26:9100:167016690ba8dbdfaa069c72634dfbaa].', '2024-01-29T11:48:38.701-08:00 [Fatal] metadataClient::updateTopology: index 5146726825432286104 on node ec8c44e38694c88cf0ff83310aebaa38 which is not yet valid for tracker 1. Valid indexers in topology - map[172.23.104.102:9100:172.23.104.102:9100_Indexer_Id_1 172.23.105.33:9100:172.23.105.33:9100_Indexer_Id_2 172.23.106.112:9100:5863c6461bed95e0b661a717a2b0bf54 172.23.107.26:9100:167016690ba8dbdfaa069c72634dfbaa].', '2024-01-29T11:48:38.708-08:00 [Fatal] metadataClient::updateTopology: index 5673763664465736853 on node 99745027e623695f1e5747bd5821f2ca which is not yet valid for tracker 2. Valid indexers in topology - map[172.23.104.102:9100:ec8c44e38694c88cf0ff83310aebaa38 172.23.105.33:9100:172.23.105.33:9100_Indexer_Id_2 172.23.106.112:9100:5863c6461bed95e0b661a717a2b0bf54 172.23.107.26:9100:167016690ba8dbdfaa069c72634dfbaa].', '2024-01-29T11:48:38.708-08:00 [Fatal] metadataClient::updateTopology: index 12230768353147897350 on node 99745027e623695f1e5747bd5821f2ca which is not yet valid for tracker 2. Valid indexers in topology - map[172.23.104.102:9100:ec8c44e38694c88cf0ff83310aebaa38 172.23.105.33:9100:172.23.105.33:9100_Indexer_Id_2 172.23.106.112:9100:5863c6461bed95e0b661a717a2b0bf54 172.23.107.26:9100:167016690ba8dbdfaa069c72634dfbaa].', '2024-01-29T11:48:38.708-08:00 [Fatal] metadataClient::updateTopology: index 12719682417778041975 on node 99745027e623695f1e5747bd5821f2ca which is not yet valid for tracker 2. Valid indexers in topology - map[172.23.104.102:9100:ec8c44e38694c88cf0ff83310aebaa38 172.23.105.33:9100:172.23.105.33:9100_Indexer_Id_2 172.23.106.112:9100:5863c6461bed95e0b661a717a2b0bf54 172.23.107.26:9100:167016690ba8dbdfaa069c72634dfbaa].', '2024-01-29T11:48:38.708-08:00 [Fatal] metadataClient::updateTopology: index 10303464563694372308 on node 99745027e623695f1e5747bd5821f2ca which is not yet valid for tracker 2. Valid indexers in topology - map[172.23.104.102:9100:ec8c44e38694c88cf0ff83310aebaa38 172.23.105.33:9100:172.23.105.33:9100_Indexer_Id_2 172.23.106.112:9100:5863c6461bed95e0b661a717a2b0bf54 172.23.107.26:9100:167016690ba8dbdfaa069c72634dfbaa].', '2024-01-29T11:48:38.708-08:00 [Fatal] metadataClient::updateTopology: index 6786070803911553049 on node 99745027e623695f1e5747bd5821f2ca which is not yet valid for tracker 2. Valid indexers in topology - map[172.23.104.102:9100:ec8c44e38694c88cf0ff83310aebaa38 172.23.105.33:9100:172.23.105.33:9100_Indexer_Id_2 172.23.106.112:9100:5863c6461bed95e0b661a717a2b0bf54 172.23.107.26:9100:167016690ba8dbdfaa069c72634dfbaa].', '2024-01-29T11:48:38.708-08:00 [Fatal] metadataClient::updateTopology: index 711131104148014940 on node 99745027e623695f1e5747bd5821f2ca which is not yet valid for tracker 2. Valid indexers in topology - map[172.23.104.102:9100:ec8c44e38694c88cf0ff83310aebaa38 172.23.105.33:9100:172.23.105.33:9100_Indexer_Id_2 172.23.106.112:9100:5863c6461bed95e0b661a717a2b0bf54 172.23.107.26:9100:167016690ba8dbdfaa069c72634dfbaa].', '2024-01-29T11:48:38.708-08:00 [Fatal] metadataClient::updateTopology: index 17637709398337244807 on node 99745027e623695f1e5747bd5821f2ca which is not yet valid for tracker 2. Valid indexers in topology - map[172.23.104.102:9100:ec8c44e38694c88cf0ff83310aebaa38 172.23.105.33:9100:172.23.105.33:9100_Indexer_Id_2 172.23.106.112:9100:5863c6461bed95e0b661a717a2b0bf54 172.23.107.26:9100:167016690ba8dbdfaa069c72634dfbaa].', '2024-01-29T11:48:38.708-08:00 [Fatal] metadataClient::updateTopology: index 6896286666461423440 on node 99745027e623695f1e5747bd5821f2ca which is not yet valid for tracker 2. Valid indexers in topology - map[172.23.104.102:9100:ec8c44e38694c88cf0ff83310aebaa38 172.23.105.33:9100:172.23.105.33:9100_Indexer_Id_2 172.23.106.112:9100:5863c6461bed95e0b661a717a2b0bf54 172.23.107.26:9100:167016690ba8dbdfaa069c72634dfbaa].', '2024-01-29T11:48:38.708-08:00 [Fatal] metadataClient::updateTopology: index 9402338192706680353 on node 99745027e623695f1e5747bd5821f2ca which is not yet valid for tracker 2. Valid indexers in topology - map[172.23.104.102:9100:ec8c44e38694c88cf0ff83310aebaa38 172.23.105.33:9100:172.23.105.33:9100_Indexer_Id_2 172.23.106.112:9100:5863c6461bed95e0b661a717a2b0bf54 172.23.107.26:9100:167016690ba8dbdfaa069c72634dfbaa].', '2024-01-29T11:48:38.708-08:00 [Fatal] metadataClient::updateTopology: index 2746386609218607170 on node 99745027e623695f1e5747bd5821f2ca which is not yet valid for tracker 2. Valid indexers in topology - map[172.23.104.102:9100:ec8c44e38694c88cf0ff83310aebaa38 172.23.105.33:9100:172.23.105.33:9100_Indexer_Id_2 172.23.106.112:9100:5863c6461bed95e0b661a717a2b0bf54 172.23.107.26:9100:167016690ba8dbdfaa069c72634dfbaa].'] 2024-01-29 11:53:25,675 - root - INFO - closing all ssh connections 2024-01-29 11:53:25,676 - root - INFO - closing all memcached connections Cluster instance shutdown with force 2024-01-29 11:53:25,678 - root - INFO - SSH Connecting to 172.23.107.26 with username:root, attempt#1 of 5 2024-01-29 11:53:25,680 - root - INFO - SSH Connecting to 172.23.104.102 with username:root, attempt#1 of 5 2024-01-29 11:53:25,681 - root - INFO - SSH Connecting to 172.23.105.33 with username:root, attempt#1 of 5 2024-01-29 11:53:25,682 - root - INFO - SSH Connecting to 172.23.106.112 with username:root, attempt#1 of 5 2024-01-29 11:53:25,752 - root - INFO - SSH Connected to 172.23.105.33 as root 2024-01-29 11:53:25,862 - root - INFO - SSH Connected to 172.23.104.102 as root 2024-01-29 11:53:25,863 - root - INFO - SSH Connected to 172.23.107.26 as root 2024-01-29 11:53:25,866 - root - INFO - SSH Connected to 172.23.106.112 as root 2024-01-29 11:53:25,879 - root - INFO - os_distro: Ubuntu, os_version: debian 10, is_linux_distro: True 2024-01-29 11:53:26,025 - root - INFO - os_distro: Ubuntu, os_version: debian 11, is_linux_distro: True 2024-01-29 11:53:26,028 - root - INFO - os_distro: Ubuntu, os_version: debian 11, is_linux_distro: True 2024-01-29 11:53:26,029 - root - INFO - os_distro: Ubuntu, os_version: debian 11, is_linux_distro: True 2024-01-29 11:53:26,136 - root - INFO - extract_remote_info-->distribution_type: Ubuntu, distribution_version: debian 10 Collecting logs from 172.23.105.33 2024-01-29 11:53:26,137 - root - INFO - running command.raw on 172.23.105.33: /opt/couchbase/bin/cbcollect_info 172.23.105.33-20240129-1153-diag.zip 2024-01-29 11:53:26,327 - root - INFO - extract_remote_info-->distribution_type: Ubuntu, distribution_version: debian 11 Collecting logs from 172.23.104.102 2024-01-29 11:53:26,327 - root - INFO - running command.raw on 172.23.104.102: /opt/couchbase/bin/cbcollect_info 172.23.104.102-20240129-1153-diag.zip 2024-01-29 11:53:26,333 - root - INFO - extract_remote_info-->distribution_type: Ubuntu, distribution_version: debian 11 Collecting logs from 172.23.107.26 2024-01-29 11:53:26,333 - root - INFO - running command.raw on 172.23.107.26: /opt/couchbase/bin/cbcollect_info 172.23.107.26-20240129-1153-diag.zip 2024-01-29 11:53:26,335 - root - INFO - extract_remote_info-->distribution_type: Ubuntu, distribution_version: debian 11 Collecting logs from 172.23.106.112 2024-01-29 11:53:26,336 - root - INFO - running command.raw on 172.23.106.112: /opt/couchbase/bin/cbcollect_info 172.23.106.112-20240129-1153-diag.zip 2024-01-29 11:57:27,753 - root - INFO - command executed successfully with root 2024-01-29 11:57:27,827 - root - INFO - found the file /root/172.23.105.33-20240129-1153-diag.zip Downloading zipped logs from 172.23.105.33 2024-01-29 11:57:28,056 - root - INFO - running command.raw on 172.23.105.33: rm -f /root/172.23.105.33-20240129-1153-diag.zip 2024-01-29 11:57:28,065 - root - INFO - command executed successfully with root 2024-01-29 11:58:04,717 - root - INFO - command executed successfully with root 2024-01-29 11:58:04,813 - root - INFO - found the file /root/172.23.107.26-20240129-1153-diag.zip Downloading zipped logs from 172.23.107.26 2024-01-29 11:58:05,028 - root - INFO - running command.raw on 172.23.107.26: rm -f /root/172.23.107.26-20240129-1153-diag.zip 2024-01-29 11:58:05,044 - root - INFO - command executed successfully with root 2024-01-29 11:58:06,971 - root - INFO - command executed successfully with root 2024-01-29 11:58:07,056 - root - INFO - found the file /root/172.23.104.102-20240129-1153-diag.zip Downloading zipped logs from 172.23.104.102 2024-01-29 11:58:07,271 - root - INFO - running command.raw on 172.23.104.102: rm -f /root/172.23.104.102-20240129-1153-diag.zip 2024-01-29 11:58:07,287 - root - INFO - command executed successfully with root 2024-01-29 11:58:32,961 - root - INFO - command executed successfully with root 2024-01-29 11:58:33,050 - root - INFO - found the file /root/172.23.106.112-20240129-1153-diag.zip Downloading zipped logs from 172.23.106.112 2024-01-29 11:58:33,275 - root - INFO - running command.raw on 172.23.106.112: rm -f /root/172.23.106.112-20240129-1153-diag.zip 2024-01-29 11:58:33,291 - root - INFO - command executed successfully with root *** TestRunner *** workspace is /data/workspace/debian-p0-query-vset00-00-upgrade_7.1_n1ql fails is 1 1 Desc1: 7.6.0-2054 - query upgrade_7.1_n1ql - debian (0/1) python3 scripts/rerun_jobs.py 7.6.0-2054 --executor_jenkins_job --run_params=gsi_type=plasma,nodes_init=4,initial_version=7.1.6-3955,upgrade_version=7.6.0-2054,GROUP=N1QL,get-cbcollect-info=True,get-cbcollect-info=True,bucket_storage=couchstore,sirius_url=http://172.23.120.103:4000 INFO:merge_reports:Merging of report files from ['Old_Report_mergedreport-24-Jan-28_09-17-03-tuqquery.n1ql_upgrade.QueriesUpgradeTests.xml', 'logs/**/*.xml'] INFO:merge_reports:-- Old_Report_mergedreport-24-Jan-28_09-17-03-tuqquery.n1ql_upgrade.QueriesUpgradeTests.xml -- INFO:merge_reports:-- logs/testrunner-24-Jan-29_11-40-24/report-24-Jan-29_11-40-24-tuqquery.n1ql_upgrade.QueriesUpgradeTests.xml -- INFO:merge_reports: Number of TestSuites=1 INFO:merge_reports: TestSuite#1) tuqquery.n1ql_upgrade.QueriesUpgradeTests, Number of Tests=4 INFO:merge_reports:Summary file is at /data/workspace/debian-p0-query-vset00-00-upgrade_7.1_n1ql/logs/testrunner-24-Jan-29_11-58-33/merged_summary/mergedreport-24-Jan-29_11-58-33-tuqquery.n1ql_upgrade.QueriesUpgradeTests.xml Merging xmls summary so far suite tuqquery.n1ql_upgrade.QueriesUpgradeTests , pass 3 , fail 1 failures so far... tuqquery.n1ql_upgrade.QueriesUpgradeTests.test_upgrade merged xmls No more failed tests. Stopping reruns [description-setter] Description set: 7.6.0-2054 - query upgrade_7.1_n1ql - debian (0/1) [EnvInject] - Injecting environment variables from a build step. [EnvInject] - Injecting as environment variables the properties file path 'propfile' [EnvInject] - Variables injected successfully. Archiving artifacts Recording test results Build step 'Publish JUnit test result report' changed build result to UNSTABLE [BFA] Scanning build for known causes... [BFA] No failure causes found [BFA] Done. 0s Notifying upstream projects of job completion Email was triggered for: Unstable (Test Failures) Sending email for trigger: Unstable (Test Failures) Sending email to: pierre.regazzoni@couchbase.com Triggering a new build of savejoblogs Triggering a new build of test-executor-cleanup Triggering a new build of test-executor-cleanup-aws Triggering a new build of post-to-gerrit Finished: UNSTABLE