Started by user Balakumaran Gopal Rebuilds build #289 Running as SYSTEM [EnvInject] - Loading node environment variables. Building remotely on magma-sd2501-30g-20c (magma_slave magma_sdk2) in workspace /data/workspace/temp_rebalance_magma [WS-CLEANUP] Deleting project workspace... [WS-CLEANUP] Deferred wipeout is used... [WS-CLEANUP] Done No credentials specified Cloning the remote Git repository Cloning repository https://github.com/couchbaselabs/TAF/ > /usr/bin/git init /data/workspace/temp_rebalance_magma # timeout=10 Fetching upstream changes from https://github.com/couchbaselabs/TAF/ > /usr/bin/git --version # timeout=10 > /usr/bin/git fetch --tags --progress https://github.com/couchbaselabs/TAF/ +refs/heads/*:refs/remotes/origin/* # timeout=10 > /usr/bin/git config remote.origin.url https://github.com/couchbaselabs/TAF/ # timeout=10 > /usr/bin/git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10 > /usr/bin/git config remote.origin.url https://github.com/couchbaselabs/TAF/ # timeout=10 Fetching upstream changes from https://github.com/couchbaselabs/TAF/ > /usr/bin/git fetch --tags --progress https://github.com/couchbaselabs/TAF/ +refs/heads/*:refs/remotes/origin/* # timeout=10 > /usr/bin/git rev-parse refs/remotes/origin/master^{commit} # timeout=10 > /usr/bin/git rev-parse refs/remotes/origin/origin/master^{commit} # timeout=10 Checking out Revision 582c86a5666f288a13a5e41af2154ca81d5ea88b (refs/remotes/origin/master) > /usr/bin/git config core.sparsecheckout # timeout=10 > /usr/bin/git checkout -f 582c86a5666f288a13a5e41af2154ca81d5ea88b # timeout=10 Commit message: "Adding bloomfilter stats" > /usr/bin/git rev-list --no-walk 582c86a5666f288a13a5e41af2154ca81d5ea88b # timeout=10 [temp_rebalance_magma] $ /bin/sh -xe /tmp/jenkins5827770975919072239.sh + echo '[global] username:root password:couchbase port:8091 n1ql_port:8093 index_port:9102 data_path:/data [membase] rest_username:Administrator rest_password:password [servers] 1:_1 2:_2 3:_3 4:_4 5:_5 6:_6 7:_7 [_1] ip:172.23.105.164 [_2] ip:172.23.105.206 [_3] ip:172.23.106.177 [_4] ip:172.23.100.35 [_5] ip:172.23.100.34 [_6] ip:172.23.100.36 [_7] ip:172.23.100.37' + git checkout master Switched to a new branch 'master' Branch master set up to track remote branch master from origin. + git pull origin master From https://github.com/couchbaselabs/TAF * branch master -> FETCH_HEAD Already up-to-date. + git clone https://github.com/sumedhpb/guides.git Cloning into 'guides'... + jython_path=/opt/jython/bin/jython + /opt/jython/bin/pip install -r requirements.txt Requirement already satisfied (use --upgrade to upgrade): futures==3.3.0 in /opt/jython/Lib/site-packages (from -r requirements.txt (line 1)) Requirement already satisfied (use --upgrade to upgrade): requests==2.24.0 in /opt/jython/Lib/site-packages (from -r requirements.txt (line 2)) Requirement already satisfied (use --upgrade to upgrade): urllib3==1.25.10 in /opt/jython/Lib/site-packages (from -r requirements.txt (line 3)) Requirement already satisfied (use --upgrade to upgrade): ruamel.yaml==0.16.12 in /opt/jython/Lib/site-packages (from -r requirements.txt (line 4)) Requirement already satisfied (use --upgrade to upgrade): six==1.15.0 in /opt/jython/Lib/site-packages (from -r requirements.txt (line 5)) Cleaning up... + sdk_client_params='-P transaction_version=1.1.8 -P client_version=3.1.6' + mkdir tr_for_install + cd tr_for_install + git clone https://github.com/couchbase/testrunner.git Cloning into 'testrunner'... + cd testrunner + git checkout master Already on 'master' + git pull origin master From https://github.com/couchbase/testrunner * branch master -> FETCH_HEAD Already up-to-date. + py_executable=python + 7.1.0-2440 + grep 7 /tmp/jenkins5827770975919072239.sh: line 65: 7.1.0-2440: command not found + python3 scripts/new_install.py -i /tmp/win10-bucket-ops-temp_rebalance_magma.ini -p timeout=1200,get-cbcollect-info=True,version=7.1.0-2440,product=cb,debug_logs=False,url= 2022-03-06 03:32:21,479 - root - WARNING - URL: is not valid, will use version to locate build 2022-03-06 03:32:21,480 - root - INFO - SSH Connecting to 172.23.105.164 with username:root, attempt#1 of 5 2022-03-06 03:32:21,586 - root - INFO - SSH Connected to 172.23.105.164 as root 2022-03-06 03:32:21,858 - root - INFO - os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2022-03-06 03:32:22,106 - root - INFO - extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2022-03-06 03:32:22,107 - root - INFO - SSH Connecting to 172.23.105.206 with username:root, attempt#1 of 5 2022-03-06 03:32:22,210 - root - INFO - SSH Connected to 172.23.105.206 as root 2022-03-06 03:32:22,475 - root - INFO - os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2022-03-06 03:32:22,718 - root - INFO - extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2022-03-06 03:32:22,719 - root - INFO - SSH Connecting to 172.23.106.177 with username:root, attempt#1 of 5 2022-03-06 03:32:22,821 - root - INFO - SSH Connected to 172.23.106.177 as root 2022-03-06 03:32:23,088 - root - INFO - os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2022-03-06 03:32:23,331 - root - INFO - extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2022-03-06 03:32:23,332 - root - INFO - SSH Connecting to 172.23.100.35 with username:root, attempt#1 of 5 2022-03-06 03:32:23,437 - root - INFO - SSH Connected to 172.23.100.35 as root 2022-03-06 03:32:23,705 - root - INFO - os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2022-03-06 03:32:23,949 - root - INFO - extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2022-03-06 03:32:23,950 - root - INFO - SSH Connecting to 172.23.100.34 with username:root, attempt#1 of 5 2022-03-06 03:32:24,054 - root - INFO - SSH Connected to 172.23.100.34 as root 2022-03-06 03:32:24,320 - root - INFO - os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2022-03-06 03:32:24,564 - root - INFO - extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2022-03-06 03:32:24,565 - root - INFO - SSH Connecting to 172.23.100.36 with username:root, attempt#1 of 5 2022-03-06 03:32:24,667 - root - INFO - SSH Connected to 172.23.100.36 as root 2022-03-06 03:32:24,929 - root - INFO - os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2022-03-06 03:32:25,171 - root - INFO - extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2022-03-06 03:32:25,173 - root - INFO - SSH Connecting to 172.23.100.37 with username:root, attempt#1 of 5 2022-03-06 03:32:25,273 - root - INFO - SSH Connected to 172.23.100.37 as root 2022-03-06 03:32:25,539 - root - INFO - os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2022-03-06 03:32:25,781 - root - INFO - extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2022-03-06 03:32:25,782 - root - INFO - SSH Connecting to 172.23.105.164 with username:root, attempt#1 of 5 2022-03-06 03:32:25,883 - root - INFO - SSH Connected to 172.23.105.164 as root 2022-03-06 03:32:26,144 - root - INFO - os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2022-03-06 03:32:26,392 - root - INFO - extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2022-03-06 03:32:26,394 - root - INFO - SSH Connecting to 172.23.105.206 with username:root, attempt#1 of 5 2022-03-06 03:32:26,493 - root - INFO - SSH Connected to 172.23.105.206 as root 2022-03-06 03:32:26,745 - root - INFO - os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2022-03-06 03:32:26,988 - root - INFO - extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2022-03-06 03:32:26,990 - root - INFO - SSH Connecting to 172.23.106.177 with username:root, attempt#1 of 5 2022-03-06 03:32:27,088 - root - INFO - SSH Connected to 172.23.106.177 as root 2022-03-06 03:32:27,342 - root - INFO - os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2022-03-06 03:32:27,583 - root - INFO - extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2022-03-06 03:32:27,585 - root - INFO - SSH Connecting to 172.23.100.35 with username:root, attempt#1 of 5 2022-03-06 03:32:27,686 - root - INFO - SSH Connected to 172.23.100.35 as root 2022-03-06 03:32:27,939 - root - INFO - os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2022-03-06 03:32:28,185 - root - INFO - extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2022-03-06 03:32:28,186 - root - INFO - SSH Connecting to 172.23.100.34 with username:root, attempt#1 of 5 2022-03-06 03:32:28,290 - root - INFO - SSH Connected to 172.23.100.34 as root 2022-03-06 03:32:28,546 - root - INFO - os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2022-03-06 03:32:28,792 - root - INFO - extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2022-03-06 03:32:28,793 - root - INFO - SSH Connecting to 172.23.100.36 with username:root, attempt#1 of 5 2022-03-06 03:32:28,891 - root - INFO - SSH Connected to 172.23.100.36 as root 2022-03-06 03:32:29,140 - root - INFO - os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2022-03-06 03:32:29,381 - root - INFO - extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2022-03-06 03:32:29,383 - root - INFO - SSH Connecting to 172.23.100.37 with username:root, attempt#1 of 5 2022-03-06 03:32:29,480 - root - INFO - SSH Connected to 172.23.100.37 as root 2022-03-06 03:32:29,739 - root - INFO - os_distro: CentOS, os_version: centos 7, is_linux_distro: True 2022-03-06 03:32:29,982 - root - INFO - extract_remote_info-->distribution_type: CentOS, distribution_version: centos 7 2022-03-06 03:32:29,982 - root - INFO - Trying to check is this url alive: http://172.23.126.166/builds/latestbuilds/couchbase-server/neo/2440/couchbase-server-enterprise-7.1.0-2440-centos7.x86_64.rpm 2022-03-06 03:32:29,986 - root - INFO - This url http://172.23.126.166/builds/latestbuilds/couchbase-server/neo/2440/couchbase-server-enterprise-7.1.0-2440-centos7.x86_64.rpm is live 2022-03-06 03:32:29,986 - root - INFO - Trying to check is this url alive: http://172.23.126.166/builds/latestbuilds/couchbase-server/neo/2440/couchbase-server-enterprise-7.1.0-2440-centos7.x86_64.rpm 2022-03-06 03:32:29,992 - root - INFO - This url http://172.23.126.166/builds/latestbuilds/couchbase-server/neo/2440/couchbase-server-enterprise-7.1.0-2440-centos7.x86_64.rpm is live 2022-03-06 03:32:29,992 - root - INFO - Trying to check is this url alive: http://172.23.126.166/builds/latestbuilds/couchbase-server/neo/2440/couchbase-server-enterprise-7.1.0-2440-centos7.x86_64.rpm 2022-03-06 03:32:29,993 - root - INFO - This url http://172.23.126.166/builds/latestbuilds/couchbase-server/neo/2440/couchbase-server-enterprise-7.1.0-2440-centos7.x86_64.rpm is live 2022-03-06 03:32:29,993 - root - INFO - Trying to check is this url alive: http://172.23.126.166/builds/latestbuilds/couchbase-server/neo/2440/couchbase-server-enterprise-7.1.0-2440-centos7.x86_64.rpm 2022-03-06 03:32:29,994 - root - INFO - This url http://172.23.126.166/builds/latestbuilds/couchbase-server/neo/2440/couchbase-server-enterprise-7.1.0-2440-centos7.x86_64.rpm is live 2022-03-06 03:32:29,995 - root - INFO - Trying to check is this url alive: http://172.23.126.166/builds/latestbuilds/couchbase-server/neo/2440/couchbase-server-enterprise-7.1.0-2440-centos7.x86_64.rpm 2022-03-06 03:32:29,996 - root - INFO - This url http://172.23.126.166/builds/latestbuilds/couchbase-server/neo/2440/couchbase-server-enterprise-7.1.0-2440-centos7.x86_64.rpm is live 2022-03-06 03:32:29,996 - root - INFO - Trying to check is this url alive: http://172.23.126.166/builds/latestbuilds/couchbase-server/neo/2440/couchbase-server-enterprise-7.1.0-2440-centos7.x86_64.rpm 2022-03-06 03:32:29,997 - root - INFO - This url http://172.23.126.166/builds/latestbuilds/couchbase-server/neo/2440/couchbase-server-enterprise-7.1.0-2440-centos7.x86_64.rpm is live 2022-03-06 03:32:29,997 - root - INFO - Trying to check is this url alive: http://172.23.126.166/builds/latestbuilds/couchbase-server/neo/2440/couchbase-server-enterprise-7.1.0-2440-centos7.x86_64.rpm 2022-03-06 03:32:29,998 - root - INFO - This url http://172.23.126.166/builds/latestbuilds/couchbase-server/neo/2440/couchbase-server-enterprise-7.1.0-2440-centos7.x86_64.rpm is live 2022-03-06 03:32:43,105 - root - INFO - Done with uninstall on 172.23.105.164. 2022-03-06 03:32:43,538 - root - INFO - Done with uninstall on 172.23.106.177. 2022-03-06 03:32:44,466 - root - INFO - Done with uninstall on 172.23.105.206. 2022-03-06 03:32:55,924 - root - INFO - Done with uninstall on 172.23.100.34. 2022-03-06 03:32:55,930 - root - INFO - Done with uninstall on 172.23.100.36. 2022-03-06 03:32:57,824 - root - INFO - Done with uninstall on 172.23.100.37. 2022-03-06 03:33:31,708 - root - INFO - Done with uninstall on 172.23.100.35. 2022-03-06 03:33:36,169 - root - INFO - running command.raw on 172.23.105.164: cd /tmp/ && wc -c couchbase-server-enterprise-7.1.0-2440-centos7.x86_64.rpm 2022-03-06 03:33:36,184 - root - INFO - command executed successfully with root 2022-03-06 03:33:37,330 - root - INFO - running command.raw on 172.23.105.206: cd /tmp/ && wc -c couchbase-server-enterprise-7.1.0-2440-centos7.x86_64.rpm 2022-03-06 03:33:37,337 - root - INFO - command executed successfully with root 2022-03-06 03:33:38,383 - root - INFO - running command.raw on 172.23.106.177: cd /tmp/ && wc -c couchbase-server-enterprise-7.1.0-2440-centos7.x86_64.rpm 2022-03-06 03:33:38,390 - root - INFO - command executed successfully with root 2022-03-06 03:33:39,606 - root - INFO - running command.raw on 172.23.100.35: cd /tmp/ && wc -c couchbase-server-enterprise-7.1.0-2440-centos7.x86_64.rpm 2022-03-06 03:33:39,645 - root - INFO - command executed successfully with root 2022-03-06 03:33:40,803 - root - INFO - running command.raw on 172.23.100.34: cd /tmp/ && wc -c couchbase-server-enterprise-7.1.0-2440-centos7.x86_64.rpm 2022-03-06 03:33:40,809 - root - INFO - command executed successfully with root 2022-03-06 03:33:41,761 - root - INFO - running command.raw on 172.23.100.36: cd /tmp/ && wc -c couchbase-server-enterprise-7.1.0-2440-centos7.x86_64.rpm 2022-03-06 03:33:41,767 - root - INFO - command executed successfully with root 2022-03-06 03:33:42,680 - root - INFO - running command.raw on 172.23.100.37: cd /tmp/ && wc -c couchbase-server-enterprise-7.1.0-2440-centos7.x86_64.rpm 2022-03-06 03:33:42,687 - root - INFO - command executed successfully with root 2022-03-06 03:34:38,917 - root - INFO - Done with install on 172.23.105.206. 2022-03-06 03:34:38,918 - root - INFO - Waiting for couchbase to be reachable 2022-03-06 03:34:38,919 - root - ERROR - socket error while connecting to http://172.23.105.206:8091/nodes/self error [Errno 111] Connection refused 2022-03-06 03:34:39,933 - root - INFO - Done with install on 172.23.106.177. 2022-03-06 03:34:39,933 - root - INFO - Waiting for couchbase to be reachable 2022-03-06 03:34:39,934 - root - ERROR - socket error while connecting to http://172.23.106.177:8091/nodes/self error [Errno 111] Connection refused 2022-03-06 03:34:40,410 - root - INFO - Done with install on 172.23.105.164. 2022-03-06 03:34:40,411 - root - INFO - Waiting for couchbase to be reachable 2022-03-06 03:34:40,412 - root - ERROR - socket error while connecting to http://172.23.105.164:8091/nodes/self error [Errno 111] Connection refused 2022-03-06 03:34:41,923 - root - ERROR - socket error while connecting to http://172.23.105.206:8091/nodes/self error [Errno 111] Connection refused 2022-03-06 03:34:42,938 - root - ERROR - socket error while connecting to http://172.23.106.177:8091/nodes/self error [Errno 111] Connection refused 2022-03-06 03:34:43,415 - root - ERROR - socket error while connecting to http://172.23.105.164:8091/nodes/self error [Errno 111] Connection refused 2022-03-06 03:34:47,935 - root - INFO - running command.raw on 172.23.105.206: /opt/couchbase/bin/couchbase-cli node-init -c 172.23.105.206 -u Administrator -p password > /dev/null && echo 1 || echo 0; 2022-03-06 03:34:48,338 - root - INFO - command executed successfully with root 2022-03-06 03:34:48,341 - root - INFO - running command.raw on 172.23.105.206: rm -rf /data/* 2022-03-06 03:34:48,950 - root - INFO - running command.raw on 172.23.106.177: /opt/couchbase/bin/couchbase-cli node-init -c 172.23.106.177 -u Administrator -p password > /dev/null && echo 1 || echo 0; 2022-03-06 03:34:49,347 - root - INFO - command executed successfully with root 2022-03-06 03:34:49,350 - root - INFO - running command.raw on 172.23.106.177: rm -rf /data/* 2022-03-06 03:34:49,428 - root - INFO - running command.raw on 172.23.105.164: /opt/couchbase/bin/couchbase-cli node-init -c 172.23.105.164 -u Administrator -p password > /dev/null && echo 1 || echo 0; 2022-03-06 03:34:49,832 - root - INFO - command executed successfully with root 2022-03-06 03:34:49,835 - root - INFO - running command.raw on 172.23.105.164: rm -rf /data/* 2022-03-06 03:34:54,053 - root - INFO - command executed successfully with root 2022-03-06 03:34:54,053 - root - INFO - running command.raw on 172.23.105.206: chown -R couchbase:couchbase /data 2022-03-06 03:34:54,061 - root - INFO - command executed successfully with root 2022-03-06 03:34:54,061 - root - INFO - /nodes/self/controller/settings : path=%2Fdata 2022-03-06 03:34:55,636 - root - INFO - command executed successfully with root 2022-03-06 03:34:55,636 - root - INFO - running command.raw on 172.23.106.177: chown -R couchbase:couchbase /data 2022-03-06 03:34:55,684 - root - INFO - command executed successfully with root 2022-03-06 03:34:55,684 - root - INFO - /nodes/self/controller/settings : path=%2Fdata 2022-03-06 03:34:55,701 - root - INFO - command executed successfully with root 2022-03-06 03:34:55,701 - root - INFO - running command.raw on 172.23.105.164: chown -R couchbase:couchbase /data 2022-03-06 03:34:55,795 - root - INFO - command executed successfully with root 2022-03-06 03:34:55,795 - root - INFO - /nodes/self/controller/settings : path=%2Fdata 2022-03-06 03:34:56,562 - root - INFO - Setting data_path: /data: status True 2022-03-06 03:34:57,568 - root - INFO - Setting KV memory quota as 7855 MB on 172.23.105.206 2022-03-06 03:34:57,569 - root - INFO - pools/default params : memoryQuota=7855 2022-03-06 03:34:57,571 - root - INFO - --> init_node_services(Administrator,password,None,8091,['kv']) 2022-03-06 03:34:57,572 - root - INFO - /node/controller/setupServices params on 172.23.105.206: 8091:hostname=None&user=Administrator&password=password&services=kv 2022-03-06 03:34:57,600 - root - INFO - --> in init_cluster...Administrator,password,8091 2022-03-06 03:34:57,601 - root - INFO - settings/web params on 172.23.105.206:8091:port=8091&username=Administrator&password=password 2022-03-06 03:34:57,660 - root - INFO - --> status:True 2022-03-06 03:34:57,660 - root - INFO - Done with init on 172.23.105.206. 2022-03-06 03:34:57,771 - root - INFO - Done with cleanup on 172.23.105.206. 2022-03-06 03:34:58,138 - root - INFO - Setting data_path: /data: status True 2022-03-06 03:34:58,145 - root - INFO - Setting data_path: /data: status True 2022-03-06 03:34:59,142 - root - INFO - Setting KV memory quota as 7855 MB on 172.23.105.164 2022-03-06 03:34:59,142 - root - INFO - pools/default params : memoryQuota=7855 2022-03-06 03:34:59,145 - root - INFO - --> init_node_services(Administrator,password,None,8091,['kv']) 2022-03-06 03:34:59,145 - root - INFO - /node/controller/setupServices params on 172.23.105.164: 8091:hostname=None&user=Administrator&password=password&services=kv 2022-03-06 03:34:59,149 - root - INFO - Setting KV memory quota as 7855 MB on 172.23.106.177 2022-03-06 03:34:59,149 - root - INFO - pools/default params : memoryQuota=7855 2022-03-06 03:34:59,152 - root - INFO - --> init_node_services(Administrator,password,None,8091,['kv']) 2022-03-06 03:34:59,152 - root - INFO - /node/controller/setupServices params on 172.23.106.177: 8091:hostname=None&user=Administrator&password=password&services=kv 2022-03-06 03:34:59,174 - root - INFO - --> in init_cluster...Administrator,password,8091 2022-03-06 03:34:59,174 - root - INFO - settings/web params on 172.23.105.164:8091:port=8091&username=Administrator&password=password 2022-03-06 03:34:59,181 - root - INFO - --> in init_cluster...Administrator,password,8091 2022-03-06 03:34:59,182 - root - INFO - settings/web params on 172.23.106.177:8091:port=8091&username=Administrator&password=password 2022-03-06 03:34:59,233 - root - INFO - --> status:True 2022-03-06 03:34:59,233 - root - INFO - Done with init on 172.23.105.164. 2022-03-06 03:34:59,240 - root - INFO - --> status:True 2022-03-06 03:34:59,240 - root - INFO - Done with init on 172.23.106.177. 2022-03-06 03:34:59,342 - root - INFO - Done with cleanup on 172.23.106.177. 2022-03-06 03:34:59,346 - root - INFO - Done with cleanup on 172.23.105.164. 2022-03-06 03:38:41,010 - root - INFO - Done with install on 172.23.100.34. 2022-03-06 03:38:41,010 - root - INFO - Waiting for couchbase to be reachable 2022-03-06 03:38:41,012 - root - ERROR - socket error while connecting to http://172.23.100.34:8091/nodes/self error [Errno 111] Connection refused 2022-03-06 03:38:44,015 - root - ERROR - socket error while connecting to http://172.23.100.34:8091/nodes/self error [Errno 111] Connection refused 2022-03-06 03:38:48,185 - root - INFO - Done with install on 172.23.100.36. 2022-03-06 03:38:48,185 - root - INFO - Waiting for couchbase to be reachable 2022-03-06 03:38:48,186 - root - ERROR - socket error while connecting to http://172.23.100.36:8091/nodes/self error [Errno 111] Connection refused 2022-03-06 03:38:50,624 - root - INFO - running command.raw on 172.23.100.34: /opt/couchbase/bin/couchbase-cli node-init -c 172.23.100.34 -u Administrator -p password > /dev/null && echo 1 || echo 0; 2022-03-06 03:38:51,190 - root - ERROR - socket error while connecting to http://172.23.100.36:8091/nodes/self error [Errno 111] Connection refused 2022-03-06 03:38:52,136 - root - INFO - command executed successfully with root 2022-03-06 03:38:52,139 - root - INFO - running command.raw on 172.23.100.34: rm -rf /data/* 2022-03-06 03:38:52,145 - root - INFO - command executed successfully with root 2022-03-06 03:38:52,145 - root - INFO - running command.raw on 172.23.100.34: chown -R couchbase:couchbase /data 2022-03-06 03:38:52,148 - root - INFO - Done with install on 172.23.100.35. 2022-03-06 03:38:52,149 - root - INFO - Waiting for couchbase to be reachable 2022-03-06 03:38:52,149 - root - ERROR - socket error while connecting to http://172.23.100.35:8091/nodes/self error [Errno 111] Connection refused 2022-03-06 03:38:52,192 - root - INFO - command executed successfully with root 2022-03-06 03:38:52,192 - root - INFO - /nodes/self/controller/settings : path=%2Fdata 2022-03-06 03:38:55,153 - root - ERROR - socket error while connecting to http://172.23.100.35:8091/nodes/self error [Errno 111] Connection refused 2022-03-06 03:38:55,592 - root - INFO - Setting data_path: /data: status True 2022-03-06 03:38:56,597 - root - INFO - Setting KV memory quota as 7855 MB on 172.23.100.34 2022-03-06 03:38:56,597 - root - INFO - pools/default params : memoryQuota=7855 2022-03-06 03:38:56,600 - root - INFO - --> init_node_services(Administrator,password,None,8091,['kv']) 2022-03-06 03:38:56,600 - root - INFO - /node/controller/setupServices params on 172.23.100.34: 8091:hostname=None&user=Administrator&password=password&services=kv 2022-03-06 03:38:57,197 - root - ERROR - socket error while connecting to http://172.23.100.36:8091/nodes/self error [Errno 111] Connection refused 2022-03-06 03:39:00,888 - root - INFO - --> in init_cluster...Administrator,password,8091 2022-03-06 03:39:00,889 - root - INFO - settings/web params on 172.23.100.34:8091:port=8091&username=Administrator&password=password 2022-03-06 03:39:00,947 - root - INFO - --> status:True 2022-03-06 03:39:00,947 - root - INFO - Done with init on 172.23.100.34. 2022-03-06 03:39:01,045 - root - INFO - Done with cleanup on 172.23.100.34. 2022-03-06 03:39:01,160 - root - ERROR - socket error while connecting to http://172.23.100.35:8091/nodes/self error [Errno 111] Connection refused 2022-03-06 03:39:04,037 - root - INFO - Done with install on 172.23.100.37. 2022-03-06 03:39:04,037 - root - INFO - Waiting for couchbase to be reachable 2022-03-06 03:39:04,038 - root - ERROR - socket error while connecting to http://172.23.100.37:8091/nodes/self error [Errno 111] Connection refused 2022-03-06 03:39:07,042 - root - ERROR - socket error while connecting to http://172.23.100.37:8091/nodes/self error [Errno 111] Connection refused 2022-03-06 03:39:09,215 - root - INFO - running command.raw on 172.23.100.36: /opt/couchbase/bin/couchbase-cli node-init -c 172.23.100.36 -u Administrator -p password > /dev/null && echo 1 || echo 0; 2022-03-06 03:39:11,861 - root - INFO - command executed successfully with root 2022-03-06 03:39:11,865 - root - INFO - running command.raw on 172.23.100.36: rm -rf /data/* 2022-03-06 03:39:11,870 - root - INFO - command executed successfully with root 2022-03-06 03:39:11,870 - root - INFO - running command.raw on 172.23.100.36: chown -R couchbase:couchbase /data 2022-03-06 03:39:11,917 - root - INFO - command executed successfully with root 2022-03-06 03:39:11,917 - root - INFO - /nodes/self/controller/settings : path=%2Fdata 2022-03-06 03:39:13,049 - root - ERROR - socket error while connecting to http://172.23.100.37:8091/nodes/self error [Errno 111] Connection refused 2022-03-06 03:39:13,178 - root - INFO - running command.raw on 172.23.100.35: /opt/couchbase/bin/couchbase-cli node-init -c 172.23.100.35 -u Administrator -p password > /dev/null && echo 1 || echo 0; 2022-03-06 03:39:13,899 - root - INFO - command executed successfully with root 2022-03-06 03:39:13,902 - root - INFO - running command.raw on 172.23.100.35: rm -rf /data/* 2022-03-06 03:39:17,877 - root - INFO - Setting data_path: /data: status True 2022-03-06 03:39:18,882 - root - INFO - Setting KV memory quota as 7855 MB on 172.23.100.36 2022-03-06 03:39:18,882 - root - INFO - pools/default params : memoryQuota=7855 2022-03-06 03:39:18,885 - root - INFO - --> init_node_services(Administrator,password,None,8091,['kv']) 2022-03-06 03:39:18,885 - root - INFO - /node/controller/setupServices params on 172.23.100.36: 8091:hostname=None&user=Administrator&password=password&services=kv 2022-03-06 03:39:19,475 - root - INFO - --> in init_cluster...Administrator,password,8091 2022-03-06 03:39:19,475 - root - INFO - settings/web params on 172.23.100.36:8091:port=8091&username=Administrator&password=password 2022-03-06 03:39:19,533 - root - INFO - --> status:True 2022-03-06 03:39:19,533 - root - INFO - Done with init on 172.23.100.36. 2022-03-06 03:39:19,634 - root - INFO - Done with cleanup on 172.23.100.36. 2022-03-06 03:39:19,803 - root - INFO - command executed successfully with root 2022-03-06 03:39:19,803 - root - INFO - running command.raw on 172.23.100.35: chown -R couchbase:couchbase /data 2022-03-06 03:39:19,959 - root - INFO - command executed successfully with root 2022-03-06 03:39:19,959 - root - INFO - /nodes/self/controller/settings : path=%2Fdata 2022-03-06 03:39:23,556 - root - INFO - Setting data_path: /data: status True 2022-03-06 03:39:24,562 - root - INFO - Setting KV memory quota as 7855 MB on 172.23.100.35 2022-03-06 03:39:24,562 - root - INFO - pools/default params : memoryQuota=7855 2022-03-06 03:39:24,565 - root - INFO - --> init_node_services(Administrator,password,None,8091,['kv']) 2022-03-06 03:39:24,565 - root - INFO - /node/controller/setupServices params on 172.23.100.35: 8091:hostname=None&user=Administrator&password=password&services=kv 2022-03-06 03:39:24,987 - root - INFO - --> in init_cluster...Administrator,password,8091 2022-03-06 03:39:24,987 - root - INFO - settings/web params on 172.23.100.35:8091:port=8091&username=Administrator&password=password 2022-03-06 03:39:25,045 - root - INFO - --> status:True 2022-03-06 03:39:25,046 - root - INFO - Done with init on 172.23.100.35. 2022-03-06 03:39:25,067 - root - INFO - running command.raw on 172.23.100.37: /opt/couchbase/bin/couchbase-cli node-init -c 172.23.100.37 -u Administrator -p password > /dev/null && echo 1 || echo 0; 2022-03-06 03:39:25,088 - root - INFO - Done with cleanup on 172.23.100.35. 2022-03-06 03:39:26,930 - root - INFO - command executed successfully with root 2022-03-06 03:39:26,933 - root - INFO - running command.raw on 172.23.100.37: rm -rf /data/* 2022-03-06 03:39:26,940 - root - INFO - command executed successfully with root 2022-03-06 03:39:26,940 - root - INFO - running command.raw on 172.23.100.37: chown -R couchbase:couchbase /data 2022-03-06 03:39:26,986 - root - INFO - command executed successfully with root 2022-03-06 03:39:26,986 - root - INFO - /nodes/self/controller/settings : path=%2Fdata 2022-03-06 03:39:31,384 - root - INFO - Setting data_path: /data: status True 2022-03-06 03:39:32,390 - root - INFO - Setting KV memory quota as 7855 MB on 172.23.100.37 2022-03-06 03:39:32,390 - root - INFO - pools/default params : memoryQuota=7855 2022-03-06 03:39:32,393 - root - INFO - --> init_node_services(Administrator,password,None,8091,['kv']) 2022-03-06 03:39:32,393 - root - INFO - /node/controller/setupServices params on 172.23.100.37: 8091:hostname=None&user=Administrator&password=password&services=kv 2022-03-06 03:39:32,530 - root - INFO - --> in init_cluster...Administrator,password,8091 2022-03-06 03:39:32,531 - root - INFO - settings/web params on 172.23.100.37:8091:port=8091&username=Administrator&password=password 2022-03-06 03:39:32,588 - root - INFO - --> status:True 2022-03-06 03:39:32,588 - root - INFO - Done with init on 172.23.100.37. 2022-03-06 03:39:32,689 - root - INFO - Done with cleanup on 172.23.100.37. 2022-03-06 03:39:33,088 - root - INFO - ---------------------------------------------------------------------------------------------------- 2022-03-06 03:39:33,115 - root - INFO - cluster:C1 node:172.23.105.164:8091 version:7.1.0-2440-enterprise aFamily:inet services:['kv'] 2022-03-06 03:39:33,115 - root - INFO - cluster:C2 node:172.23.105.206:8091 version:7.1.0-2440-enterprise aFamily:inet services:['kv'] 2022-03-06 03:39:33,115 - root - INFO - cluster:C3 node:172.23.106.177:8091 version:7.1.0-2440-enterprise aFamily:inet services:['kv'] 2022-03-06 03:39:33,115 - root - INFO - cluster:C4 node:172.23.100.35:8091 version:7.1.0-2440-enterprise aFamily:inet services:['kv'] 2022-03-06 03:39:33,115 - root - INFO - cluster:C5 node:172.23.100.34:8091 version:7.1.0-2440-enterprise aFamily:inet services:['kv'] 2022-03-06 03:39:33,115 - root - INFO - cluster:C6 node:172.23.100.36:8091 version:7.1.0-2440-enterprise aFamily:inet services:['kv'] 2022-03-06 03:39:33,115 - root - INFO - cluster:C7 node:172.23.100.37:8091 version:7.1.0-2440-enterprise aFamily:inet services:['kv'] 2022-03-06 03:39:33,115 - root - INFO - ---------------------------------------------------------------------------------------------------- 2022-03-06 03:39:33,115 - root - INFO - ---------------------------------------------------------------------------------------------------- 2022-03-06 03:39:33,115 - root - INFO - ---------------------------------------------------------------------------------------------------- 2022-03-06 03:39:33,115 - root - INFO - INSTALL COMPLETED ON: 172.23.105.164 2022-03-06 03:39:33,116 - root - INFO - INSTALL COMPLETED ON: 172.23.105.206 2022-03-06 03:39:33,116 - root - INFO - INSTALL COMPLETED ON: 172.23.106.177 2022-03-06 03:39:33,116 - root - INFO - INSTALL COMPLETED ON: 172.23.100.35 2022-03-06 03:39:33,116 - root - INFO - INSTALL COMPLETED ON: 172.23.100.34 2022-03-06 03:39:33,116 - root - INFO - INSTALL COMPLETED ON: 172.23.100.36 2022-03-06 03:39:33,116 - root - INFO - INSTALL COMPLETED ON: 172.23.100.37 2022-03-06 03:39:33,116 - root - INFO - ---------------------------------------------------------------------------------------------------- 2022-03-06 03:39:33,116 - root - INFO - TOTAL INSTALL TIME = 432 seconds + status=0 + cd ../.. + rm -rf tr_for_install + git fetch https://review.couchbase.org/TAF refs/changes/33/171733/7 From https://review.couchbase.org/TAF * branch refs/changes/33/171733/7 -> FETCH_HEAD + git cherry-pick FETCH_HEAD [master 22f587f] Add clusterMembership to Rebalance Overview which helps in failover testing Author: Balakumaran G Committer: root Your name and email address were configured automatically based on your username and hostname. Please check that they are accurate. You can suppress this message by setting them explicitly: git config --global user.name "Your Name" git config --global user.email you@example.com After doing this, you may fix the identity used for this commit with: git commit --amend --reset-author 2 files changed, 6 insertions(+), 4 deletions(-) + git fetch https://review.couchbase.org/TAF refs/changes/24/171624/3 From https://review.couchbase.org/TAF * branch refs/changes/24/171624/3 -> FETCH_HEAD + git cherry-pick FETCH_HEAD [master cbc99bf] Add ns_serv tests for bloom filter support in magma Author: Balakumaran G Committer: root Your name and email address were configured automatically based on your username and hostname. Please check that they are accurate. You can suppress this message by setting them explicitly: git config --global user.name "Your Name" git config --global user.email you@example.com After doing this, you may fix the identity used for this commit with: git commit --amend --reset-author 4 files changed, 66 insertions(+), 1 deletion(-) create mode 100644 conf/magma/dgm_collections_bloom_filter.conf create mode 100644 pytests/bucket_collections/bucket_templates/magma_dgm/bloom_filter/4_node_1_replica_magma_1024.py create mode 100644 pytests/bucket_collections/bucket_templates/magma_dgm/bloom_filter/__init__.py + git fetch https://review.couchbase.org/TAF refs/changes/58/171158/2 From https://review.couchbase.org/TAF * branch refs/changes/58/171158/2 -> FETCH_HEAD + git cherry-pick FETCH_HEAD [master b7c7b1f] Fetch cbcollect when cleanup of the cluster fails due to rebalance Author: Balakumaran G Committer: root Your name and email address were configured automatically based on your username and hostname. Please check that they are accurate. You can suppress this message by setting them explicitly: git config --global user.name "Your Name" git config --global user.email you@example.com After doing this, you may fix the identity used for this commit with: git commit --amend --reset-author 1 file changed, 8 insertions(+), 4 deletions(-) + guides/gradlew --refresh-dependencies testrunner -P jython=/opt/jython/bin/jython -P transaction_version=1.1.8 -P client_version=3.1.6 -P 'args=-i /tmp/win10-bucket-ops-temp_rebalance_magma.ini -p rerun=False,get-cbcollect-info=True -c conf/magma/dgm_collections_bloom_filter.conf -m rest' Starting a Gradle Daemon, 82 busy Daemons could not be reused, use --status for details > Configure project : Using Transaction_client :: 1.1.8 Using Java_client :: 3.1.6 Running: /opt/jython/bin/jython -J-cp /data/workspace/temp_rebalance_magma/build/classes/java/main:/data/workspace/temp_rebalance_magma/build/resources/main:/root/.gradle/caches/modules-2/files-2.1/com.jcraft/jsch/0.1.54/da3584329a263616e277e15462b387addd1b208d/jsch-0.1.54.jar:/root/.gradle/caches/modules-2/files-2.1/org.apache.logging.log4j/log4j-slf4j-impl/2.11.1/4b41b53a3a2d299ce381a69d165381ca19f62912/log4j-slf4j-impl-2.11.1.jar:/root/.gradle/caches/modules-2/files-2.1/commons-cli/commons-cli/1.4/c51c00206bb913cd8612b24abd9fa98ae89719b1/commons-cli-1.4.jar:/root/.gradle/caches/modules-2/files-2.1/com.azure/azure-storage-blob/12.14.0/75d45d21dc208fa369abc832354470464c859d67/azure-storage-blob-12.14.0.jar:/root/.gradle/caches/modules-2/files-2.1/com.couchbase.client/couchbase-transactions/1.1.8/e69a6013e59f498f76671c1acc43df14f1163180/couchbase-transactions-1.1.8.jar:/root/.gradle/caches/modules-2/files-2.1/com.couchbase.client/java-client/3.1.6/f065d71963e08bd5577838592e33f2ca35f5d64a/java-client-3.1.6.jar:/root/.gradle/caches/modules-2/files-2.1/com.azure/azure-storage-internal-avro/12.1.0/c5882c519f8ee15c528e9d0396ec7535da7d4ecd/azure-storage-internal-avro-12.1.0.jar:/root/.gradle/caches/modules-2/files-2.1/com.azure/azure-storage-common/12.13.0/ef8910849375def2b715ea64429136e694fb7ad6/azure-storage-common-12.13.0.jar:/root/.gradle/caches/modules-2/files-2.1/com.azure/azure-core-http-netty/1.11.0/2177ae2dc9cff849e49f9180bf6b95b8e2c78e1b/azure-core-http-netty-1.11.0.jar:/root/.gradle/caches/modules-2/files-2.1/com.azure/azure-core/1.20.0/a98c6bd18aa2066ecd8b39bf7ac51bd8e7307851/azure-core-1.20.0.jar:/root/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-api/1.7.32/cdcff33940d9f2de763bc41ea05a0be5941176c3/slf4j-api-1.7.32.jar:/root/.gradle/caches/modules-2/files-2.1/org.apache.logging.log4j/log4j-core/2.11.1/592a48674c926b01a9a747c7831bcd82a9e6d6e4/log4j-core-2.11.1.jar:/root/.gradle/caches/modules-2/files-2.1/org.apache.logging.log4j/log4j-api/2.11.1/268f0fe4df3eefe052b57c87ec48517d64fb2a10/log4j-api-2.11.1.jar:/root/.gradle/caches/modules-2/files-2.1/com.couchbase.client/core-io/2.1.6/b3ece73ab7069b1e97669500edc448163c2f4304/core-io-2.1.6.jar:/root/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.dataformat/jackson-dataformat-xml/2.12.4/15c743856696c0239f2c51d8d19d9f97f034713/jackson-dataformat-xml-2.12.4.jar:/root/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.module/jackson-module-jaxb-annotations/2.12.4/5e43703aae1a9843dfd7df0a0ad6cbfedcaff67f/jackson-module-jaxb-annotations-2.12.4.jar:/root/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.core/jackson-core/2.12.4/6a1bd259b6c4e3f9219ec8ec0be55ed11eed0c/jackson-core-2.12.4.jar:/root/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.datatype/jackson-datatype-jsr310/2.12.4/b1174c05d4ded121a7eaeed3f148709f9585b981/jackson-datatype-jsr310-2.12.4.jar:/root/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.core/jackson-databind/2.12.4/69206e02e6a696034f06a59d3ddbfbba5a4cd81/jackson-databind-2.12.4.jar:/root/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.core/jackson-annotations/2.12.4/752cf9a2562ac2c012e48057e3a4c17dad66c66e/jackson-annotations-2.12.4.jar:/root/.gradle/caches/modules-2/files-2.1/io.projectreactor.netty/reactor-netty/1.0.10/c54f6c92628f396076f74945424319746bbb37f8/reactor-netty-1.0.10.jar:/root/.gradle/caches/modules-2/files-2.1/io.projectreactor.netty/reactor-netty-http-brave/1.0.10/30aebc029ee25f5e0c2d1c0a798b17e926f61b8/reactor-netty-http-brave-1.0.10.jar:/root/.gradle/caches/modules-2/files-2.1/io.projectreactor.netty/reactor-netty-http/1.0.10/f34196d8778243d2ac4a4ae7cc74f110d1073803/reactor-netty-http-1.0.10.jar:/root/.gradle/caches/modules-2/files-2.1/io.projectreactor.netty/reactor-netty-core/1.0.10/dcb5d53dc07a6f660060f0b55c40a1e9944a6fe1/reactor-netty-core-1.0.10.jar:/root/.gradle/caches/modules-2/files-2.1/io.projectreactor/reactor-core/3.4.9/820332aa7b0fe3a8dfe14f58fc16e49ad178291/reactor-core-3.4.9.jar:/root/.gradle/caches/modules-2/files-2.1/io.netty/netty-tcnative-boringssl-static/2.0.40.Final/6b73a163c13ed76921892d28eb81235f4b41e40a/netty-tcnative-boringssl-static-2.0.40.Final.jar:/root/.gradle/caches/modules-2/files-2.1/io.netty/netty-handler-proxy/4.1.67.Final/2528292c49bd15f1b328dda25a7a75744d6f0991/netty-handler-proxy-4.1.67.Final.jar:/root/.gradle/caches/modules-2/files-2.1/io.netty/netty-codec-http2/4.1.67.Final/e2a0c27f396035fab7d932031d6f337244369495/netty-codec-http2-4.1.67.Final.jar:/root/.gradle/caches/modules-2/files-2.1/io.netty/netty-codec-http/4.1.67.Final/e282137917c67332fa9a414df89f89a93487aede/netty-codec-http-4.1.67.Final.jar:/root/.gradle/caches/modules-2/files-2.1/io.netty/netty-resolver-dns-native-macos/4.1.66.Final/8ca314d87b202a4dd94d810d2385b22555dd801c/netty-resolver-dns-native-macos-4.1.66.Final-osx-x86_64.jar:/root/.gradle/caches/modules-2/files-2.1/io.netty/netty-resolver-dns/4.1.66.Final/7b74815fb1403e5747c872c6eee2a07e7a700d30/netty-resolver-dns-4.1.66.Final.jar:/root/.gradle/caches/modules-2/files-2.1/io.netty/netty-handler/4.1.67.Final/62640a6524e1c08d6e8ac06556892b5e1362392f/netty-handler-4.1.67.Final.jar:/root/.gradle/caches/modules-2/files-2.1/io.netty/netty-transport-native-epoll/4.1.67.Final/ff955604e1edb8b13dc855ccbc84acb5fc1d989/netty-transport-native-epoll-4.1.67.Final-linux-x86_64.jar:/root/.gradle/caches/modules-2/files-2.1/io.netty/netty-transport-native-kqueue/4.1.67.Final/7a88ccb4237d1ff091be544860ae729f5b568ac/netty-transport-native-kqueue-4.1.67.Final-osx-x86_64.jar:/root/.gradle/caches/modules-2/files-2.1/io.netty/netty-transport-native-unix-common/4.1.67.Final/c6cca7ba897e1ac65e0742654ed10064401dbd2f/netty-transport-native-unix-common-4.1.67.Final.jar:/root/.gradle/caches/modules-2/files-2.1/io.netty/netty-codec-socks/4.1.67.Final/a4fe0487451dc262daaa024224fa207d088b8687/netty-codec-socks-4.1.67.Final.jar:/root/.gradle/caches/modules-2/files-2.1/io.netty/netty-codec-dns/4.1.66.Final/390ff96a0e1f7c626cb52c119a1a1dfd0784d193/netty-codec-dns-4.1.66.Final.jar:/root/.gradle/caches/modules-2/files-2.1/io.netty/netty-codec/4.1.67.Final/292e818822a4fca4e9f3ad711cdd46362f69590a/netty-codec-4.1.67.Final.jar:/root/.gradle/caches/modules-2/files-2.1/io.netty/netty-transport/4.1.67.Final/e8b502618f960cb1f8cdb7dca92d0878576ad7c6/netty-transport-4.1.67.Final.jar:/root/.gradle/caches/modules-2/files-2.1/io.netty/netty-buffer/4.1.67.Final/4b6217b05792fe6e2bd080c054138b2c20ae1b37/netty-buffer-4.1.67.Final.jar:/root/.gradle/caches/modules-2/files-2.1/com.fasterxml.woodstox/woodstox-core/6.2.4/16b9f8ab972e67eb21872ea2c40046249d543989/woodstox-core-6.2.4.jar:/root/.gradle/caches/modules-2/files-2.1/org.codehaus.woodstox/stax2-api/4.2.1/a3f7325c52240418c2ba257b103c3c550e140c83/stax2-api-4.2.1.jar:/root/.gradle/caches/modules-2/files-2.1/org.reactivestreams/reactive-streams/1.0.3/d9fb7a7926ffa635b3dcaa5049fb2bfa25b3e7d0/reactive-streams-1.0.3.jar:/root/.gradle/caches/modules-2/files-2.1/io.netty/netty-resolver/4.1.67.Final/bb9041d5f85f9a6270f5378d0becd243de8cfeac/netty-resolver-4.1.67.Final.jar:/root/.gradle/caches/modules-2/files-2.1/io.netty/netty-common/4.1.67.Final/a86a19033588a07e159ff95818845b1ec86b2281/netty-common-4.1.67.Final.jar:/root/.gradle/caches/modules-2/files-2.1/jakarta.xml.bind/jakarta.xml.bind-api/2.3.2/8d49996a4338670764d7ca4b85a1c4ccf7fe665d/jakarta.xml.bind-api-2.3.2.jar:/root/.gradle/caches/modules-2/files-2.1/jakarta.activation/jakarta.activation-api/1.2.1/562a587face36ec7eff2db7f2fc95425c6602bc1/jakarta.activation-api-1.2.1.jar:/root/.gradle/caches/modules-2/files-2.1/io.zipkin.brave/brave-instrumentation-http/5.13.3/fe70809f06c786b171e4597747c6f5a8c911fe5f/brave-instrumentation-http-5.13.3.jar:/root/.gradle/caches/modules-2/files-2.1/io.zipkin.brave/brave/5.13.3/2d8ecb2352108f95b3e66ccc8027e9343ad9852c/brave-5.13.3.jar:/root/.gradle/caches/modules-2/files-2.1/io.zipkin.reporter2/zipkin-reporter-brave/2.16.3/4d5017d71e4de139b6a31612cfd837b7c71d288c/zipkin-reporter-brave-2.16.3.jar:/root/.gradle/caches/modules-2/files-2.1/io.zipkin.reporter2/zipkin-reporter/2.16.3/7e43d8be3376d305c355d969e8b9f3a62221380/zipkin-reporter-2.16.3.jar:/root/.gradle/caches/modules-2/files-2.1/io.zipkin.zipkin2/zipkin/2.23.2/1c2c7f2e91a3749311f7f75d0535d14ba2e2f6/zipkin-2.23.2.jar scripts/ssh.py -i /tmp/win10-bucket-ops-temp_rebalance_magma.ini -p rerun=False,get-cbcollect-info=True -c conf/magma/dgm_collections_bloom_filter.conf -m rest Running: /opt/jython/bin/jython -J-cp /data/workspace/temp_rebalance_magma/build/classes/java/main:/data/workspace/temp_rebalance_magma/build/resources/main:/root/.gradle/caches/modules-2/files-2.1/com.jcraft/jsch/0.1.54/da3584329a263616e277e15462b387addd1b208d/jsch-0.1.54.jar:/root/.gradle/caches/modules-2/files-2.1/org.apache.logging.log4j/log4j-slf4j-impl/2.11.1/4b41b53a3a2d299ce381a69d165381ca19f62912/log4j-slf4j-impl-2.11.1.jar:/root/.gradle/caches/modules-2/files-2.1/commons-cli/commons-cli/1.4/c51c00206bb913cd8612b24abd9fa98ae89719b1/commons-cli-1.4.jar:/root/.gradle/caches/modules-2/files-2.1/com.azure/azure-storage-blob/12.14.0/75d45d21dc208fa369abc832354470464c859d67/azure-storage-blob-12.14.0.jar:/root/.gradle/caches/modules-2/files-2.1/com.couchbase.client/couchbase-transactions/1.1.8/e69a6013e59f498f76671c1acc43df14f1163180/couchbase-transactions-1.1.8.jar:/root/.gradle/caches/modules-2/files-2.1/com.couchbase.client/java-client/3.1.6/f065d71963e08bd5577838592e33f2ca35f5d64a/java-client-3.1.6.jar:/root/.gradle/caches/modules-2/files-2.1/com.azure/azure-storage-internal-avro/12.1.0/c5882c519f8ee15c528e9d0396ec7535da7d4ecd/azure-storage-internal-avro-12.1.0.jar:/root/.gradle/caches/modules-2/files-2.1/com.azure/azure-storage-common/12.13.0/ef8910849375def2b715ea64429136e694fb7ad6/azure-storage-common-12.13.0.jar:/root/.gradle/caches/modules-2/files-2.1/com.azure/azure-core-http-netty/1.11.0/2177ae2dc9cff849e49f9180bf6b95b8e2c78e1b/azure-core-http-netty-1.11.0.jar:/root/.gradle/caches/modules-2/files-2.1/com.azure/azure-core/1.20.0/a98c6bd18aa2066ecd8b39bf7ac51bd8e7307851/azure-core-1.20.0.jar:/root/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-api/1.7.32/cdcff33940d9f2de763bc41ea05a0be5941176c3/slf4j-api-1.7.32.jar:/root/.gradle/caches/modules-2/files-2.1/org.apache.logging.log4j/log4j-core/2.11.1/592a48674c926b01a9a747c7831bcd82a9e6d6e4/log4j-core-2.11.1.jar:/root/.gradle/caches/modules-2/files-2.1/org.apache.logging.log4j/log4j-api/2.11.1/268f0fe4df3eefe052b57c87ec48517d64fb2a10/log4j-api-2.11.1.jar:/root/.gradle/caches/modules-2/files-2.1/com.couchbase.client/core-io/2.1.6/b3ece73ab7069b1e97669500edc448163c2f4304/core-io-2.1.6.jar:/root/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.dataformat/jackson-dataformat-xml/2.12.4/15c743856696c0239f2c51d8d19d9f97f034713/jackson-dataformat-xml-2.12.4.jar:/root/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.module/jackson-module-jaxb-annotations/2.12.4/5e43703aae1a9843dfd7df0a0ad6cbfedcaff67f/jackson-module-jaxb-annotations-2.12.4.jar:/root/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.core/jackson-core/2.12.4/6a1bd259b6c4e3f9219ec8ec0be55ed11eed0c/jackson-core-2.12.4.jar:/root/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.datatype/jackson-datatype-jsr310/2.12.4/b1174c05d4ded121a7eaeed3f148709f9585b981/jackson-datatype-jsr310-2.12.4.jar:/root/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.core/jackson-databind/2.12.4/69206e02e6a696034f06a59d3ddbfbba5a4cd81/jackson-databind-2.12.4.jar:/root/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.core/jackson-annotations/2.12.4/752cf9a2562ac2c012e48057e3a4c17dad66c66e/jackson-annotations-2.12.4.jar:/root/.gradle/caches/modules-2/files-2.1/io.projectreactor.netty/reactor-netty/1.0.10/c54f6c92628f396076f74945424319746bbb37f8/reactor-netty-1.0.10.jar:/root/.gradle/caches/modules-2/files-2.1/io.projectreactor.netty/reactor-netty-http-brave/1.0.10/30aebc029ee25f5e0c2d1c0a798b17e926f61b8/reactor-netty-http-brave-1.0.10.jar:/root/.gradle/caches/modules-2/files-2.1/io.projectreactor.netty/reactor-netty-http/1.0.10/f34196d8778243d2ac4a4ae7cc74f110d1073803/reactor-netty-http-1.0.10.jar:/root/.gradle/caches/modules-2/files-2.1/io.projectreactor.netty/reactor-netty-core/1.0.10/dcb5d53dc07a6f660060f0b55c40a1e9944a6fe1/reactor-netty-core-1.0.10.jar:/root/.gradle/caches/modules-2/files-2.1/io.projectreactor/reactor-core/3.4.9/820332aa7b0fe3a8dfe14f58fc16e49ad178291/reactor-core-3.4.9.jar:/root/.gradle/caches/modules-2/files-2.1/io.netty/netty-tcnative-boringssl-static/2.0.40.Final/6b73a163c13ed76921892d28eb81235f4b41e40a/netty-tcnative-boringssl-static-2.0.40.Final.jar:/root/.gradle/caches/modules-2/files-2.1/io.netty/netty-handler-proxy/4.1.67.Final/2528292c49bd15f1b328dda25a7a75744d6f0991/netty-handler-proxy-4.1.67.Final.jar:/root/.gradle/caches/modules-2/files-2.1/io.netty/netty-codec-http2/4.1.67.Final/e2a0c27f396035fab7d932031d6f337244369495/netty-codec-http2-4.1.67.Final.jar:/root/.gradle/caches/modules-2/files-2.1/io.netty/netty-codec-http/4.1.67.Final/e282137917c67332fa9a414df89f89a93487aede/netty-codec-http-4.1.67.Final.jar:/root/.gradle/caches/modules-2/files-2.1/io.netty/netty-resolver-dns-native-macos/4.1.66.Final/8ca314d87b202a4dd94d810d2385b22555dd801c/netty-resolver-dns-native-macos-4.1.66.Final-osx-x86_64.jar:/root/.gradle/caches/modules-2/files-2.1/io.netty/netty-resolver-dns/4.1.66.Final/7b74815fb1403e5747c872c6eee2a07e7a700d30/netty-resolver-dns-4.1.66.Final.jar:/root/.gradle/caches/modules-2/files-2.1/io.netty/netty-handler/4.1.67.Final/62640a6524e1c08d6e8ac06556892b5e1362392f/netty-handler-4.1.67.Final.jar:/root/.gradle/caches/modules-2/files-2.1/io.netty/netty-transport-native-epoll/4.1.67.Final/ff955604e1edb8b13dc855ccbc84acb5fc1d989/netty-transport-native-epoll-4.1.67.Final-linux-x86_64.jar:/root/.gradle/caches/modules-2/files-2.1/io.netty/netty-transport-native-kqueue/4.1.67.Final/7a88ccb4237d1ff091be544860ae729f5b568ac/netty-transport-native-kqueue-4.1.67.Final-osx-x86_64.jar:/root/.gradle/caches/modules-2/files-2.1/io.netty/netty-transport-native-unix-common/4.1.67.Final/c6cca7ba897e1ac65e0742654ed10064401dbd2f/netty-transport-native-unix-common-4.1.67.Final.jar:/root/.gradle/caches/modules-2/files-2.1/io.netty/netty-codec-socks/4.1.67.Final/a4fe0487451dc262daaa024224fa207d088b8687/netty-codec-socks-4.1.67.Final.jar:/root/.gradle/caches/modules-2/files-2.1/io.netty/netty-codec-dns/4.1.66.Final/390ff96a0e1f7c626cb52c119a1a1dfd0784d193/netty-codec-dns-4.1.66.Final.jar:/root/.gradle/caches/modules-2/files-2.1/io.netty/netty-codec/4.1.67.Final/292e818822a4fca4e9f3ad711cdd46362f69590a/netty-codec-4.1.67.Final.jar:/root/.gradle/caches/modules-2/files-2.1/io.netty/netty-transport/4.1.67.Final/e8b502618f960cb1f8cdb7dca92d0878576ad7c6/netty-transport-4.1.67.Final.jar:/root/.gradle/caches/modules-2/files-2.1/io.netty/netty-buffer/4.1.67.Final/4b6217b05792fe6e2bd080c054138b2c20ae1b37/netty-buffer-4.1.67.Final.jar:/root/.gradle/caches/modules-2/files-2.1/com.fasterxml.woodstox/woodstox-core/6.2.4/16b9f8ab972e67eb21872ea2c40046249d543989/woodstox-core-6.2.4.jar:/root/.gradle/caches/modules-2/files-2.1/org.codehaus.woodstox/stax2-api/4.2.1/a3f7325c52240418c2ba257b103c3c550e140c83/stax2-api-4.2.1.jar:/root/.gradle/caches/modules-2/files-2.1/org.reactivestreams/reactive-streams/1.0.3/d9fb7a7926ffa635b3dcaa5049fb2bfa25b3e7d0/reactive-streams-1.0.3.jar:/root/.gradle/caches/modules-2/files-2.1/io.netty/netty-resolver/4.1.67.Final/bb9041d5f85f9a6270f5378d0becd243de8cfeac/netty-resolver-4.1.67.Final.jar:/root/.gradle/caches/modules-2/files-2.1/io.netty/netty-common/4.1.67.Final/a86a19033588a07e159ff95818845b1ec86b2281/netty-common-4.1.67.Final.jar:/root/.gradle/caches/modules-2/files-2.1/jakarta.xml.bind/jakarta.xml.bind-api/2.3.2/8d49996a4338670764d7ca4b85a1c4ccf7fe665d/jakarta.xml.bind-api-2.3.2.jar:/root/.gradle/caches/modules-2/files-2.1/jakarta.activation/jakarta.activation-api/1.2.1/562a587face36ec7eff2db7f2fc95425c6602bc1/jakarta.activation-api-1.2.1.jar:/root/.gradle/caches/modules-2/files-2.1/io.zipkin.brave/brave-instrumentation-http/5.13.3/fe70809f06c786b171e4597747c6f5a8c911fe5f/brave-instrumentation-http-5.13.3.jar:/root/.gradle/caches/modules-2/files-2.1/io.zipkin.brave/brave/5.13.3/2d8ecb2352108f95b3e66ccc8027e9343ad9852c/brave-5.13.3.jar:/root/.gradle/caches/modules-2/files-2.1/io.zipkin.reporter2/zipkin-reporter-brave/2.16.3/4d5017d71e4de139b6a31612cfd837b7c71d288c/zipkin-reporter-brave-2.16.3.jar:/root/.gradle/caches/modules-2/files-2.1/io.zipkin.reporter2/zipkin-reporter/2.16.3/7e43d8be3376d305c355d969e8b9f3a62221380/zipkin-reporter-2.16.3.jar:/root/.gradle/caches/modules-2/files-2.1/io.zipkin.zipkin2/zipkin/2.23.2/1c2c7f2e91a3749311f7f75d0535d14ba2e2f6/zipkin-2.23.2.jar scripts/eagles_all_around.py -i /tmp/win10-bucket-ops-temp_rebalance_magma.ini -p rerun=False,get-cbcollect-info=True -c conf/magma/dgm_collections_bloom_filter.conf -m rest Running: /opt/jython/bin/jython -J-cp /data/workspace/temp_rebalance_magma/build/classes/java/main:/data/workspace/temp_rebalance_magma/build/resources/main:/root/.gradle/caches/modules-2/files-2.1/com.jcraft/jsch/0.1.54/da3584329a263616e277e15462b387addd1b208d/jsch-0.1.54.jar:/root/.gradle/caches/modules-2/files-2.1/org.apache.logging.log4j/log4j-slf4j-impl/2.11.1/4b41b53a3a2d299ce381a69d165381ca19f62912/log4j-slf4j-impl-2.11.1.jar:/root/.gradle/caches/modules-2/files-2.1/commons-cli/commons-cli/1.4/c51c00206bb913cd8612b24abd9fa98ae89719b1/commons-cli-1.4.jar:/root/.gradle/caches/modules-2/files-2.1/com.azure/azure-storage-blob/12.14.0/75d45d21dc208fa369abc832354470464c859d67/azure-storage-blob-12.14.0.jar:/root/.gradle/caches/modules-2/files-2.1/com.couchbase.client/couchbase-transactions/1.1.8/e69a6013e59f498f76671c1acc43df14f1163180/couchbase-transactions-1.1.8.jar:/root/.gradle/caches/modules-2/files-2.1/com.couchbase.client/java-client/3.1.6/f065d71963e08bd5577838592e33f2ca35f5d64a/java-client-3.1.6.jar:/root/.gradle/caches/modules-2/files-2.1/com.azure/azure-storage-internal-avro/12.1.0/c5882c519f8ee15c528e9d0396ec7535da7d4ecd/azure-storage-internal-avro-12.1.0.jar:/root/.gradle/caches/modules-2/files-2.1/com.azure/azure-storage-common/12.13.0/ef8910849375def2b715ea64429136e694fb7ad6/azure-storage-common-12.13.0.jar:/root/.gradle/caches/modules-2/files-2.1/com.azure/azure-core-http-netty/1.11.0/2177ae2dc9cff849e49f9180bf6b95b8e2c78e1b/azure-core-http-netty-1.11.0.jar:/root/.gradle/caches/modules-2/files-2.1/com.azure/azure-core/1.20.0/a98c6bd18aa2066ecd8b39bf7ac51bd8e7307851/azure-core-1.20.0.jar:/root/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-api/1.7.32/cdcff33940d9f2de763bc41ea05a0be5941176c3/slf4j-api-1.7.32.jar:/root/.gradle/caches/modules-2/files-2.1/org.apache.logging.log4j/log4j-core/2.11.1/592a48674c926b01a9a747c7831bcd82a9e6d6e4/log4j-core-2.11.1.jar:/root/.gradle/caches/modules-2/files-2.1/org.apache.logging.log4j/log4j-api/2.11.1/268f0fe4df3eefe052b57c87ec48517d64fb2a10/log4j-api-2.11.1.jar:/root/.gradle/caches/modules-2/files-2.1/com.couchbase.client/core-io/2.1.6/b3ece73ab7069b1e97669500edc448163c2f4304/core-io-2.1.6.jar:/root/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.dataformat/jackson-dataformat-xml/2.12.4/15c743856696c0239f2c51d8d19d9f97f034713/jackson-dataformat-xml-2.12.4.jar:/root/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.module/jackson-module-jaxb-annotations/2.12.4/5e43703aae1a9843dfd7df0a0ad6cbfedcaff67f/jackson-module-jaxb-annotations-2.12.4.jar:/root/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.core/jackson-core/2.12.4/6a1bd259b6c4e3f9219ec8ec0be55ed11eed0c/jackson-core-2.12.4.jar:/root/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.datatype/jackson-datatype-jsr310/2.12.4/b1174c05d4ded121a7eaeed3f148709f9585b981/jackson-datatype-jsr310-2.12.4.jar:/root/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.core/jackson-databind/2.12.4/69206e02e6a696034f06a59d3ddbfbba5a4cd81/jackson-databind-2.12.4.jar:/root/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.core/jackson-annotations/2.12.4/752cf9a2562ac2c012e48057e3a4c17dad66c66e/jackson-annotations-2.12.4.jar:/root/.gradle/caches/modules-2/files-2.1/io.projectreactor.netty/reactor-netty/1.0.10/c54f6c92628f396076f74945424319746bbb37f8/reactor-netty-1.0.10.jar:/root/.gradle/caches/modules-2/files-2.1/io.projectreactor.netty/reactor-netty-http-brave/1.0.10/30aebc029ee25f5e0c2d1c0a798b17e926f61b8/reactor-netty-http-brave-1.0.10.jar:/root/.gradle/caches/modules-2/files-2.1/io.projectreactor.netty/reactor-netty-http/1.0.10/f34196d8778243d2ac4a4ae7cc74f110d1073803/reactor-netty-http-1.0.10.jar:/root/.gradle/caches/modules-2/files-2.1/io.projectreactor.netty/reactor-netty-core/1.0.10/dcb5d53dc07a6f660060f0b55c40a1e9944a6fe1/reactor-netty-core-1.0.10.jar:/root/.gradle/caches/modules-2/files-2.1/io.projectreactor/reactor-core/3.4.9/820332aa7b0fe3a8dfe14f58fc16e49ad178291/reactor-core-3.4.9.jar:/root/.gradle/caches/modules-2/files-2.1/io.netty/netty-tcnative-boringssl-static/2.0.40.Final/6b73a163c13ed76921892d28eb81235f4b41e40a/netty-tcnative-boringssl-static-2.0.40.Final.jar:/root/.gradle/caches/modules-2/files-2.1/io.netty/netty-handler-proxy/4.1.67.Final/2528292c49bd15f1b328dda25a7a75744d6f0991/netty-handler-proxy-4.1.67.Final.jar:/root/.gradle/caches/modules-2/files-2.1/io.netty/netty-codec-http2/4.1.67.Final/e2a0c27f396035fab7d932031d6f337244369495/netty-codec-http2-4.1.67.Final.jar:/root/.gradle/caches/modules-2/files-2.1/io.netty/netty-codec-http/4.1.67.Final/e282137917c67332fa9a414df89f89a93487aede/netty-codec-http-4.1.67.Final.jar:/root/.gradle/caches/modules-2/files-2.1/io.netty/netty-resolver-dns-native-macos/4.1.66.Final/8ca314d87b202a4dd94d810d2385b22555dd801c/netty-resolver-dns-native-macos-4.1.66.Final-osx-x86_64.jar:/root/.gradle/caches/modules-2/files-2.1/io.netty/netty-resolver-dns/4.1.66.Final/7b74815fb1403e5747c872c6eee2a07e7a700d30/netty-resolver-dns-4.1.66.Final.jar:/root/.gradle/caches/modules-2/files-2.1/io.netty/netty-handler/4.1.67.Final/62640a6524e1c08d6e8ac06556892b5e1362392f/netty-handler-4.1.67.Final.jar:/root/.gradle/caches/modules-2/files-2.1/io.netty/netty-transport-native-epoll/4.1.67.Final/ff955604e1edb8b13dc855ccbc84acb5fc1d989/netty-transport-native-epoll-4.1.67.Final-linux-x86_64.jar:/root/.gradle/caches/modules-2/files-2.1/io.netty/netty-transport-native-kqueue/4.1.67.Final/7a88ccb4237d1ff091be544860ae729f5b568ac/netty-transport-native-kqueue-4.1.67.Final-osx-x86_64.jar:/root/.gradle/caches/modules-2/files-2.1/io.netty/netty-transport-native-unix-common/4.1.67.Final/c6cca7ba897e1ac65e0742654ed10064401dbd2f/netty-transport-native-unix-common-4.1.67.Final.jar:/root/.gradle/caches/modules-2/files-2.1/io.netty/netty-codec-socks/4.1.67.Final/a4fe0487451dc262daaa024224fa207d088b8687/netty-codec-socks-4.1.67.Final.jar:/root/.gradle/caches/modules-2/files-2.1/io.netty/netty-codec-dns/4.1.66.Final/390ff96a0e1f7c626cb52c119a1a1dfd0784d193/netty-codec-dns-4.1.66.Final.jar:/root/.gradle/caches/modules-2/files-2.1/io.netty/netty-codec/4.1.67.Final/292e818822a4fca4e9f3ad711cdd46362f69590a/netty-codec-4.1.67.Final.jar:/root/.gradle/caches/modules-2/files-2.1/io.netty/netty-transport/4.1.67.Final/e8b502618f960cb1f8cdb7dca92d0878576ad7c6/netty-transport-4.1.67.Final.jar:/root/.gradle/caches/modules-2/files-2.1/io.netty/netty-buffer/4.1.67.Final/4b6217b05792fe6e2bd080c054138b2c20ae1b37/netty-buffer-4.1.67.Final.jar:/root/.gradle/caches/modules-2/files-2.1/com.fasterxml.woodstox/woodstox-core/6.2.4/16b9f8ab972e67eb21872ea2c40046249d543989/woodstox-core-6.2.4.jar:/root/.gradle/caches/modules-2/files-2.1/org.codehaus.woodstox/stax2-api/4.2.1/a3f7325c52240418c2ba257b103c3c550e140c83/stax2-api-4.2.1.jar:/root/.gradle/caches/modules-2/files-2.1/org.reactivestreams/reactive-streams/1.0.3/d9fb7a7926ffa635b3dcaa5049fb2bfa25b3e7d0/reactive-streams-1.0.3.jar:/root/.gradle/caches/modules-2/files-2.1/io.netty/netty-resolver/4.1.67.Final/bb9041d5f85f9a6270f5378d0becd243de8cfeac/netty-resolver-4.1.67.Final.jar:/root/.gradle/caches/modules-2/files-2.1/io.netty/netty-common/4.1.67.Final/a86a19033588a07e159ff95818845b1ec86b2281/netty-common-4.1.67.Final.jar:/root/.gradle/caches/modules-2/files-2.1/jakarta.xml.bind/jakarta.xml.bind-api/2.3.2/8d49996a4338670764d7ca4b85a1c4ccf7fe665d/jakarta.xml.bind-api-2.3.2.jar:/root/.gradle/caches/modules-2/files-2.1/jakarta.activation/jakarta.activation-api/1.2.1/562a587face36ec7eff2db7f2fc95425c6602bc1/jakarta.activation-api-1.2.1.jar:/root/.gradle/caches/modules-2/files-2.1/io.zipkin.brave/brave-instrumentation-http/5.13.3/fe70809f06c786b171e4597747c6f5a8c911fe5f/brave-instrumentation-http-5.13.3.jar:/root/.gradle/caches/modules-2/files-2.1/io.zipkin.brave/brave/5.13.3/2d8ecb2352108f95b3e66ccc8027e9343ad9852c/brave-5.13.3.jar:/root/.gradle/caches/modules-2/files-2.1/io.zipkin.reporter2/zipkin-reporter-brave/2.16.3/4d5017d71e4de139b6a31612cfd837b7c71d288c/zipkin-reporter-brave-2.16.3.jar:/root/.gradle/caches/modules-2/files-2.1/io.zipkin.reporter2/zipkin-reporter/2.16.3/7e43d8be3376d305c355d969e8b9f3a62221380/zipkin-reporter-2.16.3.jar:/root/.gradle/caches/modules-2/files-2.1/io.zipkin.zipkin2/zipkin/2.23.2/1c2c7f2e91a3749311f7f75d0535d14ba2e2f6/zipkin-2.23.2.jar:build/classes/java/main scripts/install.py -i /tmp/win10-bucket-ops-temp_rebalance_magma.ini -p rerun=False,get-cbcollect-info=True -c conf/magma/dgm_collections_bloom_filter.conf -m rest Running: /opt/jython/bin/jython -J-cp /data/workspace/temp_rebalance_magma/build/classes/java/main:/data/workspace/temp_rebalance_magma/build/resources/main:/root/.gradle/caches/modules-2/files-2.1/com.jcraft/jsch/0.1.54/da3584329a263616e277e15462b387addd1b208d/jsch-0.1.54.jar:/root/.gradle/caches/modules-2/files-2.1/org.apache.logging.log4j/log4j-slf4j-impl/2.11.1/4b41b53a3a2d299ce381a69d165381ca19f62912/log4j-slf4j-impl-2.11.1.jar:/root/.gradle/caches/modules-2/files-2.1/commons-cli/commons-cli/1.4/c51c00206bb913cd8612b24abd9fa98ae89719b1/commons-cli-1.4.jar:/root/.gradle/caches/modules-2/files-2.1/com.azure/azure-storage-blob/12.14.0/75d45d21dc208fa369abc832354470464c859d67/azure-storage-blob-12.14.0.jar:/root/.gradle/caches/modules-2/files-2.1/com.couchbase.client/couchbase-transactions/1.1.8/e69a6013e59f498f76671c1acc43df14f1163180/couchbase-transactions-1.1.8.jar:/root/.gradle/caches/modules-2/files-2.1/com.couchbase.client/java-client/3.1.6/f065d71963e08bd5577838592e33f2ca35f5d64a/java-client-3.1.6.jar:/root/.gradle/caches/modules-2/files-2.1/com.azure/azure-storage-internal-avro/12.1.0/c5882c519f8ee15c528e9d0396ec7535da7d4ecd/azure-storage-internal-avro-12.1.0.jar:/root/.gradle/caches/modules-2/files-2.1/com.azure/azure-storage-common/12.13.0/ef8910849375def2b715ea64429136e694fb7ad6/azure-storage-common-12.13.0.jar:/root/.gradle/caches/modules-2/files-2.1/com.azure/azure-core-http-netty/1.11.0/2177ae2dc9cff849e49f9180bf6b95b8e2c78e1b/azure-core-http-netty-1.11.0.jar:/root/.gradle/caches/modules-2/files-2.1/com.azure/azure-core/1.20.0/a98c6bd18aa2066ecd8b39bf7ac51bd8e7307851/azure-core-1.20.0.jar:/root/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-api/1.7.32/cdcff33940d9f2de763bc41ea05a0be5941176c3/slf4j-api-1.7.32.jar:/root/.gradle/caches/modules-2/files-2.1/org.apache.logging.log4j/log4j-core/2.11.1/592a48674c926b01a9a747c7831bcd82a9e6d6e4/log4j-core-2.11.1.jar:/root/.gradle/caches/modules-2/files-2.1/org.apache.logging.log4j/log4j-api/2.11.1/268f0fe4df3eefe052b57c87ec48517d64fb2a10/log4j-api-2.11.1.jar:/root/.gradle/caches/modules-2/files-2.1/com.couchbase.client/core-io/2.1.6/b3ece73ab7069b1e97669500edc448163c2f4304/core-io-2.1.6.jar:/root/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.dataformat/jackson-dataformat-xml/2.12.4/15c743856696c0239f2c51d8d19d9f97f034713/jackson-dataformat-xml-2.12.4.jar:/root/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.module/jackson-module-jaxb-annotations/2.12.4/5e43703aae1a9843dfd7df0a0ad6cbfedcaff67f/jackson-module-jaxb-annotations-2.12.4.jar:/root/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.core/jackson-core/2.12.4/6a1bd259b6c4e3f9219ec8ec0be55ed11eed0c/jackson-core-2.12.4.jar:/root/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.datatype/jackson-datatype-jsr310/2.12.4/b1174c05d4ded121a7eaeed3f148709f9585b981/jackson-datatype-jsr310-2.12.4.jar:/root/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.core/jackson-databind/2.12.4/69206e02e6a696034f06a59d3ddbfbba5a4cd81/jackson-databind-2.12.4.jar:/root/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.core/jackson-annotations/2.12.4/752cf9a2562ac2c012e48057e3a4c17dad66c66e/jackson-annotations-2.12.4.jar:/root/.gradle/caches/modules-2/files-2.1/io.projectreactor.netty/reactor-netty/1.0.10/c54f6c92628f396076f74945424319746bbb37f8/reactor-netty-1.0.10.jar:/root/.gradle/caches/modules-2/files-2.1/io.projectreactor.netty/reactor-netty-http-brave/1.0.10/30aebc029ee25f5e0c2d1c0a798b17e926f61b8/reactor-netty-http-brave-1.0.10.jar:/root/.gradle/caches/modules-2/files-2.1/io.projectreactor.netty/reactor-netty-http/1.0.10/f34196d8778243d2ac4a4ae7cc74f110d1073803/reactor-netty-http-1.0.10.jar:/root/.gradle/caches/modules-2/files-2.1/io.projectreactor.netty/reactor-netty-core/1.0.10/dcb5d53dc07a6f660060f0b55c40a1e9944a6fe1/reactor-netty-core-1.0.10.jar:/root/.gradle/caches/modules-2/files-2.1/io.projectreactor/reactor-core/3.4.9/820332aa7b0fe3a8dfe14f58fc16e49ad178291/reactor-core-3.4.9.jar:/root/.gradle/caches/modules-2/files-2.1/io.netty/netty-tcnative-boringssl-static/2.0.40.Final/6b73a163c13ed76921892d28eb81235f4b41e40a/netty-tcnative-boringssl-static-2.0.40.Final.jar:/root/.gradle/caches/modules-2/files-2.1/io.netty/netty-handler-proxy/4.1.67.Final/2528292c49bd15f1b328dda25a7a75744d6f0991/netty-handler-proxy-4.1.67.Final.jar:/root/.gradle/caches/modules-2/files-2.1/io.netty/netty-codec-http2/4.1.67.Final/e2a0c27f396035fab7d932031d6f337244369495/netty-codec-http2-4.1.67.Final.jar:/root/.gradle/caches/modules-2/files-2.1/io.netty/netty-codec-http/4.1.67.Final/e282137917c67332fa9a414df89f89a93487aede/netty-codec-http-4.1.67.Final.jar:/root/.gradle/caches/modules-2/files-2.1/io.netty/netty-resolver-dns-native-macos/4.1.66.Final/8ca314d87b202a4dd94d810d2385b22555dd801c/netty-resolver-dns-native-macos-4.1.66.Final-osx-x86_64.jar:/root/.gradle/caches/modules-2/files-2.1/io.netty/netty-resolver-dns/4.1.66.Final/7b74815fb1403e5747c872c6eee2a07e7a700d30/netty-resolver-dns-4.1.66.Final.jar:/root/.gradle/caches/modules-2/files-2.1/io.netty/netty-handler/4.1.67.Final/62640a6524e1c08d6e8ac06556892b5e1362392f/netty-handler-4.1.67.Final.jar:/root/.gradle/caches/modules-2/files-2.1/io.netty/netty-transport-native-epoll/4.1.67.Final/ff955604e1edb8b13dc855ccbc84acb5fc1d989/netty-transport-native-epoll-4.1.67.Final-linux-x86_64.jar:/root/.gradle/caches/modules-2/files-2.1/io.netty/netty-transport-native-kqueue/4.1.67.Final/7a88ccb4237d1ff091be544860ae729f5b568ac/netty-transport-native-kqueue-4.1.67.Final-osx-x86_64.jar:/root/.gradle/caches/modules-2/files-2.1/io.netty/netty-transport-native-unix-common/4.1.67.Final/c6cca7ba897e1ac65e0742654ed10064401dbd2f/netty-transport-native-unix-common-4.1.67.Final.jar:/root/.gradle/caches/modules-2/files-2.1/io.netty/netty-codec-socks/4.1.67.Final/a4fe0487451dc262daaa024224fa207d088b8687/netty-codec-socks-4.1.67.Final.jar:/root/.gradle/caches/modules-2/files-2.1/io.netty/netty-codec-dns/4.1.66.Final/390ff96a0e1f7c626cb52c119a1a1dfd0784d193/netty-codec-dns-4.1.66.Final.jar:/root/.gradle/caches/modules-2/files-2.1/io.netty/netty-codec/4.1.67.Final/292e818822a4fca4e9f3ad711cdd46362f69590a/netty-codec-4.1.67.Final.jar:/root/.gradle/caches/modules-2/files-2.1/io.netty/netty-transport/4.1.67.Final/e8b502618f960cb1f8cdb7dca92d0878576ad7c6/netty-transport-4.1.67.Final.jar:/root/.gradle/caches/modules-2/files-2.1/io.netty/netty-buffer/4.1.67.Final/4b6217b05792fe6e2bd080c054138b2c20ae1b37/netty-buffer-4.1.67.Final.jar:/root/.gradle/caches/modules-2/files-2.1/com.fasterxml.woodstox/woodstox-core/6.2.4/16b9f8ab972e67eb21872ea2c40046249d543989/woodstox-core-6.2.4.jar:/root/.gradle/caches/modules-2/files-2.1/org.codehaus.woodstox/stax2-api/4.2.1/a3f7325c52240418c2ba257b103c3c550e140c83/stax2-api-4.2.1.jar:/root/.gradle/caches/modules-2/files-2.1/org.reactivestreams/reactive-streams/1.0.3/d9fb7a7926ffa635b3dcaa5049fb2bfa25b3e7d0/reactive-streams-1.0.3.jar:/root/.gradle/caches/modules-2/files-2.1/io.netty/netty-resolver/4.1.67.Final/bb9041d5f85f9a6270f5378d0becd243de8cfeac/netty-resolver-4.1.67.Final.jar:/root/.gradle/caches/modules-2/files-2.1/io.netty/netty-common/4.1.67.Final/a86a19033588a07e159ff95818845b1ec86b2281/netty-common-4.1.67.Final.jar:/root/.gradle/caches/modules-2/files-2.1/jakarta.xml.bind/jakarta.xml.bind-api/2.3.2/8d49996a4338670764d7ca4b85a1c4ccf7fe665d/jakarta.xml.bind-api-2.3.2.jar:/root/.gradle/caches/modules-2/files-2.1/jakarta.activation/jakarta.activation-api/1.2.1/562a587face36ec7eff2db7f2fc95425c6602bc1/jakarta.activation-api-1.2.1.jar:/root/.gradle/caches/modules-2/files-2.1/io.zipkin.brave/brave-instrumentation-http/5.13.3/fe70809f06c786b171e4597747c6f5a8c911fe5f/brave-instrumentation-http-5.13.3.jar:/root/.gradle/caches/modules-2/files-2.1/io.zipkin.brave/brave/5.13.3/2d8ecb2352108f95b3e66ccc8027e9343ad9852c/brave-5.13.3.jar:/root/.gradle/caches/modules-2/files-2.1/io.zipkin.reporter2/zipkin-reporter-brave/2.16.3/4d5017d71e4de139b6a31612cfd837b7c71d288c/zipkin-reporter-brave-2.16.3.jar:/root/.gradle/caches/modules-2/files-2.1/io.zipkin.reporter2/zipkin-reporter/2.16.3/7e43d8be3376d305c355d969e8b9f3a62221380/zipkin-reporter-2.16.3.jar:/root/.gradle/caches/modules-2/files-2.1/io.zipkin.zipkin2/zipkin/2.23.2/1c2c7f2e91a3749311f7f75d0535d14ba2e2f6/zipkin-2.23.2.jar:build/classes/java/main:src/main/resources testrunner.py -i /tmp/win10-bucket-ops-temp_rebalance_magma.ini -p rerun=False,get-cbcollect-info=True -c conf/magma/dgm_collections_bloom_filter.conf -m rest Running: /opt/jython/bin/jython -J-cp /data/workspace/temp_rebalance_magma/build/classes/java/main:/data/workspace/temp_rebalance_magma/build/resources/main:/root/.gradle/caches/modules-2/files-2.1/com.jcraft/jsch/0.1.54/da3584329a263616e277e15462b387addd1b208d/jsch-0.1.54.jar:/root/.gradle/caches/modules-2/files-2.1/org.apache.logging.log4j/log4j-slf4j-impl/2.11.1/4b41b53a3a2d299ce381a69d165381ca19f62912/log4j-slf4j-impl-2.11.1.jar:/root/.gradle/caches/modules-2/files-2.1/commons-cli/commons-cli/1.4/c51c00206bb913cd8612b24abd9fa98ae89719b1/commons-cli-1.4.jar:/root/.gradle/caches/modules-2/files-2.1/com.azure/azure-storage-blob/12.14.0/75d45d21dc208fa369abc832354470464c859d67/azure-storage-blob-12.14.0.jar:/root/.gradle/caches/modules-2/files-2.1/com.couchbase.client/couchbase-transactions/1.1.8/e69a6013e59f498f76671c1acc43df14f1163180/couchbase-transactions-1.1.8.jar:/root/.gradle/caches/modules-2/files-2.1/com.couchbase.client/java-client/3.1.6/f065d71963e08bd5577838592e33f2ca35f5d64a/java-client-3.1.6.jar:/root/.gradle/caches/modules-2/files-2.1/com.azure/azure-storage-internal-avro/12.1.0/c5882c519f8ee15c528e9d0396ec7535da7d4ecd/azure-storage-internal-avro-12.1.0.jar:/root/.gradle/caches/modules-2/files-2.1/com.azure/azure-storage-common/12.13.0/ef8910849375def2b715ea64429136e694fb7ad6/azure-storage-common-12.13.0.jar:/root/.gradle/caches/modules-2/files-2.1/com.azure/azure-core-http-netty/1.11.0/2177ae2dc9cff849e49f9180bf6b95b8e2c78e1b/azure-core-http-netty-1.11.0.jar:/root/.gradle/caches/modules-2/files-2.1/com.azure/azure-core/1.20.0/a98c6bd18aa2066ecd8b39bf7ac51bd8e7307851/azure-core-1.20.0.jar:/root/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-api/1.7.32/cdcff33940d9f2de763bc41ea05a0be5941176c3/slf4j-api-1.7.32.jar:/root/.gradle/caches/modules-2/files-2.1/org.apache.logging.log4j/log4j-core/2.11.1/592a48674c926b01a9a747c7831bcd82a9e6d6e4/log4j-core-2.11.1.jar:/root/.gradle/caches/modules-2/files-2.1/org.apache.logging.log4j/log4j-api/2.11.1/268f0fe4df3eefe052b57c87ec48517d64fb2a10/log4j-api-2.11.1.jar:/root/.gradle/caches/modules-2/files-2.1/com.couchbase.client/core-io/2.1.6/b3ece73ab7069b1e97669500edc448163c2f4304/core-io-2.1.6.jar:/root/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.dataformat/jackson-dataformat-xml/2.12.4/15c743856696c0239f2c51d8d19d9f97f034713/jackson-dataformat-xml-2.12.4.jar:/root/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.module/jackson-module-jaxb-annotations/2.12.4/5e43703aae1a9843dfd7df0a0ad6cbfedcaff67f/jackson-module-jaxb-annotations-2.12.4.jar:/root/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.core/jackson-core/2.12.4/6a1bd259b6c4e3f9219ec8ec0be55ed11eed0c/jackson-core-2.12.4.jar:/root/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.datatype/jackson-datatype-jsr310/2.12.4/b1174c05d4ded121a7eaeed3f148709f9585b981/jackson-datatype-jsr310-2.12.4.jar:/root/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.core/jackson-databind/2.12.4/69206e02e6a696034f06a59d3ddbfbba5a4cd81/jackson-databind-2.12.4.jar:/root/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.core/jackson-annotations/2.12.4/752cf9a2562ac2c012e48057e3a4c17dad66c66e/jackson-annotations-2.12.4.jar:/root/.gradle/caches/modules-2/files-2.1/io.projectreactor.netty/reactor-netty/1.0.10/c54f6c92628f396076f74945424319746bbb37f8/reactor-netty-1.0.10.jar:/root/.gradle/caches/modules-2/files-2.1/io.projectreactor.netty/reactor-netty-http-brave/1.0.10/30aebc029ee25f5e0c2d1c0a798b17e926f61b8/reactor-netty-http-brave-1.0.10.jar:/root/.gradle/caches/modules-2/files-2.1/io.projectreactor.netty/reactor-netty-http/1.0.10/f34196d8778243d2ac4a4ae7cc74f110d1073803/reactor-netty-http-1.0.10.jar:/root/.gradle/caches/modules-2/files-2.1/io.projectreactor.netty/reactor-netty-core/1.0.10/dcb5d53dc07a6f660060f0b55c40a1e9944a6fe1/reactor-netty-core-1.0.10.jar:/root/.gradle/caches/modules-2/files-2.1/io.projectreactor/reactor-core/3.4.9/820332aa7b0fe3a8dfe14f58fc16e49ad178291/reactor-core-3.4.9.jar:/root/.gradle/caches/modules-2/files-2.1/io.netty/netty-tcnative-boringssl-static/2.0.40.Final/6b73a163c13ed76921892d28eb81235f4b41e40a/netty-tcnative-boringssl-static-2.0.40.Final.jar:/root/.gradle/caches/modules-2/files-2.1/io.netty/netty-handler-proxy/4.1.67.Final/2528292c49bd15f1b328dda25a7a75744d6f0991/netty-handler-proxy-4.1.67.Final.jar:/root/.gradle/caches/modules-2/files-2.1/io.netty/netty-codec-http2/4.1.67.Final/e2a0c27f396035fab7d932031d6f337244369495/netty-codec-http2-4.1.67.Final.jar:/root/.gradle/caches/modules-2/files-2.1/io.netty/netty-codec-http/4.1.67.Final/e282137917c67332fa9a414df89f89a93487aede/netty-codec-http-4.1.67.Final.jar:/root/.gradle/caches/modules-2/files-2.1/io.netty/netty-resolver-dns-native-macos/4.1.66.Final/8ca314d87b202a4dd94d810d2385b22555dd801c/netty-resolver-dns-native-macos-4.1.66.Final-osx-x86_64.jar:/root/.gradle/caches/modules-2/files-2.1/io.netty/netty-resolver-dns/4.1.66.Final/7b74815fb1403e5747c872c6eee2a07e7a700d30/netty-resolver-dns-4.1.66.Final.jar:/root/.gradle/caches/modules-2/files-2.1/io.netty/netty-handler/4.1.67.Final/62640a6524e1c08d6e8ac06556892b5e1362392f/netty-handler-4.1.67.Final.jar:/root/.gradle/caches/modules-2/files-2.1/io.netty/netty-transport-native-epoll/4.1.67.Final/ff955604e1edb8b13dc855ccbc84acb5fc1d989/netty-transport-native-epoll-4.1.67.Final-linux-x86_64.jar:/root/.gradle/caches/modules-2/files-2.1/io.netty/netty-transport-native-kqueue/4.1.67.Final/7a88ccb4237d1ff091be544860ae729f5b568ac/netty-transport-native-kqueue-4.1.67.Final-osx-x86_64.jar:/root/.gradle/caches/modules-2/files-2.1/io.netty/netty-transport-native-unix-common/4.1.67.Final/c6cca7ba897e1ac65e0742654ed10064401dbd2f/netty-transport-native-unix-common-4.1.67.Final.jar:/root/.gradle/caches/modules-2/files-2.1/io.netty/netty-codec-socks/4.1.67.Final/a4fe0487451dc262daaa024224fa207d088b8687/netty-codec-socks-4.1.67.Final.jar:/root/.gradle/caches/modules-2/files-2.1/io.netty/netty-codec-dns/4.1.66.Final/390ff96a0e1f7c626cb52c119a1a1dfd0784d193/netty-codec-dns-4.1.66.Final.jar:/root/.gradle/caches/modules-2/files-2.1/io.netty/netty-codec/4.1.67.Final/292e818822a4fca4e9f3ad711cdd46362f69590a/netty-codec-4.1.67.Final.jar:/root/.gradle/caches/modules-2/files-2.1/io.netty/netty-transport/4.1.67.Final/e8b502618f960cb1f8cdb7dca92d0878576ad7c6/netty-transport-4.1.67.Final.jar:/root/.gradle/caches/modules-2/files-2.1/io.netty/netty-buffer/4.1.67.Final/4b6217b05792fe6e2bd080c054138b2c20ae1b37/netty-buffer-4.1.67.Final.jar:/root/.gradle/caches/modules-2/files-2.1/com.fasterxml.woodstox/woodstox-core/6.2.4/16b9f8ab972e67eb21872ea2c40046249d543989/woodstox-core-6.2.4.jar:/root/.gradle/caches/modules-2/files-2.1/org.codehaus.woodstox/stax2-api/4.2.1/a3f7325c52240418c2ba257b103c3c550e140c83/stax2-api-4.2.1.jar:/root/.gradle/caches/modules-2/files-2.1/org.reactivestreams/reactive-streams/1.0.3/d9fb7a7926ffa635b3dcaa5049fb2bfa25b3e7d0/reactive-streams-1.0.3.jar:/root/.gradle/caches/modules-2/files-2.1/io.netty/netty-resolver/4.1.67.Final/bb9041d5f85f9a6270f5378d0becd243de8cfeac/netty-resolver-4.1.67.Final.jar:/root/.gradle/caches/modules-2/files-2.1/io.netty/netty-common/4.1.67.Final/a86a19033588a07e159ff95818845b1ec86b2281/netty-common-4.1.67.Final.jar:/root/.gradle/caches/modules-2/files-2.1/jakarta.xml.bind/jakarta.xml.bind-api/2.3.2/8d49996a4338670764d7ca4b85a1c4ccf7fe665d/jakarta.xml.bind-api-2.3.2.jar:/root/.gradle/caches/modules-2/files-2.1/jakarta.activation/jakarta.activation-api/1.2.1/562a587face36ec7eff2db7f2fc95425c6602bc1/jakarta.activation-api-1.2.1.jar:/root/.gradle/caches/modules-2/files-2.1/io.zipkin.brave/brave-instrumentation-http/5.13.3/fe70809f06c786b171e4597747c6f5a8c911fe5f/brave-instrumentation-http-5.13.3.jar:/root/.gradle/caches/modules-2/files-2.1/io.zipkin.brave/brave/5.13.3/2d8ecb2352108f95b3e66ccc8027e9343ad9852c/brave-5.13.3.jar:/root/.gradle/caches/modules-2/files-2.1/io.zipkin.reporter2/zipkin-reporter-brave/2.16.3/4d5017d71e4de139b6a31612cfd837b7c71d288c/zipkin-reporter-brave-2.16.3.jar:/root/.gradle/caches/modules-2/files-2.1/io.zipkin.reporter2/zipkin-reporter/2.16.3/7e43d8be3376d305c355d969e8b9f3a62221380/zipkin-reporter-2.16.3.jar:/root/.gradle/caches/modules-2/files-2.1/io.zipkin.zipkin2/zipkin/2.23.2/1c2c7f2e91a3749311f7f75d0535d14ba2e2f6/zipkin-2.23.2.jar:build/classes/java/main:src/main/resources scripts/rerun_jobs.py -i /tmp/win10-bucket-ops-temp_rebalance_magma.ini -p rerun=False,get-cbcollect-info=True -c conf/magma/dgm_collections_bloom_filter.conf -m rest > Task :compileJava warning: unknown enum constant When.MAYBE reason: class file for javax.annotation.meta.When not found Note: Some input files use or override a deprecated API. Note: Recompile with -Xlint:deprecation for details. Note: Some input files use unchecked or unsafe operations. Note: Recompile with -Xlint:unchecked for details. 1 warning > Task :testrunner Filename: conf/magma/dgm_collections_bloom_filter.conf Prefix: bucket_collections.collections_rebalance.CollectionsRebalance Global Test input params: {'cluster_name': 'win10-bucket-ops-temp_rebalance_magma', 'conf_file': 'conf/magma/dgm_collections_bloom_filter.conf', 'get-cbcollect-info': 'True', 'ini': '/tmp/win10-bucket-ops-temp_rebalance_magma.ini', 'num_nodes': 7, 'rerun': 'False', 'spec': 'dgm_collections_bloom_filter'} Logs will be stored at /data/workspace/temp_rebalance_magma/logs/testrunner-22-Mar-06_03-39-55/test_1 guides/gradlew --refresh-dependencies testrunner -P jython=/opt/jython/bin/jython -P 'args=-i /tmp/win10-bucket-ops-temp_rebalance_magma.ini rerun=False,get-cbcollect-info=True -t bucket_collections.collections_rebalance.CollectionsRebalance.test_data_load_collections_with_graceful_failover_recovery,nodes_init=4,nodes_failover=1,recovery_type=full,bucket_spec=magma_dgm.bloom_filter.4_node_1_replica_magma_1024,doc_size=30,randomize_value=True,data_load_stage=during,skip_validations=False,data_load_spec=volume_test_load_with_CRUD_on_collections,change_magma_quota=True,GROUP=failover' Test Input params: {'doc_size': '30', 'data_load_stage': 'during', 'cluster_name': 'win10-bucket-ops-temp_rebalance_magma', 'nodes_failover': '1', 'ini': '/tmp/win10-bucket-ops-temp_rebalance_magma.ini', 'get-cbcollect-info': 'True', 'recovery_type': 'full', 'conf_file': 'conf/magma/dgm_collections_bloom_filter.conf', 'change_magma_quota': 'True', 'skip_validations': 'False', 'spec': 'dgm_collections_bloom_filter', 'rerun': 'False', 'num_nodes': 7, 'logs_folder': '/data/workspace/temp_rebalance_magma/logs/testrunner-22-Mar-06_03-39-55/test_1', 'nodes_init': '4', 'GROUP': 'failover', 'bucket_spec': 'magma_dgm.bloom_filter.4_node_1_replica_magma_1024', 'case_number': 1, 'randomize_value': 'True', 'data_load_spec': 'volume_test_load_with_CRUD_on_collections'} test_data_load_collections_with_graceful_failover_recovery (bucket_collections.collections_rebalance.CollectionsRebalance) ... 2022-03-06 03:39:59,269 | test | INFO | MainThread | [basetestcase:log_setup_status:647] ========= CollectionsRebalance setup started for test #1 test_data_load_collections_with_graceful_failover_recovery ========= 2022-03-06 03:39:59,520 | test | INFO | MainThread | [basetestcase:log_setup_status:647] ========= BaseTestCase setup started for test #1 test_data_load_collections_with_graceful_failover_recovery ========= 2022-03-06 03:40:00,051 | test | INFO | MainThread | [basetestcase:initialize_cluster:467] Initializing cluster : C1 Got this failure java.net.ConnectException: Connection refused: /172.23.105.164:8091 during connect (<_realsocket at 0xd type=client open_count=1 channel=[id: 0xce19d241, 0.0.0.0/0.0.0.0:46021] timeout=300.0>) 2022-03-06 03:40:02,551 | test | ERROR | pool-3-thread-15 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.105.164:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.100.36:8091 during connect (<_realsocket at 0xe type=client open_count=1 channel=[id: 0x1ee86077, 0.0.0.0/0.0.0.0:57963] timeout=300.0>) 2022-03-06 03:40:02,640 | test | ERROR | pool-3-thread-20 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.100.36:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.100.34:8091 during connect (<_realsocket at 0xf type=client open_count=1 channel=[id: 0x96004b8b, 0.0.0.0/0.0.0.0:33523] timeout=300.0>) 2022-03-06 03:40:02,798 | test | ERROR | pool-3-thread-19 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.100.34:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.106.177:8091 during connect (<_realsocket at 0x10 type=client open_count=1 channel=[id: 0x5f15a04e, 0.0.0.0/0.0.0.0:52091] timeout=300.0>) 2022-03-06 03:40:02,947 | test | ERROR | pool-3-thread-17 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.106.177:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.100.35:8091 during connect (<_realsocket at 0x11 type=client open_count=1 channel=[id: 0xe52a386f, 0.0.0.0/0.0.0.0:42496] timeout=300.0>) 2022-03-06 03:40:03,207 | test | ERROR | pool-3-thread-18 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.100.35:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.100.37:8091 during connect (<_realsocket at 0x12 type=client open_count=1 channel=[id: 0xc689aede, 0.0.0.0/0.0.0.0:35462] timeout=300.0>) 2022-03-06 03:40:03,362 | test | ERROR | pool-3-thread-21 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.100.37:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.105.206:8091 during connect (<_realsocket at 0x13 type=client open_count=1 channel=[id: 0xe598a158, 0.0.0.0/0.0.0.0:37666] timeout=300.0>) 2022-03-06 03:40:03,371 | test | ERROR | pool-3-thread-16 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.105.206:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.106.177:8091 during connect (<_realsocket at 0x14 type=client open_count=1 channel=[id: 0x1c9f5143, 0.0.0.0/0.0.0.0:56768] timeout=300.0>) 2022-03-06 03:40:04,976 | test | ERROR | pool-3-thread-17 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.106.177:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.100.36:8091 during connect (<_realsocket at 0x15 type=client open_count=1 channel=[id: 0xdf395112, 0.0.0.0/0.0.0.0:35390] timeout=300.0>) 2022-03-06 03:40:05,055 | test | ERROR | pool-3-thread-20 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.100.36:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.100.34:8091 during connect (<_realsocket at 0x16 type=client open_count=1 channel=[id: 0x516b5773, 0.0.0.0/0.0.0.0:45968] timeout=300.0>) 2022-03-06 03:40:05,069 | test | ERROR | pool-3-thread-19 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.100.34:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.105.164:8091 during connect (<_realsocket at 0x17 type=client open_count=1 channel=[id: 0x6eda7782, 0.0.0.0/0.0.0.0:60980] timeout=300.0>) 2022-03-06 03:40:05,071 | test | ERROR | pool-3-thread-15 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.105.164:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.100.35:8091 during connect (<_realsocket at 0x18 type=client open_count=1 channel=[id: 0x565a176e, 0.0.0.0/0.0.0.0:59145] timeout=300.0>) 2022-03-06 03:40:05,217 | test | ERROR | pool-3-thread-18 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.100.35:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.100.37:8091 during connect (<_realsocket at 0x19 type=client open_count=1 channel=[id: 0x087c1a5c, 0.0.0.0/0.0.0.0:54787] timeout=300.0>) 2022-03-06 03:40:05,375 | test | ERROR | pool-3-thread-21 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.100.37:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.105.206:8091 during connect (<_realsocket at 0x1a type=client open_count=1 channel=[id: 0xf76dafdc, 0.0.0.0/0.0.0.0:45083] timeout=300.0>) 2022-03-06 03:40:05,381 | test | ERROR | pool-3-thread-16 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.105.206:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.106.177:8091 during connect (<_realsocket at 0x1b type=client open_count=1 channel=[id: 0xb58b1b37, 0.0.0.0/0.0.0.0:34331] timeout=300.0>) 2022-03-06 03:40:06,989 | test | ERROR | pool-3-thread-17 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.106.177:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.100.36:8091 during connect (<_realsocket at 0x1c type=client open_count=1 channel=[id: 0x983d5944, 0.0.0.0/0.0.0.0:36566] timeout=300.0>) 2022-03-06 03:40:07,063 | test | ERROR | pool-3-thread-20 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.100.36:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.100.34:8091 during connect (<_realsocket at 0x1d type=client open_count=1 channel=[id: 0xd8aaea87, 0.0.0.0/0.0.0.0:44081] timeout=300.0>) Got this failure java.net.ConnectException: Connection refused: /172.23.105.164:8091 during connect (<_realsocket at 0x1e type=client open_count=1 channel=[id: 0xe90630f7, 0.0.0.0/0.0.0.0:47562] timeout=300.0>) 2022-03-06 03:40:07,081 | test | ERROR | pool-3-thread-19 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.100.34:8091/nodes/self error [Errno 111] Connection refused 2022-03-06 03:40:07,082 | test | ERROR | pool-3-thread-15 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.105.164:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.100.35:8091 during connect (<_realsocket at 0x1f type=client open_count=1 channel=[id: 0x2e7816ef, 0.0.0.0/0.0.0.0:43864] timeout=300.0>) 2022-03-06 03:40:07,224 | test | ERROR | pool-3-thread-18 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.100.35:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.100.37:8091 during connect (<_realsocket at 0x20 type=client open_count=1 channel=[id: 0x37a6ffd5, 0.0.0.0/0.0.0.0:34211] timeout=300.0>) 2022-03-06 03:40:07,384 | test | ERROR | pool-3-thread-21 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.100.37:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.105.206:8091 during connect (<_realsocket at 0x21 type=client open_count=1 channel=[id: 0x31d764c9, 0.0.0.0/0.0.0.0:59242] timeout=300.0>) 2022-03-06 03:40:07,391 | test | ERROR | pool-3-thread-16 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.105.206:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.106.177:8091 during connect (<_realsocket at 0x22 type=client open_count=1 channel=[id: 0x84c92c8e, 0.0.0.0/0.0.0.0:46977] timeout=300.0>) 2022-03-06 03:40:09,000 | test | ERROR | pool-3-thread-17 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.106.177:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.100.36:8091 during connect (<_realsocket at 0x23 type=client open_count=1 channel=[id: 0x061961f4, 0.0.0.0/0.0.0.0:47915] timeout=300.0>) 2022-03-06 03:40:09,072 | test | ERROR | pool-3-thread-20 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.100.36:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.100.34:8091 during connect (<_realsocket at 0x24 type=client open_count=1 channel=[id: 0x38f0d9d3, 0.0.0.0/0.0.0.0:44271] timeout=300.0>)Got this failure java.net.ConnectException: Connection refused: /172.23.105.164:8091 during connect (<_realsocket at 0x25 type=client open_count=1 channel=[id: 0xf18dfef7, 0.0.0.0/0.0.0.0:40824] timeout=300.0>) 2022-03-06 03:40:09,092 | test | ERROR | pool-3-thread-19 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.100.34:8091/nodes/self error [Errno 111] Connection refused 2022-03-06 03:40:09,092 | test | ERROR | pool-3-thread-15 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.105.164:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.100.35:8091 during connect (<_realsocket at 0x26 type=client open_count=1 channel=[id: 0xbbc415ed, 0.0.0.0/0.0.0.0:38417] timeout=300.0>) 2022-03-06 03:40:09,233 | test | ERROR | pool-3-thread-18 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.100.35:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.100.37:8091 during connect (<_realsocket at 0x27 type=client open_count=1 channel=[id: 0x3a94eb2c, 0.0.0.0/0.0.0.0:59461] timeout=300.0>) 2022-03-06 03:40:09,391 | test | ERROR | pool-3-thread-21 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.100.37:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.105.206:8091 during connect (<_realsocket at 0x28 type=client open_count=1 channel=[id: 0xa7d4ba2d, 0.0.0.0/0.0.0.0:41919] timeout=300.0>) 2022-03-06 03:40:09,398 | test | ERROR | pool-3-thread-16 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.105.206:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.100.36:8091 during connect (<_realsocket at 0x29 type=client open_count=1 channel=[id: 0x002e0064, 0.0.0.0/0.0.0.0:34262] timeout=300.0>) 2022-03-06 03:40:11,082 | test | ERROR | pool-3-thread-20 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.100.36:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.100.34:8091 during connect (<_realsocket at 0x2a type=client open_count=1 channel=[id: 0xf1e9fe03, 0.0.0.0/0.0.0.0:39901] timeout=300.0>) 2022-03-06 03:40:11,105 | test | ERROR | pool-3-thread-19 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.100.34:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.100.35:8091 during connect (<_realsocket at 0x2b type=client open_count=1 channel=[id: 0xd5b3f9df, 0.0.0.0/0.0.0.0:41864] timeout=300.0>) 2022-03-06 03:40:11,240 | test | ERROR | pool-3-thread-18 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.100.35:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.100.37:8091 during connect (<_realsocket at 0x2c type=client open_count=1 channel=[id: 0x444a1e71, 0.0.0.0/0.0.0.0:51218] timeout=300.0>) 2022-03-06 03:40:11,400 | test | ERROR | pool-3-thread-21 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.100.37:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.100.37:8091 during connect (<_realsocket at 0x2d type=client open_count=1 channel=[id: 0xc5bda4db, 0.0.0.0/0.0.0.0:34914] timeout=300.0>) 2022-03-06 03:40:13,407 | test | ERROR | pool-3-thread-21 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.100.37:8091/nodes/self error [Errno 111] Connection refused 2022-03-06 03:40:23,571 | infra | CRITICAL | pool-3-thread-22 | [task:call:6181] mcdMemoryReserved reported in nodes/self is: 9379 2022-03-06 03:40:23,573 | test | CRITICAL | pool-3-thread-22 | [table_view:display:72] Memory quota allocated: +---------------------+---------+ | Service | RAM MiB | +---------------------+---------+ | eventingMemoryQuota | 256 | | cbasMemoryQuota | 1024 | | indexMemoryQuota | 256 | | ftsMemoryQuota | 256 | | memoryQuota | 9279 | +---------------------+---------+ 2022-03-06 03:40:38,099 | test | WARNING | MainThread | [basetestcase:_initialize_nodes:702] RAM quota was defined less than 100 MB: 2022-03-06 03:40:50,611 | test | INFO | MainThread | [basetestcase:initialize_cluster:493] Cluster C1 initialized 2022-03-06 03:40:50,612 | test | INFO | MainThread | [basetestcase:log_setup_status:647] ========= BaseTestCase setup finished for test #1 test_data_load_collections_with_graceful_failover_recovery ========= 2022-03-06 03:40:50,631 | test | INFO | MainThread | [basetestcase:log_setup_status:647] ========= ClusterSetup setup started for test #1 test_data_load_collections_with_graceful_failover_recovery ========= 2022-03-06 03:41:11,509 | test | INFO | pool-3-thread-23 | [table_view:display:72] Rebalance Overview +----------------+----------+-----------------------+----------------+--------------+-----------------------+ | Nodes | Services | Version | CPU | Status | Membership / Recovery | +----------------+----------+-----------------------+----------------+--------------+-----------------------+ | 172.23.105.164 | kv | 7.1.0-2440-enterprise | 0.663246151921 | Cluster node | active / none | | 172.23.105.206 | None | | | <--- IN --- | | | 172.23.106.177 | None | | | <--- IN --- | | | 172.23.100.35 | None | | | <--- IN --- | | +----------------+----------+-----------------------+----------------+--------------+-----------------------+ 2022-03-06 03:41:11,526 | test | INFO | pool-3-thread-23 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 03:41:26,559 | test | INFO | pool-3-thread-23 | [task:check:415] Rebalance - status: none, progress: 100 2022-03-06 03:41:26,572 | test | INFO | pool-3-thread-23 | [task:check:474] Rebalance completed with progress: 100% in 15.0640001297 sec 2022-03-06 03:41:26,687 | test | INFO | MainThread | [table_view:display:72] Cluster statistics +----------------+----------+-----------------+-----------+-----------+----------------------+-------------------+-----------------------+ | Node | Services | CPU_utilization | Mem_total | Mem_free | Swap_mem_used | Active / Replica | Version | +----------------+----------+-----------------+-----------+-----------+----------------------+-------------------+-----------------------+ | 172.23.105.206 | kv | 0.325651302605 | 11.45 GiB | 10.54 GiB | 23.59 MiB / 3.50 GiB | 0 / 0 | 7.1.0-2440-enterprise | | 172.23.106.177 | kv | 0.28818443804 | 11.45 GiB | 10.44 GiB | 23.15 MiB / 3.50 GiB | 0 / 0 | 7.1.0-2440-enterprise | | 172.23.100.35 | kv | 0.212872526922 | 11.45 GiB | 10.61 GiB | 0.0 Byte / 3.50 GiB | 0 / 0 | 7.1.0-2440-enterprise | | 172.23.105.164 | kv | 1.14035087719 | 11.45 GiB | 10.49 GiB | 0.0 Byte / 3.50 GiB | 0 / 0 | 7.1.0-2440-enterprise | +----------------+----------+-----------------+-----------+-----------+----------------------+-------------------+-----------------------+ 2022-03-06 03:41:26,688 | test | INFO | MainThread | [basetestcase:log_setup_status:647] ========= ClusterSetup setup complete for test #1 test_data_load_collections_with_graceful_failover_recovery ========= 2022-03-06 03:41:26,688 | test | INFO | MainThread | [basetestcase:log_setup_status:647] ========= CollectionBase setup started for test #1 test_data_load_collections_with_graceful_failover_recovery ========= 2022-03-06 03:41:26,852 | test | INFO | MainThread | [bucket_ready_functions:create_buckets_using_json_data:1913] Creating required buckets from template 2022-03-06 03:41:27,144 | test | INFO | MainThread | [bucket_ready_functions:wait_for_collection_creation_to_complete:2527] Waiting for all collections to be created 2022-03-06 03:41:27,401 | test | INFO | MainThread | [table_view:display:72] Bucket statistics +---------+-----------+-----------------+----------+------------+-----+-------+-----------+----------+-----------+-----+ | Bucket | Type | Storage Backend | Replicas | Durability | TTL | Items | RAM Quota | RAM Used | Disk Used | ARR | +---------+-----------+-----------------+----------+------------+-----+-------+-----------+----------+-----------+-----+ | bucket1 | couchbase | couchstore | 1 | none | 0 | 0 | 0.0 Byte | 0.0 Byte | 0.0 Byte | 100 | | bucket2 | couchbase | magma | 1 | none | 0 | 0 | 4.00 GiB | 0.0 Byte | 0.0 Byte | 100 | | default | couchbase | magma | 1 | none | 0 | 0 | 0.0 Byte | 0.0 Byte | 0.0 Byte | 100 | +---------+-----------+-----------------+----------+------------+-----+-------+-----------+----------+-----------+-----+ 2022-03-06 03:41:28,193 | test | INFO | MainThread | [common_lib:sleep:23] Sleep 30 seconds. Reason: Sleep while magma storage quota setting is taking effect 2022-03-06 03:41:58,194 | test | INFO | MainThread | [collections_base:collection_setup:174] Creating required SDK clients for client_pool 2022-03-06 03:42:05,062 | test | INFO | MainThread | [bucket_ready_functions:perform_tasks_from_spec:4993] Performing scope/collection specific operations 2022-03-06 03:42:05,078 | test | INFO | MainThread | [bucket_ready_functions:perform_tasks_from_spec:5083] Done Performing scope/collection specific operations 2022-03-06 04:39:08,911 | test | INFO | pool-3-thread-28 | [table_view:display:72] Ops trend for bucket 'default' +-----------+----------------------------------------------------------------------------------------------------------------------------------+-----------+ | Min | Trend | Max | +-----------+----------------------------------------------------------------------------------------------------------------------------------+-----------+ | 0.000 | *****....................................................********...........................................********....*******X | 86468.800 | | 85578.900 | .................................................................................................*******X | 85578.900 | | 85065.500 | ................................................................................................********...*******X | 87293.500 | | 87064.900 | ..................................................................................................*******X | 87064.900 | | 85144.700 | ................................................................................................*******X | 85144.700 | | 83600.500 | ..............................................................................................********...*******X | 85711.800 | | 83838.500 | ...............................................................................................********..*******X | 86064.800 | | 84835.200 | ................................................................................................******X | 84835.200 | | 82699.600 | .............................................................................................********.*******X | 83481.700 | | 82587.600 | .............................................................................................********..********..*******X | 85571.200 | | 84974.200 | ................................................................................................*******X | 84974.200 | | 84514.400 | ...............................................................................................*******X | 84514.400 | | 81805.400 | ............................................................................................********.*******X | 82328.000 | | 82160.000 | .............................................................................................*******X | 82160.000 | | 81676.800 | ............................................................................................*******.***************X | 82843.000 | | 82350.300 | .............................................................................................****************..*******X | 84663.600 | | 81798.700 | ............................................................................................********.******X | 82678.700 | | 82641.000 | .............................................................................................********.*******X | 83047.100 | | 82863.300 | .............................................................................................*******X | 82863.300 | | 82267.100 | .............................................................................................*******X | 82267.100 | | 80172.600 | ..........................................................................................*******X | 80172.600 | | 79511.100 | ..........................................................................................******X | 79511.100 | | 78102.500 | ........................................................................................X | 78102.500 | | 76680.700 | ......................................................................................*******....********.********...*******X | 83770.500 | | 83266.700 | ..............................................................................................*******X | 83266.700 | | 79733.400 | ..........................................................................................*******.*******X | 80440.100 | | 79826.000 | ..........................................................................................*******X | 79826.000 | | 78767.700 | .........................................................................................*******X | 78767.700 | | 77152.000 | .......................................................................................***************.....*******X | 81487.300 | | 78586.100 | .........................................................................................*******X | 78586.100 | | 78260.000 | ........................................................................................**************X | 78457.200 | | 75746.600 | .....................................................................................****************...********.*******X | 79417.000 | | 77872.600 | ........................................................................................********.*******X | 78770.300 | | 76928.800 | .......................................................................................******X | 76928.800 | | 74032.900 | ...................................................................................*******X | 74032.900 | | 73827.000 | ...................................................................................********....****************....*******X | 81093.200 | | 79904.700 | ..........................................................................................******X | 79904.700 | | 79613.200 | ..........................................................................................X | 79613.200 | | 78771.600 | .........................................................................................******X | 78771.600 | | 77276.400 | .......................................................................................***************X | 77519.100 | | 71217.900 | ................................................................................*******X | 71217.900 | | 70836.600 | ................................................................................********.....*******...********..*******X | 79600.800 | | 74181.800 | ...................................................................................*******X | 74181.800 | | 73436.800 | ...................................................................................*******X | 73436.800 | | 67959.300 | ............................................................................******X | 67959.300 | | 67504.400 | ............................................................................********.........********....*******X | 79091.800 | | 78860.100 | .........................................................................................*******X | 78860.100 | | 76367.500 | ......................................................................................******X | 76367.500 | | 75598.400 | .....................................................................................*******X | 75598.400 | | 69302.500 | ..............................................................................*******X | 69302.500 | | 68221.500 | .............................................................................********....********.******X | 73060.500 | | 67366.100 | ............................................................................*******X | 67366.100 | | 64970.800 | .........................................................................*******X | 64970.800 | | 63814.500 | ........................................................................********.......*******........*...******X | 79585.100 | | 70282.600 | ...............................................................................*******X | 70282.600 | | 65548.600 | ..........................................................................********.....*******X | 69882.900 | | 69548.200 | ..............................................................................*******.*******X | 70625.000 | | 66257.300 | ..........................................................................******X | 66257.300 | | 62260.300 | ......................................................................********......********.......******X | 74086.000 | | 73278.000 | ..................................................................................********...*******X | 75923.900 | | 71486.300 | ................................................................................******X | 71486.300 | | 65575.400 | ..........................................................................********.********....********..******X | 71886.600 | | 69098.400 | ..............................................................................*******X | 69098.400 | | 62227.200 | ......................................................................*******..*******X | 64297.200 | | 63191.600 | .......................................................................********..........********..*******X | 73749.800 | | 67482.400 | ............................................................................******X | 67482.400 | | 66234.500 | ..........................................................................********..********....*******.********.*******X | 72841.700 | | 67816.800 | ............................................................................*******X | 67816.800 | | 63292.300 | .......................................................................*******.....********....*******X | 71478.600 | | 69580.000 | ..............................................................................******X | 69580.000 | | 66953.800 | ...........................................................................*******X | 66953.800 | | 61711.200 | .....................................................................*******...********......*******X | 69243.700 | | 68906.300 | .............................................................................**************X | 68945.100 | | 61259.400 | .....................................................................*******X | 61259.400 | | 57334.300 | ................................................................*******.......********.....********....******X | 70923.400 | | 69124.000 | ..............................................................................*******X | 69124.000 | | 63221.300 | .......................................................................********.....********..*******...*******X | 71885.500 | | 71780.200 | .................................................................................*******X | 71780.200 | | 69801.900 | ..............................................................................*******X | 69801.900 | | 68161.500 | .............................................................................******X | 68161.500 | | 61515.200 | .....................................................................********....******X | 65116.000 | | 64434.900 | ........................................................................*******X | 64434.900 | | 57825.400 | .................................................................******X | 57825.400 | | 55756.100 | ..............................................................********........********.*******..*******X | 65062.200 | | 60173.200 | ...................................................................******X | 60173.200 | | 53931.100 | ............................................................*******X | 53931.100 | | 48530.400 | ......................................................********......................********......******X | 72740.500 | | 68850.000 | .............................................................................*******X | 68850.000 | | 63756.400 | ........................................................................******X | 63756.400 | | 58875.600 | ..................................................................X | 58875.600 | | 57280.300 | ................................................................*******..********.....********............*******X | 74090.400 | | 71969.300 | .................................................................................******X | 71969.300 | | 60129.400 | ...................................................................*******X | 60129.400 | | 57768.400 | .................................................................********...******X | 60635.600 | | 57138.200 | ................................................................*******X | 57138.200 | | 53138.700 | ...........................................................*******...********.....********...*******...*******X | 65320.300 | | 59517.100 | ...................................................................*******X | 59517.100 | | 58853.700 | ..................................................................*******....*....**************X | 65973.400 | | 60134.900 | ...................................................................******X | 60134.900 | | 58901.100 | ..................................................................********.*******....********....********...*******X | 69053.300 | | 68269.500 | .............................................................................******X | 68269.500 | | 67565.300 | ............................................................................*******X | 67565.300 | | 56351.600 | ...............................................................******X | 56351.600 | | 52997.200 | ...........................................................********.........*******.........********.....*******X | 73016.600 | | 68045.900 | ............................................................................******X | 68045.900 | | 51517.100 | ..........................................................*******X | 51517.100 | | 46363.200 | ....................................................*******..........********..........*******X | 64361.400 | | 61992.000 | ......................................................................******X | 61992.000 | | 59137.700 | ..................................................................*******X | 59137.700 | | 53906.000 | ............................................................********.....*******...***************X | 60568.200 | | 55020.300 | ..............................................................********......******X | 60239.000 | | 58403.000 | .................................................................********...*******X | 60896.600 | | 48793.000 | ......................................................******X | 48793.000 | | 48407.600 | ......................................................********....********.......*******....*******X | 61676.300 | | 51494.800 | .........................................................********.******X | 51861.900 | | 42823.800 | ................................................********...............********.....*******X | 60705.800 | | 60277.100 | ....................................................................******X | 60277.100 | | 54337.600 | .............................................................**************X | 54954.400 | | 49865.100 | ........................................................********.....*******X | 54308.900 | | 44758.400 | ..................................................*******........********......*******X | 56766.700 | | 50026.900 | ........................................................******X | 50026.900 | | 46972.800 | ....................................................********..............*******.........********.*******X | 67710.600 | | 65131.600 | .........................................................................*******...*...*******..*******X | 71928.000 | | 70278.100 | ...............................................................................********....*******.......********...*******X | 82718.600 | | 80856.500 | ...........................................................................................********....*******X | 84256.900 | | 81680.500 | ............................................................................................*******X | 81680.500 | | 80731.600 | ...........................................................................................********...**************X | 83501.300 | | 83195.500 | ..............................................................................................********.*******X | 83987.300 | | 83436.400 | ..............................................................................................**************X | 83719.300 | | 83646.100 | ..............................................................................................X | 83646.100 | | 83371.300 | ..............................................................................................******X | 83371.300 | | 82671.500 | .............................................................................................********.*******X | 83423.700 | | 82895.900 | .............................................................................................*******X | 82895.900 | | 81568.400 | ............................................................................................*******X | 81568.400 | | 81537.500 | ............................................................................................*******..***************X | 83400.500 | | 80723.800 | ...........................................................................................********.********..******X | 82951.400 | | 81430.900 | ............................................................................................********.*******X | 82748.800 | | 81090.100 | ...........................................................................................*******X | 81090.100 | | 80798.100 | ...........................................................................................********..*******X | 82652.100 | | 80931.100 | ...........................................................................................*******..*******X | 82182.600 | | 82132.600 | .............................................................................................*******X | 82132.600 | | 80949.300 | ...........................................................................................********...**************X | 83150.500 | | 81897.900 | ............................................................................................********...*******X | 84225.400 | | 82394.200 | .............................................................................................********.********.*******X | 84148.800 | | 83844.300 | ...............................................................................................*******X | 83844.300 | | 83740.600 | ..............................................................................................*******X | 83740.600 | | 82976.200 | ..............................................................................................*******X | 82976.200 | | 81497.600 | ............................................................................................***************X | 81926.400 | | 81857.100 | ............................................................................................**************X | 81903.900 | | 81557.300 | ............................................................................................********...*******X | 84209.400 | | 83286.300 | ..............................................................................................*******X | 83286.300 | | 80857.800 | ...........................................................................................********...*******X | 83313.000 | | 83136.000 | ..............................................................................................*******X | 83136.000 | | 82196.600 | .............................................................................................*******X | 82196.600 | | 80309.900 | ..........................................................................................********..******X | 81712.700 | | 79714.100 | ..........................................................................................***************X | 79850.300 | | 75733.600 | .....................................................................................********.....*******X | 80214.800 | | 79758.900 | ..........................................................................................********..******X | 81660.300 | | 81017.700 | ...........................................................................................********.*******X | 81899.700 | | 81037.100 | ...........................................................................................********...*******X | 83112.200 | | 81074.500 | ...........................................................................................********...******X | 83496.700 | | 83371.100 | ..............................................................................................X | 83371.100 | | 83171.800 | ..............................................................................................******X | 83171.800 | | 81369.000 | ............................................................................................***************X | 81395.500 | | 81369.300 | ............................................................................................********.*******X | 82548.000 | | 82250.100 | .............................................................................................*******.*******X | 82943.300 | | 82740.800 | .............................................................................................*******X | 82740.800 | | 81943.100 | ............................................................................................********...******X | 84195.400 | | 81649.300 | ............................................................................................********..*******X | 83063.400 | | 82884.500 | .............................................................................................*******X | 82884.500 | | 81020.100 | ...........................................................................................********..******X | 82850.200 | | 80891.500 | ...........................................................................................X | 80891.500 | | 80197.200 | ..........................................................................................*******...*******X | 82288.200 | | 79618.800 | ..........................................................................................********..***************X | 81423.000 | | 80626.800 | ...........................................................................................*******X | 80626.800 | | 80562.900 | ...........................................................................................********..********.*******X | 83781.000 | | 80933.500 | ...........................................................................................******X | 80933.500 | | 79627.100 | ..........................................................................................********...********.*******X | 83195.600 | | 81207.200 | ............................................................................................********.****************.******X | 83172.000 | | 79782.600 | ..........................................................................................********.*******X | 80634.600 | | 77944.400 | ........................................................................................*******X | 77944.400 | | 75058.600 | ....................................................................................*******X | 75058.600 | | 74750.200 | ....................................................................................********..*******X | 76359.600 | | 73674.600 | ...................................................................................*******X | 74121.700 | | 73332.300 | ...................................................................................*******.********.*******X | 75793.500 | | 72562.000 | ..................................................................................***X | 72562.000 | +-----------+----------------------------------------------------------------------------------------------------------------------------------+-----------+ 2022-03-06 04:39:10,055 | test | INFO | pool-3-thread-29 | [table_view:display:72] Ops trend for bucket 'bucket2' +---------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------+ | Min | Trend | Max | +---------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------+ | 0.000 | *****...........................................................................*******X | 900.000 | | 0.000 | **************************************************************************************************************************************************************************************************************************************........********............................................................................................*******X | 1200.000 | | 0.000 | *****************************************************************************************************************************************************************************************************************************************************.........................********........*******X | 400.000 | | 200.000 | ...............*******X | 200.000 | | 0.000 | ****************************************************************************************************************************************************************************************************************************************************************..........................................................******X | 700.000 | | 600.000 | .................................................*******X | 600.000 | | 200.000 | ...............*******X | 200.000 | | 0.000 | ************************************************************************************************************************************************************************************************************************************************************************................*******.........***********************X | 300.000 | | 100.000 | .......*******X | 100.000 | | 0.000 | ******************************************************************************************************************************************************************************************************************************************************************************************************.........................*******X | 300.000 | | 100.000 | .......******X | 100.000 | | 0.000 | ***************************************************************************************************************************************************************************************************************************************************************************************************************************************************...................................................................................*******X | 1000.000 | | 300.000 | ........................*******................*.................******X | 700.000 | | 0.000 | *******************************************************************************************************************************************************************************************************************************........********........********.........*******X | 300.000 | | 100.000 | .......*******X | 100.000 | | 0.000 | ************************************************************************************************************************************************************************************************************************************.........................................*******X | 500.000 | | 100.000 | .......***************X | 100.000 | | 0.000 | ***********************************************************************************************************************************************************************************************************************................******X | 200.000 | | 0.000 | ********................*******X | 200.000 | | 0.000 | **********************************************************************************************************************************************************X | 0.000 | +---------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------+ 2022-03-06 04:39:11,193 | test | INFO | pool-3-thread-30 | [table_view:display:72] Ops trend for bucket 'bucket1' +---------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------+ | Min | Trend | Max | +---------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------+ | 0.000 | *****....................................................................................................*******X | 1800.000 | | 0.000 | *********************************************************************************************************************************************************************************************************************************************.......................................................*******X | 1000.000 | | 0.000 | *****************************************************************************************************************************************************************************************************************************************************...........********.......................................*******X | 900.000 | | 0.000 | ************************************************************************************************************************************************************************************************************************************************************************......................*******...........*******X | 600.000 | | 255.200 | .............*******X | 255.200 | | 44.800 | .*******X | 44.800 | | 0.000 | ***************************************************************************************************************************************************************************************************************************************************************.......********...................................******X | 760.800 | | 696.000 | .....................................X | 696.000 | | 500.000 | ..........................******X | 500.000 | | 100.000 | ....***************X | 100.000 | | 0.000 | **********************************************************************************************************************************************************************************************************************************************************************************************.....*******.................*******X | 400.000 | | 0.000 | ********.....***************X | 100.000 | | 0.000 | ********************************************************************************************************************************************************************************************************************************************************************************************************************.....***************......*******X | 200.000 | | 0.000 | ********...........*******X | 200.000 | | 0.000 | ***********************************************************************************************************************************************************************************************************************************************................*******X | 300.000 | | 100.000 | ....*******X | 100.000 | | 0.000 | ************************************************************************************************************************************************************************************************************************************...........*******************************X | 200.000 | | 0.000 | *******************************************************************************************************************************************************************************************************.....*******........*...******X | 305.500 | | 0.000 | ********................*******X | 300.000 | | 0.000 | ***********************************************************************************************************************************************************X | 0.000 | +---------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------+ 2022-03-06 04:39:12,414 | test | INFO | pool-3-thread-8 | [table_view:display:72] Ops trend for bucket 'default' +-----------+------------------------------------------------------------------------------------------------------+-----------+ | Min | Trend | Max | +-----------+------------------------------------------------------------------------------------------------------+-----------+ | 72562.000 | ...................................................................................................X | 72562.000 | +-----------+------------------------------------------------------------------------------------------------------+-----------+ 2022-03-06 04:39:12,588 | test | INFO | MainThread | [table_view:display:72] Cluster statistics +----------------+----------+-----------------+-----------+----------+----------------------+---------------------+-----------------------+ | Node | Services | CPU_utilization | Mem_total | Mem_free | Swap_mem_used | Active / Replica | Version | +----------------+----------+-----------------+-----------+----------+----------------------+---------------------+-----------------------+ | 172.23.105.206 | kv | 80.4175852678 | 11.45 GiB | 8.85 GiB | 23.59 MiB / 3.50 GiB | 62247107 / 62208923 | 7.1.0-2440-enterprise | | 172.23.106.177 | kv | 79.4990963078 | 11.45 GiB | 8.71 GiB | 23.15 MiB / 3.50 GiB | 62417075 / 62394661 | 7.1.0-2440-enterprise | | 172.23.100.35 | kv | 83.3870344117 | 11.45 GiB | 8.79 GiB | 0.0 Byte / 3.50 GiB | 62364733 / 62376639 | 7.1.0-2440-enterprise | | 172.23.105.164 | kv | 87.3429084381 | 11.45 GiB | 8.74 GiB | 0.0 Byte / 3.50 GiB | 61779021 / 61759917 | 7.1.0-2440-enterprise | +----------------+----------+-----------------+-----------+----------+----------------------+---------------------+-----------------------+ 2022-03-06 04:39:26,292 | test | INFO | MainThread | [bucket_ready_functions:validate_docs_per_collections_all_buckets:4742] Validating collection stats and item counts 2022-03-06 04:39:34,094 | test | ERROR | MainThread | [bucket_ready_functions:validate_manifest_uid:4605] Bucket UID mismatch. Expected: 0, Actual: 1 2022-03-06 04:39:34,095 | test | WARNING | MainThread | [bucket_ready_functions:validate_docs_per_collections_all_buckets:4759] Bucket manifest UID mismatch! 2022-03-06 04:39:38,055 | test | ERROR | MainThread | [bucket_ready_functions:validate_manifest_uid:4605] Bucket UID mismatch. Expected: 0, Actual: 1 2022-03-06 04:39:38,055 | test | WARNING | MainThread | [bucket_ready_functions:validate_docs_per_collections_all_buckets:4759] Bucket manifest UID mismatch! 2022-03-06 04:39:42,036 | test | ERROR | MainThread | [bucket_ready_functions:validate_manifest_uid:4605] Bucket UID mismatch. Expected: 0, Actual: 1 2022-03-06 04:39:42,036 | test | WARNING | MainThread | [bucket_ready_functions:validate_docs_per_collections_all_buckets:4759] Bucket manifest UID mismatch! 2022-03-06 04:39:51,990 | test | INFO | MainThread | [table_view:display:72] Bucket statistics +---------+-----------+-----------------+----------+------------+-----+-----------+-----------+------------+------------+-----------+ | Bucket | Type | Storage Backend | Replicas | Durability | TTL | Items | RAM Quota | RAM Used | Disk Used | ARR | +---------+-----------+-----------------+----------+------------+-----+-----------+-----------+------------+------------+-----------+ | bucket1 | couchbase | couchstore | 1 | none | 0 | 100000 | 7.81 GiB | 112.67 MiB | 101.89 MiB | 100 | | bucket2 | couchbase | magma | 1 | none | 0 | 100000 | 4.00 GiB | 301.22 MiB | 403.25 MiB | 100 | | default | couchbase | magma | 1 | none | 0 | 250000000 | 4.00 GiB | 2.51 GiB | 41.23 GiB | 4.4068484 | +---------+-----------+-----------------+----------+------------+-----+-----------+-----------+------------+------------+-----------+ 2022-03-06 04:39:51,990 | test | INFO | MainThread | [basetestcase:log_setup_status:647] ========= CollectionBase setup complete for test #1 test_data_load_collections_with_graceful_failover_recovery ========= 2022-03-06 04:39:55,013 | test | INFO | MainThread | [collections_rebalance:load_collections_with_rebalance:1003] Doing collection data load during graceful_failover_recovery 2022-03-06 04:39:55,015 | test | INFO | MainThread | [collections_rebalance:rebalance_operation:361] Starting rebalance operation of type : graceful_failover_recovery 2022-03-06 04:39:55,016 | test | INFO | MainThread | [collections_rebalance:rebalance_operation:671] failing over nodes [ip:172.23.100.35 port:8091 ssh_username:root] 2022-03-06 04:40:13,223 | test | INFO | pool-3-thread-24 | [rest_client:monitorRebalance:1610] Rebalance done. Taken 18.0950000286 seconds to complete 2022-03-06 04:40:13,229 | test | INFO | pool-3-thread-24 | [common_lib:sleep:23] Sleep 5 seconds. Reason: Wait after rebalance complete 2022-03-06 04:45:18,230 | test | INFO | MainThread | [collections_rebalance:data_load_after_failover:330] Starting a sync data load after failover 2022-03-06 04:45:18,236 | test | INFO | MainThread | [bucket_ready_functions:perform_tasks_from_spec:4993] Performing scope/collection specific operations 2022-03-06 04:45:22,446 | test | INFO | MainThread | [bucket_ready_functions:perform_tasks_from_spec:5083] Done Performing scope/collection specific operations 2022-03-06 05:27:53,315 | test | INFO | pool-3-thread-25 | [table_view:display:72] Ops trend for bucket 'default' +------------+---------------------------------------------------------------------------------------------------------------------------+------------+ | Min | Trend | Max | +------------+---------------------------------------------------------------------------------------------------------------------------+------------+ | 0.000 | ******.................................................................*******...................................*******X | 112551.900 | | 109582.700 | ................................................................................................*******X | 109582.700 | | 92251.500 | ................................................................................******X | 92251.500 | | 77110.000 | ...................................................................*******X | 77110.000 | | 52353.800 | .............................................*******...*******.......*******......********..******X | 73102.400 | | 66642.100 | ..........................................................******X | 66642.100 | | 60310.300 | ....................................................****X | 60310.300 | | 60310.300 | ....................................................******X | 60709.900 | | 60709.900 | ....................................................**X | 60709.900 | | 58784.900 | ...................................................******X | 58784.900 | | 56961.000 | .................................................******X | 56961.000 | | 51583.000 | ............................................******X | 51583.000 | | 50375.800 | ...........................................********..***X | 52214.400 | | 52214.400 | .............................................**X | 52214.400 | | 50983.700 | ............................................*******...******X | 54716.000 | | 53675.600 | ..............................................*X | 53675.600 | | 53675.600 | ..............................................*X | 53675.600 | | 53675.600 | ..............................................***.********.***X | 56031.100 | | 56031.100 | ................................................**X | 56031.100 | | 53167.200 | ..............................................*******....*******.*******..******X | 61079.300 | | 54578.700 | ...............................................*******X | 54578.700 | | 50632.600 | ...........................................*******.....********..*X | 57910.100 | | 57910.100 | ..................................................*****.********.....*X | 64680.600 | | 64680.600 | ........................................................***********X | 64783.100 | | 61571.300 | .....................................................******X | 61571.300 | | 58175.800 | ..................................................********.******X | 59356.200 | | 57975.900 | ..................................................*X | 57975.900 | | 57975.900 | ..................................................*X | 57975.900 | | 57975.900 | ..................................................*X | 57975.900 | | 57975.900 | ..................................................*X | 57975.900 | | 55341.800 | ................................................******X | 55341.800 | | 54878.000 | ...............................................******X | 54878.000 | | 54520.700 | ...............................................********.*******...*******.*******X | 60359.000 | | 54484.300 | ...............................................******X | 54484.300 | | 53904.700 | ..............................................*X | 53904.700 | | 53904.700 | ..............................................*X | 53904.700 | | 53904.700 | ..............................................**X | 53904.700 | | 53397.100 | ..............................................******X | 53397.100 | | 50372.800 | ...........................................*******X | 50372.800 | | 49140.800 | ..........................................*******..***X | 51200.600 | | 51200.600 | ............................................*X | 51200.600 | | 51200.600 | ............................................X | 51200.600 | | 48768.800 | ..........................................************X | 48790.600 | | 48790.600 | ..........................................**...*X | 52096.700 | | 52096.700 | .............................................*****........*******X | 60931.600 | | 58341.600 | ..................................................**X | 58341.600 | | 58341.600 | ..................................................**X | 58341.600 | | 58341.600 | ..................................................X | 58341.600 | | 53079.900 | ..............................................**************...*******X | 57157.700 | | 55062.300 | ...............................................*******..********..*******......*******......******X | 72390.000 | | 64695.200 | ........................................................X | 64695.200 | | 61046.000 | .....................................................**X | 61046.000 | | 61046.000 | .....................................................***X | 61046.000 | | 57861.000 | ..................................................*******...*******X | 61149.500 | | 59419.300 | ...................................................******X | 59419.300 | | 57846.400 | ..................................................********...******X | 61296.300 | | 60536.500 | ....................................................***X | 60536.500 | | 60536.500 | ....................................................**X | 60536.500 | | 59541.800 | ...................................................***X | 59541.800 | | 59541.800 | ...................................................****.******X | 60393.800 | | 41249.700 | ...................................********.******X | 41837.800 | | 38075.800 | ................................******X | 38075.800 | | 36118.100 | ...............................**X | 36118.100 | | 36118.100 | ...............................*X | 36118.100 | | 36118.100 | ...............................***..*******..******X | 40966.900 | | 34351.100 | .............................*******.*..****X | 37420.900 | | 37420.900 | ................................*X | 37420.900 | | 36139.400 | ...............................*X | 36139.400 | | 36139.400 | ...............................*****........**X | 45728.400 | | 45728.400 | .......................................***X | 45728.400 | | 38494.100 | .................................********....******X | 43868.000 | | 37478.400 | ................................*******.....******X | 43816.800 | | 39368.100 | .................................********...******X | 42050.800 | | 40170.900 | ..................................***X | 40170.900 | | 40170.900 | ..................................**X | 40170.900 | | 40170.900 | ..................................X | 40170.900 | | 39667.000 | ..................................*******.*******X | 40832.300 | | 39868.500 | ..................................**************..*******X | 42500.300 | | 39688.600 | ..................................**************X | 40082.200 | | 38466.600 | .................................*****X | 38466.600 | | 38466.600 | .................................*.********.******X | 40526.400 | | 32841.900 | ............................*******......*******X | 39448.600 | | 38647.700 | .................................******X | 38647.700 | | 33949.500 | .............................*******.*******X | 35576.500 | | 34965.000 | ..............................******X | 34965.000 | | 33919.300 | .............................*******.********..**************X | 37757.400 | | 35081.800 | ..............................*******...X | 38841.900 | | 38841.900 | .................................******..******X | 40524.000 | | 36170.200 | ...............................*******X | 36170.200 | | 35350.600 | ..............................*X | 35350.600 | | 35350.600 | ..............................**X | 35350.600 | | 35350.600 | ..............................*X | 35350.600 | | 34256.400 | .............................******X | 34256.400 | | 33178.800 | ............................********.******X | 34560.300 | | 33933.500 | .............................*******..***X | 36914.600 | | 36914.600 | ...............................**X | 36914.600 | | 36576.800 | ...............................*******X | 36576.800 | | 35807.400 | ..............................*******..*********X | 38206.400 | | 38206.400 | ................................****...******X | 40720.000 | | 38319.300 | .................................**X | 38622.400 | | 38622.400 | .................................***X | 38622.400 | | 38622.400 | .................................X | 38622.400 | | 35656.400 | ..............................*X | 35656.400 | | 35656.400 | ..............................**X | 35656.400 | | 35656.400 | ..............................**.....******X | 40899.600 | | 38303.200 | .................................******X | 38303.200 | | 37375.100 | ................................*******X | 37375.100 | | 37373.100 | ................................******X | 37373.100 | | 36692.800 | ...............................******X | 36692.800 | | 34856.600 | .............................*******X | 34856.600 | | 33402.800 | ............................*******.*******...*******X | 37235.900 | | 37164.800 | ................................******X | 37164.800 | | 35786.900 | ..............................******X | 35786.900 | | 35323.400 | ..............................***X | 35323.400 | | 35323.400 | ..............................****...******X | 38486.500 | | 33952.900 | .............................*******.*************X | 35949.100 | | 35949.100 | ..............................*.*******X | 36945.900 | | 35500.200 | ..............................******X | 35500.200 | | 35399.300 | ..............................********.******X | 36189.700 | | 30623.800 | ..........................*******.....*******X | 36343.100 | | 34397.300 | .............................*******.*******X | 35034.500 | | 33979.200 | .............................*******..**X | 36497.100 | | 36497.100 | ...............................***X | 36497.100 | | 35982.700 | ..............................X | 35982.700 | | 34069.800 | .............................*************X | 34465.200 | | 34347.600 | .............................*******X | 34347.600 | | 33456.900 | ............................*****X | 33456.900 | | 33456.900 | ............................*.*******X | 34408.400 | | 34167.700 | .............................**************X | 34173.600 | | 33308.000 | ............................*******....X | 37424.400 | | 37424.400 | ................................******X | 37424.400 | | 36820.400 | ...............................*******.*******.*X | 38708.400 | | 38708.400 | .................................*X | 38708.400 | | 38708.400 | .................................****.******X | 40017.200 | | 36549.400 | ...............................*X | 36549.400 | | 36549.400 | ...............................******.******X | 38214.200 | | 34862.800 | .............................********...*X | 37989.600 | | 37989.600 | ................................*X | 37989.600 | | 37989.600 | ................................***.******X | 38515.800 | | 34787.400 | .............................********...*******.******X | 38453.200 | | 37078.300 | ...............................*******X | 37078.300 | | 33551.400 | ............................*******...*******X | 36635.100 | | 33603.800 | ............................*******.*******X | 34388.500 | | 33461.100 | ............................*******.********...******X | 37779.400 | | 33229.600 | ............................********.**************X | 34864.300 | | 34476.300 | .............................********.******X | 35709.600 | | 34102.900 | .............................*******X | 34102.900 | | 32963.000 | ............................*******....X | 37177.100 | | 37177.100 | ................................*******.......********.............********.......X | 67893.200 | | 67893.200 | ...........................................................*******.********.*******X | 69920.800 | | 67354.600 | ..........................................................*******X | 67354.600 | | 65563.200 | .........................................................X | 65563.200 | | 65563.200 | .........................................................***X | 65563.200 | | 65563.200 | .........................................................**.*******X | 66839.700 | | 65771.000 | .........................................................****************X | 66215.800 | | 66215.800 | .........................................................****X | 66215.800 | | 66215.800 | .........................................................**X | 66265.600 | | 66265.600 | .........................................................*X | 66265.600 | | 66265.600 | .........................................................**X | 66265.600 | | 66265.600 | .........................................................*X | 66265.600 | | 65835.600 | .........................................................**************X | 66195.000 | | 64015.000 | .......................................................*****X | 64015.000 | | 64015.000 | .......................................................**...X | 67263.600 | | 67263.600 | ..........................................................*****X | 67263.600 | | 67263.600 | ..........................................................*.*******X | 67682.800 | | 65959.500 | .........................................................****X | 65959.500 | | 65959.500 | .........................................................***.*******X | 67313.500 | | 62986.500 | ......................................................********.********.*******X | 64903.900 | | 63638.800 | .......................................................***************...********...*******X | 70581.400 | | 63790.800 | .......................................................***************X | 64112.600 | | 61566.600 | .....................................................********..*******..*******X | 66132.800 | | 56789.900 | .................................................********.....*******X | 62101.800 | | 60378.400 | ....................................................*******X | 60378.400 | | 56984.200 | .................................................*******..*..******X | 61313.500 | | 58057.400 | ..................................................***************X | 58180.500 | | 57842.500 | ..................................................********..**************X | 59858.800 | | 59195.300 | ...................................................********.******X | 59881.900 | | 57030.400 | .................................................********...*******X | 60150.300 | | 58513.300 | ..................................................*******X | 58513.300 | | 58137.200 | ..................................................********....*******X | 62864.300 | | 61470.900 | .....................................................******X | 61470.900 | | 61448.200 | .....................................................*******X | 61448.200 | | 57625.000 | ..................................................********..*******X | 60356.400 | | 56529.200 | .................................................********....*******X | 60990.000 | | 57875.200 | ..................................................********.******X | 59190.700 | | 57738.400 | ..................................................*******X | 57738.400 | | 56236.500 | ................................................********...*******X | 58713.300 | | 57660.800 | ..................................................********.*******..***************..*******X | 63521.800 | | 61793.400 | .....................................................*******X | 61793.400 | +------------+---------------------------------------------------------------------------------------------------------------------------+------------+ 2022-03-06 05:27:53,661 | test | INFO | pool-3-thread-12 | [table_view:display:72] Ops trend for bucket 'bucket2' +---------+----------------------------------------------------------------------------------------------------------------------------------+---------+ | Min | Trend | Max | +---------+----------------------------------------------------------------------------------------------------------------------------------+---------+ | 0.000 | *****...*..******X | 32.000 | | 0.000 | ********.....*******......................................................................................*******X | 532.000 | | 0.000 | *********************.....*******..*.*******.............................................................................******X | 500.000 | | 84.000 | .............********..........................................................................******X | 516.000 | | 16.000 | .***************...............................................................................................******X | 568.000 | | 0.000 | *******.....*******X | 32.000 | | 0.000 | *******...........................................................................................*******X | 532.000 | | 64.000 | .........******X | 64.000 | | 0.000 | *******.....*...******X | 48.000 | | 0.000 | *******....................................................................................................******X | 584.000 | | 51.500 | .......*******X | 51.500 | | 32.500 | ....*******...******X | 52.000 | | 16.000 | .*******...............................*...............................................................******X | 564.000 | | 68.000 | ..........******X | 68.000 | | 0.000 | ********...........******X | 68.000 | | 16.000 | .******X | 16.000 | | 0.000 | *******.............*******X | 80.000 | | 16.000 | .******X | 16.000 | | 0.000 | ***************........*******X | 48.000 | | 0.000 | *******..******X | 16.000 | | 0.000 | ********..........*******...*******X | 80.000 | | 32.000 | ....******X | 32.000 | | 0.000 | *******..********........******X | 64.000 | | 0.000 | *******.*.******X | 16.000 | | 0.000 | *******...............*******X | 88.000 | | 16.000 | .**************...*******...*******X | 48.000 | | 20.000 | ..******X | 20.000 | | 16.000 | .*******...*******X | 32.000 | | 0.000 | **************..........********.......................................................................................******X | 572.000 | | 100.000 | ................******X | 100.000 | | 48.000 | .......********..........******X | 108.000 | | 0.000 | ***********************..........******X | 64.000 | | 0.000 | ********.....******X | 32.000 | | 16.000 | .********....*******.................******X | 136.000 | | 0.000 | ***************.....*******X | 32.000 | | 16.000 | .******X | 16.000 | | 0.000 | ***************.....******X | 32.000 | | 16.000 | .**************X | 16.000 | | 0.000 | *******...........********...******X | 84.000 | | 0.000 | ********......******X | 40.000 | | 16.000 | .***************......*******X | 52.000 | | 0.000 | *******.....******X | 32.000 | | 0.000 | ********...******X | 20.000 | | 0.000 | ***************..*******X | 16.000 | | 0.000 | ***************........******X | 52.000 | | 0.000 | *******...*..******X | 32.000 | | 0.000 | **************************************...******X | 20.000 | | 0.000 | ***********************..*******.............******X | 92.000 | | 20.000 | ..********..******X | 32.000 | | 16.000 | .***************......*******X | 52.000 | | 40.000 | .....******X | 40.000 | | 16.000 | .******X | 16.000 | | 0.000 | *********************************************.............*******X | 80.000 | | 0.000 | **********************..*******X | 16.000 | | 0.000 | ***************************************************************************......*******X | 36.000 | | 16.000 | .******X | 16.000 | | 0.000 | ***************..*******X | 16.000 | | 0.000 | **********************..********....******X | 36.000 | | 0.000 | ********...******X | 20.000 | | 0.000 | ***************...*******X | 20.000 | | 0.000 | *******...*******.....*******X | 48.000 | | 0.000 | **************......***************X | 36.000 | | 0.000 | ********************************************..********.*******.........*******X | 72.000 | | 16.000 | .*******.*******X | 20.000 | | 0.000 | **************************************........******X | 52.000 | | 16.000 | .*******.............***************X | 92.000 | | 16.000 | .******X | 16.000 | | 0.000 | ********...********...******X | 40.000 | | 32.000 | ....*******X | 32.000 | | 0.000 | ****************....................*******X | 120.000 | | 36.000 | .....********..*******X | 52.000 | | 0.000 | ************************............*******X | 72.000 | | 16.000 | .********.********...***************X | 36.000 | | 16.000 | .********......*******X | 52.000 | | 16.000 | .********...*******X | 32.000 | | 20.000 | ..*******X | 20.000 | | 0.000 | ********..*******X | 16.000 | | 0.000 | ********..*******X | 16.000 | | 0.000 | ****************...********.....*******X | 52.000 | | 16.000 | .*******X | 16.000 | | 0.000 | ********.........********........*******X | 100.000 | | 72.000 | ...........*******X | 72.000 | | 0.000 | ********........******X | 52.000 | | 32.000 | ....********..........*******X | 92.000 | | 36.000 | .....*******X | 36.000 | | 0.000 | ********..*******X | 16.000 | | 0.000 | ********.....*******X | 32.000 | | 0.000 | ***************......****************..********......*******X | 84.000 | | 0.000 | ********...*******X | 20.000 | | 0.000 | ****************..********...*******X | 32.000 | | 0.000 | ********......********..*******X | 52.000 | | 16.000 | .********...........*******X | 80.000 | | 16.000 | .********....*******X | 36.000 | | 32.000 | ....**************X | 32.000 | | 0.000 | ***********************X | 0.000 | +---------+----------------------------------------------------------------------------------------------------------------------------------+---------+ 2022-03-06 05:27:54,255 | test | INFO | pool-3-thread-9 | [table_view:display:72] Ops trend for bucket 'bucket1' +---------+-------------------------------------------------------------------------------------------------------------------------------------------+----------+ | Min | Trend | Max | +---------+-------------------------------------------------------------------------------------------------------------------------------------------+----------+ | 0.000 | *****..............*******X | 148.000 | | 0.000 | ********.....................................................******X | 548.000 | | 0.000 | ***************.*******..*******.................................................................................................*******X | 1016.000 | | 568.000 | ......................................................******X | 568.000 | | 0.000 | ***************..*....*****X | 64.000 | | 32.000 | ..*******X | 32.000 | | 16.000 | *******....................................................*******X | 548.000 | | 36.000 | ..******X | 36.000 | | 0.000 | ***************............******X | 128.000 | | 36.000 | ..******X | 36.000 | | 0.000 | ***************...********.........******X | 128.000 | | 0.000 | ***************....*******..******X | 64.000 | | 32.000 | ..********.******X | 48.000 | | 0.000 | *******.....*******X | 52.000 | | 48.000 | ...******X | 48.000 | | 36.000 | ..******X | 36.000 | | 0.000 | ***************....******X | 48.000 | | 0.000 | **************************************.....*******......******X | 120.000 | | 0.000 | ********.*******................................................******X | 500.000 | | 0.000 | ***************.********..******X | 32.000 | | 16.000 | ***************............******X | 140.000 | | 16.000 | ******X | 16.000 | | 0.000 | ********.******X | 16.000 | | 0.000 | *******.********..*******...******X | 68.000 | | 16.000 | ********..******X | 36.000 | | 16.000 | *******...*******X | 48.000 | | 0.000 | *****************************........*******X | 84.000 | | 36.000 | ..******X | 36.000 | | 0.000 | ********.******X | 16.000 | | 0.000 | ********.******X | 16.000 | | 0.000 | *****************************.*******X | 16.000 | | 0.000 | *******...*******X | 32.000 | | 0.000 | *******..........*******X | 108.000 | | 0.000 | *******...*******...*.......******X | 140.000 | | 16.000 | ******X | 16.000 | | 0.000 | ****************.*******..*******X | 32.000 | | 0.000 | *****************************......*******...*****X | 100.000 | | 100.000 | ........*X | 100.000 | | 68.000 | .....******X | 68.000 | | 0.000 | ***************.******X | 20.000 | | 0.000 | ********...*******X | 32.000 | | 0.000 | ***************.******X | 16.000 | | 0.000 | ***************...*******X | 32.000 | | 0.000 | ******************************...******X | 32.000 | | 0.000 | ***************.******************************.....******X | 64.000 | | 20.000 | *******..*******X | 40.000 | | 0.000 | *************************************.********..******X | 36.000 | | 0.000 | ********.....*******....*******X | 100.000 | | 0.000 | ***********************************************************************************...*******........******X | 112.000 | | 0.000 | ********************************************************************.******X | 16.000 | | 0.000 | **********************......................*******X | 224.000 | | 16.000 | **************X | 20.000 | | 0.000 | *******.*******X | 16.000 | | 0.000 | *******...*******X | 36.000 | | 16.000 | ******X | 16.000 | | 5.100 | X | 5.100 | | 0.000 | *******.******X | 16.000 | | 0.000 | ********...******X | 36.000 | | 16.000 | ********...............******X | 168.000 | | 111.200 | .........X | 111.200 | | 0.000 | *****************************************************.******X | 20.000 | | 16.000 | ********..........******X | 116.000 | | 83.000 | .......X | 83.000 | | 16.000 | ******X | 16.000 | | 0.000 | ****************.***************........*******X | 100.000 | | 16.000 | ********....*******X | 52.000 | | 0.000 | ****************....*******X | 48.000 | | 0.000 | ********.*******X | 16.000 | | 0.000 | ********...............*******X | 156.000 | | 20.000 | *******X | 20.000 | | 16.000 | ****************..***************X | 36.000 | | 16.000 | *******X | 16.000 | | 0.000 | ********.....*******X | 56.000 | | 20.000 | ****************......*******X | 72.000 | | 20.000 | ****************..******X | 32.000 | | 21.800 | .X | 21.800 | | 0.000 | *******.****************....*******X | 52.000 | | 0.000 | ********...*******X | 32.000 | | 20.000 | ********........*******X | 92.000 | | 0.000 | ********.********....********...*******X | 88.000 | | 32.000 | ..*******X | 32.000 | | 0.000 | ********...**********************X | 32.000 | | 0.000 | ****************.........*******X | 100.000 | | 16.000 | *******X | 16.000 | | 0.000 | ********........*******X | 88.000 | | 0.000 | ********.**************X | 20.000 | | 0.000 | ********.*******************************X | 20.000 | | 16.000 | ********..********.......*******X | 104.000 | | 0.000 | ********...********.*******X | 48.000 | | 16.000 | *******X | 16.000 | | 0.000 | ***********************X | 0.000 | +---------+-------------------------------------------------------------------------------------------------------------------------------------------+----------+ 2022-03-06 05:27:55,520 | test | INFO | pool-3-thread-20 | [table_view:display:72] Ops trend for bucket 'default' +-----------+------------------------------------------------------------------------------------------------------+-----------+ | Min | Trend | Max | +-----------+------------------------------------------------------------------------------------------------------+-----------+ | 35472.600 | ...................................................................................................X | 35472.600 | +-----------+------------------------------------------------------------------------------------------------------+-----------+ 2022-03-06 05:27:56,835 | test | WARNING | MainThread | [rest_client:get_nodes:1896] 172.23.100.35 - Node not part of cluster inactiveFailed 2022-03-06 05:28:11,473 | test | INFO | MainThread | [bucket_ready_functions:perform_tasks_from_spec:4993] Performing scope/collection specific operations 2022-03-06 05:28:11,579 | test | INFO | pool-3-thread-1 | [table_view:display:72] Rebalance Overview +----------------+----------+-----------------------+---------------+--------------+-----------------------+ | Nodes | Services | Version | CPU | Status | Membership / Recovery | +----------------+----------+-----------------------+---------------+--------------+-----------------------+ | 172.23.105.206 | kv | 7.1.0-2440-enterprise | 41.9606854839 | Cluster node | active / none | | 172.23.106.177 | kv | 7.1.0-2440-enterprise | 26.31843927 | Cluster node | active / none | | 172.23.100.35 | kv | 7.1.0-2440-enterprise | 0.1627644923 | Cluster node | inactiveAdded / full | | 172.23.105.164 | kv | 7.1.0-2440-enterprise | 66.9031032299 | Cluster node | active / none | +----------------+----------+-----------------------+---------------+--------------+-----------------------+ 2022-03-06 05:28:11,598 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 05:28:15,700 | test | INFO | MainThread | [bucket_ready_functions:perform_tasks_from_spec:5083] Done Performing scope/collection specific operations 2022-03-06 05:28:21,628 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 05:28:31,670 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 05:28:41,730 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 05:28:51,767 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 05:29:01,825 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 05:29:11,855 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 05:29:21,891 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 05:29:31,934 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 05:29:41,969 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 05:29:52,002 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 05:30:02,032 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 05:30:12,072 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 05:30:22,109 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 05:30:32,141 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 05:30:42,167 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 05:30:52,194 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 05:31:02,226 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 05:31:12,266 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 05:31:22,299 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 05:31:32,341 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 05:31:42,380 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 0.07 2022-03-06 05:31:52,411 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 0.07 2022-03-06 05:32:02,464 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 0.07 2022-03-06 05:32:12,503 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 0.07 2022-03-06 05:32:22,542 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 0.07 2022-03-06 05:32:32,582 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 0.07 2022-03-06 05:32:42,617 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 0.07 2022-03-06 05:32:52,657 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 0.07 2022-03-06 05:33:02,691 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 0.07 2022-03-06 05:33:12,739 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 0.07 2022-03-06 05:33:22,779 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 0.07 2022-03-06 05:33:32,826 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 0.07 2022-03-06 05:33:42,891 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 0.07 2022-03-06 05:33:52,930 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 0.07 2022-03-06 05:34:02,970 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 0.07 2022-03-06 05:34:13,029 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 0.07 2022-03-06 05:34:23,065 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 0.07 2022-03-06 05:34:33,092 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 0.07 2022-03-06 05:34:43,138 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 0.07 2022-03-06 05:34:53,170 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 0.07 2022-03-06 05:35:03,206 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 0.07 2022-03-06 05:35:13,250 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 0.07 2022-03-06 05:35:23,293 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 0.07 2022-03-06 05:35:33,326 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 0.07 2022-03-06 05:35:43,377 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 0.07 2022-03-06 05:35:53,420 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 0.07 2022-03-06 05:36:03,453 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 0.07 2022-03-06 05:36:13,490 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 0.07 2022-03-06 05:36:23,526 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 0.07 2022-03-06 05:36:33,563 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 0.07 2022-03-06 05:36:43,602 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 0.07 2022-03-06 05:36:53,640 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 0.07 2022-03-06 05:37:03,680 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 0.07 2022-03-06 05:37:13,723 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 0.07 2022-03-06 05:37:23,763 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 0.07 2022-03-06 05:37:33,811 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 0.07 2022-03-06 05:37:43,849 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 0.07 2022-03-06 05:37:53,884 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 0.07 2022-03-06 05:38:03,934 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 0.07 2022-03-06 05:38:13,977 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 0.07 2022-03-06 05:38:24,019 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 0.07 2022-03-06 05:38:34,071 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 0.07 2022-03-06 05:38:44,111 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 0.07 2022-03-06 05:38:54,153 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 0.07 2022-03-06 05:39:04,191 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 0.07 2022-03-06 05:39:14,243 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 0.07 2022-03-06 05:39:24,288 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 0.07 2022-03-06 05:39:34,348 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 0.07 2022-03-06 05:39:44,388 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 0.07 2022-03-06 05:39:54,437 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 0.07 2022-03-06 05:40:04,473 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 0.07 2022-03-06 05:40:14,526 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 0.07 2022-03-06 05:40:24,559 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 0.07 2022-03-06 05:40:34,611 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 0.07 2022-03-06 05:40:44,655 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 0.07 2022-03-06 05:40:54,700 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 0.07 2022-03-06 05:41:04,742 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 0.07 2022-03-06 05:41:14,773 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 0.07 2022-03-06 05:41:24,809 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 0.07 2022-03-06 05:41:34,852 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 0.13 2022-03-06 05:41:44,890 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 0.13 2022-03-06 05:41:54,930 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 0.13 2022-03-06 05:42:04,973 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 0.13 2022-03-06 05:42:15,003 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 0.13 2022-03-06 05:42:25,038 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 0.13 2022-03-06 05:42:35,072 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 0.13 2022-03-06 05:42:45,118 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 0.13 2022-03-06 05:42:55,178 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 0.13 2022-03-06 05:43:05,220 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 0.13 2022-03-06 05:43:15,265 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 0.13 2022-03-06 05:43:25,305 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 0.13 2022-03-06 05:43:35,336 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 0.13 2022-03-06 05:43:45,368 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 0.13 2022-03-06 05:43:55,394 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 0.13 2022-03-06 05:44:05,430 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 0.13 2022-03-06 05:44:15,460 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 0.13 2022-03-06 05:44:25,503 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 0.13 2022-03-06 05:44:35,542 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 0.13 2022-03-06 05:44:45,569 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 0.13 2022-03-06 05:44:55,605 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 0.13 2022-03-06 05:45:05,634 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 0.13 2022-03-06 05:45:15,670 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 0.13 2022-03-06 05:45:25,703 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 0.13 2022-03-06 05:45:35,755 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 0.13 2022-03-06 05:45:45,789 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 0.13 2022-03-06 05:45:55,823 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 0.13 2022-03-06 05:46:05,861 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 2.61 2022-03-06 05:46:15,907 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 9.47 2022-03-06 05:46:25,943 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 9.47 2022-03-06 05:46:35,989 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 9.47 2022-03-06 05:46:46,023 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 9.47 2022-03-06 05:46:56,052 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 15.16 2022-03-06 05:47:06,082 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 33.33 2022-03-06 05:47:16,119 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 49.22 2022-03-06 05:47:26,154 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 66.67 2022-03-06 05:47:36,200 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 66.67 2022-03-06 05:47:46,234 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 67.06 2022-03-06 05:47:56,278 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 67.32 2022-03-06 05:48:06,335 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 67.71 2022-03-06 05:48:16,385 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 67.9 2022-03-06 05:48:26,433 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 68.23 2022-03-06 05:48:36,474 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 68.55 2022-03-06 05:48:46,512 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 68.88 2022-03-06 05:48:56,553 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 69.27 2022-03-06 05:49:06,598 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 69.4 2022-03-06 05:49:16,634 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 69.79 2022-03-06 05:49:26,667 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 70.05 2022-03-06 05:49:36,701 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 70.44 2022-03-06 05:49:46,739 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 70.83 2022-03-06 05:49:56,779 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 70.96 2022-03-06 05:50:06,811 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 71.35 2022-03-06 05:50:16,844 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 71.61 2022-03-06 05:50:26,884 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 71.94 2022-03-06 05:50:36,915 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 72.27 2022-03-06 05:50:46,956 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 72.53 2022-03-06 05:50:56,993 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 72.79 2022-03-06 05:51:07,039 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 73.18 2022-03-06 05:51:17,085 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 73.44 2022-03-06 05:51:27,121 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 73.76 2022-03-06 05:51:37,161 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 73.96 2022-03-06 05:51:47,198 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 74.15 2022-03-06 05:51:57,239 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 74.55 2022-03-06 05:52:07,289 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 74.81 2022-03-06 05:52:17,328 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 75.07 2022-03-06 05:52:27,377 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 75.39 2022-03-06 05:52:37,423 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 75.59 2022-03-06 05:52:47,450 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 75.85 2022-03-06 05:52:57,493 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 76.11 2022-03-06 05:53:07,532 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 76.37 2022-03-06 05:53:17,573 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 76.69 2022-03-06 05:53:27,611 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 76.95 2022-03-06 05:53:37,644 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 77.15 2022-03-06 05:53:47,677 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 77.28 2022-03-06 05:53:57,711 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 77.61 2022-03-06 05:54:07,762 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 77.93 2022-03-06 05:54:17,802 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 78.19 2022-03-06 05:54:27,851 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 78.45 2022-03-06 05:54:37,884 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 78.58 2022-03-06 05:54:47,914 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 78.84 2022-03-06 05:54:57,947 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 79.1 2022-03-06 05:55:07,982 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 79.36 2022-03-06 05:55:18,010 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 79.56 2022-03-06 05:55:28,053 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 79.82 2022-03-06 05:55:38,084 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 80.02 2022-03-06 05:55:48,118 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 80.15 2022-03-06 05:55:58,164 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 80.41 2022-03-06 05:56:08,209 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 80.67 2022-03-06 05:56:18,263 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 80.86 2022-03-06 05:56:28,305 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 81.12 2022-03-06 05:56:38,351 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 81.32 2022-03-06 05:56:48,384 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 81.58 2022-03-06 05:56:58,421 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 81.71 2022-03-06 05:57:08,457 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 81.9 2022-03-06 05:57:18,500 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 82.16 2022-03-06 05:57:28,548 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 82.36 2022-03-06 05:57:38,591 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 82.62 2022-03-06 05:57:48,635 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 82.82 2022-03-06 05:57:58,681 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 83.01 2022-03-06 05:58:08,713 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 83.27 2022-03-06 05:58:18,744 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 83.34 2022-03-06 05:58:28,792 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 83.53 2022-03-06 05:58:38,831 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 83.79 2022-03-06 05:58:48,861 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 83.99 2022-03-06 05:58:58,895 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 84.12 2022-03-06 05:59:04,921 | test | INFO | pool-3-thread-30 | [table_view:display:72] Ops trend for bucket 'default' +------------+------------------------------------------------------------------------------------------------------------------------+------------+ | Min | Trend | Max | +------------+------------------------------------------------------------------------------------------------------------------------+------------+ | 0.000 | ***....................................********................................................................******X | 106383.500 | | 100606.600 | .............................................................................................*******X | 100606.600 | | 70288.900 | .................................................................*******X | 70288.900 | | 52360.200 | ................................................******X | 52360.200 | | 49169.200 | .............................................*******X | 49169.200 | | 40673.300 | .....................................********.******X | 42176.800 | | 36865.200 | .................................******X | 36865.200 | | 34873.600 | ...............................X | 34873.600 | | 33866.200 | ..............................******X | 33866.200 | | 30364.900 | ...........................*******X | 30364.900 | | 25132.100 | ......................*******.*******X | 26117.500 | | 23478.300 | .....................******X | 23478.300 | | 22308.400 | ...................********..........********.*******..********..........*******..*******X | 48284.700 | | 43367.300 | .......................................*******..........********...******X | 57256.200 | | 53468.600 | .................................................******X | 53468.600 | | 50320.200 | ..............................................X | 50320.200 | | 48720.900 | ............................................******X | 48720.900 | | 46228.300 | ..........................................*******.******X | 47724.100 | | 46878.100 | ...........................................*******X | 46878.100 | | 44585.600 | ........................................******X | 44585.600 | | 40990.400 | .....................................X | 40990.400 | | 39285.600 | ...................................*******...*******...********...*******..*******.*X | 51726.500 | | 51726.500 | ...............................................******.******X | 52505.900 | | 51675.700 | ...............................................******X | 51675.700 | | 42862.000 | .......................................*******......*******.......********..X | 59429.900 | | 59429.900 | ......................................................X | 59429.900 | | 59429.900 | ......................................................**X | 59429.900 | | 59429.900 | ......................................................**...****X | 62282.400 | | 62282.400 | .........................................................**X | 62282.400 | | 50709.300 | ..............................................*******...*******X | 53403.500 | | 53263.400 | .................................................*******..***X | 55700.900 | | 55700.900 | ...................................................*X | 55700.900 | | 55700.900 | ...................................................X | 55700.900 | | 55033.500 | ..................................................**************X | 55048.300 | | 50866.100 | ..............................................*******..******X | 52263.400 | | 51343.500 | ...............................................*****X | 51343.500 | | 51343.500 | ...............................................*...***************...******X | 57617.700 | | 55801.800 | ...................................................********..*******..******X | 59999.100 | | 56262.500 | ...................................................*******X | 56262.500 | | 52079.600 | ...............................................X | 52079.600 | | 52079.600 | ...............................................*****X | 52079.600 | | 50121.100 | ..............................................*******.*******X | 51955.200 | | 51150.500 | ...............................................*******.*******..********...*******..****X | 60334.400 | | 60334.400 | .......................................................**..*.******X | 63780.200 | | 63709.600 | ..........................................................*X | 63709.600 | | 63709.600 | ..........................................................**X | 63709.600 | | 63709.600 | ..........................................................*X | 63709.600 | | 61586.300 | ........................................................******X | 61586.300 | | 49592.800 | .............................................******X | 49592.800 | | 41787.100 | ......................................*X | 41787.100 | | 41787.100 | ......................................*X | 41787.100 | | 41787.100 | ......................................***.******X | 42837.400 | | 40059.000 | ....................................*******X | 40059.000 | | 39284.000 | ...................................******X | 39284.000 | | 35765.300 | ................................******X | 35765.300 | | 33541.400 | ..............................********....******X | 37713.500 | | 36543.800 | .................................********..******X | 38696.400 | | 37094.700 | .................................********...**************X | 40344.600 | | 34216.000 | ...............................*******...******X | 38154.800 | | 37076.600 | .................................X | 37076.600 | | 37046.900 | .................................******X | 37046.900 | | 35538.600 | ................................*X | 35538.600 | | 35538.600 | ................................*****....******X | 39878.300 | | 39206.300 | ...................................*******X | 39206.300 | | 36321.800 | .................................******X | 36321.800 | | 34205.500 | ...............................********...******X | 38047.300 | | 33290.800 | ..............................*******...*******X | 36751.300 | | 33958.500 | ..............................*******..******X | 35759.900 | | 34246.700 | ...............................******X | 34246.700 | | 30639.800 | ...........................********....******X | 34503.900 | | 33179.800 | ..............................******X | 33179.800 | | 29951.200 | ...........................********.*******..******X | 33215.800 | | 32957.800 | .............................********.******X | 33420.900 | | 31621.800 | ............................******X | 31621.800 | | 29810.000 | ...........................******X | 29810.000 | | 29258.300 | ..........................*******.*******X | 30606.000 | | 29790.100 | ...........................*********************X | 30649.500 | | 30635.100 | ...........................******X | 30635.100 | | 26751.200 | ........................*******...*........*******.................**X | 57402.000 | | 57402.000 | ....................................................*****.********.....*******X | 63049.100 | | 56599.000 | ....................................................******X | 56599.000 | | 54171.700 | .................................................********....*******X | 57984.200 | | 56895.700 | ....................................................*******...*******X | 60116.600 | | 55506.900 | ...................................................*******X | 55506.900 | | 54914.500 | ..................................................********.....********..*******X | 62667.900 | | 55483.800 | ...................................................*******...********.*******X | 60374.900 | | 56542.100 | ....................................................*******X | 56542.100 | | 54297.700 | ..................................................********.********..******X | 58347.800 | | 56078.100 | ...................................................********.********..*******X | 58916.000 | | 55258.200 | ..................................................********.....*******X | 60498.600 | | 51608.700 | ...............................................********.......******X | 59460.700 | | 59375.000 | ......................................................*******X | 59375.000 | | 59065.700 | ......................................................*******X | 59065.700 | | 58896.300 | ......................................................********.*******X | 59883.300 | | 56597.800 | ....................................................********..*******.*******X | 59635.600 | | 55429.700 | ...................................................********..*******..********.*******X | 61015.500 | | 60543.300 | .......................................................*******X | 60543.300 | | 59760.600 | .......................................................*******X | 59760.600 | | 58656.000 | ......................................................*******...*******X | 61793.300 | | 57327.500 | ....................................................********.********..******X | 60634.900 | | 57124.100 | ....................................................*******X | 57124.100 | | 55909.700 | ...................................................********....******X | 60608.200 | | 58716.100 | ......................................................********.********..******X | 62109.000 | | 61588.800 | ........................................................X | 61588.800 | | 60671.500 | ........................................................******X | 60671.500 | | 56339.700 | ...................................................********.....*******X | 61023.600 | | 57925.900 | .....................................................*******.......*******X | 64971.800 | | 59836.200 | .......................................................********...*******..*******X | 64978.100 | | 60708.200 | ........................................................********..******X | 63454.700 | | 62656.100 | .........................................................*******X | 62656.100 | | 59285.600 | ......................................................********.*******X | 59821.900 | | 56564.400 | ....................................................*******....*******X | 61256.800 | | 55640.600 | ...................................................*******....*******X | 59755.100 | | 56148.200 | ...................................................********....******X | 59992.400 | | 55398.700 | ...................................................*******X | 55398.700 | | 45320.700 | .........................................X | 45320.700 | +------------+------------------------------------------------------------------------------------------------------------------------+------------+ 2022-03-06 05:59:06,157 | test | INFO | pool-3-thread-6 | [table_view:display:72] Ops trend for bucket 'bucket2' +---------+-----------------------------------------------------------------------------------------------------------------+----------+ | Min | Trend | Max | +---------+-----------------------------------------------------------------------------------------------------------------+----------+ | 0.000 | ***....................................................................................................*******X | 1183.600 | | 0.000 | ********.....................................................******X | 637.700 | | 0.000 | **************************************.........................*******X | 297.700 | | 0.000 | ***************........*******X | 95.300 | | 0.000 | ********.******X | 15.300 | | 0.000 | ****************.*******..........*******X | 141.200 | | 30.600 | .*******................................................********........******X | 687.100 | | 500.000 | .........................................*******X | 500.000 | | 0.000 | ***************........******X | 95.700 | | 15.300 | ********....******X | 61.200 | | 0.000 | ***************..........*******X | 122.400 | | 84.300 | ......X | 84.300 | | 84.300 | ......*X | 84.300 | | 84.300 | ......*X | 84.300 | | 84.300 | ......*X | 84.300 | | 15.300 | ********...........................................................******X | 721.200 | | 76.500 | .....******X | 76.500 | | 61.200 | ....******X | 61.200 | | 39.200 | ..X | 39.200 | | 0.000 | ************************************.*******X | 15.300 | | 0.000 | *******...............................................*******X | 561.200 | | 34.500 | .**************.******X | 45.900 | | 14.700 | X | 14.700 | | 0.000 | **************....*******X | 49.800 | | 34.500 | .******X | 34.500 | | 0.000 | *************************************.*******...............*******X | 191.000 | | 160.400 | ............******X | 160.400 | | 30.600 | .******X | 30.600 | | 0.000 | ******************************.*******...*******X | 49.800 | | 0.000 | *******............*******X | 149.400 | | 0.000 | *********************.*******X | 15.300 | | 0.000 | *******........********....******X | 149.800 | | 19.200 | ******X | 19.200 | | 0.000 | ********.*******...........*******X | 149.800 | | 65.100 | ....*******X | 65.100 | | 0.000 | *******.******X | 15.300 | | 4.900 | X | 4.900 | | 0.000 | *************************************.*******.*******X | 34.500 | | 0.000 | ********................******X | 194.900 | | 15.300 | *******X | 15.300 | | 0.000 | *******..*******X | 30.600 | | 0.000 | **************.******X | 15.300 | | 0.000 | ********.********......******X | 84.300 | | 0.000 | ********.............................******X | 352.500 | | 0.000 | ***************.********..........******X | 134.100 | | 0.000 | *******.*******X | 15.300 | | 0.000 | ***************..******X | 34.500 | | 0.000 | *******************************..*******X | 31.000 | | 0.000 | ********....******X | 53.700 | | 34.500 | .********..........................*******X | 336.400 | | 15.300 | *******X | 15.300 | | 0.000 | ***********************..*******X | 30.600 | | 15.300 | ****************.......*******X | 100.000 | | 84.300 | ......*******X | 84.300 | | 0.000 | *******..*******X | 34.500 | | 0.000 | ********................*******X | 194.500 | | 15.300 | ********...******X | 49.800 | | 19.200 | ********..*******X | 38.400 | | 0.000 | ************************.*******X | 19.200 | | 0.000 | ********...******X | 38.400 | | 15.300 | ********...........*******X | 149.800 | | 95.700 | .......*******X | 95.700 | | 0.000 | ***************.........*******X | 115.300 | | 0.000 | ********.***************......*******X | 92.100 | | 0.000 | ********************************.......*******X | 88.200 | | 65.100 | ....**************X | 69.000 | | 0.000 | ********...********......*******X | 114.500 | | 80.000 | .....*******X | 80.000 | | 69.000 | ....*******X | 69.000 | | 19.200 | *******...*******X | 49.800 | | 30.600 | .*******X | 30.600 | | 0.000 | ***************.***************.....*******X | 80.400 | | 49.800 | ...*******X | 49.800 | | 34.500 | .********.......*******X | 110.600 | | 15.300 | *******X | 15.300 | | 0.000 | *******..********.....*******X | 93.000 | | 60.000 | ....******X | 60.000 | | 15.300 | *******X | 15.300 | | 0.000 | ************************X | 0.000 | +---------+-----------------------------------------------------------------------------------------------------------------+----------+ 2022-03-06 05:59:06,352 | test | INFO | pool-3-thread-4 | [table_view:display:72] Ops trend for bucket 'bucket1' +---------+------------------------------------------------------------------------------------------------------------------------------------------+----------+ | Min | Trend | Max | +---------+------------------------------------------------------------------------------------------------------------------------------------------+----------+ | 0.000 | ***..............................*******X | 323.100 | | 253.400 | .......................********.......................*******X | 500.000 | | 45.900 | ...******X | 45.900 | | 0.000 | ******************************..*******X | 30.600 | | 15.300 | ******X | 15.300 | | 0.000 | ********.***********************....******X | 61.200 | | 15.300 | *******X | 15.300 | | 0.000 | ********.******X | 15.300 | | 0.000 | ***************.*.******X | 30.600 | | 15.300 | ***************...*******.*******X | 61.200 | | 0.000 | *****************************....................................................................................................******X | 1052.300 | | 154.000 | .............*******X | 154.000 | | 15.300 | ***************......*******....******X | 122.400 | | 0.000 | ***************..******X | 30.600 | | 0.000 | **************.********...................................................******X | 549.800 | | 30.600 | .*******X | 30.600 | | 15.300 | ******X | 15.300 | | 0.000 | *******.******X | 19.200 | | 15.300 | *******X | 15.300 | | 0.000 | *******..*******........*******X | 111.000 | | 15.300 | *******.******X | 30.600 | | 0.000 | ********.******X | 13.800 | | 8.300 | X | 8.300 | | 1.500 | ******...............................................*******X | 500.000 | | 76.500 | ......*******...******X | 114.500 | | 0.000 | **********************....******X | 45.900 | | 0.000 | ********.*******......*******......******X | 141.200 | | 124.000 | ..........X | 124.000 | | 114.900 | .........******X | 114.900 | | 0.000 | *******.******X | 15.300 | | 0.000 | **********************....******X | 49.800 | | 34.500 | ..*******X | 34.500 | | 0.000 | *******.......*******X | 80.000 | | 15.300 | *********************X | 15.300 | | 0.000 | ***************.******X | 15.300 | | 0.000 | *******..*******.********..******X | 61.200 | | 0.000 | *******.********.*******....*******X | 65.100 | | 0.000 | *******.......*******X | 80.000 | | 15.300 | ******X | 15.300 | | 0.000 | **************...******X | 34.500 | | 0.000 | ********.*******.............******X | 149.400 | | 0.000 | ********........******X | 84.300 | | 0.000 | **************.**************X | 15.300 | | 0.000 | *******.**************X | 15.300 | | 0.000 | *******......*******X | 69.000 | | 15.300 | *******..******X | 38.400 | | 0.000 | ***********************.********..*******X | 38.400 | | 15.300 | *******.....................*******X | 238.000 | | 0.000 | ********.....*******X | 53.700 | | 19.200 | ********..********..********.*******X | 65.100 | | 0.000 | ****************..............*******X | 153.300 | | 15.300 | *******...*******X | 49.800 | | 30.600 | .********..................*******X | 218.400 | | 0.000 | ****************..******************************X | 30.600 | | 15.300 | *******X | 15.300 | | 0.000 | ***************.*..*******..********.................*******X | 234.300 | | 187.800 | ................*******X | 187.800 | | 38.400 | ..******X | 38.400 | | 0.000 | ********...*******X | 34.500 | | 0.000 | *******.****************.********..*******X | 45.900 | | 15.300 | ***********************...*******X | 49.800 | | 19.200 | ********..*******X | 34.500 | | 15.300 | ***************..*******X | 38.400 | | 0.000 | ********.....*******X | 61.200 | | 38.400 | ..******X | 38.400 | | 19.200 | ********.........*******X | 111.000 | | 30.600 | .******X | 30.600 | | 0.000 | ********.*******X | 19.200 | | 0.000 | ********.......*******X | 80.400 | | 19.200 | ********..............******X | 168.200 | | 15.300 | *******X | 15.300 | | 0.000 | ********.********.*******..............*******X | 172.200 | | 30.600 | .*******..*******X | 45.900 | | 0.000 | *************************X | 0.000 | +---------+------------------------------------------------------------------------------------------------------------------------------------------+----------+ 2022-03-06 05:59:07,634 | test | INFO | pool-3-thread-23 | [table_view:display:72] Ops trend for bucket 'default' +-----------+------------------------------------------------------------------------------------------------------+-----------+ | Min | Trend | Max | +-----------+------------------------------------------------------------------------------------------------------+-----------+ | 45320.700 | ...................................................................................................X | 45320.700 | +-----------+------------------------------------------------------------------------------------------------------+-----------+ 2022-03-06 05:59:08,924 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 84.44 2022-03-06 05:59:18,956 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 84.84 2022-03-06 05:59:28,983 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 85.16 2022-03-06 05:59:39,009 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 85.62 2022-03-06 05:59:49,035 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 85.94 2022-03-06 05:59:59,061 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 86.33 2022-03-06 06:00:09,086 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 86.73 2022-03-06 06:00:19,115 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 87.12 2022-03-06 06:00:29,140 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 87.51 2022-03-06 06:00:39,165 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 87.83 2022-03-06 06:00:49,193 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 88.22 2022-03-06 06:00:59,217 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 88.55 2022-03-06 06:01:09,244 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 88.94 2022-03-06 06:01:19,272 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 89.33 2022-03-06 06:01:29,296 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 89.72 2022-03-06 06:01:39,325 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 90.18 2022-03-06 06:01:49,354 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 90.5 2022-03-06 06:01:59,380 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 90.89 2022-03-06 06:02:09,404 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 91.28 2022-03-06 06:02:19,430 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 91.67 2022-03-06 06:02:29,456 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 92.06 2022-03-06 06:02:39,482 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 92.46 2022-03-06 06:02:49,506 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 93.04 2022-03-06 06:02:59,532 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 93.43 2022-03-06 06:03:09,558 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 93.82 2022-03-06 06:03:19,582 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 94.21 2022-03-06 06:03:29,608 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 94.6 2022-03-06 06:03:39,634 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 95.0 2022-03-06 06:03:49,661 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 95.39 2022-03-06 06:03:59,687 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 95.78 2022-03-06 06:04:09,714 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 96.17 2022-03-06 06:04:19,740 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 96.69 2022-03-06 06:04:29,766 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 97.08 2022-03-06 06:04:39,792 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 97.4 2022-03-06 06:04:49,818 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 97.66 2022-03-06 06:04:59,842 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 97.92 2022-03-06 06:05:09,868 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 98.25 2022-03-06 06:05:19,894 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 98.57 2022-03-06 06:05:29,920 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 98.83 2022-03-06 06:05:39,947 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 99.09 2022-03-06 06:05:49,971 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 99.35 2022-03-06 06:05:59,997 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 99.48 2022-03-06 06:06:10,023 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 99.68 2022-03-06 06:06:20,049 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: running, progress: 99.74 2022-03-06 06:06:35,084 | test | INFO | pool-3-thread-1 | [task:check:415] Rebalance - status: none, progress: 100 2022-03-06 06:06:35,096 | test | INFO | pool-3-thread-1 | [task:check:474] Rebalance completed with progress: 100% in 2303.51699996 sec 2022-03-06 06:06:35,186 | test | INFO | MainThread | [collections_rebalance:get_zone_info:969] nodes in zone inside get_zone_info():{'Group 1': ['172.23.105.164', '172.23.100.35', '172.23.106.177', '172.23.105.206']} 2022-03-06 06:06:36,957 | test | INFO | MainThread | [bucket_ready_functions:validate_docs_per_collections_all_buckets:4742] Validating collection stats and item counts 2022-03-06 06:06:44,663 | test | ERROR | MainThread | [bucket_ready_functions:validate_manifest_uid:4605] Bucket UID mismatch. Expected: 46, Actual: 2f 2022-03-06 06:06:44,663 | test | WARNING | MainThread | [bucket_ready_functions:validate_docs_per_collections_all_buckets:4759] Bucket manifest UID mismatch! 2022-03-06 06:06:48,209 | test | ERROR | MainThread | [bucket_ready_functions:validate_manifest_uid:4605] Bucket UID mismatch. Expected: 47, Actual: 30 2022-03-06 06:06:48,210 | test | WARNING | MainThread | [bucket_ready_functions:validate_docs_per_collections_all_buckets:4759] Bucket manifest UID mismatch! 2022-03-06 06:06:51,615 | test | ERROR | MainThread | [bucket_ready_functions:validate_manifest_uid:4605] Bucket UID mismatch. Expected: 69, Actual: 46 2022-03-06 06:06:51,615 | test | WARNING | MainThread | [bucket_ready_functions:validate_docs_per_collections_all_buckets:4759] Bucket manifest UID mismatch! 2022-03-06 06:06:56,706 | test | INFO | MainThread | [table_view:display:72] Bucket statistics +---------+-----------+-----------------+----------+------------+-----+-----------+-----------+------------+------------+---------------+ | Bucket | Type | Storage Backend | Replicas | Durability | TTL | Items | RAM Quota | RAM Used | Disk Used | ARR | +---------+-----------+-----------------+----------+------------+-----+-----------+-----------+------------+------------+---------------+ | bucket1 | couchbase | couchstore | 1 | none | 0 | 112554 | 7.81 GiB | 104.50 MiB | 161.76 MiB | 100 | | bucket2 | couchbase | magma | 1 | none | 0 | 142475 | 4.00 GiB | 275.09 MiB | 405.68 MiB | 100 | | default | couchbase | magma | 1 | none | 0 | 126323000 | 4.00 GiB | 2.55 GiB | 36.58 GiB | 8.86635292069 | +---------+-----------+-----------------+----------+------------+-----+-----------+-----------+------------+------------+---------------+ 2022-03-06 06:06:57,118 | test | INFO | MainThread | [collections_base:tearDown:84] Bucket: bucket2, Active Resident ratio(DGM): 100% 2022-03-06 06:06:57,118 | test | INFO | MainThread | [collections_base:tearDown:87] Bucket: bucket2, Replica Resident ratio(DGM): 100% 2022-03-06 06:07:03,405 | test | INFO | MainThread | [collections_base:tearDown:84] Bucket: bucket1, Active Resident ratio(DGM): 100% 2022-03-06 06:07:03,405 | test | INFO | MainThread | [collections_base:tearDown:87] Bucket: bucket1, Replica Resident ratio(DGM): 100% 2022-03-06 06:07:09,023 | test | INFO | MainThread | [collections_base:tearDown:84] Bucket: default, Active Resident ratio(DGM): 2% 2022-03-06 06:07:09,025 | test | INFO | MainThread | [collections_base:tearDown:87] Bucket: default, Replica Resident ratio(DGM): 2% ok ---------------------------------------------------------------------- Ran 1 test in 8846.042s During the test, Remote Connections: 209, Disconnections: 209 SDK Connections: 30, Disconnections: 30 OK summary so far suite bucket_collections.collections_rebalance.CollectionsRebalance , pass 1 , fail 0 testrunner logs, diags and results are available under /data/workspace/temp_rebalance_magma/logs/testrunner-22-Mar-06_03-39-55/test_1 Logs will be stored at /data/workspace/temp_rebalance_magma/logs/testrunner-22-Mar-06_03-39-55/test_2 guides/gradlew --refresh-dependencies testrunner -P jython=/opt/jython/bin/jython -P 'args=-i /tmp/win10-bucket-ops-temp_rebalance_magma.ini rerun=False,get-cbcollect-info=True -t bucket_collections.collections_rebalance.CollectionsRebalance.test_data_load_collections_with_graceful_failover_recovery,nodes_init=4,nodes_failover=1,recovery_type=delta,bucket_spec=magma_dgm.bloom_filter.4_node_1_replica_magma_1024,doc_size=30,randomize_value=True,data_load_stage=during,skip_validations=False,data_load_spec=volume_test_load_with_CRUD_on_collections,change_magma_quota=True,GROUP=failover' Test Input params: {'doc_size': '30', 'data_load_stage': 'during', 'cluster_name': 'win10-bucket-ops-temp_rebalance_magma', 'nodes_failover': '1', 'ini': '/tmp/win10-bucket-ops-temp_rebalance_magma.ini', 'get-cbcollect-info': 'True', 'recovery_type': 'delta', 'conf_file': 'conf/magma/dgm_collections_bloom_filter.conf', 'change_magma_quota': 'True', 'skip_validations': 'False', 'spec': 'dgm_collections_bloom_filter', 'rerun': 'False', 'num_nodes': 7, 'logs_folder': '/data/workspace/temp_rebalance_magma/logs/testrunner-22-Mar-06_03-39-55/test_2', 'nodes_init': '4', 'GROUP': 'failover', 'bucket_spec': 'magma_dgm.bloom_filter.4_node_1_replica_magma_1024', 'case_number': 2, 'randomize_value': 'True', 'data_load_spec': 'volume_test_load_with_CRUD_on_collections'} test_data_load_collections_with_graceful_failover_recovery (bucket_collections.collections_rebalance.CollectionsRebalance) ... 2022-03-06 06:07:24,944 | test | INFO | MainThread | [basetestcase:log_setup_status:647] ========= CollectionsRebalance setup started for test #2 test_data_load_collections_with_graceful_failover_recovery ========= 2022-03-06 06:07:25,030 | test | INFO | MainThread | [basetestcase:log_setup_status:647] ========= BaseTestCase setup started for test #2 test_data_load_collections_with_graceful_failover_recovery ========= 2022-03-06 06:07:56,529 | infra | ERROR | MainThread | [Rest_Connection:_http_request:286] DELETE http://172.23.105.164:8091/pools/default/buckets/bucket2 body: headers: {'Accept': '*/*', 'Connection': 'close', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==', 'Content-Type': 'application/x-www-form-urlencoded'} error: 500 reason: unknown {"_":"Bucket deletion not yet complete, but will continue.\r\n"} auth: Administrator:password 2022-03-06 06:07:56,918 | infra | ERROR | MainThread | [remote_util:log_command_output:3012] wc: /data: Is a directory 2022-03-06 06:07:56,918 | test | ERROR | MainThread | [bucket_ready_functions:delete_bucket:1617] Unable to get timings for bucket: global name 'StatsCommon' is not defined 2022-03-06 06:08:27,078 | infra | ERROR | MainThread | [Rest_Connection:_http_request:286] DELETE http://172.23.105.164:8091/pools/default/buckets/default body: headers: {'Accept': '*/*', 'Connection': 'close', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==', 'Content-Type': 'application/x-www-form-urlencoded'} error: 500 reason: unknown {"_":"Bucket deletion not yet complete, but will continue.\r\n"} auth: Administrator:password 2022-03-06 06:08:27,398 | infra | ERROR | MainThread | [remote_util:log_command_output:3012] wc: /data: Is a directory 2022-03-06 06:08:27,398 | test | ERROR | MainThread | [bucket_ready_functions:delete_bucket:1617] Unable to get timings for bucket: global name 'StatsCommon' is not defined 2022-03-06 06:10:28,543 | test | ERROR | MainThread | [rest_client:_rebalance_status_and_progress:1677] {u'errorMessage': u'Rebalance failed. See logs for detailed reason. You can try again.', u'type': u'rebalance', u'masterRequestTimedOut': False, u'statusId': u'7bfdfe2905af13c5b9d5bbabb8468bc6', u'subtype': u'rebalance', u'statusIsStale': False, u'lastReportURI': u'/logs/rebalanceReport?reportID=854009218c08fe5095036b306d13e409', u'status': u'notRunning'} - rebalance failed 2022-03-06 06:10:28,565 | test | INFO | MainThread | [rest_client:print_UI_logs:2842] Latest logs from UI on 172.23.105.164: 2022-03-06 06:10:28,565 | test | ERROR | MainThread | [rest_client:print_UI_logs:2844] {u'code': 0, u'module': u'ns_orchestrator', u'type': u'critical', u'node': u'ns_1@172.23.105.164', u'tstamp': 1646575827642L, u'shortText': u'message', u'serverTime': u'2022-03-06T06:10:27.642Z', u'text': u'Rebalance exited with reason {buckets_shutdown_wait_failed,\n [{\'ns_1@172.23.105.164\',\n {\'EXIT\',\n {old_buckets_shutdown_wait_failed,\n ["default","bucket2"]}}}]}.\nRebalance Operation Id = e7cdd07c96d27e28ae8715c06235bd8e'} 2022-03-06 06:10:28,566 | test | ERROR | MainThread | [rest_client:print_UI_logs:2844] {u'code': 0, u'module': u'ns_rebalancer', u'type': u'critical', u'node': u'ns_1@172.23.105.164', u'tstamp': 1646575827641L, u'shortText': u'message', u'serverTime': u'2022-03-06T06:10:27.641Z', u'text': u'Failed to wait deletion of some buckets on some nodes: [{\'ns_1@172.23.105.164\',\n {\'EXIT\',\n {old_buckets_shutdown_wait_failed,\n ["default",\n "bucket2"]}}}]\n'} 2022-03-06 06:10:28,566 | test | ERROR | MainThread | [rest_client:print_UI_logs:2844] {u'code': 0, u'module': u'ns_orchestrator', u'type': u'info', u'node': u'ns_1@172.23.105.164', u'tstamp': 1646575707630L, u'shortText': u'message', u'serverTime': u'2022-03-06T06:08:27.630Z', u'text': u"Starting rebalance, KeepNodes = ['ns_1@172.23.105.164'], EjectNodes = ['ns_1@172.23.105.206',\n 'ns_1@172.23.106.177',\n 'ns_1@172.23.100.35'], Failed over and being ejected nodes = []; no delta recovery nodes; Operation Id = e7cdd07c96d27e28ae8715c06235bd8e"} 2022-03-06 06:10:28,566 | test | ERROR | MainThread | [rest_client:print_UI_logs:2844] {u'code': 102, u'module': u'menelaus_web', u'type': u'warning', u'node': u'ns_1@172.23.105.164', u'tstamp': 1646575707622L, u'shortText': u'client-side error report', u'serverTime': u'2022-03-06T06:08:27.622Z', u'text': u'Client-side error-report for user "Administrator" on node \'ns_1@172.23.105.164\':\nUser-Agent:Python-httplib2/$Rev: 259 $\nStarting rebalance from test, ejected nodes [u\'ns_1@172.23.105.206\', u\'ns_1@172.23.106.177\', u\'ns_1@172.23.100.35\']'} 2022-03-06 06:10:28,566 | test | ERROR | MainThread | [rest_client:print_UI_logs:2844] {u'code': 0, u'module': u'ns_memcached', u'type': u'info', u'node': u'ns_1@172.23.100.35', u'tstamp': 1646575646844L, u'shortText': u'message', u'serverTime': u'2022-03-06T06:07:26.844Z', u'text': u'Shutting down bucket "bucket2" on \'ns_1@172.23.100.35\' for deletion'} 2022-03-06 06:10:28,568 | test | ERROR | MainThread | [rest_client:print_UI_logs:2844] {u'code': 0, u'module': u'ns_memcached', u'type': u'info', u'node': u'ns_1@172.23.105.164', u'tstamp': 1646575646582L, u'shortText': u'message', u'serverTime': u'2022-03-06T06:07:26.582Z', u'text': u'Shutting down bucket "bucket2" on \'ns_1@172.23.105.164\' for deletion'} 2022-03-06 06:10:28,568 | test | ERROR | MainThread | [rest_client:print_UI_logs:2844] {u'code': 0, u'module': u'ns_memcached', u'type': u'info', u'node': u'ns_1@172.23.106.177', u'tstamp': 1646575646574L, u'shortText': u'message', u'serverTime': u'2022-03-06T06:07:26.574Z', u'text': u'Shutting down bucket "bucket2" on \'ns_1@172.23.106.177\' for deletion'} 2022-03-06 06:10:28,568 | test | ERROR | MainThread | [rest_client:print_UI_logs:2844] {u'code': 0, u'module': u'ns_memcached', u'type': u'info', u'node': u'ns_1@172.23.105.206', u'tstamp': 1646575646572L, u'shortText': u'message', u'serverTime': u'2022-03-06T06:07:26.572Z', u'text': u'Shutting down bucket "bucket2" on \'ns_1@172.23.105.206\' for deletion'} 2022-03-06 06:10:28,568 | test | ERROR | MainThread | [rest_client:print_UI_logs:2844] {u'code': 11, u'module': u'menelaus_web', u'type': u'info', u'node': u'ns_1@172.23.105.164', u'tstamp': 1646575646403L, u'shortText': u'message', u'serverTime': u'2022-03-06T06:07:26.403Z', u'text': u'Deleted bucket "bucket1"\n'} 2022-03-06 06:10:28,569 | test | ERROR | MainThread | [rest_client:print_UI_logs:2844] {u'code': 0, u'module': u'ns_memcached', u'type': u'info', u'node': u'ns_1@172.23.100.35', u'tstamp': 1646575645398L, u'shortText': u'message', u'serverTime': u'2022-03-06T06:07:25.398Z', u'text': u'Shutting down bucket "bucket1" on \'ns_1@172.23.100.35\' for deletion'} Traceback (most recent call last): File "pytests/basetestcase.py", line 363, in setUp self.cluster_util.cluster_cleanup(cluster, File "couchbase_utils/cluster_utils/cluster_ready_functions.py", line 169, in cluster_cleanup self.cleanup_cluster(cluster, master=cluster.master) File "couchbase_utils/cluster_utils/cluster_ready_functions.py", line 222, in cleanup_cluster ejectedNodes=[node.id for node in nodes File "lib/membase/api/rest_client.py", line 143, in remove_nodes return self.rest.monitorRebalance() File "lib/membase/api/rest_client.py", line 1580, in monitorRebalance progress = self._rebalance_progress() File "lib/membase/api/rest_client.py", line 1717, in _rebalance_progress return self._rebalance_status_and_progress()[1] File "lib/membase/api/rest_client.py", line 1679, in _rebalance_status_and_progress raise RebalanceFailedException(msg) RebalanceFailedException: Rebalance Failed: {u'errorMessage': u'Rebalance failed. See logs for detailed reason. You can try again.', u'type': u'rebalance', u'masterRequestTimedOut': False, u'statusId': u'7bfdfe2905af13c5b9d5bbabb8468bc6', u'subtype': u'rebalance', u'statusIsStale': False, u'lastReportURI': u'/logs/rebalanceReport?reportID=854009218c08fe5095036b306d13e409', u'status': u'notRunning'} - rebalance failed 2022-03-06 06:10:28,594 | test | INFO | MainThread | [cluster_ready_functions:trigger_cb_collect_on_cluster:1130] Running cbcollect on node ns_1@172.23.100.35,ns_1@172.23.105.164,ns_1@172.23.105.206,ns_1@172.23.106.177 2022-03-06 06:10:38,637 | test | INFO | MainThread | [cluster_ready_functions:trigger_cb_collect_on_cluster:1133] ns_1@172.23.100.35,ns_1@172.23.105.164,ns_1@172.23.105.206,ns_1@172.23.106.177 - cbcollect status: True 2022-03-06 06:10:38,637 | test | INFO | MainThread | [cluster_ready_functions:wait_for_cb_collect_to_complete:1138] Polling active_tasks to check cbcollect status 2022-03-06 06:13:39,003 | test | INFO | MainThread | [cluster_ready_functions:copy_cb_collect_logs:1165] ns_1@172.23.100.35: Copying cbcollect ZIP file to Client 2022-03-06 06:13:39,641 | test | INFO | MainThread | [cluster_ready_functions:copy_cb_collect_logs:1165] ns_1@172.23.105.164: Copying cbcollect ZIP file to Client 2022-03-06 06:13:40,591 | test | INFO | MainThread | [cluster_ready_functions:copy_cb_collect_logs:1165] ns_1@172.23.105.206: Copying cbcollect ZIP file to Client 2022-03-06 06:13:41,290 | test | INFO | MainThread | [cluster_ready_functions:copy_cb_collect_logs:1165] ns_1@172.23.106.177: Copying cbcollect ZIP file to Client 2022-03-06 06:13:41,999 | test | ERROR | MainThread | [basetestcase:setUp:368] Rebalance Failed: {u'errorMessage': u'Rebalance failed. See logs for detailed reason. You can try again.', u'type': u'rebalance', u'masterRequestTimedOut': False, u'statusId': u'7bfdfe2905af13c5b9d5bbabb8468bc6', u'subtype': u'rebalance', u'statusIsStale': False, u'lastReportURI': u'/logs/rebalanceReport?reportID=854009218c08fe5095036b306d13e409', u'status': u'notRunning'} - rebalance failed 2022-03-06 06:13:42,000 | test | INFO | MainThread | [basetestcase:initialize_cluster:467] Initializing cluster : C1 Got this failure java.net.ConnectException: Connection refused: /172.23.100.34:8091 during connect (<_realsocket at 0x3c9e07 type=client open_count=1 channel=[id: 0xbc716341, 0.0.0.0/0.0.0.0:39135] timeout=300.0>) 2022-03-06 06:13:45,607 | test | ERROR | pool-5-thread-12 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.100.34:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.100.37:8091 during connect (<_realsocket at 0x3c9e08 type=client open_count=1 channel=[id: 0x92c8a68e, 0.0.0.0/0.0.0.0:54348] timeout=300.0>) 2022-03-06 06:13:46,104 | test | ERROR | pool-5-thread-14 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.100.37:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.100.36:8091 during connect (<_realsocket at 0x3c9e09 type=client open_count=1 channel=[id: 0xee5e5789, 0.0.0.0/0.0.0.0:42405] timeout=300.0>) 2022-03-06 06:13:46,108 | test | ERROR | pool-5-thread-13 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.100.36:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.100.34:8091 during connect (<_realsocket at 0x3c9e0a type=client open_count=1 channel=[id: 0xee29bbb4, 0.0.0.0/0.0.0.0:44635] timeout=300.0>) 2022-03-06 06:13:47,614 | test | ERROR | pool-5-thread-12 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.100.34:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.100.37:8091 during connect (<_realsocket at 0x3c9e0b type=client open_count=1 channel=[id: 0xfffe4ec9, 0.0.0.0/0.0.0.0:43176] timeout=300.0>) 2022-03-06 06:13:48,111 | test | ERROR | pool-5-thread-14 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.100.37:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.100.36:8091 during connect (<_realsocket at 0x3c9e0c type=client open_count=1 channel=[id: 0x73b5751b, 0.0.0.0/0.0.0.0:38382] timeout=300.0>) 2022-03-06 06:13:48,115 | test | ERROR | pool-5-thread-13 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.100.36:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.100.34:8091 during connect (<_realsocket at 0x3c9e0d type=client open_count=1 channel=[id: 0x467c04ed, 0.0.0.0/0.0.0.0:40853] timeout=300.0>) 2022-03-06 06:13:49,618 | test | ERROR | pool-5-thread-12 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.100.34:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.100.37:8091 during connect (<_realsocket at 0x3c9e0e type=client open_count=1 channel=[id: 0xec5e3507, 0.0.0.0/0.0.0.0:39034] timeout=300.0>) 2022-03-06 06:13:50,115 | test | ERROR | pool-5-thread-14 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.100.37:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.100.36:8091 during connect (<_realsocket at 0x3c9e0f type=client open_count=1 channel=[id: 0x2bdb1047, 0.0.0.0/0.0.0.0:39049] timeout=300.0>) 2022-03-06 06:13:50,119 | test | ERROR | pool-5-thread-13 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.100.36:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.100.34:8091 during connect (<_realsocket at 0x3c9e10 type=client open_count=1 channel=[id: 0xc907f70d, 0.0.0.0/0.0.0.0:55895] timeout=300.0>) 2022-03-06 06:13:51,624 | test | ERROR | pool-5-thread-12 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.100.34:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.100.37:8091 during connect (<_realsocket at 0x3c9e11 type=client open_count=1 channel=[id: 0xb61667db, 0.0.0.0/0.0.0.0:53836] timeout=300.0>) 2022-03-06 06:13:52,121 | test | ERROR | pool-5-thread-14 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.100.37:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.100.36:8091 during connect (<_realsocket at 0x3c9e12 type=client open_count=1 channel=[id: 0x8a54c4da, 0.0.0.0/0.0.0.0:46061] timeout=300.0>) 2022-03-06 06:13:52,125 | test | ERROR | pool-5-thread-13 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.100.36:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.100.34:8091 during connect (<_realsocket at 0x3c9e13 type=client open_count=1 channel=[id: 0x6c6362f5, 0.0.0.0/0.0.0.0:56279] timeout=300.0>) 2022-03-06 06:13:53,628 | test | ERROR | pool-5-thread-12 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.100.34:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.100.37:8091 during connect (<_realsocket at 0x3c9e14 type=client open_count=1 channel=[id: 0xf015cc01, 0.0.0.0/0.0.0.0:53639] timeout=300.0>) 2022-03-06 06:13:54,125 | test | ERROR | pool-5-thread-14 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.100.37:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.100.36:8091 during connect (<_realsocket at 0x3c9e15 type=client open_count=1 channel=[id: 0x2f01c7eb, 0.0.0.0/0.0.0.0:33416] timeout=300.0>) 2022-03-06 06:13:54,130 | test | ERROR | pool-5-thread-13 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.100.36:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.100.34:8091 during connect (<_realsocket at 0x3c9e16 type=client open_count=1 channel=[id: 0xfa352e39, 0.0.0.0/0.0.0.0:51690] timeout=300.0>) 2022-03-06 06:13:55,632 | test | ERROR | pool-5-thread-12 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.100.34:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.100.35:8091 during connect (<_realsocket at 0x3c9e17 type=client open_count=1 channel=[id: 0x19843b31, 0.0.0.0/0.0.0.0:50381] timeout=300.0>) 2022-03-06 06:13:55,650 | test | ERROR | pool-5-thread-11 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.100.35:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.105.206:8091 during connect (<_realsocket at 0x3c9e18 type=client open_count=1 channel=[id: 0x3a431663, 0.0.0.0/0.0.0.0:58846] timeout=300.0>) 2022-03-06 06:13:56,969 | test | ERROR | pool-5-thread-9 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.105.206:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.105.164:8091 during connect (<_realsocket at 0x3c9e19 type=client open_count=1 channel=[id: 0x0d1819de, 0.0.0.0/0.0.0.0:44343] timeout=300.0>) 2022-03-06 06:13:57,039 | test | ERROR | pool-5-thread-8 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.105.164:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.106.177:8091 during connect (<_realsocket at 0x3c9e1a type=client open_count=1 channel=[id: 0x8d50d6a9, 0.0.0.0/0.0.0.0:50243] timeout=300.0>) 2022-03-06 06:13:57,207 | test | ERROR | pool-5-thread-10 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.106.177:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.100.35:8091 during connect (<_realsocket at 0x3c9e1b type=client open_count=1 channel=[id: 0x4a80ba13, 0.0.0.0/0.0.0.0:49916] timeout=300.0>) 2022-03-06 06:13:57,654 | test | ERROR | pool-5-thread-11 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.100.35:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.105.206:8091 during connect (<_realsocket at 0x3c9e1c type=client open_count=1 channel=[id: 0x3389e9c2, 0.0.0.0/0.0.0.0:54777] timeout=300.0>) 2022-03-06 06:13:58,973 | test | ERROR | pool-5-thread-9 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.105.206:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.105.164:8091 during connect (<_realsocket at 0x3c9e1d type=client open_count=1 channel=[id: 0x2c9a9715, 0.0.0.0/0.0.0.0:55063] timeout=300.0>) 2022-03-06 06:13:59,045 | test | ERROR | pool-5-thread-8 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.105.164:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.106.177:8091 during connect (<_realsocket at 0x3c9e1e type=client open_count=1 channel=[id: 0x7a68fb64, 0.0.0.0/0.0.0.0:44834] timeout=300.0>) 2022-03-06 06:13:59,213 | test | ERROR | pool-5-thread-10 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.106.177:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.100.35:8091 during connect (<_realsocket at 0x3c9e1f type=client open_count=1 channel=[id: 0x2c2b33db, 0.0.0.0/0.0.0.0:58042] timeout=300.0>) 2022-03-06 06:13:59,660 | test | ERROR | pool-5-thread-11 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.100.35:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.105.206:8091 during connect (<_realsocket at 0x3c9e20 type=client open_count=1 channel=[id: 0x906b6104, 0.0.0.0/0.0.0.0:47757] timeout=300.0>) 2022-03-06 06:14:00,977 | test | ERROR | pool-5-thread-9 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.105.206:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.105.164:8091 during connect (<_realsocket at 0x3c9e21 type=client open_count=1 channel=[id: 0x514ee0ce, 0.0.0.0/0.0.0.0:50623] timeout=300.0>) 2022-03-06 06:14:01,049 | test | ERROR | pool-5-thread-8 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.105.164:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.106.177:8091 during connect (<_realsocket at 0x3c9e22 type=client open_count=1 channel=[id: 0xf502c411, 0.0.0.0/0.0.0.0:55624] timeout=300.0>) 2022-03-06 06:14:01,217 | test | ERROR | pool-5-thread-10 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.106.177:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.100.35:8091 during connect (<_realsocket at 0x3c9e23 type=client open_count=1 channel=[id: 0xc8468688, 0.0.0.0/0.0.0.0:42087] timeout=300.0>) 2022-03-06 06:14:01,664 | test | ERROR | pool-5-thread-11 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.100.35:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.105.206:8091 during connect (<_realsocket at 0x3c9e24 type=client open_count=1 channel=[id: 0x6448c565, 0.0.0.0/0.0.0.0:49250] timeout=300.0>) 2022-03-06 06:14:02,983 | test | ERROR | pool-5-thread-9 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.105.206:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.105.164:8091 during connect (<_realsocket at 0x3c9e25 type=client open_count=1 channel=[id: 0x7352df7b, 0.0.0.0/0.0.0.0:36004] timeout=300.0>) 2022-03-06 06:14:03,053 | test | ERROR | pool-5-thread-8 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.105.164:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.106.177:8091 during connect (<_realsocket at 0x3c9e26 type=client open_count=1 channel=[id: 0xdf957708, 0.0.0.0/0.0.0.0:56514] timeout=300.0>) 2022-03-06 06:14:03,221 | test | ERROR | pool-5-thread-10 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.106.177:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.100.35:8091 during connect (<_realsocket at 0x3c9e27 type=client open_count=1 channel=[id: 0x4143a812, 0.0.0.0/0.0.0.0:50451] timeout=300.0>) 2022-03-06 06:14:03,667 | test | ERROR | pool-5-thread-11 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.100.35:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.100.35:8091 during connect (<_realsocket at 0x3c9e28 type=client open_count=1 channel=[id: 0x0d049b22, 0.0.0.0/0.0.0.0:45218] timeout=300.0>) 2022-03-06 06:14:05,673 | test | ERROR | pool-5-thread-11 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.100.35:8091/nodes/self error [Errno 111] Connection refused 2022-03-06 06:14:09,963 | infra | CRITICAL | pool-5-thread-15 | [task:call:6181] mcdMemoryReserved reported in nodes/self is: 9379 2022-03-06 06:14:09,964 | test | CRITICAL | pool-5-thread-15 | [table_view:display:72] Memory quota allocated: +---------------------+---------+ | Service | RAM MiB | +---------------------+---------+ | eventingMemoryQuota | 256 | | cbasMemoryQuota | 1024 | | indexMemoryQuota | 256 | | ftsMemoryQuota | 256 | | memoryQuota | 9279 | +---------------------+---------+ 2022-03-06 06:14:10,062 | test | WARNING | MainThread | [basetestcase:_initialize_nodes:702] RAM quota was defined less than 100 MB: 2022-03-06 06:14:21,947 | test | INFO | MainThread | [basetestcase:initialize_cluster:493] Cluster C1 initialized 2022-03-06 06:14:21,947 | test | INFO | MainThread | [basetestcase:log_setup_status:647] ========= BaseTestCase setup finished for test #2 test_data_load_collections_with_graceful_failover_recovery ========= 2022-03-06 06:14:21,963 | test | INFO | MainThread | [basetestcase:log_setup_status:647] ========= ClusterSetup setup started for test #2 test_data_load_collections_with_graceful_failover_recovery ========= 2022-03-06 06:14:35,742 | test | INFO | pool-5-thread-16 | [table_view:display:72] Rebalance Overview +----------------+----------+-----------------------+--------------+--------------+-----------------------+ | Nodes | Services | Version | CPU | Status | Membership / Recovery | +----------------+----------+-----------------------+--------------+--------------+-----------------------+ | 172.23.105.164 | kv | 7.1.0-2440-enterprise | 80.429022082 | Cluster node | active / none | | 172.23.105.206 | None | | | <--- IN --- | | | 172.23.106.177 | None | | | <--- IN --- | | | 172.23.100.35 | None | | | <--- IN --- | | +----------------+----------+-----------------------+--------------+--------------+-----------------------+ 2022-03-06 06:14:35,756 | test | INFO | pool-5-thread-16 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 06:14:50,789 | test | INFO | pool-5-thread-16 | [task:check:415] Rebalance - status: none, progress: 100 2022-03-06 06:14:50,799 | test | INFO | pool-5-thread-16 | [task:check:474] Rebalance completed with progress: 100% in 15.0579998493 sec 2022-03-06 06:14:50,908 | test | INFO | MainThread | [table_view:display:72] Cluster statistics +----------------+----------+-----------------+-----------+-----------+----------------------+-------------------+-----------------------+ | Node | Services | CPU_utilization | Mem_total | Mem_free | Swap_mem_used | Active / Replica | Version | +----------------+----------+-----------------+-----------+-----------+----------------------+-------------------+-----------------------+ | 172.23.105.206 | kv | 0.702194357367 | 11.45 GiB | 10.45 GiB | 23.59 MiB / 3.50 GiB | 0 / 0 | 7.1.0-2440-enterprise | | 172.23.106.177 | kv | 0.739626425975 | 11.45 GiB | 10.36 GiB | 23.15 MiB / 3.50 GiB | 0 / 0 | 7.1.0-2440-enterprise | | 172.23.100.35 | kv | 0.225338007011 | 11.45 GiB | 10.52 GiB | 0.0 Byte / 3.50 GiB | 0 / 0 | 7.1.0-2440-enterprise | | 172.23.105.164 | kv | 0.514364571572 | 11.45 GiB | 10.43 GiB | 0.0 Byte / 3.50 GiB | 0 / 0 | 7.1.0-2440-enterprise | +----------------+----------+-----------------+-----------+-----------+----------------------+-------------------+-----------------------+ 2022-03-06 06:14:50,910 | test | INFO | MainThread | [basetestcase:log_setup_status:647] ========= ClusterSetup setup complete for test #2 test_data_load_collections_with_graceful_failover_recovery ========= 2022-03-06 06:14:50,910 | test | INFO | MainThread | [basetestcase:log_setup_status:647] ========= CollectionBase setup started for test #2 test_data_load_collections_with_graceful_failover_recovery ========= 2022-03-06 06:14:51,042 | test | INFO | MainThread | [bucket_ready_functions:create_buckets_using_json_data:1913] Creating required buckets from template 2022-03-06 06:14:51,295 | test | INFO | MainThread | [bucket_ready_functions:wait_for_collection_creation_to_complete:2527] Waiting for all collections to be created 2022-03-06 06:14:51,576 | test | INFO | MainThread | [table_view:display:72] Bucket statistics +---------+-----------+-----------------+----------+------------+-----+-------+-----------+----------+-----------+-----+ | Bucket | Type | Storage Backend | Replicas | Durability | TTL | Items | RAM Quota | RAM Used | Disk Used | ARR | +---------+-----------+-----------------+----------+------------+-----+-------+-----------+----------+-----------+-----+ | bucket1 | couchbase | couchstore | 1 | none | 0 | 0 | 0.0 Byte | 0.0 Byte | 0.0 Byte | 100 | | bucket2 | couchbase | magma | 1 | none | 0 | 0 | 4.00 GiB | 0.0 Byte | 0.0 Byte | 100 | | default | couchbase | magma | 1 | none | 0 | 0 | 0.0 Byte | 0.0 Byte | 0.0 Byte | 100 | +---------+-----------+-----------------+----------+------------+-----+-------+-----------+----------+-----------+-----+ 2022-03-06 06:14:52,338 | test | INFO | MainThread | [common_lib:sleep:23] Sleep 30 seconds. Reason: Sleep while magma storage quota setting is taking effect 2022-03-06 06:15:22,338 | test | INFO | MainThread | [collections_base:collection_setup:174] Creating required SDK clients for client_pool 2022-03-06 06:15:28,102 | test | INFO | MainThread | [bucket_ready_functions:perform_tasks_from_spec:4993] Performing scope/collection specific operations 2022-03-06 06:15:28,108 | test | INFO | MainThread | [bucket_ready_functions:perform_tasks_from_spec:5083] Done Performing scope/collection specific operations 2022-03-06 07:14:42,460 | test | INFO | pool-5-thread-21 | [table_view:display:72] Ops trend for bucket 'bucket2' +---------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------+ | Min | Trend | Max | +---------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------+ | 0.000 | ***....................................................................................................*******X | 5200.000 | | 0.000 | ******************************************************************************************************************************************************************************************************************************************************.............*******X | 700.000 | | 300.000 | ....*******X | 300.000 | | 0.000 | ************************************************************************************************************************************************************************************************************************************************************.....********............*******X | 900.000 | | 0.000 | *************************************************************************************************************************************************************************************************************************************************************************.****************....******X | 300.000 | | 250.000 | ...X | 250.000 | | 200.000 | ..******X | 200.000 | | 0.000 | *************************************************************************************************************************************************************************************************************************************************************************.****************..*******X | 200.000 | | 0.000 | ********************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************.*******X | 100.000 | | 0.000 | *******.**********************X | 100.000 | | 0.000 | *************************************************************************************************************************************************************************************************************************************.********..*******X | 200.000 | | 100.000 | *******X | 100.000 | | 0.000 | ************************************************************************************************************************************************************************************************************************************...*******X | 200.000 | | 0.000 | ********.******X | 100.000 | | 0.000 | *************************************************************************************************************************************************************************************************************************************.*******..*******X | 200.000 | | 0.000 | ********.*******X | 100.000 | | 0.000 | ********************************************************************************************************************************************X | 0.000 | +---------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------+ 2022-03-06 07:14:42,657 | test | INFO | pool-5-thread-22 | [table_view:display:72] Ops trend for bucket 'default' +-----------+----------------------------------------------------------------------------------------------------------------------------------------------------+-----------+ | Min | Trend | Max | +-----------+----------------------------------------------------------------------------------------------------------------------------------------------------+-----------+ | 0.000 | ***................................********................................................................*******X | 84895.600 | | 80703.300 | ...........................................................................................***************X | 80902.800 | | 79919.100 | ..........................................................................................***************...********.*******X | 83449.700 | | 83151.000 | .............................................................................................********..********.********...*******X | 87647.800 | | 85013.900 | ...............................................................................................*******X | 85013.900 | | 83681.500 | ..............................................................................................*******X | 83681.500 | | 82766.200 | .............................................................................................*******X | 82766.200 | | 81665.300 | ............................................................................................*******X | 81665.300 | | 81115.500 | ...........................................................................................*******...********.*******X | 84881.700 | | 82955.400 | .............................................................................................*******X | 82955.400 | | 79568.500 | .........................................................................................********..*******X | 80662.800 | | 77889.300 | .......................................................................................********.....*******...*******X | 84321.300 | | 83001.300 | .............................................................................................*******X | 83001.300 | | 81011.100 | ...........................................................................................*******X | 81011.100 | | 80726.500 | ...........................................................................................*******X | 80726.500 | | 78581.500 | ........................................................................................******X | 78581.500 | | 74097.000 | ...................................................................................*******X | 74097.000 | | 70540.300 | ...............................................................................*******X | 70540.300 | | 69350.600 | ..............................................................................*******X | 69350.600 | | 61797.500 | .....................................................................********............********.*******X | 73379.900 | | 70360.000 | ...............................................................................******X | 70360.000 | | 60556.400 | ....................................................................********.********......................********.*******X | 81639.300 | | 79488.000 | .........................................................................................******X | 79488.000 | | 76782.400 | ......................................................................................*******X | 76782.400 | | 72695.600 | .................................................................................********...********.....*******.....*******X | 83775.500 | | 83742.300 | ..............................................................................................*******X | 83742.300 | | 83142.800 | .............................................................................................***************X | 83200.100 | | 75762.600 | .....................................................................................******X | 75762.600 | | 73864.300 | ...................................................................................*******X | 73864.300 | | 72807.300 | ..................................................................................********..********........*******.*******X | 83174.600 | | 82596.800 | .............................................................................................*******X | 82596.800 | | 80720.400 | ...........................................................................................*******X | 80720.400 | | 74089.500 | ...................................................................................*******....*******X | 77694.300 | | 75074.500 | ....................................................................................*******X | 75074.500 | | 71326.900 | ................................................................................********...*******........*******X | 80737.600 | | 79880.100 | ..........................................................................................*******X | 79880.100 | | 75873.800 | .....................................................................................***************X | 76090.500 | | 71886.600 | .................................................................................*******..*******X | 74287.800 | | 73914.900 | ...................................................................................*******X | 73914.900 | | 72678.700 | .................................................................................******X | 72678.700 | | 68371.200 | .............................................................................*******X | 68371.200 | | 65766.200 | ..........................................................................*******X | 65766.200 | | 64948.500 | .........................................................................*******.....*******X | 70047.800 | | 62867.600 | ......................................................................********.....********......********..******X | 74077.800 | | 70116.500 | ..............................................................................*******X | 70116.500 | | 64715.300 | ........................................................................*******X | 64715.300 | | 60660.200 | ....................................................................*******........********..........*******X | 76585.800 | | 74770.500 | ....................................................................................******X | 74770.500 | | 67172.800 | ...........................................................................********..*******X | 69238.200 | | 68104.400 | ............................................................................*******...********........*******X | 77983.500 | | 71819.600 | ................................................................................******X | 71819.600 | | 71008.700 | ................................................................................********.*******X | 72256.300 | | 70991.600 | ...............................................................................******X | 70991.600 | | 69486.200 | ..............................................................................X | 69486.200 | | 68968.100 | .............................................................................******X | 68968.100 | | 64347.300 | ........................................................................********...*******X | 67115.700 | | 65046.800 | .........................................................................*******.......********.......*******X | 77456.000 | | 75337.700 | ....................................................................................*******X | 75337.700 | | 67609.200 | ............................................................................**************X | 67954.500 | | 63368.900 | .......................................................................********............********......********..******X | 81111.600 | | 79785.100 | ..........................................................................................***************X | 80438.900 | | 71781.400 | ................................................................................*******X | 71781.400 | | 70332.900 | ...............................................................................******X | 70332.900 | | 70157.200 | ...............................................................................***************X | 70634.700 | | 68579.000 | .............................................................................*******......********...*******X | 77003.100 | | 71098.600 | ................................................................................******X | 71098.600 | | 69182.200 | .............................................................................*******X | 69182.200 | | 68676.400 | .............................................................................*******X | 68676.400 | | 63979.300 | .......................................................................******X | 63979.300 | | 61009.300 | ....................................................................********......********...*******...*******X | 71517.100 | | 68432.300 | .............................................................................******X | 68432.300 | | 65179.700 | .........................................................................X | 65179.700 | | 61954.400 | .....................................................................******X | 61954.400 | | 51344.400 | .........................................................********............*******............********..*******X | 73772.800 | | 69596.300 | ..............................................................................*******X | 69596.300 | | 63000.400 | ......................................................................******X | 63000.400 | | 57217.000 | ................................................................*******.............********..........*******X | 77447.500 | | 69263.900 | ..............................................................................********......******X | 75189.100 | | 74402.000 | ...................................................................................*******X | 74402.000 | | 59083.400 | ..................................................................******X | 59083.400 | | 53290.200 | ...........................................................********............********......*******..***************X | 70570.100 | | 62970.000 | ......................................................................*******.*******X | 63746.400 | | 55251.300 | ..............................................................********....**************X | 59322.500 | | 57427.600 | ................................................................********......**************X | 62991.800 | | 57608.400 | ................................................................*******..********...*******......*******X | 66619.100 | | 63352.400 | .......................................................................******X | 63352.400 | | 62562.000 | ......................................................................X | 62562.000 | | 61741.500 | .....................................................................******X | 61741.500 | | 40429.100 | .............................................********..........*******..........********..............*******X | 70353.200 | | 60410.300 | ...................................................................******X | 60410.300 | | 54109.000 | ............................................................*******X | 54109.000 | | 44577.300 | .................................................*******.......................*******X | 64124.300 | | 57063.500 | ................................................................*******..*******X | 59291.000 | | 59239.100 | ..................................................................******X | 59239.100 | | 54831.300 | .............................................................*******X | 54831.300 | | 50668.600 | ........................................................******X | 50668.600 | | 49586.300 | .......................................................X | 49586.300 | | 49207.800 | .......................................................*******.............********.........********......******X | 73992.900 | | 66002.900 | ..........................................................................*******X | 66002.900 | | 48479.000 | ......................................................********.*******.................*******X | 64025.800 | | 62478.800 | ......................................................................*******X | 62478.800 | | 56629.200 | ...............................................................*******......*******X | 61659.100 | | 55949.000 | ..............................................................*******X | 55949.000 | | 41936.600 | ..............................................*******........********........*******X | 55791.200 | | 47667.800 | .....................................................*******.........*******X | 55849.600 | | 54872.400 | .............................................................******X | 54872.400 | | 38023.300 | ..........................................********.....********......................*******........*******X | 68762.100 | | 52553.700 | ..........................................................******X | 52553.700 | | 36386.000 | ........................................********............*******................********.*******X | 61481.800 | | 40621.900 | .............................................*******.*******X | 41625.900 | | 37627.900 | .........................................*******..........********......********..........******X | 60240.700 | | 59514.700 | ..................................................................*******X | 59514.700 | | 51537.900 | .........................................................******X | 51537.900 | | 37326.900 | .........................................********...........*******.......*******X | 52716.800 | | 44374.100 | .................................................********......******X | 49240.800 | | 47122.500 | ....................................................*******X | 47122.500 | | 43437.200 | ................................................********...........**************X | 52890.100 | | 49404.100 | .......................................................********......*******.......*******X | 60514.900 | | 55058.100 | .............................................................*******X | 55058.100 | | 48590.300 | ......................................................*******..********...............******X | 63479.300 | | 61300.100 | ....................................................................*******X | 61300.100 | | 46815.700 | ....................................................*******....*******X | 50751.600 | | 44755.700 | ..................................................*******X | 44755.700 | | 39427.900 | ...........................................*******...********...********......*******........********........*******X | 63368.100 | | 52259.300 | ..........................................................******X | 52259.300 | | 49390.800 | .......................................................*******X | 49390.800 | | 45832.600 | ...................................................********...*******..........*******X | 57146.500 | | 53385.500 | ...........................................................******X | 53385.500 | | 51025.700 | .........................................................********....*******.........********.......*******.........*******X | 76716.200 | | 74164.400 | ...................................................................................********....****************.*******..********.***************X | 81282.800 | | 80380.600 | ..........................................................................................********.*******X | 81358.900 | | 81159.300 | ...........................................................................................***************.*******X | 82386.900 | | 82079.400 | ............................................................................................********..***************X | 83733.900 | | 82792.000 | .............................................................................................*******X | 82792.000 | | 81633.800 | ............................................................................................***************X | 81890.500 | | 80092.500 | ..........................................................................................*******.********.*******X | 82060.100 | | 80971.600 | ...........................................................................................********..*******X | 82523.400 | | 81815.700 | ............................................................................................********..*******X | 83499.500 | | 81825.100 | ............................................................................................*******X | 81825.100 | | 81683.700 | ............................................................................................*******X | 81683.700 | | 80082.900 | ..........................................................................................********.*******.********..*******X | 83995.100 | | 82575.000 | .............................................................................................***************X | 82635.300 | | 80819.300 | ...........................................................................................*******X | 80819.300 | | 80224.800 | ..........................................................................................********..*******X | 82223.200 | | 81097.100 | ...........................................................................................*******.********.*******X | 83202.400 | | 81627.800 | ............................................................................................*******X | 81627.800 | | 80887.200 | ...........................................................................................********.********..*******X | 83870.900 | | 82324.000 | ............................................................................................********.*******X | 82410.400 | | 81304.900 | ...........................................................................................*******.********..*******X | 83480.100 | | 83366.200 | ..............................................................................................*******X | 83366.200 | | 82959.000 | .............................................................................................*******X | 82959.000 | | 81980.600 | ............................................................................................*******X | 81980.600 | | 81504.200 | ...........................................................................................*******X | 81504.200 | | 79437.600 | .........................................................................................***************.********..****************.*******X | 82499.500 | | 80454.500 | ..........................................................................................*******X | 80454.500 | | 78720.100 | ........................................................................................******X | 78720.100 | | 78597.200 | ........................................................................................X | 78597.200 | | 78442.500 | ........................................................................................******X | 78442.500 | | 76119.900 | .....................................................................................*******X | 76119.900 | | 74814.800 | ....................................................................................********..********..*******X | 78471.600 | | 76985.800 | ......................................................................................********..*******X | 78237.200 | | 77809.900 | .......................................................................................*******..*...******X | 82118.500 | | 80168.200 | ..........................................................................................*******X | 80168.200 | | 77232.800 | .......................................................................................********...********.********.********.*******X | 83003.500 | | 82406.500 | .............................................................................................******X | 82406.500 | | 82212.800 | ............................................................................................*******X | 82212.800 | | 81442.500 | ...........................................................................................*******X | 81442.500 | | 80565.400 | ..........................................................................................*******X | 80565.400 | | 78298.400 | ........................................................................................*******X | 78298.400 | | 75581.200 | .....................................................................................********.******X | 76948.400 | | 76329.800 | ......................................................................................*******X | 76329.800 | | 74834.200 | ....................................................................................********..*******X | 76696.900 | | 74109.200 | ...................................................................................********..*******X | 75781.700 | | 73159.800 | ..................................................................................********..*******X | 74987.900 | | 74166.000 | ...................................................................................******X | 74166.000 | | 72337.700 | .................................................................................********.********....********..********.********.********.******X | 81432.800 | | 80738.900 | ...........................................................................................************************...*******X | 83795.200 | | 79039.500 | .........................................................................................********..********.********..******X | 84062.500 | | 81379.900 | ...........................................................................................*******X | 81379.900 | | 80680.400 | ...........................................................................................*******X | 80680.400 | | 79858.400 | ..........................................................................................********...*******X | 82815.800 | | 81615.800 | ............................................................................................*******..*******X | 83697.700 | | 82510.300 | .............................................................................................*******X | 82510.300 | | 80702.900 | ...........................................................................................******X | 80702.900 | +-----------+----------------------------------------------------------------------------------------------------------------------------------------------------+-----------+ 2022-03-06 07:14:43,493 | test | INFO | pool-5-thread-23 | [table_view:display:72] Ops trend for bucket 'bucket1' +---------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------+ | Min | Trend | Max | +---------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------+ | 0.000 | ***....................................................................................................*******X | 1200.000 | | 0.000 | ******************************************************************************************************************************************************************************************************************************************************.........................*******X | 300.000 | | 200.000 | ...............*******X | 200.000 | | 0.000 | *************************************************************************************************************************************************************************************************************************************************************................********...........................................................*******X | 900.000 | | 0.000 | **************************************************************************************************************************************************************************************************************************************************************************........********........********.................*******X | 400.000 | | 100.000 | .......******X | 100.000 | | 24.900 | .X | 24.900 | | 0.000 | *****************************************************************************************************************************************************************************************************************************************************************.........................*******X | 300.000 | | 0.000 | *******..............*............................................******X | 700.000 | | 200.000 | ...............*******X | 200.000 | | 0.000 | **********************************************************************************************************************************************************************************************************************************************************************************************************************************........******X | 100.000 | | 0.000 | ********.................................******X | 400.000 | | 0.000 | ******************************************************************************************************************************************************************************************************************************************************************************************************************************************************************................********.................*******X | 400.000 | | 0.000 | *******.........................********........******X | 400.000 | | 200.000 | ...............**************X | 200.000 | | 50.200 | ...X | 50.200 | | 0.000 | *********************************************************************************************************************************************************************************************************************................********.........********.........................*******X | 600.000 | | 400.000 | ................................*******X | 400.000 | | 100.000 | .......******X | 100.000 | | 0.000 | *********************************************************************************************************************************************************************************************************************.................................*******X | 400.000 | | 242.900 | ...................*******X | 242.900 | | 57.100 | ...********....*******..*......******X | 200.000 | | 0.000 | *****************************************************************************************************************************************************************************************************************************........*******.................*******X | 300.000 | | 0.000 | *************************************************************************************************************************************************************X | 0.000 | +---------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------+ 2022-03-06 07:14:44,755 | test | INFO | pool-5-thread-30 | [table_view:display:72] Ops trend for bucket 'default' +-----------+------------------------------------------------------------------------------------------------------+-----------+ | Min | Trend | Max | +-----------+------------------------------------------------------------------------------------------------------+-----------+ | 54293.900 | ...................................................................................................X | 54293.900 | +-----------+------------------------------------------------------------------------------------------------------+-----------+ 2022-03-06 07:14:44,874 | test | INFO | MainThread | [table_view:display:72] Cluster statistics +----------------+----------+-----------------+-----------+----------+----------------------+---------------------+-----------------------+ | Node | Services | CPU_utilization | Mem_total | Mem_free | Swap_mem_used | Active / Replica | Version | +----------------+----------+-----------------+-----------+----------+----------------------+---------------------+-----------------------+ | 172.23.105.206 | kv | 86.0170586715 | 11.45 GiB | 8.73 GiB | 23.59 MiB / 3.50 GiB | 61876689 / 61885187 | 7.1.0-2440-enterprise | | 172.23.106.177 | kv | 82.7746861654 | 11.45 GiB | 8.68 GiB | 23.15 MiB / 3.50 GiB | 61933841 / 61940862 | 7.1.0-2440-enterprise | | 172.23.100.35 | kv | 75.8727296148 | 11.45 GiB | 8.72 GiB | 0.0 Byte / 3.50 GiB | 62375911 / 62384525 | 7.1.0-2440-enterprise | | 172.23.105.164 | kv | 77.2381811155 | 11.45 GiB | 8.65 GiB | 0.0 Byte / 3.50 GiB | 62249267 / 62294386 | 7.1.0-2440-enterprise | +----------------+----------+-----------------+-----------+----------+----------------------+---------------------+-----------------------+ 2022-03-06 07:14:52,704 | test | INFO | MainThread | [bucket_ready_functions:validate_docs_per_collections_all_buckets:4742] Validating collection stats and item counts 2022-03-06 07:15:00,365 | test | ERROR | MainThread | [bucket_ready_functions:validate_manifest_uid:4605] Bucket UID mismatch. Expected: 0, Actual: 1 2022-03-06 07:15:00,367 | test | WARNING | MainThread | [bucket_ready_functions:validate_docs_per_collections_all_buckets:4759] Bucket manifest UID mismatch! 2022-03-06 07:15:04,276 | test | ERROR | MainThread | [bucket_ready_functions:validate_manifest_uid:4605] Bucket UID mismatch. Expected: 0, Actual: 1 2022-03-06 07:15:04,276 | test | WARNING | MainThread | [bucket_ready_functions:validate_docs_per_collections_all_buckets:4759] Bucket manifest UID mismatch! 2022-03-06 07:15:08,226 | test | ERROR | MainThread | [bucket_ready_functions:validate_manifest_uid:4605] Bucket UID mismatch. Expected: 0, Actual: 1 2022-03-06 07:15:08,226 | test | WARNING | MainThread | [bucket_ready_functions:validate_docs_per_collections_all_buckets:4759] Bucket manifest UID mismatch! 2022-03-06 07:15:18,108 | test | INFO | MainThread | [table_view:display:72] Bucket statistics +---------+-----------+-----------------+----------+------------+-----+-----------+-----------+------------+------------+-----------+ | Bucket | Type | Storage Backend | Replicas | Durability | TTL | Items | RAM Quota | RAM Used | Disk Used | ARR | +---------+-----------+-----------------+----------+------------+-----+-----------+-----------+------------+------------+-----------+ | bucket1 | couchbase | couchstore | 1 | none | 0 | 100000 | 7.81 GiB | 113.25 MiB | 115.07 MiB | 100 | | bucket2 | couchbase | magma | 1 | none | 0 | 100000 | 4.00 GiB | 269.97 MiB | 237.03 MiB | 100 | | default | couchbase | magma | 1 | none | 0 | 250000000 | 4.00 GiB | 2.10 GiB | 41.22 GiB | 3.2507352 | +---------+-----------+-----------------+----------+------------+-----+-----------+-----------+------------+------------+-----------+ 2022-03-06 07:15:18,109 | test | INFO | MainThread | [basetestcase:log_setup_status:647] ========= CollectionBase setup complete for test #2 test_data_load_collections_with_graceful_failover_recovery ========= 2022-03-06 07:15:21,153 | test | INFO | MainThread | [collections_rebalance:load_collections_with_rebalance:1003] Doing collection data load during graceful_failover_recovery 2022-03-06 07:15:21,153 | test | INFO | MainThread | [collections_rebalance:rebalance_operation:361] Starting rebalance operation of type : graceful_failover_recovery 2022-03-06 07:15:21,153 | test | INFO | MainThread | [collections_rebalance:rebalance_operation:671] failing over nodes [ip:172.23.100.35 port:8091 ssh_username:root] 2022-03-06 07:15:39,365 | test | INFO | pool-5-thread-10 | [rest_client:monitorRebalance:1610] Rebalance done. Taken 18.1009998322 seconds to complete 2022-03-06 07:15:39,369 | test | INFO | pool-5-thread-10 | [common_lib:sleep:23] Sleep 5 seconds. Reason: Wait after rebalance complete 2022-03-06 07:20:44,371 | test | INFO | MainThread | [collections_rebalance:data_load_after_failover:330] Starting a sync data load after failover 2022-03-06 07:20:44,374 | test | INFO | MainThread | [bucket_ready_functions:perform_tasks_from_spec:4993] Performing scope/collection specific operations 2022-03-06 07:20:48,390 | test | INFO | MainThread | [bucket_ready_functions:perform_tasks_from_spec:5083] Done Performing scope/collection specific operations 2022-03-06 08:04:18,145 | test | INFO | pool-5-thread-8 | [table_view:display:72] Ops trend for bucket 'bucket2' +---------+---------------------------------------------------------------------------------------------------------------+----------+ | Min | Trend | Max | +---------+---------------------------------------------------------------------------------------------------------------+----------+ | 0.000 | *....................................................................................................*******X | 1863.800 | | 308.200 | ...............*******.........******X | 480.000 | | 72.000 | ..*******X | 72.000 | | 0.000 | *******...............................******X | 580.000 | | 0.000 | **********************....*******...******X | 132.000 | | 0.000 | ********.........*******..................*******X | 516.000 | | 16.000 | *******.*******X | 20.000 | | 16.000 | ******X | 16.000 | | 0.000 | ***************..............*******X | 276.000 | | 16.000 | ******X | 16.000 | | 0.000 | **********************X | 16.000 | | 0.000 | *************X | 16.000 | | 0.000 | ***************...******X | 68.000 | | 0.000 | ***********************.....******X | 104.000 | | 0.000 | ********.******X | 36.000 | | 32.000 | *******X | 32.000 | | 0.000 | *******..******X | 52.000 | | 0.000 | ***************...........................*******X | 516.000 | | 84.000 | ...******X | 84.000 | | 16.000 | **************X | 16.000 | | 0.000 | **************X | 16.000 | | 0.000 | *******..*******X | 48.000 | | 0.000 | *******.......******X | 136.000 | | 0.000 | **************X | 16.000 | | 0.000 | ******************************......******X | 116.000 | | 32.000 | *******X | 32.000 | | 0.000 | *******...*******X | 64.000 | | 0.000 | *******................................******X | 612.000 | | 16.000 | ********..******X | 52.000 | | 36.000 | ******X | 36.000 | | 0.000 | *****************************.....******X | 100.000 | | 0.000 | ********.******X | 32.000 | | 0.000 | ********************************************.....*******X | 108.100 | | 11.900 | ******X | 11.900 | | 0.000 | ***************...*******X | 60.000 | | 40.000 | .******X | 40.000 | | 0.000 | ********..******X | 40.000 | | 0.000 | **********************..******X | 52.000 | | 0.000 | ********.*******.******X | 48.000 | | 0.000 | ***************..*******X | 52.000 | | 0.000 | ***************..******X | 48.000 | | 16.000 | ***************.*******X | 20.000 | | 0.000 | ***************.******X | 20.000 | | 0.000 | *****************************X | 16.000 | | 0.000 | ********..******X | 48.000 | | 0.000 | *******..******X | 52.000 | | 17.100 | X | 17.100 | | 0.000 | ***************************************************......*******X | 124.000 | | 0.000 | *********************.*******X | 36.000 | | 0.000 | **************...******X | 56.000 | | 0.000 | ********..******X | 48.000 | | 0.000 | ***********************.******X | 20.000 | | 0.000 | **********************X | 16.000 | | 0.000 | ******************************.******X | 20.000 | | 0.000 | **************X | 16.000 | | 0.000 | ************************************X | 16.000 | | 0.000 | *******....*******X | 84.000 | | 0.000 | ***************...*******X | 56.000 | | 0.000 | **********************..*******X | 40.000 | | 0.000 | **************************************************************************X | 16.000 | | 0.000 | ********.**************X | 20.000 | | 0.000 | ******************************.**************X | 36.000 | | 0.000 | *******.....*******X | 96.000 | | 0.000 | ****************************X | 16.000 | | 0.000 | ****************..*******X | 52.000 | | 0.000 | ****************.********.*******X | 52.000 | | 0.000 | ****************....*******X | 88.000 | | 0.000 | ********************************.*******X | 20.000 | | 0.000 | ************************.********........*******X | 172.000 | | 32.000 | *******X | 32.000 | | 0.000 | ********.********.*******X | 40.000 | | 20.000 | *******X | 20.000 | | 0.000 | *******.***************X | 20.000 | | 0.000 | *********...*******X | 68.000 | | 20.000 | *******X | 20.000 | | 0.000 | ***************************************.*******X | 36.000 | | 16.000 | ********.********.*******X | 52.000 | | 20.000 | ********..*******X | 72.000 | | 0.000 | ************************....*******X | 84.000 | | 56.000 | ..*******X | 56.000 | | 16.000 | *******X | 16.000 | | 0.000 | ********.....*******X | 108.000 | | 68.000 | ..*******X | 68.000 | | 0.000 | ********.********.*******X | 48.000 | | 0.000 | ***************....********X | 88.000 | | 0.000 | ***********************.*******X | 20.000 | | 0.000 | ********..*******X | 48.000 | | 0.000 | ****************...***************X | 72.000 | | 16.000 | ********..*******X | 48.000 | | 16.000 | ********.******X | 36.000 | | 0.000 | ****************....*******X | 80.000 | | 48.000 | .*******X | 48.000 | | 16.000 | ******X | 16.000 | | 0.000 | ***********X | 0.000 | +---------+---------------------------------------------------------------------------------------------------------------+----------+ 2022-03-06 08:04:18,792 | test | INFO | pool-5-thread-11 | [table_view:display:72] Ops trend for bucket 'default' +-----------+----------------------------------------------------------------------------------------------------------------+-----------+ | Min | Trend | Max | +-----------+----------------------------------------------------------------------------------------------------------------+-----------+ | 0.000 | *....*X | 4523.300 | | 4523.300 | ...*X | 4523.300 | | 4523.300 | ...***................................................................................................*******X | 95364.000 | | 85386.600 | ........................................................................................******X | 85386.600 | | 66986.900 | .....................................................................******X | 66986.900 | | 60029.100 | .............................................................*******X | 60029.100 | | 55267.800 | ........................................................X | 55267.800 | | 55267.800 | ........................................................*****X | 55267.800 | | 49189.100 | ..................................................*******...*******X | 51682.000 | | 44594.700 | .............................................*******..*******............*******..***X | 59905.900 | | 59905.900 | .............................................................**X | 59905.900 | | 54292.900 | .......................................................*******X | 54292.900 | | 45586.700 | ..............................................*******.**X | 45980.400 | | 45980.400 | ...............................................***X | 45980.400 | | 43939.300 | .............................................*******X | 43939.300 | | 42267.400 | ...........................................X | 42267.400 | | 42267.400 | ...........................................******......*******.*******X | 49505.800 | | 48169.700 | .................................................*******.********..******X | 50655.400 | | 49768.400 | ...................................................********....*******.....******X | 59043.500 | | 54014.600 | .......................................................******X | 54014.600 | | 49939.300 | ...................................................********........*******..*******.X | 60764.400 | | 60764.400 | ..............................................................*****X | 60764.400 | | 52438.200 | .....................................................*******X | 52438.200 | | 49736.300 | ...................................................******X | 49736.300 | | 48681.400 | ..................................................*******..******X | 51268.300 | | 48537.700 | .................................................*******X | 48537.700 | | 46440.300 | ...............................................*******......******X | 52345.000 | | 50732.600 | ....................................................********..******X | 53079.000 | | 46949.500 | ................................................********.X | 48367.600 | | 48367.600 | .................................................**X | 48367.600 | | 48367.600 | .................................................**X | 48367.600 | | 47737.200 | .................................................********..*******..*******X | 52000.500 | | 51457.600 | ....................................................*******.............********...******X | 65909.600 | | 64855.600 | ...................................................................*X | 64855.600 | | 64855.600 | ...................................................................*X | 64855.600 | | 64855.600 | ...................................................................*X | 64855.600 | | 64855.600 | ...................................................................X | 64855.600 | | 61128.900 | ...............................................................X | 61128.900 | | 61128.900 | ...............................................................****X | 61128.900 | | 61128.900 | ...............................................................*X | 61128.900 | | 56147.700 | .........................................................******X | 56147.700 | | 55564.800 | .........................................................******X | 55564.800 | | 55261.200 | ........................................................X | 55261.200 | | 54492.900 | ........................................................******X | 54492.900 | | 52161.800 | .....................................................*******...*******......*******X | 60507.500 | | 57233.700 | ...........................................................******X | 57233.700 | | 54889.500 | ........................................................*******.....*******..*******X | 61038.600 | | 60479.500 | ..............................................................**X | 60479.500 | | 60479.500 | ..............................................................**X | 60479.500 | | 60479.500 | ..............................................................*..*************X | 62212.800 | | 60298.700 | ..............................................................*******X | 60298.700 | | 59788.300 | .............................................................******X | 59788.300 | | 57777.400 | ...........................................................X | 57777.400 | | 57777.400 | ...........................................................*X | 57777.400 | | 57777.400 | ...........................................................****...*******.****X | 61337.000 | | 61337.000 | ...............................................................**...******X | 63928.800 | | 63799.900 | .................................................................**X | 63799.900 | | 63799.900 | .................................................................***X | 63799.900 | | 63799.900 | .................................................................X | 63799.900 | | 63150.800 | .................................................................*******..******X | 65796.000 | | 64487.300 | ..................................................................*******..******X | 66541.600 | | 65862.400 | ....................................................................*******X | 65912.500 | | 62374.300 | ................................................................*******..*X | 63947.100 | | 63947.100 | ..................................................................***X | 63947.100 | | 63947.100 | ..................................................................*******X | 64404.500 | | 49504.800 | ..................................................******X | 49504.800 | | 45587.500 | ..............................................*******X | 45587.500 | | 41346.400 | ..........................................X | 41346.400 | | 41346.400 | ..........................................**X | 41346.400 | | 41346.400 | ..........................................***..**************X | 43432.900 | | 39096.900 | .......................................X | 39096.900 | | 39096.900 | .......................................***X | 39096.900 | | 39096.900 | .......................................**..*******X | 40840.100 | | 40840.100 | .........................................****X | 40840.100 | | 40840.100 | .........................................**..******X | 42042.300 | | 41233.200 | ..........................................******X | 41233.200 | | 38430.900 | .......................................*X | 38430.900 | | 38430.900 | .......................................**X | 38430.900 | | 38430.900 | .......................................***..******X | 40109.300 | | 35330.900 | ....................................*******..*******.****X | 38233.600 | | 38233.600 | .......................................**X | 38233.600 | | 37954.400 | ......................................******X | 37954.400 | | 35627.200 | ....................................*******...*******X | 38462.500 | | 34926.700 | ...................................*******..*******.******X | 37467.100 | | 37467.100 | ......................................*..******X | 39710.500 | | 36985.400 | .....................................*******.....*******X | 41031.500 | | 36619.600 | .....................................X | 36619.600 | | 36619.600 | .....................................***X | 36619.600 | | 36619.600 | .....................................**.********.******X | 38424.500 | | 34021.900 | ..................................*******....*******X | 38024.900 | | 36637.800 | .....................................*******.*******.*******X | 38782.300 | | 35989.100 | ....................................*******...******X | 38920.400 | | 36790.500 | .....................................*************X | 37156.700 | | 35893.400 | ....................................*******X | 35893.400 | | 34100.700 | ..................................******X | 34100.700 | | 33067.400 | .................................*******..******X | 35023.400 | | 32500.600 | .................................********..******X | 34437.400 | | 33672.900 | ..................................******X | 33672.900 | | 30612.800 | ...............................********.*************X | 31877.900 | | 29464.400 | .............................*******....**************X | 32861.600 | | 29390.100 | .............................*******...******X | 31768.900 | | 31153.000 | ...............................*******X | 31153.000 | | 30364.900 | ..............................*******...******X | 33240.600 | | 30956.600 | ...............................********..*******.******X | 33935.400 | | 30688.700 | ...............................********.*******.******X | 32632.400 | | 30439.500 | ..............................********..******X | 32082.800 | | 31674.700 | ................................*********************X | 32116.200 | | 30867.600 | ...............................*******.....******X | 35572.800 | | 31667.600 | ................................X | 31667.600 | | 30189.400 | ..............................*******....******X | 33590.900 | | 29696.200 | ..............................********..******X | 32413.500 | | 31822.800 | ................................********.******X | 32473.300 | | 31402.500 | ...............................********.******X | 31759.200 | | 31411.700 | ...............................********..******X | 32873.100 | | 31531.000 | ................................*******X | 31531.000 | | 30231.500 | ..............................********..*******.******X | 32598.200 | | 32506.600 | .................................X | 32506.600 | | 31208.100 | ...............................*******..*******.*******X | 33796.000 | | 31596.200 | ................................*******.******X | 32455.600 | | 29793.300 | ..............................********...******X | 32978.200 | | 30015.300 | ..............................********.....******X | 34794.400 | | 31836.800 | ................................*******.X | 32749.900 | | 32235.300 | ................................*******..******X | 33755.400 | | 31478.900 | ................................**************X | 31583.600 | | 30723.800 | ...............................********..*************X | 32839.800 | | 31590.900 | ................................********.******X | 32643.300 | | 30522.100 | ...............................***************.******X | 31719.700 | | 29103.600 | .............................********....******X | 33295.900 | | 31817.500 | ................................******X | 31817.500 | | 31442.900 | ...............................********............********.............********........********..*******X | 64574.700 | | 62821.500 | ................................................................*******X | 62821.500 | | 62103.300 | ................................................................********.*******X | 63612.700 | | 62607.700 | ................................................................********..**************X | 64583.200 | | 63983.300 | ..................................................................********.*******X | 65101.800 | | 63361.700 | .................................................................********..**************X | 65757.300 | | 63778.600 | .................................................................********.....*******X | 68054.700 | | 67625.500 | .....................................................................*******X | 67625.500 | | 64280.800 | ..................................................................********...********.*******X | 68499.300 | | 62646.400 | ................................................................********.....*******X | 67117.900 | | 63585.800 | .................................................................*******X | 63585.800 | | 63357.000 | .................................................................***************.*******X | 64119.800 | | 61539.500 | ...............................................................********..*******X | 63071.800 | | 62454.300 | ................................................................*******X | 62454.300 | | 59581.100 | .............................................................********....*******X | 63877.700 | | 61791.300 | ...............................................................******X | 61791.300 | | 60316.300 | ..............................................................***********************X | 61004.200 | | 57275.800 | ...........................................................********.*******X | 59100.300 | | 57749.900 | ...........................................................*******..*******X | 59186.000 | | 59058.200 | ............................................................********..********.*******X | 61356.700 | | 59848.700 | .............................................................********.******X | 60329.100 | | 58032.900 | ...........................................................********...*******X | 60092.900 | | 57919.500 | ...........................................................********..********.*******X | 60492.100 | | 59803.900 | .............................................................*******X | 59803.900 | | 57563.300 | ...........................................................********....******X | 61282.700 | | 59425.300 | .............................................................****************.*******X | 60918.000 | | 58063.600 | ...........................................................*******X | 58063.600 | | 57322.700 | ...........................................................*******...***************X | 60994.200 | | 58233.200 | ............................................................********.********..*******X | 61908.700 | | 59021.100 | ............................................................********..***************..***************X | 62464.400 | | 59424.800 | .............................................................********.*******X | 60433.400 | | 60403.100 | ..............................................................*******X | 60403.100 | | 57303.300 | ...........................................................********....********...*******.*******X | 65239.100 | | 62930.500 | ................................................................*******X | 62930.500 | | 60631.100 | ..............................................................***X | 60631.100 | +-----------+----------------------------------------------------------------------------------------------------------------+-----------+ 2022-03-06 08:04:19,790 | test | INFO | pool-5-thread-14 | [table_view:display:72] Ops trend for bucket 'bucket1' +--------+------------------------------------------------------------------------------------------------------------------------------------+----------+ | Min | Trend | Max | +--------+------------------------------------------------------------------------------------------------------------------------------------+----------+ | 0.000 | *....................................................................................................*******X | 1204.000 | | 52.000 | ...******X | 52.000 | | 16.000 | *******.*******X | 32.000 | | 0.000 | *******.******X | 16.000 | | 0.000 | **********************................................................*******X | 588.000 | | 84.000 | .....******X | 84.000 | | 0.000 | *******......**************X | 80.000 | | 48.000 | ..******X | 48.000 | | 0.000 | ***********************.*******...........................................................................................*******X | 1116.000 | | 0.000 | ******************************.******X | 16.000 | | 0.000 | ***********************.............................................******X | 548.000 | | 16.000 | ******X | 16.000 | | 5.500 | X | 5.500 | | 0.000 | **************............................................*******X | 532.000 | | 0.000 | *******.********.******X | 36.000 | | 16.000 | **************...*******X | 52.000 | | 0.000 | *******..********.******X | 48.000 | | 0.000 | ********.******X | 16.000 | | 0.000 | ***************.********...******X | 52.000 | | 0.000 | *******......*******X | 80.000 | | 0.000 | ***************.*******.*******....*******X | 80.000 | | 16.000 | *******....*******X | 68.000 | | 0.000 | ***************.....******X | 64.000 | | 0.000 | *******.********.*******.*******X | 48.000 | | 0.000 | *****************************..*******......*******X | 108.000 | | 48.000 | ..******X | 48.000 | | 20.000 | *******X | 20.000 | | 16.000 | ******X | 16.000 | | 0.000 | ***************.*******.******X | 32.000 | | 0.000 | ********..........******X | 132.000 | | 0.000 | ******************************..******X | 32.000 | | 0.000 | ********..*******X | 36.000 | | 16.000 | *************X | 16.000 | | 0.000 | ********.......******X | 96.000 | | 48.000 | ..******X | 48.000 | | 16.200 | X | 16.200 | | 0.000 | *****************************.....******X | 72.000 | | 0.000 | ***************.....*******X | 64.000 | | 0.000 | *******....*******X | 60.000 | | 16.000 | ******X | 16.000 | | 0.000 | ********..******X | 36.000 | | 16.000 | *******X | 16.000 | | 0.000 | *******.**************X | 20.000 | | 0.000 | ********.*******......******X | 88.000 | | 0.000 | *******.....******X | 68.000 | | 22.500 | X | 22.500 | | 0.000 | ************************************.******X | 20.000 | | 0.000 | ********..**************X | 36.000 | | 0.000 | **********************.******X | 20.000 | | 0.000 | **************.*******.*..******X | 52.000 | | 0.000 | *******.*******X | 16.000 | | 0.000 | ********..******X | 36.000 | | 0.000 | ***************.******X | 20.000 | | 0.000 | ****************.******X | 16.000 | | 0.000 | *******..*******X | 32.000 | | 0.000 | *******.*******X | 20.000 | | 0.000 | *******..******X | 36.000 | | 0.000 | ***************.******X | 16.000 | | 0.000 | ********...........******X | 136.000 | | 16.000 | *******X | 16.000 | | 0.000 | ******************************.*******X | 16.000 | | 0.000 | *********************************************.*******..*******X | 40.000 | | 0.000 | ********.******X | 20.000 | | 0.000 | ***************..*******X | 32.000 | | 0.000 | *************************************.*.******X | 32.000 | | 0.000 | **********************.*******X | 20.000 | | 0.000 | *******************************..*******X | 36.000 | | 0.000 | ********.*******X | 20.000 | | 0.000 | ********...********...*******X | 76.000 | | 0.000 | ****************.......*******X | 92.000 | | 0.000 | ****************..*******X | 36.000 | | 0.000 | ********..*******X | 36.000 | | 16.000 | ******X | 16.000 | | 0.000 | ************************..*********.*******X | 40.000 | | 0.000 | ********.********........*******X | 112.000 | | 0.000 | ***************..*******X | 36.000 | | 20.000 | ****************........*******X | 112.000 | | 20.000 | ****************..*******X | 40.000 | | 0.000 | ********************************..********.*******X | 48.000 | | 40.000 | ..*******X | 40.000 | | 16.000 | ***************X | 20.000 | | 0.000 | ********..****************......******X | 104.000 | | 0.000 | ********..*******X | 36.000 | | 0.000 | ********....*******X | 52.000 | | 0.000 | ****************...***************X | 48.000 | | 16.000 | *******X | 16.000 | | 0.000 | ********..*******X | 32.000 | | 0.000 | ***********************..*******X | 32.000 | | 16.000 | ********.*******X | 32.000 | | 0.000 | ********.********.***************X | 32.000 | | 0.000 | ********..********.*******X | 48.000 | | 32.000 | .******X | 32.000 | | 0.000 | ****************.**************X | 16.000 | | 0.000 | ********************X | 0.000 | +--------+------------------------------------------------------------------------------------------------------------------------------------+----------+ 2022-03-06 08:04:21,062 | test | INFO | pool-5-thread-24 | [table_view:display:72] Ops trend for bucket 'default' +-----------+------------------------------------------------------------------------------------------------------+-----------+ | Min | Trend | Max | +-----------+------------------------------------------------------------------------------------------------------+-----------+ | 60631.100 | ...................................................................................................X | 60631.100 | +-----------+------------------------------------------------------------------------------------------------------+-----------+ 2022-03-06 08:04:21,174 | test | WARNING | MainThread | [rest_client:get_nodes:1896] 172.23.100.35 - Node not part of cluster inactiveFailed 2022-03-06 08:04:35,693 | test | INFO | MainThread | [bucket_ready_functions:perform_tasks_from_spec:4993] Performing scope/collection specific operations 2022-03-06 08:04:35,841 | test | INFO | pool-5-thread-15 | [table_view:display:72] Rebalance Overview +----------------+----------+-----------------------+----------------+--------------+-----------------------+ | Nodes | Services | Version | CPU | Status | Membership / Recovery | +----------------+----------+-----------------------+----------------+--------------+-----------------------+ | 172.23.105.206 | kv | 7.1.0-2440-enterprise | 78.2705389848 | Cluster node | active / none | | 172.23.106.177 | kv | 7.1.0-2440-enterprise | 44.8232163224 | Cluster node | active / none | | 172.23.100.35 | kv | 7.1.0-2440-enterprise | 0.212899185974 | Cluster node | inactiveAdded / delta | | 172.23.105.164 | kv | 7.1.0-2440-enterprise | 6.90218757858 | Cluster node | active / none | +----------------+----------+-----------------------+----------------+--------------+-----------------------+ 2022-03-06 08:04:35,859 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:04:40,323 | test | INFO | MainThread | [bucket_ready_functions:perform_tasks_from_spec:5083] Done Performing scope/collection specific operations 2022-03-06 08:04:45,898 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:04:55,973 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:05:06,026 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:05:16,082 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:05:26,124 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:05:36,174 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:05:46,227 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:05:56,269 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:06:06,326 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:06:16,395 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:06:26,460 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:06:36,517 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:06:46,586 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:06:56,644 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:07:06,680 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:07:16,726 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:07:26,763 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:07:36,802 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:07:46,851 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:07:56,914 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:08:06,977 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:08:17,015 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:08:27,079 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:08:37,141 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:08:47,184 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:08:57,226 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:09:07,278 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:09:17,325 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:09:27,380 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:09:37,437 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:09:47,476 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:09:57,529 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:10:07,575 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:10:17,637 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:10:27,711 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:10:37,759 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:10:47,825 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:10:57,864 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:11:07,897 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:11:17,937 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:11:27,980 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:11:38,012 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:11:48,068 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:11:58,112 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:12:08,154 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:12:18,183 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:12:28,230 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:12:38,276 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:12:48,315 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:12:58,358 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:13:08,417 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:13:18,467 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:13:28,519 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:13:38,556 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:13:48,598 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:13:58,661 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:14:08,720 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:14:18,759 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:14:28,815 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:14:38,878 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:14:48,924 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:14:58,963 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:15:08,994 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:15:19,042 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:15:29,082 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:15:39,125 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:15:49,190 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:15:59,252 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:16:09,302 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:16:19,335 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:16:29,371 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:16:39,411 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:16:49,457 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:16:59,496 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:17:09,553 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:17:19,604 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:17:29,657 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:17:39,694 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:17:49,743 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:17:59,803 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:18:09,848 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:18:19,892 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:18:29,954 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:18:40,000 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:18:50,039 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:19:00,089 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:19:10,150 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:19:20,180 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:19:30,213 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:19:40,260 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:19:50,313 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:20:00,351 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:20:10,384 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:20:20,430 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:20:30,474 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:20:40,517 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:20:50,546 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:21:00,573 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:21:10,608 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:21:20,651 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:21:30,723 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:21:40,779 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:21:50,841 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:22:00,903 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:22:10,956 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:22:21,005 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:22:31,048 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:22:41,084 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:22:51,151 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:23:01,201 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:23:11,263 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:23:21,299 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:23:31,349 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:23:41,392 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:23:51,438 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:24:01,489 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:24:11,542 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:24:21,572 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:24:31,615 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:24:41,663 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:24:51,729 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:25:01,773 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:25:11,809 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:25:21,858 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:25:31,913 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:25:41,951 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:25:52,009 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:26:02,052 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:26:12,082 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:26:22,134 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:26:32,167 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:26:42,201 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:26:52,253 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:27:02,293 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:27:12,348 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:27:22,394 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:27:32,450 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:27:42,494 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:27:52,526 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 08:28:02,561 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 5.14 2022-03-06 08:28:12,638 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 6.97 2022-03-06 08:28:22,680 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 6.97 2022-03-06 08:28:32,755 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 6.97 2022-03-06 08:28:42,809 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 6.97 2022-03-06 08:28:52,914 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 6.97 2022-03-06 08:29:02,947 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 6.97 2022-03-06 08:29:12,990 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 6.97 2022-03-06 08:29:23,026 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 6.97 2022-03-06 08:29:33,061 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 6.97 2022-03-06 08:29:43,091 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 6.97 2022-03-06 08:29:53,119 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 6.97 2022-03-06 08:30:03,161 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 6.97 2022-03-06 08:30:13,209 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 6.97 2022-03-06 08:30:23,266 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 6.97 2022-03-06 08:30:33,302 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 6.97 2022-03-06 08:30:43,345 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 6.97 2022-03-06 08:30:53,378 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 6.97 2022-03-06 08:31:03,417 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 6.97 2022-03-06 08:31:13,464 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 6.97 2022-03-06 08:31:23,506 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 6.97 2022-03-06 08:31:33,566 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 6.97 2022-03-06 08:31:43,612 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 7.2 2022-03-06 08:31:53,658 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 7.2 2022-03-06 08:32:03,691 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 7.2 2022-03-06 08:32:13,733 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 7.2 2022-03-06 08:32:23,769 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 7.2 2022-03-06 08:32:33,816 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 7.2 2022-03-06 08:32:43,865 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 7.2 2022-03-06 08:32:53,911 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 7.2 2022-03-06 08:33:03,950 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 7.2 2022-03-06 08:33:13,993 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 7.2 2022-03-06 08:33:24,026 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 7.2 2022-03-06 08:33:34,053 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 7.89 2022-03-06 08:33:44,084 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 14.63 2022-03-06 08:33:54,153 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 15.65 2022-03-06 08:34:04,214 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 19.28 2022-03-06 08:34:14,265 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 21.56 2022-03-06 08:34:24,305 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 21.67 2022-03-06 08:34:34,338 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 21.79 2022-03-06 08:34:44,398 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 36.14 2022-03-06 08:34:54,440 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 51.07 2022-03-06 08:35:02,615 | test | INFO | pool-5-thread-22 | [table_view:display:72] Ops trend for bucket 'bucket2' +---------+---------------------------------------------------------------------------------------------------------------------+----------+ | Min | Trend | Max | +---------+---------------------------------------------------------------------------------------------------------------------+----------+ | 0.000 | *******....................................................................................................*******X | 2278.900 | | 0.000 | *******...*******X | 76.500 | | 0.000 | ***************.*******.......******X | 190.600 | | 15.300 | **************..............*.......******X | 489.800 | | 0.000 | **************....*******X | 95.300 | | 49.800 | .******X | 49.800 | | 0.000 | *******......******X | 141.200 | | 0.000 | ********.................******X | 390.200 | | 80.000 | ..******X | 80.000 | | 0.000 | ********....***X | 95.700 | | 95.700 | ...**X | 95.700 | | 49.800 | .******X | 49.800 | | 0.000 | ********....******X | 91.800 | | 0.000 | ***************..******X | 61.200 | | 49.800 | .******X | 49.800 | | 30.600 | *******.*******X | 45.900 | | 0.000 | ***************..******X | 65.100 | | 0.000 | **************...........................******X | 625.900 | | 0.000 | **************.......................******X | 545.900 | | 34.500 | *******X | 34.500 | | 0.000 | ***************.......******X | 164.300 | | 45.900 | .******X | 45.900 | | 0.000 | ********........******X | 191.400 | | 0.000 | *********************..*******X | 61.200 | | 0.000 | ***************..******X | 45.900 | | 15.300 | ********...******X | 84.300 | | 0.000 | ***************..**************X | 61.200 | | 0.000 | **************.******X | 34.500 | | 30.600 | ******X | 30.600 | | 0.000 | ********.*******.........*******X | 230.200 | | 34.500 | ******X | 34.500 | | 11.400 | X | 11.400 | | 0.000 | **************.******X | 38.400 | | 0.000 | *****************************.........******X | 205.900 | | 100.000 | ...******X | 100.000 | | 45.900 | .X | 45.900 | | 19.200 | ******X | 19.200 | | 0.000 | ***************..........******X | 248.600 | | 0.000 | **************X | 19.200 | | 0.000 | ***************....******X | 99.600 | | 30.600 | ******X | 30.600 | | 0.000 | ******************************...*******X | 72.900 | | 49.800 | .******X | 49.800 | | 15.300 | **************X | 15.300 | | 0.000 | ***********************.....******X | 126.300 | | 40.400 | X | 40.400 | | 0.000 | **********************..*******..******X | 99.600 | | 0.000 | ********..******X | 65.100 | | 15.300 | *******.*******X | 34.500 | | 0.000 | *******...******X | 72.900 | | 24.200 | X | 24.200 | | 0.000 | **************......******X | 149.000 | | 130.600 | ....******X | 130.600 | | 19.200 | *******X | 19.200 | | 0.000 | *************************************...*******X | 80.400 | | 0.000 | ***************.******X | 30.600 | | 0.000 | ***************......******X | 149.000 | | 0.000 | ***************....******X | 92.100 | | 43.300 | X | 43.300 | | 19.200 | *******....******X | 99.200 | | 0.000 | ********....******X | 100.000 | | 0.000 | *******...*******X | 84.300 | | 0.000 | *******.***************..**************X | 80.000 | | 0.000 | ***************.*******X | 34.500 | | 0.000 | ************************************X | 15.300 | | 0.000 | ***************..******X | 65.100 | | 0.000 | ********..******X | 61.200 | | 0.000 | ***************......*******X | 149.800 | | 15.300 | *************X | 15.300 | | 10.500 | X | 10.500 | | 0.000 | *******.******X | 38.400 | | 13.000 | X | 13.000 | | 0.000 | *******.***************.******X | 53.700 | | 15.300 | ********..******X | 65.100 | | 0.000 | ******************************....******X | 95.300 | | 0.000 | ********.....******X | 129.800 | | 30.600 | *******.*******X | 49.800 | | 0.000 | *******..******X | 45.900 | | 0.000 | *****************************X | 0.000 | +---------+---------------------------------------------------------------------------------------------------------------------+----------+ 2022-03-06 08:35:03,361 | test | INFO | pool-5-thread-12 | [table_view:display:72] Ops trend for bucket 'default' +-----------+----------------------------------------------------------------------------------------------------------------------------+------------+ | Min | Trend | Max | +-----------+----------------------------------------------------------------------------------------------------------------------------+------------+ | 0.000 | *******.....................................................................................********...............******X | 105257.900 | | 92020.600 | ......................................................................................**X | 92020.600 | | 92020.600 | ......................................................................................**X | 92020.600 | | 92020.600 | ......................................................................................*X | 92020.600 | | 89343.400 | ...................................................................................********.******X | 89937.700 | | 76798.500 | .......................................................................******X | 76798.500 | | 69735.800 | .................................................................******X | 69735.800 | | 69455.500 | ................................................................*******.....******X | 73906.500 | | 70554.500 | ..................................................................***********X | 70934.700 | | 70934.700 | ..................................................................***.******X | 72423.300 | | 66920.800 | ..............................................................*******.*X | 68081.900 | | 68081.900 | ...............................................................**X | 68081.900 | | 68081.900 | ...............................................................**...******X | 70843.700 | | 68478.000 | ................................................................X | 68478.000 | | 67483.000 | ...............................................................*****X | 67483.000 | | 65178.200 | ............................................................*******X | 65178.200 | | 61190.800 | .........................................................*********************X | 61721.800 | | 53308.100 | .................................................*******......*******......*...******X | 68915.100 | | 59322.400 | .......................................................******X | 59322.400 | | 59040.900 | .......................................................*******....**X | 64157.900 | | 64157.900 | ...........................................................****.........******X | 73131.700 | | 71425.700 | ..................................................................******X | 71425.700 | | 61334.000 | .........................................................X | 61334.000 | | 56160.600 | ....................................................X | 56160.600 | | 56160.600 | ....................................................******X | 56263.000 | | 56263.000 | ....................................................******....******X | 60829.800 | | 60205.100 | ........................................................*******X | 60205.100 | | 53056.900 | .................................................*******....******X | 57656.600 | | 52289.200 | ................................................******X | 52289.200 | | 49431.400 | .............................................***X | 49431.400 | | 49431.400 | .............................................X | 49431.400 | | 49431.400 | .............................................*X | 49431.400 | | 47169.600 | ...........................................*X | 47169.600 | | 47169.600 | ...........................................**X | 47169.600 | | 47169.600 | ...........................................*X | 47169.600 | | 46943.300 | ...........................................*************X | 47097.600 | | 46016.500 | ..........................................*******.*******X | 46632.000 | | 46539.000 | ...........................................******X | 46539.000 | | 43878.400 | ........................................******X | 43878.400 | | 39504.800 | ....................................*******X | 39504.800 | | 39203.900 | ....................................***************..*******.******X | 42787.700 | | 41631.400 | ......................................******X | 41631.400 | | 39031.000 | ....................................********..******X | 41512.000 | | 39644.100 | ....................................X | 39644.100 | | 39644.100 | ....................................******..*******X | 41289.300 | | 39082.000 | ....................................******X | 39082.000 | | 35057.700 | ................................*******.......******X | 42667.900 | | 41041.300 | .....................................********....*******......******X | 50710.100 | | 50214.500 | ..............................................X | 50214.500 | | 50125.300 | ..............................................*****X | 50125.300 | | 49429.200 | .............................................*******...*******.******X | 53613.700 | | 52826.300 | .................................................*******X | 52826.300 | | 48335.800 | ............................................******X | 48335.800 | | 47918.600 | ............................................******X | 47918.600 | | 43139.800 | .......................................*******X | 43139.800 | | 41873.800 | ......................................******X | 41873.800 | | 40445.300 | .....................................*******X | 40445.300 | | 39636.800 | ....................................*******........****X | 47998.600 | | 47998.600 | ............................................**........*******X | 55844.000 | | 41302.400 | ......................................******X | 41302.400 | | 38290.300 | ...................................******X | 38290.300 | | 35216.200 | ................................*******..*********X | 37667.600 | | 37667.600 | ..................................****..*..******X | 41758.400 | | 39801.800 | ....................................******X | 39801.800 | | 39351.800 | ....................................*******X | 39351.800 | | 36282.000 | .................................******X | 36282.000 | | 35834.500 | .................................*******.********..******X | 39914.300 | | 36472.200 | .................................X | 36472.200 | | 35710.500 | ................................*******..*******.********..******X | 40006.800 | | 35815.100 | .................................***************..X | 37896.600 | | 37896.600 | ...................................*X | 37896.600 | | 37896.600 | ...................................*****.******X | 39201.100 | | 38846.200 | ...................................*******.******X | 39328.800 | | 38646.900 | ...................................********...******X | 41125.900 | | 37373.500 | ..................................**************X | 37835.900 | | 36346.700 | .................................*******....*******X | 40580.200 | | 38705.200 | ...................................******X | 38705.200 | | 36931.800 | ..................................******X | 36931.800 | | 36214.900 | .................................*******.**X | 37242.900 | | 37242.900 | ..................................***X | 37242.900 | | 37242.900 | ..................................*******X | 37244.300 | | 34406.500 | ...............................**X | 34406.500 | | 34406.500 | ...............................****..*******X | 36801.700 | | 34648.400 | ...............................****X | 34648.400 | | 34648.400 | ...............................**.******X | 35690.000 | | 34737.100 | ................................X | 34737.100 | | 34218.500 | ...............................******X | 34218.500 | | 33509.600 | ..............................******X | 33509.600 | | 28854.200 | ..........................*******...******X | 32609.200 | | 31462.400 | ............................******X | 31462.400 | | 31207.000 | ............................********..*******..*******X | 34971.000 | | 28507.300 | ..........................*******....*******X | 33130.700 | | 32334.300 | .............................*******...******X | 35002.100 | | 31802.600 | .............................********..******X | 34532.600 | | 32626.100 | .............................********..******X | 33974.700 | | 30823.000 | ............................*******....*******X | 35336.100 | | 34025.000 | ...............................*******.******X | 35148.300 | | 31977.400 | .............................*******.********.******X | 34001.200 | | 32148.200 | .............................*******.******X | 32779.400 | | 31506.500 | ............................********....******X | 35609.900 | | 33619.800 | ..............................******X | 33619.800 | | 33303.200 | ..............................*******X | 33303.200 | | 32695.300 | ..............................*******.******X | 33693.100 | | 32811.300 | ..............................*******X | 32811.300 | | 32720.000 | ..............................******X | 32720.000 | | 31566.300 | ............................********...******X | 34429.800 | | 33561.300 | ..............................******X | 33561.300 | | 30316.900 | ...........................*******.....*******X | 35338.900 | | 33843.500 | ...............................*************X | 34668.600 | | 31149.500 | ............................*******X | 31149.500 | | 30990.800 | ............................*******......******X | 37072.400 | | 32644.100 | ..............................*******X | 32644.100 | | 32503.100 | .............................*******..*******..******X | 35874.100 | | 34664.500 | ...............................X | 34664.500 | | 33892.700 | ...............................*******.******X | 35411.500 | | 33089.200 | ..............................*******X | 33089.200 | | 32407.600 | .............................*******.....******X | 37104.500 | | 34553.200 | ...............................********.*************X | 35288.500 | | 34766.700 | ................................**************X | 34930.600 | | 34767.700 | ................................*************X | 35594.100 | | 35194.100 | ................................*******X | 35194.100 | | 34312.600 | ...............................*******.******X | 34973.200 | | 34318.900 | ...............................********.....*******....*******........********...*******......******X | 61660.100 | | 59565.900 | .......................................................*******X | 59565.900 | | 59517.000 | .......................................................******X | 59517.000 | | 56267.300 | ....................................................*******X | 56267.300 | | 54462.500 | ..................................................******X | 54462.500 | +-----------+----------------------------------------------------------------------------------------------------------------------------+------------+ 2022-03-06 08:35:03,457 | test | INFO | pool-5-thread-20 | [table_view:display:72] Ops trend for bucket 'bucket1' +---------+-----------------------------------------------------------------------------------------------------------------------------------+----------+ | Min | Trend | Max | +---------+-----------------------------------------------------------------------------------------------------------------------------------+----------+ | 0.000 | *******.................................................................*******X | 794.600 | | 0.000 | *******...*******X | 45.900 | | 0.000 | ***************..........................................*******..........................................................******X | 1210.200 | | 30.600 | .*******X | 30.600 | | 0.000 | *******............******X | 149.400 | | 0.000 | **********************.........******X | 114.900 | | 15.300 | *******............*......******X | 240.400 | | 15.300 | *******......******X | 91.800 | | 0.000 | *************************************............*******X | 156.500 | | 0.000 | *******..*******.......******X | 111.000 | | 99.500 | .......X | 99.500 | | 95.300 | ......******X | 95.300 | | 15.300 | *******.........................................******X | 519.200 | | 351.200 | ............................X | 351.200 | | 15.300 | ******X | 15.300 | | 0.000 | *******.*.******X | 30.600 | | 0.000 | **************..******X | 30.600 | | 0.000 | ***************.........******X | 110.600 | | 19.200 | ******X | 19.200 | | 0.000 | ***************..............................*...............******X | 545.900 | | 114.500 | ........******X | 114.500 | | 0.000 | *******..*..******X | 49.800 | | 0.000 | *******.***************..******X | 45.900 | | 0.000 | ***************.....*******X | 61.200 | | 0.000 | *******...................................................*******X | 619.200 | | 15.300 | ******X | 15.300 | | 0.000 | *******...*..*******.******X | 76.500 | | 15.300 | *******..******X | 45.900 | | 30.600 | .**************X | 30.600 | | 0.000 | *******.********.........******X | 129.800 | | 30.600 | .******X | 30.600 | | 19.600 | X | 19.600 | | 0.000 | **************....*........*****X | 149.000 | | 0.000 | *******..*..******X | 49.800 | | 19.200 | **************......**X | 95.700 | | 95.700 | ......****.........******X | 194.900 | | 15.300 | *******X | 15.300 | | 0.000 | **************.......*******X | 91.800 | | 15.300 | *******X | 15.300 | | 0.000 | **************.********..........******X | 141.200 | | 0.000 | ***********************...******X | 38.400 | | 0.000 | ********..........******X | 130.200 | | 90.900 | ......X | 90.900 | | 19.200 | ******X | 19.200 | | 0.000 | **********************....*******X | 53.700 | | 0.000 | *******............*******X | 153.300 | | 0.000 | ***************.........******X | 115.300 | | 49.800 | ...*******....******X | 103.500 | | 0.000 | ********..*******................******X | 229.800 | | 15.300 | ****************..........******X | 134.500 | | 34.500 | .*******X | 34.500 | | 30.600 | .*******.*************X | 38.400 | | 15.300 | *******X | 15.300 | | 9.800 | X | 9.800 | | 0.000 | *********************.********...********.........******X | 160.400 | | 34.500 | .*******X | 34.500 | | 0.000 | *******....*******X | 49.800 | | 0.000 | *******.********.******X | 30.600 | | 0.000 | *******...*******X | 38.400 | | 15.300 | *******........*******X | 110.600 | | 15.300 | ******X | 15.300 | | 4.800 | X | 4.800 | | 0.000 | *********************....*...******X | 88.200 | | 0.000 | ***************...*******X | 45.900 | | 0.000 | *******...******X | 38.400 | | 1.600 | X | 1.600 | | 0.000 | *******.******X | 19.200 | | 15.300 | **************X | 15.300 | | 0.000 | ***********************.**************....*******X | 65.100 | | 0.000 | ***************...*******X | 38.400 | | 34.500 | .******X | 34.500 | | 0.000 | *******..*....******X | 78.300 | | 9.900 | *******....********.....*******X | 115.300 | | 0.000 | *******....******X | 53.700 | | 15.300 | ********.........******X | 122.700 | | 95.300 | ......******X | 95.300 | | 75.300 | .....X | 75.300 | | 45.900 | ..**************X | 45.900 | | 15.300 | ******X | 15.300 | | 0.000 | ***************.**************X | 19.200 | | 0.000 | *******.********.*******...********..******X | 95.700 | | 61.200 | ....******X | 61.200 | | 0.000 | *****************************X | 0.000 | +---------+-----------------------------------------------------------------------------------------------------------------------------------+----------+ 2022-03-06 08:35:04,467 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:35:04,815 | test | INFO | pool-5-thread-29 | [table_view:display:72] Ops trend for bucket 'default' +-----------+------------------------------------------------------------------------------------------------------+-----------+ | Min | Trend | Max | +-----------+------------------------------------------------------------------------------------------------------+-----------+ | 30980.900 | ...................................................................................................X | 30980.900 | +-----------+------------------------------------------------------------------------------------------------------+-----------+ 2022-03-06 08:35:14,496 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:35:24,525 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:35:34,552 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:35:44,581 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:35:54,609 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:36:04,640 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:36:14,670 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:36:24,698 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:36:34,729 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:36:44,759 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:36:54,788 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:37:04,816 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:37:14,846 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:37:24,875 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:37:34,905 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:37:44,934 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:37:54,963 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:38:04,993 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:38:15,022 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:38:25,051 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:38:35,081 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:38:45,109 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:38:55,140 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:39:05,170 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:39:15,200 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:39:25,230 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:39:35,259 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:39:45,288 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:39:55,318 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:40:05,348 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:40:15,377 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:40:25,405 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:40:35,434 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:40:45,466 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:40:55,496 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:41:05,525 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:41:15,553 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:41:25,584 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:41:35,614 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:41:45,642 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:41:55,671 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:42:05,700 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:42:15,730 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:42:25,759 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:42:35,789 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:42:45,818 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:42:55,845 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:43:05,875 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:43:15,904 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:43:25,933 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:43:35,961 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:43:45,992 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:43:56,022 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:44:06,051 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:44:16,078 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:44:26,108 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:44:36,138 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:44:46,167 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:44:56,196 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:45:06,226 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:45:16,256 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:45:26,290 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:45:36,322 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:45:46,355 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:45:56,382 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:46:06,411 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:46:16,440 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:46:26,470 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:46:36,502 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:46:46,530 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:46:56,559 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:47:06,591 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:47:16,618 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:47:26,648 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:47:36,678 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:47:46,707 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:47:56,736 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:48:06,766 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:48:16,796 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:48:26,825 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:48:36,855 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:48:46,884 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:48:56,915 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:49:06,944 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:49:16,976 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:49:27,006 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:49:37,035 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:49:47,063 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:49:57,094 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:50:07,122 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:50:17,153 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:50:27,181 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:50:37,210 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:50:47,240 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:50:57,270 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:51:07,302 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:51:17,331 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:51:27,359 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:51:37,388 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:51:47,418 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:51:57,448 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:52:07,480 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:52:17,509 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:52:27,539 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:52:37,569 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:52:47,598 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:52:57,628 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:53:07,657 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:53:17,684 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:53:27,713 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:53:37,743 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:53:47,770 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:53:57,799 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:54:07,828 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:54:17,855 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:54:27,884 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:54:37,914 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:54:47,943 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:54:57,971 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:55:08,000 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.33 2022-03-06 08:55:18,029 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.68 2022-03-06 08:55:28,058 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.68 2022-03-06 08:55:38,086 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.79 2022-03-06 08:55:48,115 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.79 2022-03-06 08:55:58,144 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.79 2022-03-06 08:56:08,174 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 58.79 2022-03-06 08:56:18,203 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 59.02 2022-03-06 08:56:28,233 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 59.02 2022-03-06 08:56:38,263 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 59.02 2022-03-06 08:56:48,292 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 59.02 2022-03-06 08:56:58,319 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 59.02 2022-03-06 08:57:08,348 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 59.93 2022-03-06 08:57:18,378 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 59.93 2022-03-06 08:57:28,407 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 60.28 2022-03-06 08:57:38,436 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 61.08 2022-03-06 08:57:48,463 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 65.31 2022-03-06 08:57:58,490 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 65.31 2022-03-06 08:58:08,519 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 65.31 2022-03-06 08:58:18,549 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 65.31 2022-03-06 08:58:28,579 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 65.31 2022-03-06 08:58:38,608 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 65.31 2022-03-06 08:58:48,638 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 65.31 2022-03-06 08:58:58,667 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 65.31 2022-03-06 08:59:08,698 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 65.31 2022-03-06 08:59:18,727 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 65.31 2022-03-06 08:59:28,757 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 65.31 2022-03-06 08:59:38,802 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 65.31 2022-03-06 08:59:48,831 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 65.31 2022-03-06 08:59:58,859 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 65.42 2022-03-06 09:00:08,888 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 65.42 2022-03-06 09:00:18,918 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 65.42 2022-03-06 09:00:28,948 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 65.42 2022-03-06 09:00:38,977 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 65.42 2022-03-06 09:00:49,007 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 65.54 2022-03-06 09:00:59,036 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 65.54 2022-03-06 09:01:09,066 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 65.54 2022-03-06 09:01:19,095 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 65.54 2022-03-06 09:01:29,125 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 65.54 2022-03-06 09:01:39,154 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 65.54 2022-03-06 09:01:49,184 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 65.54 2022-03-06 09:01:59,213 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 65.54 2022-03-06 09:02:09,240 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 72.74 2022-03-06 09:02:19,267 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 75.59 2022-03-06 09:02:29,295 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 75.71 2022-03-06 09:02:39,322 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 78.67 2022-03-06 09:02:49,349 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: running, progress: 80.14 2022-03-06 09:03:04,387 | test | INFO | pool-5-thread-15 | [task:check:415] Rebalance - status: none, progress: 100 2022-03-06 09:03:04,401 | test | INFO | pool-5-thread-15 | [task:check:474] Rebalance completed with progress: 100% in 3508.56099987 sec 2022-03-06 09:03:04,486 | test | INFO | MainThread | [collections_rebalance:get_zone_info:969] nodes in zone inside get_zone_info():{'Group 1': ['172.23.105.164', '172.23.100.35', '172.23.106.177', '172.23.105.206']} 2022-03-06 09:03:06,418 | test | INFO | MainThread | [bucket_ready_functions:validate_docs_per_collections_all_buckets:4742] Validating collection stats and item counts 2022-03-06 09:09:41,611 | test | ERROR | MainThread | [bucket_ready_functions:validate_manifest_uid:4605] Bucket UID mismatch. Expected: 46, Actual: 2f 2022-03-06 09:09:41,611 | test | WARNING | MainThread | [bucket_ready_functions:validate_docs_per_collections_all_buckets:4759] Bucket manifest UID mismatch! 2022-03-06 09:09:45,059 | test | ERROR | MainThread | [bucket_ready_functions:validate_manifest_uid:4605] Bucket UID mismatch. Expected: 43, Actual: 2c 2022-03-06 09:09:45,061 | test | WARNING | MainThread | [bucket_ready_functions:validate_docs_per_collections_all_buckets:4759] Bucket manifest UID mismatch! 2022-03-06 09:09:48,592 | test | ERROR | MainThread | [bucket_ready_functions:validate_manifest_uid:4605] Bucket UID mismatch. Expected: 73, Actual: 4a 2022-03-06 09:09:48,592 | test | WARNING | MainThread | [bucket_ready_functions:validate_docs_per_collections_all_buckets:4759] Bucket manifest UID mismatch! 2022-03-06 09:09:52,969 | test | INFO | MainThread | [table_view:display:72] Bucket statistics +---------+-----------+-----------------+----------+------------+-----+-----------+-----------+------------+------------+---------------+ | Bucket | Type | Storage Backend | Replicas | Durability | TTL | Items | RAM Quota | RAM Used | Disk Used | ARR | +---------+-----------+-----------------+----------+------------+-----+-----------+-----------+------------+------------+---------------+ | bucket1 | couchbase | couchstore | 1 | none | 0 | 129917 | 7.81 GiB | 108.98 MiB | 142.45 MiB | 100 | | bucket2 | couchbase | magma | 1 | none | 0 | 132675 | 4.00 GiB | 219.17 MiB | 186.21 MiB | 100 | | default | couchbase | magma | 1 | none | 0 | 108807800 | 4.00 GiB | 1.56 GiB | 33.06 GiB | 3.54169094495 | +---------+-----------+-----------------+----------+------------+-----+-----------+-----------+------------+------------+---------------+ 2022-03-06 09:09:53,378 | test | INFO | MainThread | [collections_base:tearDown:84] Bucket: bucket2, Active Resident ratio(DGM): 100% 2022-03-06 09:09:53,378 | test | INFO | MainThread | [collections_base:tearDown:87] Bucket: bucket2, Replica Resident ratio(DGM): 100% 2022-03-06 09:10:00,279 | test | INFO | MainThread | [collections_base:tearDown:84] Bucket: bucket1, Active Resident ratio(DGM): 100% 2022-03-06 09:10:00,279 | test | INFO | MainThread | [collections_base:tearDown:87] Bucket: bucket1, Replica Resident ratio(DGM): 100% 2022-03-06 09:10:07,178 | test | INFO | MainThread | [collections_base:tearDown:84] Bucket: default, Active Resident ratio(DGM): 2% 2022-03-06 09:10:07,178 | test | INFO | MainThread | [collections_base:tearDown:87] Bucket: default, Replica Resident ratio(DGM): 2% 2022-03-06 09:10:18,664 | test | INFO | MainThread | [basetestcase:check_logs:824] Comparing timestamps: Log's latest timestamp: 2022-03-06 06:13:45, Test's start timestamp is 2022-03-06 06:07:24.945000 2022-03-06 09:10:18,667 | test | INFO | MainThread | [basetestcase:check_coredump_exist:921] unwanted messages in /opt/couchbase/var/lib/couchbase/logs/memcached.log.000015.txt 2022-03-06 09:10:18,667 | test | INFO | MainThread | [basetestcase:check_logs:824] Comparing timestamps: Log's latest timestamp: 2022-03-06 06:13:45, Test's start timestamp is 2022-03-06 06:07:24.945000 2022-03-06 09:10:18,668 | test | CRITICAL | MainThread | [basetestcase:check_coredump_exist:924] 172.23.105.164: Found ' CRITICAL ' logs - 2022-03-06T06:13:43.591107-08:00 CRITICAL [(default) magma_0/kvstore-864/rev-000000001/localIndex]Failed to write tree state /data/default/magma.0/kvstore-864/rev-000000001/localIndex/state.1095: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0 2022-03-06T06:13:43.591474-08:00 CRITICAL [(default) magma_0/kvstore-864/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-864/rev-000000001/seqIndex/state.1096: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0 2022-03-06T06:13:43.591782-08:00 CRITICAL [(default) magma_0/kvstore-864/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-864/rev-000000001/seqIndex/state.1097: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0 2022-03-06T06:13:43.592119-08:00 CRITICAL [(default) magma_0/kvstore-864/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-864/rev-000000001/seqIndex/state.1098: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0 2022-03-06T06:13:43.592507-08:00 CRITICAL [(default) magma_0/kvstore-864/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-864/rev-000000001/seqIndex/state.1099: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0 2022-03-06T06:13:43.592973-08:00 CRITICAL [(default) magma_0/kvstore-864/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-864/rev-000000001/seqIndex/state.1100: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0 2022-03-06T06:13:43.593463-08:00 CRITICAL [(default) magma_0/kvstore-864/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-864/rev-000000001/seqIndex/state.1101: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0 2022-03-06T06:13:43.594024-08:00 CRITICAL [(default) magma_0/kvstore-864/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-864/rev-000000001/seqIndex/state.1102: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0 2022-03-06T06:13:43.594643-08:00 CRITICAL [(default) magma_0/kvstore-864/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-864/rev-000000001/seqIndex/state.1103: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0 2022-03-06T06:13:43.595344-08:00 CRITICAL [(default) magma_0/kvstore-864/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-864/rev-000000001/seqIndex/state.1104: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0 2022-03-06T06:13:43.596049-08:00 CRITICAL [(default) magma_0/kvstore-864/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-864/rev-000000001/seqIndex/state.1105: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0 2022-03-06T06:13:43.596793-08:00 CRITICAL [(default) magma_0/kvstore-864/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-864/rev-000000001/seqIndex/state.1106: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0 2022-03-06T06:13:43.597558-08:00 CRITICAL [(default) magma_0/kvstore-864/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-864/rev-000000001/seqIndex/state.1107: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0 2022-03-06T06:13:43.598424-08:00 CRITICAL [(default) magma_0/kvstore-864/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-864/rev-000000001/seqIndex/state.1108: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0 2022-03-06T06:13:43.599804-08:00 CRITICAL [(default) magma_0/kvstore-864/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-864/rev-000000001/seqIndex/state.1109: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0 2022-03-06T06:13:43.600894-08:00 CRITICAL [(default) magma_0/kvstore-864/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-864/rev-000000001/seqIndex/state.1110: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0 2022-03-06T06:13:43.602235-08:00 CRITICAL [(default) magma_0/kvstore-864/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-864/rev-000000001/seqIndex/state.1111: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0 2022-03-06T06:13:43.603541-08:00 CRITICAL [(default) magma_0/kvstore-864/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-864/rev-000000001/seqIndex/state.1112: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0 2022-03-06T06:13:43.605085-08:00 CRITICAL [(default) magma_0/kvstore-864/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-864/rev-000000001/seqIndex/state.1113: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0 2022-03-06T06:13:43.606921-08:00 CRITICAL [(default) magma_0/kvstore-864/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-864/rev-000000001/seqIndex/state.1114: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0 2022-03-06T06:13:43.608644-08:00 CRITICAL [(default) magma_0/kvstore-864/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-864/rev-000000001/seqIndex/state.1115: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0 2022-03-06T06:13:45.062993-08:00 CRITICAL [(default) magma_0/kvstore-48/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-48/rev-000000001/seqIndex/state.834: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0/kvstore-48/rev-000000001 2022-03-06T06:13:45.068642-08:00 CRITICAL [(default) magma_0/kvstore-56/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-56/rev-000000001/seqIndex/state.837: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0 2022-03-06T06:13:45.073736-08:00 CRITICAL [(default) magma_0/kvstore-64/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-64/rev-000000001/seqIndex/state.827: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0 2022-03-06T06:13:45.080689-08:00 CRITICAL [(default) magma_0/kvstore-72/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-72/rev-000000001/seqIndex/state.832: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0 2022-03-06T06:13:45.084886-08:00 CRITICAL [(default) magma_0/kvstore-80/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-80/rev-000000001/seqIndex/state.840: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0 2022-03-06T06:13:45.089529-08:00 CRITICAL [(default) magma_0/kvstore-256/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-256/rev-000000001/seqIndex/state.839: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0 2022-03-06T06:13:45.096475-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.839: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0 2022-03-06T06:13:45.102642-08:00 CRITICAL [(default) magma_0/kvstore-272/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-272/rev-000000001/seqIndex/state.826: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0 2022-03-06T06:13:45.110251-08:00 CRITICAL [(default) magma_0/kvstore-280/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-280/rev-000000001/seqIndex/state.851: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0 2022-03-06T06:13:45.116847-08:00 CRITICAL [(default) magma_0/kvstore-288/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-288/rev-000000001/seqIndex/state.837: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0 2022-03-06T06:13:45.123701-08:00 CRITICAL [(default) magma_0/kvstore-296/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-296/rev-000000001/seqIndex/state.834: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0 2022-03-06T06:13:45.130909-08:00 CRITICAL [(default) magma_0/kvstore-304/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-304/rev-000000001/seqIndex/state.844: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0 2022-03-06T06:13:45.139625-08:00 CRITICAL [(default) magma_0/kvstore-312/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-312/rev-000000001/seqIndex/state.839: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0 2022-03-06T06:13:45.147167-08:00 CRITICAL [(default) magma_0/kvstore-320/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-320/rev-000000001/seqIndex/state.834: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0 2022-03-06T06:13:45.153396-08:00 CRITICAL [(default) magma_0/kvstore-328/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-328/rev-000000001/seqIndex/state.845: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0 2022-03-06T06:13:45.159833-08:00 CRITICAL [(default) magma_0/kvstore-336/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-336/rev-000000001/seqIndex/state.830: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0 2022-03-06T06:13:45.168409-08:00 CRITICAL [(default) magma_0/kvstore-344/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-344/rev-000000001/seqIndex/state.838: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0 2022-03-06T06:13:45.173526-08:00 CRITICAL [(default) magma_0/kvstore-352/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-352/rev-000000001/seqIndex/state.837: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0 2022-03-06T06:13:45.177540-08:00 CRITICAL [(default) magma_0/kvstore-360/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-360/rev-000000001/seqIndex/state.843: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0 2022-03-06T06:13:45.181248-08:00 CRITICAL [(default) magma_0/kvstore-368/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-368/rev-000000001/seqIndex/state.844: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0 2022-03-06T06:13:45.184947-08:00 CRITICAL [(default) magma_0/kvstore-376/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-376/rev-000000001/seqIndex/state.834: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0 2022-03-06T06:13:45.189012-08:00 CRITICAL [(default) magma_0/kvstore-384/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-384/rev-000000001/seqIndex/state.836: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0 2022-03-06T06:13:45.192843-08:00 CRITICAL [(default) magma_0/kvstore-392/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-392/rev-000000001/seqIndex/state.829: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0 2022-03-06T06:13:45.196629-08:00 CRITICAL [(default) magma_0/kvstore-400/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-400/rev-000000001/seqIndex/state.827: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0 2022-03-06T06:13:45.200173-08:00 CRITICAL [(default) magma_0/kvstore-408/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-408/rev-000000001/seqIndex/state.833: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0 2022-03-06T06:13:45.203700-08:00 CRITICAL [(default) magma_0/kvstore-416/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-416/rev-000000001/seqIndex/state.1136: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0 2022-03-06T06:13:45.205825-08:00 CRITICAL [(default) magma_0/kvstore-424/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-424/rev-000000001/seqIndex/state.1143: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0 2022-03-06T06:13:45.208286-08:00 CRITICAL [(default) magma_0/kvstore-432/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-432/rev-000000001/seqIndex/state.1140: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0 2022-03-06T06:13:45.210111-08:00 CRITICAL [(default) magma_0/kvstore-440/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-440/rev-000000001/seqIndex/state.1136: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0 2022-03-06T06:13:45.212001-08:00 CRITICAL [(default) magma_0/kvstore-448/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-448/rev-000000001/seqIndex/state.1134: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0 2022-03-06T06:13:45.213931-08:00 CRITICAL [(default) magma_0/kvstore-456/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-456/rev-000000001/seqIndex/state.1138: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0 2022-03-06T06:13:45.215743-08:00 CRITICAL [(default) magma_0/kvstore-464/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-464/rev-000000001/seqIndex/state.1131: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0 2022-03-06T06:13:45.217268-08:00 CRITICAL [(default) magma_0/kvstore-472/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-472/rev-000000001/seqIndex/state.1136: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0 2022-03-06T06:13:45.218897-08:00 CRITICAL [(default) magma_0/kvstore-480/rev-000000001/localIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-480/rev-000000001/localIndex/sstable.1542.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0). 2022-03-06T06:13:45.220118-08:00 CRITICAL [(default) magma_0/kvstore-488/rev-000000001/localIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-488/rev-000000001/localIndex/sstable.1569.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0). 2022-03-06T06:13:45.221443-08:00 CRITICAL [(default) magma_0/kvstore-496/rev-000000001/localIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-496/rev-000000001/localIndex/sstable.1559.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0). 2022-03-06T06:13:45.222704-08:00 CRITICAL [(default) magma_0/kvstore-504/rev-000000001/localIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-504/rev-000000001/localIndex/sstable.1563.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0). 2022-03-06T06:13:45.224323-08:00 CRITICAL [(default) magma_0/kvstore-600/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-600/rev-000000001/seqIndex/state.828: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0 2022-03-06T06:13:45.228554-08:00 CRITICAL [(default) magma_0/kvstore-608/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-608/rev-000000001/seqIndex/state.833: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0 2022-03-06T06:13:45.231852-08:00 CRITICAL [(default) magma_0/kvstore-616/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-616/rev-000000001/seqIndex/state.837: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0 2022-03-06T06:13:45.237264-08:00 CRITICAL [(default) magma_0/kvstore-624/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-624/rev-000000001/seqIndex/state.829: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0 2022-03-06T06:13:45.241691-08:00 CRITICAL [(default) magma_0/kvstore-632/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-632/rev-000000001/seqIndex/state.820: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0 2022-03-06T06:13:45.247240-08:00 CRITICAL [(default) magma_0/kvstore-640/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-640/rev-000000001/seqIndex/state.841: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0 2022-03-06T06:13:45.251089-08:00 CRITICAL [(default) magma_0/kvstore-648/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-648/rev-000000001/seqIndex/state.837: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0 2022-03-06T06:13:45.254835-08:00 CRITICAL [(default) magma_0/kvstore-656/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-656/rev-000000001/seqIndex/state.824: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0 2022-03-06T06:13:45.259840-08:00 CRITICAL [(default) magma_0/kvstore-664/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-664/rev-000000001/seqIndex/state.832: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0 2022-03-06T06:13:45.265706-08:00 CRITICAL [(default) magma_0/kvstore-672/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-672/rev-000000001/seqIndex/state.1127: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0 2022-03-06T06:13:45.269707-08:00 CRITICAL [(default) magma_0/kvstore-680/rev-000000001/localIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-680/rev-000000001/localIndex/sstable.1557.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0). 2022-03-06T06:13:45.271151-08:00 CRITICAL [(default) magma_0/kvstore-856/rev-000000001/localIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-856/rev-000000001/localIndex/sstable.1566.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0). 2022-03-06T06:13:45.272353-08:00 CRITICAL [(default) magma_0/kvstore-864/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-864/rev-000000001/seqIndex/state.1116: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0 2022-03-06T06:13:45.274461-08:00 CRITICAL [(default) magma_0/kvstore-872/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-872/rev-000000001/seqIndex/state.832: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0 2022-03-06T06:13:45.278921-08:00 CRITICAL [(default) magma_0/kvstore-880/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-880/rev-000000001/seqIndex/state.825: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0 2022-03-06T06:13:45.283659-08:00 CRITICAL [(default) magma_0/kvstore-888/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-888/rev-000000001/seqIndex/state.817: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0 2022-03-06T06:13:45.289457-08:00 CRITICAL [(default) magma_0/kvstore-896/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-896/rev-000000001/seqIndex/state.839: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0 2022-03-06T06:13:45.294132-08:00 CRITICAL [(default) magma_0/kvstore-904/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-904/rev-000000001/seqIndex/state.838: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0 2022-03-06T06:13:45.297494-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.825: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0 2022-03-06T06:13:45.302502-08:00 CRITICAL [(default) magma_0/kvstore-920/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-920/rev-000000001/seqIndex/state.836: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0 2022-03-06T06:13:45.305653-08:00 CRITICAL [(default) magma_0/kvstore-928/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-928/rev-000000001/seqIndex/state.827: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0 2022-03-06 09:10:18,734 | test | INFO | MainThread | [basetestcase:check_logs:824] Comparing timestamps: Log's latest timestamp: 2022-03-06 06:13:45, Test's start timestamp is 2022-03-06 06:07:24.945000 2022-03-06 09:10:18,736 | test | INFO | MainThread | [basetestcase:check_logs:824] Comparing timestamps: Log's latest timestamp: 2022-03-06 06:13:45, Test's start timestamp is 2022-03-06 06:07:24.945000 2022-03-06 09:10:18,736 | test | INFO | MainThread | [basetestcase:check_coredump_exist:933] unwanted messages in /opt/couchbase/var/lib/couchbase/logs/babysitter.log 2022-03-06 09:10:18,737 | test | CRITICAL | MainThread | [basetestcase:check_coredump_exist:935] 172.23.105.164: Found ' CRITICAL ' logs - ['[ns_server:info,2022-03-06T06:13:43.800-08:00,babysitter_of_ns_1@cb.local:<0.133.0>:ns_port_server:log:226]memcached<0.133.0>: 2022-03-06T06:13:43.591107-08:00 CRITICAL [(default) magma_0/kvstore-864/rev-000000001/localIndex]Failed to write tree state /data/default/magma.0/kvstore-864/rev-000000001/localIndex/state.1095: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.591474-08:00 CRITICAL [(default) magma_0/kvstore-864/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-864/rev-000000001/seqIndex/state.1096: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.591782-08:00 CRITICAL [(default) magma_0/kvstore-864/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-864/rev-000000001/seqIndex/state.1097: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.592119-08:00 CRITICAL [(default) magma_0/kvstore-864/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-864/rev-000000001/seqIndex/state.1098: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.592507-08:00 CRITICAL [(default) magma_0/kvstore-864/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-864/rev-000000001/seqIndex/state.1099: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.592973-08:00 CRITICAL [(default) magma_0/kvstore-864/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-864/rev-000000001/seqIndex/state.1100: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.593463-08:00 CRITICAL [(default) magma_0/kvstore-864/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-864/rev-000000001/seqIndex/state.1101: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.594024-08:00 CRITICAL [(default) magma_0/kvstore-864/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-864/rev-000000001/seqIndex/state.1102: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.594643-08:00 CRITICAL [(default) magma_0/kvstore-864/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-864/rev-000000001/seqIndex/state.1103: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.595344-08:00 CRITICAL [(default) magma_0/kvstore-864/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-864/rev-000000001/seqIndex/state.1104: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.596049-08:00 CRITICAL [(default) magma_0/kvstore-864/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-864/rev-000000001/seqIndex/state.1105: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.596793-08:00 CRITICAL [(default) magma_0/kvstore-864/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-864/rev-000000001/seqIndex/state.1106: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.597558-08:00 CRITICAL [(default) magma_0/kvstore-864/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-864/rev-000000001/seqIndex/state.1107: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.598424-08:00 CRITICAL [(default) magma_0/kvstore-864/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-864/rev-000000001/seqIndex/state.1108: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.599804-08:00 CRITICAL [(default) magma_0/kvstore-864/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-864/rev-000000001/seqIndex/state.1109: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.600894-08:00 CRITICAL [(default) magma_0/kvstore-864/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-864/rev-000000001/seqIndex/state.1110: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.602235-08:00 CRITICAL [(default) magma_0/kvstore-864/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-864/rev-000000001/seqIndex/state.1111: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.603541-08:00 CRITICAL [(default) magma_0/kvstore-864/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-864/rev-000000001/seqIndex/state.1112: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.605085-08:00 CRITICAL [(default) magma_0/kvstore-864/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-864/rev-000000001/seqIndex/state.1113: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.606921-08:00 CRITICAL [(default) magma_0/kvstore-864/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-864/rev-000000001/seqIndex/state.1114: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.608644-08:00 CRITICAL [(default) magma_0/kvstore-864/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-864/rev-000000001/seqIndex/state.1115: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', '[ns_server:info,2022-03-06T06:13:45.263-08:00,babysitter_of_ns_1@cb.local:<0.133.0>:ns_port_server:log:226]memcached<0.133.0>: 2022-03-06T06:13:45.062993-08:00 CRITICAL [(default) magma_0/kvstore-48/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-48/rev-000000001/seqIndex/state.834: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0/kvstore-48/rev-000000001\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.068642-08:00 CRITICAL [(default) magma_0/kvstore-56/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-56/rev-000000001/seqIndex/state.837: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.073736-08:00 CRITICAL [(default) magma_0/kvstore-64/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-64/rev-000000001/seqIndex/state.827: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.080689-08:00 CRITICAL [(default) magma_0/kvstore-72/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-72/rev-000000001/seqIndex/state.832: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.084886-08:00 CRITICAL [(default) magma_0/kvstore-80/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-80/rev-000000001/seqIndex/state.840: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.089529-08:00 CRITICAL [(default) magma_0/kvstore-256/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-256/rev-000000001/seqIndex/state.839: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.096475-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.839: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.102642-08:00 CRITICAL [(default) magma_0/kvstore-272/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-272/rev-000000001/seqIndex/state.826: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.110251-08:00 CRITICAL [(default) magma_0/kvstore-280/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-280/rev-000000001/seqIndex/state.851: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.116847-08:00 CRITICAL [(default) magma_0/kvstore-288/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-288/rev-000000001/seqIndex/state.837: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.123701-08:00 CRITICAL [(default) magma_0/kvstore-296/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-296/rev-000000001/seqIndex/state.834: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.130909-08:00 CRITICAL [(default) magma_0/kvstore-304/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-304/rev-000000001/seqIndex/state.844: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.139625-08:00 CRITICAL [(default) magma_0/kvstore-312/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-312/rev-000000001/seqIndex/state.839: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.147167-08:00 CRITICAL [(default) magma_0/kvstore-320/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-320/rev-000000001/seqIndex/state.834: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.153396-08:00 CRITICAL [(default) magma_0/kvstore-328/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-328/rev-000000001/seqIndex/state.845: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.159833-08:00 CRITICAL [(default) magma_0/kvstore-336/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-336/rev-000000001/seqIndex/state.830: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.168409-08:00 CRITICAL [(default) magma_0/kvstore-344/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-344/rev-000000001/seqIndex/state.838: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.173526-08:00 CRITICAL [(default) magma_0/kvstore-352/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-352/rev-000000001/seqIndex/state.837: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.177540-08:00 CRITICAL [(default) magma_0/kvstore-360/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-360/rev-000000001/seqIndex/state.843: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.181248-08:00 CRITICAL [(default) magma_0/kvstore-368/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-368/rev-000000001/seqIndex/state.844: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.184947-08:00 CRITICAL [(default) magma_0/kvstore-376/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-376/rev-000000001/seqIndex/state.834: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.189012-08:00 CRITICAL [(default) magma_0/kvstore-384/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-384/rev-000000001/seqIndex/state.836: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.192843-08:00 CRITICAL [(default) magma_0/kvstore-392/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-392/rev-000000001/seqIndex/state.829: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.196629-08:00 CRITICAL [(default) magma_0/kvstore-400/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-400/rev-000000001/seqIndex/state.827: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.200173-08:00 CRITICAL [(default) magma_0/kvstore-408/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-408/rev-000000001/seqIndex/state.833: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.203700-08:00 CRITICAL [(default) magma_0/kvstore-416/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-416/rev-000000001/seqIndex/state.1136: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.205825-08:00 CRITICAL [(default) magma_0/kvstore-424/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-424/rev-000000001/seqIndex/state.1143: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.208286-08:00 CRITICAL [(default) magma_0/kvstore-432/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-432/rev-000000001/seqIndex/state.1140: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.210111-08:00 CRITICAL [(default) magma_0/kvstore-440/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-440/rev-000000001/seqIndex/state.1136: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.212001-08:00 CRITICAL [(default) magma_0/kvstore-448/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-448/rev-000000001/seqIndex/state.1134: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.213931-08:00 CRITICAL [(default) magma_0/kvstore-456/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-456/rev-000000001/seqIndex/state.1138: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.215743-08:00 CRITICAL [(default) magma_0/kvstore-464/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-464/rev-000000001/seqIndex/state.1131: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.217268-08:00 CRITICAL [(default) magma_0/kvstore-472/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-472/rev-000000001/seqIndex/state.1136: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.218897-08:00 CRITICAL [(default) magma_0/kvstore-480/rev-000000001/localIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-480/rev-000000001/localIndex/sstable.1542.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.220118-08:00 CRITICAL [(default) magma_0/kvstore-488/rev-000000001/localIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-488/rev-000000001/localIndex/sstable.1569.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.221443-08:00 CRITICAL [(default) magma_0/kvstore-496/rev-000000001/localIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-496/rev-000000001/localIndex/sstable.1559.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.222704-08:00 CRITICAL [(default) magma_0/kvstore-504/rev-000000001/localIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-504/rev-000000001/localIndex/sstable.1563.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.224323-08:00 CRITICAL [(default) magma_0/kvstore-600/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-600/rev-000000001/seqIndex/state.828: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.228554-08:00 CRITICAL [(default) magma_0/kvstore-608/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-608/rev-000000001/seqIndex/state.833: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.231852-08:00 CRITICAL [(default) magma_0/kvstore-616/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-616/rev-000000001/seqIndex/state.837: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.237264-08:00 CRITICAL [(default) magma_0/kvstore-624/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-624/rev-000000001/seqIndex/state.829: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.241691-08:00 CRITICAL [(default) magma_0/kvstore-632/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-632/rev-000000001/seqIndex/state.820: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.247240-08:00 CRITICAL [(default) magma_0/kvstore-640/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-640/rev-000000001/seqIndex/state.841: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.251089-08:00 CRITICAL [(default) magma_0/kvstore-648/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-648/rev-000000001/seqIndex/state.837: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.254835-08:00 CRITICAL [(default) magma_0/kvstore-656/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-656/rev-000000001/seqIndex/state.824: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.259840-08:00 CRITICAL [(default) magma_0/kvstore-664/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-664/rev-000000001/seqIndex/state.832: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', '[ns_server:info,2022-03-06T06:13:45.466-08:00,babysitter_of_ns_1@cb.local:<0.133.0>:ns_port_server:log:226]memcached<0.133.0>: 2022-03-06T06:13:45.265706-08:00 CRITICAL [(default) magma_0/kvstore-672/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-672/rev-000000001/seqIndex/state.1127: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.269707-08:00 CRITICAL [(default) magma_0/kvstore-680/rev-000000001/localIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-680/rev-000000001/localIndex/sstable.1557.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.271151-08:00 CRITICAL [(default) magma_0/kvstore-856/rev-000000001/localIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-856/rev-000000001/localIndex/sstable.1566.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.272353-08:00 CRITICAL [(default) magma_0/kvstore-864/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-864/rev-000000001/seqIndex/state.1116: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.274461-08:00 CRITICAL [(default) magma_0/kvstore-872/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-872/rev-000000001/seqIndex/state.832: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.278921-08:00 CRITICAL [(default) magma_0/kvstore-880/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-880/rev-000000001/seqIndex/state.825: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.283659-08:00 CRITICAL [(default) magma_0/kvstore-888/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-888/rev-000000001/seqIndex/state.817: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.289457-08:00 CRITICAL [(default) magma_0/kvstore-896/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-896/rev-000000001/seqIndex/state.839: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.294132-08:00 CRITICAL [(default) magma_0/kvstore-904/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-904/rev-000000001/seqIndex/state.838: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.297494-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.825: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.302502-08:00 CRITICAL [(default) magma_0/kvstore-920/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-920/rev-000000001/seqIndex/state.836: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.305653-08:00 CRITICAL [(default) magma_0/kvstore-928/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-928/rev-000000001/seqIndex/state.827: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.309798-08:00 CRITICAL [(default) magma_0/kvstore-936/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-936/rev-000000001/seqIndex/state.826: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n'] 2022-03-06 09:10:19,153 | test | INFO | MainThread | [basetestcase:check_logs:824] Comparing timestamps: Log's latest timestamp: 2022-03-06 06:13:45, Test's start timestamp is 2022-03-06 06:07:24.945000 2022-03-06 09:10:19,154 | test | INFO | MainThread | [basetestcase:check_logs:824] Comparing timestamps: Log's latest timestamp: 2022-03-06 06:13:45, Test's start timestamp is 2022-03-06 06:07:24.945000 2022-03-06 09:10:19,154 | test | INFO | MainThread | [basetestcase:check_coredump_exist:933] unwanted messages in /opt/couchbase/var/lib/couchbase/logs/babysitter.log 2022-03-06 09:10:19,154 | test | CRITICAL | MainThread | [basetestcase:check_coredump_exist:935] 172.23.105.206: Found ' CRITICAL ' logs - ['[ns_server:info,2022-03-06T06:13:45.017-08:00,babysitter_of_ns_1@cb.local:<0.133.0>:ns_port_server:log:226]memcached<0.133.0>: 2022-03-06T06:13:44.816833-08:00 CRITICAL [(default) magma_0/kvstore-168/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-168/rev-000000001/seqIndex/state.993: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.822945-08:00 CRITICAL [(default) magma_0/kvstore-344/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-344/rev-000000001/seqIndex/state.977: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.829153-08:00 CRITICAL [(default) magma_0/kvstore-352/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-352/rev-000000001/seqIndex/state.977: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.832937-08:00 CRITICAL [(default) magma_0/kvstore-360/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-360/rev-000000001/seqIndex/state.986: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.838185-08:00 CRITICAL [(default) magma_0/kvstore-368/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-368/rev-000000001/seqIndex/state.980: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.842357-08:00 CRITICAL [(default) magma_0/kvstore-376/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-376/rev-000000001/seqIndex/state.984: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.845678-08:00 CRITICAL [(default) magma_0/kvstore-384/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-384/rev-000000001/seqIndex/state.986: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.851009-08:00 CRITICAL [(default) magma_0/kvstore-392/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-392/rev-000000001/seqIndex/state.984: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.855360-08:00 CRITICAL [(default) magma_0/kvstore-400/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-400/rev-000000001/seqIndex/state.1276: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.857938-08:00 CRITICAL [(default) magma_0/kvstore-408/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-408/rev-000000001/seqIndex/state.1269: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.859975-08:00 CRITICAL [(default) magma_0/kvstore-416/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-416/rev-000000001/seqIndex/state.1273: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.861788-08:00 CRITICAL [(default) magma_0/kvstore-424/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-424/rev-000000001/seqIndex/state.1273: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.864151-08:00 CRITICAL [(default) magma_0/kvstore-512/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-512/rev-000000001/seqIndex/state.979: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.871684-08:00 CRITICAL [(default) magma_0/kvstore-520/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-520/rev-000000001/seqIndex/state.974: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.877737-08:00 CRITICAL [(default) magma_0/kvstore-528/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-528/rev-000000001/seqIndex/state.974: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.884831-08:00 CRITICAL [(default) magma_0/kvstore-536/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-536/rev-000000001/seqIndex/state.970: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.891945-08:00 CRITICAL [(default) magma_0/kvstore-544/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-544/rev-000000001/seqIndex/state.977: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.899607-08:00 CRITICAL [(default) magma_0/kvstore-552/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-552/rev-000000001/seqIndex/state.974: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.906179-08:00 CRITICAL [(default) magma_0/kvstore-560/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-560/rev-000000001/seqIndex/state.977: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.913231-08:00 CRITICAL [(default) magma_0/kvstore-568/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-568/rev-000000001/seqIndex/state.974: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.921694-08:00 CRITICAL [(default) magma_0/kvstore-576/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-576/rev-000000001/seqIndex/state.978: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.928431-08:00 CRITICAL [(default) magma_0/kvstore-584/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-584/rev-000000001/seqIndex/state.976: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.936665-08:00 CRITICAL [(default) magma_0/kvstore-592/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-592/rev-000000001/seqIndex/state.968: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.944971-08:00 CRITICAL [(default) magma_0/kvstore-600/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-600/rev-000000001/seqIndex/state.962: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.949908-08:00 CRITICAL [(default) magma_0/kvstore-608/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-608/rev-000000001/seqIndex/state.970: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.955066-08:00 CRITICAL [(default) magma_0/kvstore-616/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-616/rev-000000001/seqIndex/state.980: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.959946-08:00 CRITICAL [(default) magma_0/kvstore-624/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-624/rev-000000001/seqIndex/state.974: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.965001-08:00 CRITICAL [(default) magma_0/kvstore-632/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-632/rev-000000001/seqIndex/state.971: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.971605-08:00 CRITICAL [(default) magma_0/kvstore-640/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-640/rev-000000001/seqIndex/state.971: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.975554-08:00 CRITICAL [(default) magma_0/kvstore-648/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-648/rev-000000001/seqIndex/state.978: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.980430-08:00 CRITICAL [(default) magma_0/kvstore-656/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-656/rev-000000001/seqIndex/state.969: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.983913-08:00 CRITICAL [(default) magma_0/kvstore-664/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-664/rev-000000001/seqIndex/state.975: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.988697-08:00 CRITICAL [(default) magma_0/kvstore-672/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-672/rev-000000001/seqIndex/state.976: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.995915-08:00 CRITICAL [(default) magma_0/kvstore-680/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-680/rev-000000001/seqIndex/state.1266: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.001603-08:00 CRITICAL [(default) magma_0/kvstore-688/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-688/rev-000000001/seqIndex/state.1288: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.003724-08:00 CRITICAL [(default) magma_0/kvstore-696/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-696/rev-000000001/seqIndex/state.1280: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.008196-08:00 CRITICAL [(default) magma_0/kvstore-704/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-704/rev-000000001/seqIndex/state.1283: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.009531-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/state.1282: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.011881-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1281: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.013717-08:00 CRITICAL [(default) magma_0/kvstore-728/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-728/rev-000000001/seqIndex/state.1274: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.015612-08:00 CRITICAL [(default) magma_0/kvstore-736/rev-000000001/localIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-736/rev-000000001/localIndex/sstable.1833.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', '[ns_server:info,2022-03-06T06:13:45.218-08:00,babysitter_of_ns_1@cb.local:<0.133.0>:ns_port_server:log:226]memcached<0.133.0>: 2022-03-06T06:13:45.017464-08:00 CRITICAL [(default) magma_0/kvstore-744/rev-000000001/localIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-744/rev-000000001/localIndex/sstable.1835.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.019455-08:00 CRITICAL [(default) magma_0/kvstore-752/rev-000000001/localIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-752/rev-000000001/localIndex/sstable.1846.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.021698-08:00 CRITICAL [(default) magma_0/kvstore-760/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-760/rev-000000001/seqIndex/state.1265: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.023506-08:00 CRITICAL [(default) magma_0/kvstore-944/rev-000000001/localIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-944/rev-000000001/localIndex/sstable.1850.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.025659-08:00 CRITICAL [(default) magma_0/kvstore-952/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-952/rev-000000001/seqIndex/state.962: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.030726-08:00 CRITICAL [(default) magma_0/kvstore-960/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-960/rev-000000001/seqIndex/state.972: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.035722-08:00 CRITICAL [(default) magma_0/kvstore-968/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-968/rev-000000001/seqIndex/state.968: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.040624-08:00 CRITICAL [(default) magma_0/kvstore-976/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-976/rev-000000001/seqIndex/state.969: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.045802-08:00 CRITICAL [(default) magma_0/kvstore-984/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-984/rev-000000001/seqIndex/state.977: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.051555-08:00 CRITICAL [(default) magma_0/kvstore-992/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-992/rev-000000001/seqIndex/state.972: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.057065-08:00 CRITICAL [(default) magma_0/kvstore-1000/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-1000/rev-000000001/seqIndex/state.977: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.060951-08:00 CRITICAL [(default) magma_0/kvstore-1008/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-1008/rev-000000001/seqIndex/state.981: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.065137-08:00 CRITICAL [(default) magma_0/kvstore-1016/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-1016/rev-000000001/seqIndex/state.975: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n'] 2022-03-06 09:10:19,572 | test | INFO | MainThread | [basetestcase:check_logs:824] Comparing timestamps: Log's latest timestamp: 2022-03-06 06:13:46, Test's start timestamp is 2022-03-06 06:07:24.945000 2022-03-06 09:10:19,576 | test | INFO | MainThread | [basetestcase:check_logs:824] Comparing timestamps: Log's latest timestamp: 2022-03-06 06:13:46, Test's start timestamp is 2022-03-06 06:07:24.945000 2022-03-06 09:10:19,576 | test | INFO | MainThread | [basetestcase:check_coredump_exist:933] unwanted messages in /opt/couchbase/var/lib/couchbase/logs/babysitter.log 2022-03-06 09:10:19,578 | test | CRITICAL | MainThread | [basetestcase:check_coredump_exist:935] 172.23.106.177: Found ' CRITICAL ' logs - ['[ns_server:info,2022-03-06T06:13:43.721-08:00,babysitter_of_ns_1@cb.local:<0.133.0>:ns_port_server:log:226]memcached<0.133.0>: 2022-03-06T06:13:43.520289-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during merge sstables (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1824.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0). \n', 'memcached<0.133.0>: 2022-03-06T06:13:43.550887-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/keyIndex]Error occured during merge sstables (IOError: /data/default/magma.0/kvstore-712/rev-000000001/keyIndex/sstable.3509.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0). \n', 'memcached<0.133.0>: 2022-03-06T06:13:43.551400-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1825.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.551675-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1826.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.551859-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1827.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.552066-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1828.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.552234-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1829.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.552451-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1830.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.552743-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1831.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.552956-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1832.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.553149-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1833.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.553354-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1834.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.553604-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1835.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.553800-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1836.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.553985-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1837.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.554185-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1838.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.554360-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1839.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.554533-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1840.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.554704-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1841.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.554873-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1842.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.555074-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1843.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.555241-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1844.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.555411-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1845.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.555583-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1846.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.555752-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1847.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.555922-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1848.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.556118-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1849.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.556288-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1850.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.556458-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1851.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.556622-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1852.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.556789-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1853.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.556957-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1854.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.557149-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1855.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.557317-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1856.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.557499-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1857.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.557669-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1858.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.557840-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1859.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.558009-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1860.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.558203-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1861.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.558369-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1862.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.558536-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1863.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.558705-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1864.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.558872-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1865.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.559064-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1866.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.559235-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1867.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.559403-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1868.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.559572-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1869.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.559747-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1870.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.559917-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1871.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.560106-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1872.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.560275-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1873.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.560445-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1874.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.560615-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1875.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.560780-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1876.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.560947-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1877.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.561134-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1878.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.561373-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1879.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.561675-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1880.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.562129-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1881.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.562423-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1882.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', '[ns_server:info,2022-03-06T06:13:44.805-08:00,babysitter_of_ns_1@cb.local:<0.133.0>:ns_port_server:log:226]memcached<0.133.0>: 2022-03-06T06:13:44.604108-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1078: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.605709-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1079: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.607275-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1080: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.608316-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1081: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.609569-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1082: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.610952-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1083: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.612477-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1084: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.613793-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1085: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.615223-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1086: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.616977-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1087: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.618630-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1088: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.620380-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1089: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.621833-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1090: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.623673-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1091: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.625437-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1092: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.627751-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1093: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.629589-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1094: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.631832-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1095: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.633942-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1096: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.636264-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1097: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.638721-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1098: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.641352-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1099: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.643643-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1100: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.646240-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1101: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.648966-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1102: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.651818-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1103: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.654207-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1104: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.656761-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1105: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.659698-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1106: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.662574-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1107: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.665451-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1108: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.668326-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1109: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.671541-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1110: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.674542-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1111: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.677588-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1112: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.681338-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1113: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.684760-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1114: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.688237-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1115: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.691798-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1116: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.695605-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1117: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.699291-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1118: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.703119-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1119: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.706991-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1120: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.711273-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1121: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.714891-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1122: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.719037-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1123: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.722784-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1124: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.727046-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1125: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.731283-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1126: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.735843-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1127: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.740229-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1128: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.744630-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1129: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.748947-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1130: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.753262-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1131: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.757824-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1132: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.762476-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1133: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.767118-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1134: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.771982-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1135: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.776850-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1136: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.782853-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1137: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', '[ns_server:info,2022-03-06T06:13:45.006-08:00,babysitter_of_ns_1@cb.local:<0.133.0>:ns_port_server:log:226]memcached<0.133.0>: 2022-03-06T06:13:44.806058-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1141: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.811948-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1142: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.817492-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1143: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.823125-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1144: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.828792-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1145: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.834292-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1146: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.840819-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1147: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.847571-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1148: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.853550-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1149: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.859525-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1150: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.865645-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1151: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.871873-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1152: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.877904-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1153: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.884091-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1154: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.890327-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1155: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.896693-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1156: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.903363-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1157: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.909858-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1158: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.916779-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1159: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.923488-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1160: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.929555-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1161: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.935346-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1162: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.941766-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1163: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.947986-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1164: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.954550-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1165: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.960693-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1166: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.967442-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1167: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.973800-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1168: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.980344-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1169: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.987179-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1170: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.993706-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1171: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.000709-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1172: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', '[ns_server:info,2022-03-06T06:13:45.208-08:00,babysitter_of_ns_1@cb.local:<0.133.0>:ns_port_server:log:226]memcached<0.133.0>: 2022-03-06T06:13:45.007855-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1173: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.015646-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1174: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.022666-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1175: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.030525-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1176: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.038351-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1177: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.045598-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1178: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.053203-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1179: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.060636-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1180: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.068199-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1181: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.075727-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1182: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.083501-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1183: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.091192-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1184: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.099627-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1185: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.108553-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1186: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.117571-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1187: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.125872-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1188: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.133837-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1189: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.142533-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1190: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.151791-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1191: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.161327-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1192: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.171049-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1193: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.180470-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1194: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.189057-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1195: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.197934-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1196: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.207684-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1197: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', '[ns_server:info,2022-03-06T06:13:45.419-08:00,babysitter_of_ns_1@cb.local:<0.133.0>:ns_port_server:log:226]memcached<0.133.0>: 2022-03-06T06:13:45.218036-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1198: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.227868-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1199: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.237864-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1200: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.248405-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1201: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.258483-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1202: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.266946-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1203: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.276857-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1204: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.287113-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1205: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.297550-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1206: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.306806-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1207: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.315073-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1208: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.322360-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1209: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.331792-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1210: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.341207-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1211: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.351110-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1212: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.361486-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1213: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.372088-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1214: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.382549-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1215: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.393068-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1216: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.403484-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1217: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.414167-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1218: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', '[ns_server:info,2022-03-06T06:13:45.625-08:00,babysitter_of_ns_1@cb.local:<0.133.0>:ns_port_server:log:226]memcached<0.133.0>: 2022-03-06T06:13:45.425378-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1219: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.436074-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1220: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.446796-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1221: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.457615-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1222: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.468607-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1223: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.479487-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1224: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.490579-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1225: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.501735-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1226: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.513119-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1227: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.524394-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1228: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', '[ns_server:info,2022-03-06T06:13:45.994-08:00,babysitter_of_ns_1@cb.local:<0.133.0>:ns_port_server:log:226]memcached<0.133.0>: 2022-03-06T06:13:45.793886-08:00 CRITICAL [(default) magma_0/kvstore-176/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-176/rev-000000001/seqIndex/state.1073: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.800207-08:00 CRITICAL [(default) magma_0/kvstore-184/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-184/rev-000000001/seqIndex/state.1079: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.806224-08:00 CRITICAL [(default) magma_0/kvstore-192/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-192/rev-000000001/seqIndex/state.1082: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.812545-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.1081: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.818656-08:00 CRITICAL [(default) magma_0/kvstore-208/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-208/rev-000000001/seqIndex/state.1089: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.824592-08:00 CRITICAL [(default) magma_0/kvstore-216/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-216/rev-000000001/seqIndex/state.1083: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.832464-08:00 CRITICAL [(default) magma_0/kvstore-224/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-224/rev-000000001/seqIndex/state.1086: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.841121-08:00 CRITICAL [(default) magma_0/kvstore-232/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-232/rev-000000001/seqIndex/state.1085: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.847768-08:00 CRITICAL [(default) magma_0/kvstore-240/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-240/rev-000000001/seqIndex/state.1077: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.853901-08:00 CRITICAL [(default) magma_0/kvstore-248/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-248/rev-000000001/seqIndex/state.1076: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.860891-08:00 CRITICAL [(default) magma_0/kvstore-432/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-432/rev-000000001/seqIndex/state.1083: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.865389-08:00 CRITICAL [(default) magma_0/kvstore-440/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-440/rev-000000001/seqIndex/state.1378: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.867251-08:00 CRITICAL [(default) magma_0/kvstore-448/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-448/rev-000000001/seqIndex/state.1384: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.868606-08:00 CRITICAL [(default) magma_0/kvstore-456/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-456/rev-000000001/seqIndex/state.1385: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.869859-08:00 CRITICAL [(default) magma_0/kvstore-464/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-464/rev-000000001/seqIndex/state.1372: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.872472-08:00 CRITICAL [(default) magma_0/kvstore-472/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-472/rev-000000001/seqIndex/state.1369: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.874793-08:00 CRITICAL [(default) magma_0/kvstore-480/rev-000000001/localIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-480/rev-000000001/localIndex/sstable.2024.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default).\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.877801-08:00 CRITICAL [(default) magma_0/kvstore-488/rev-000000001/localIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-488/rev-000000001/localIndex/sstable.2021.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default).\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.880316-08:00 CRITICAL [(default) magma_0/kvstore-496/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-496/rev-000000001/seqIndex/state.1378: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.883644-08:00 CRITICAL [(default) magma_0/kvstore-504/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-504/rev-000000001/seqIndex/state.1084: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.888527-08:00 CRITICAL [(default) magma_0/kvstore-688/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-688/rev-000000001/seqIndex/state.1368: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.890998-08:00 CRITICAL [(default) magma_0/kvstore-696/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-696/rev-000000001/seqIndex/state.1374: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.892699-08:00 CRITICAL [(default) magma_0/kvstore-704/rev-000000001/localIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-704/rev-000000001/localIndex/sstable.2004.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default).\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.894413-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1931.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default).\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.899532-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1229: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.907735-08:00 CRITICAL [(default) magma_0/kvstore-728/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-728/rev-000000001/seqIndex/state.1077: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.912357-08:00 CRITICAL [(default) magma_0/kvstore-736/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-736/rev-000000001/seqIndex/state.1071: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.916746-08:00 CRITICAL [(default) magma_0/kvstore-744/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-744/rev-000000001/seqIndex/state.1077: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.921750-08:00 CRITICAL [(default) magma_0/kvstore-752/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-752/rev-000000001/seqIndex/state.1080: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.925366-08:00 CRITICAL [(default) magma_0/kvstore-760/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-760/rev-000000001/seqIndex/state.1078: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.929761-08:00 CRITICAL [(default) magma_0/kvstore-768/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-768/rev-000000001/seqIndex/state.1076: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.938837-08:00 CRITICAL [(default) magma_0/kvstore-776/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-776/rev-000000001/seqIndex/state.1090: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.944984-08:00 CRITICAL [(default) magma_0/kvstore-784/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-784/rev-000000001/seqIndex/state.1082: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.951195-08:00 CRITICAL [(default) magma_0/kvstore-792/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-792/rev-000000001/seqIndex/state.1083: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.958047-08:00 CRITICAL [(default) magma_0/kvstore-800/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-800/rev-000000001/seqIndex/state.1077: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.964941-08:00 CRITICAL [(default) magma_0/kvstore-808/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-808/rev-000000001/seqIndex/state.1081: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.971776-08:00 CRITICAL [(default) magma_0/kvstore-816/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-816/rev-000000001/seqIndex/state.1084: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.978801-08:00 CRITICAL [(default) magma_0/kvstore-824/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-824/rev-000000001/seqIndex/state.1080: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.986628-08:00 CRITICAL [(default) magma_0/kvstore-832/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-832/rev-000000001/seqIndex/state.1083: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.993831-08:00 CRITICAL [(default) magma_0/kvstore-840/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-840/rev-000000001/seqIndex/state.1090: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', '[ns_server:info,2022-03-06T06:13:46.200-08:00,babysitter_of_ns_1@cb.local:<0.133.0>:ns_port_server:log:226]memcached<0.133.0>: 2022-03-06T06:13:46.000701-08:00 CRITICAL [(default) magma_0/kvstore-848/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-848/rev-000000001/seqIndex/state.1080: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.006794-08:00 CRITICAL [(default) magma_0/kvstore-856/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-856/rev-000000001/seqIndex/state.1085: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.010415-08:00 CRITICAL [(default) magma_0/kvstore-864/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-864/rev-000000001/seqIndex/state.1078: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.015158-08:00 CRITICAL [(default) magma_0/kvstore-872/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-872/rev-000000001/seqIndex/state.1087: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.019502-08:00 CRITICAL [(default) magma_0/kvstore-880/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-880/rev-000000001/seqIndex/state.1087: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.023407-08:00 CRITICAL [(default) magma_0/kvstore-888/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-888/rev-000000001/seqIndex/state.1082: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.027634-08:00 CRITICAL [(default) magma_0/kvstore-896/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-896/rev-000000001/seqIndex/state.1081: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.039501-08:00 CRITICAL [(default) magma_0/kvstore-904/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-904/rev-000000001/seqIndex/state.1088: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.043365-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1092: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.047976-08:00 CRITICAL [(default) magma_0/kvstore-920/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-920/rev-000000001/seqIndex/state.1085: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.051897-08:00 CRITICAL [(default) magma_0/kvstore-928/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-928/rev-000000001/seqIndex/state.1091: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.056095-08:00 CRITICAL [(default) magma_0/kvstore-936/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-936/rev-000000001/seqIndex/state.1085: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.060468-08:00 CRITICAL [(default) magma_0/kvstore-944/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-944/rev-000000001/seqIndex/state.1085: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.064519-08:00 CRITICAL [(default) magma_0/kvstore-952/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-952/rev-000000001/seqIndex/state.1082: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.068800-08:00 CRITICAL [(default) magma_0/kvstore-960/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-960/rev-000000001/seqIndex/state.1087: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.073069-08:00 CRITICAL [(default) magma_0/kvstore-968/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-968/rev-000000001/seqIndex/state.1082: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.077195-08:00 CRITICAL [(default) magma_0/kvstore-976/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-976/rev-000000001/seqIndex/state.1075: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.081689-08:00 CRITICAL [(default) magma_0/kvstore-984/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-984/rev-000000001/seqIndex/state.1072: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.086877-08:00 CRITICAL [(default) magma_0/kvstore-992/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-992/rev-000000001/seqIndex/state.1084: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.090954-08:00 CRITICAL [(default) magma_0/kvstore-1000/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-1000/rev-000000001/seqIndex/state.1071: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.094751-08:00 CRITICAL [(default) magma_0/kvstore-1008/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-1008/rev-000000001/seqIndex/state.1069: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.099226-08:00 CRITICAL [(default) magma_0/kvstore-1016/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-1016/rev-000000001/seqIndex/state.1088: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', '[ns_server:info,2022-03-06T06:13:46.449-08:00,babysitter_of_ns_1@cb.local:<0.133.0>:ns_port_server:log:226]memcached<0.133.0>: 2022-03-06T06:13:46.249255-08:00 CRITICAL [(default) magma_1/kvstore-833/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-833/rev-000000001/seqIndex/state.1081: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.256030-08:00 CRITICAL [(default) magma_1/kvstore-841/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-841/rev-000000001/seqIndex/state.1074: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.265182-08:00 CRITICAL [(default) magma_1/kvstore-849/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-849/rev-000000001/seqIndex/state.1078: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.271517-08:00 CRITICAL [(default) magma_1/kvstore-857/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-857/rev-000000001/seqIndex/state.1087: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.275989-08:00 CRITICAL [(default) magma_1/kvstore-865/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-865/rev-000000001/seqIndex/state.1090: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.280316-08:00 CRITICAL [(default) magma_1/kvstore-873/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-873/rev-000000001/seqIndex/state.1084: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.284433-08:00 CRITICAL [(default) magma_1/kvstore-881/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-881/rev-000000001/seqIndex/state.1077: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.294301-08:00 CRITICAL [(default) magma_1/kvstore-889/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-889/rev-000000001/seqIndex/state.1079: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.299549-08:00 CRITICAL [(default) magma_1/kvstore-897/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-897/rev-000000001/seqIndex/state.1096: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.302884-08:00 CRITICAL [(default) magma_1/kvstore-905/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-905/rev-000000001/seqIndex/state.1081: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.307271-08:00 CRITICAL [(default) magma_1/kvstore-913/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-913/rev-000000001/seqIndex/state.1091: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.311524-08:00 CRITICAL [(default) magma_1/kvstore-921/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-921/rev-000000001/seqIndex/state.1089: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.315214-08:00 CRITICAL [(default) magma_1/kvstore-929/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-929/rev-000000001/seqIndex/state.1085: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.319675-08:00 CRITICAL [(default) magma_1/kvstore-937/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-937/rev-000000001/seqIndex/state.1093: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.323803-08:00 CRITICAL [(default) magma_1/kvstore-945/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-945/rev-000000001/seqIndex/state.1080: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.327485-08:00 CRITICAL [(default) magma_1/kvstore-953/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-953/rev-000000001/seqIndex/state.1081: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.331163-08:00 CRITICAL [(default) magma_1/kvstore-961/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-961/rev-000000001/seqIndex/state.1374: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.332898-08:00 CRITICAL [(default) magma_1/kvstore-969/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-969/rev-000000001/seqIndex/state.1382: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.334841-08:00 CRITICAL [(default) magma_1/kvstore-977/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-977/rev-000000001/seqIndex/state.1375: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.336744-08:00 CRITICAL [(default) magma_1/kvstore-985/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-985/rev-000000001/seqIndex/state.1377: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.338622-08:00 CRITICAL [(default) magma_1/kvstore-993/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-993/rev-000000001/seqIndex/state.1389: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.340264-08:00 CRITICAL [(default) magma_1/kvstore-1001/rev-000000001/localIndex]Error occured during memtable flush (IOError: /data/default/magma.1/kvstore-1001/rev-000000001/localIndex/sstable.1997.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1).\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.342754-08:00 CRITICAL [(default) magma_1/kvstore-1009/rev-000000001/localIndex]Error occured during memtable flush (IOError: /data/default/magma.1/kvstore-1009/rev-000000001/localIndex/sstable.2039.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1).\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.344185-08:00 CRITICAL [(default) magma_1/kvstore-1017/rev-000000001/localIndex]Error occured during memtable flush (IOError: /data/default/magma.1/kvstore-1017/rev-000000001/localIndex/sstable.2027.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1).\n'] 2022-03-06 09:10:20,055 | test | INFO | MainThread | [basetestcase:check_logs:824] Comparing timestamps: Log's latest timestamp: 2022-03-06 06:13:46, Test's start timestamp is 2022-03-06 06:07:24.945000 2022-03-06 09:10:20,059 | test | INFO | MainThread | [basetestcase:check_logs:824] Comparing timestamps: Log's latest timestamp: 2022-03-06 06:13:46, Test's start timestamp is 2022-03-06 06:07:24.945000 2022-03-06 09:10:20,059 | test | INFO | MainThread | [basetestcase:check_coredump_exist:933] unwanted messages in /opt/couchbase/var/lib/couchbase/logs/babysitter.log 2022-03-06 09:10:20,062 | test | CRITICAL | MainThread | [basetestcase:check_coredump_exist:935] 172.23.100.35: Found ' CRITICAL ' logs - ['[ns_server:info,2022-03-06T06:13:43.498-08:00,babysitter_of_ns_1@cb.local:<0.133.0>:ns_port_server:log:226]memcached<0.133.0>: 2022-03-06T06:13:43.297341-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/keyIndex]Error occured during merge sstables (IOError: /data/default/magma.0/kvstore-200/rev-000000001/keyIndex/sstable.1297.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0). \n', 'memcached<0.133.0>: 2022-03-06T06:13:43.297795-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.345: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.298263-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.346: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.298672-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.347: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.299150-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.348: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.299701-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.349: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.300259-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.350: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.301022-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.351: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.301844-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.352: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.302641-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.353: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.303629-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.354: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.304684-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.355: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.305841-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.356: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.307070-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.357: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.308244-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.358: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.309408-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.359: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.310805-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.360: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.312134-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.361: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.313712-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.362: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.315291-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.363: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.317005-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.364: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.318876-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.365: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.320736-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.366: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.322638-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.367: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.324606-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.368: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.326879-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.369: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.328794-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.370: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.331899-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.371: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.334765-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.372: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.337533-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.373: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.340410-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.374: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.343296-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.375: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.346058-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.376: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.348965-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.377: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.351970-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.378: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.354606-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.379: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.357544-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.380: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.360618-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.381: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.363581-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.382: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.366845-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.383: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.370179-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.384: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.373766-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.385: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.377505-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.386: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.380104-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.387: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.382512-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.388: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.384938-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.389: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.387426-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.390: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.390441-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.391: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.393355-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.392: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.395946-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.393: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.399314-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.394: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.403451-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.395: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.407749-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.396: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.412009-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.397: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.416315-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.398: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.420779-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.399: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.425281-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.400: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.429785-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.401: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.434450-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.402: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.439210-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.403: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', '[ns_server:info,2022-03-06T06:13:43.702-08:00,babysitter_of_ns_1@cb.local:<0.133.0>:ns_port_server:log:226]memcached<0.133.0>: 2022-03-06T06:13:43.500526-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.416: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.505581-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.417: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.512220-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.418: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.516429-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.419: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.520435-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.420: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.524455-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.421: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.530328-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.422: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.534456-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.423: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.539952-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.424: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.544177-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.425: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.548441-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.426: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.554138-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.427: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.560506-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.428: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.565046-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.429: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.569649-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.430: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.576090-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.431: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.580722-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.432: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.586768-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.433: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.591444-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.434: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.597613-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.435: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.604292-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.436: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.611708-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.437: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.619347-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.438: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.624300-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.439: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.629161-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.440: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.635670-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.441: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.640726-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.442: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.647438-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.443: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.653205-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.444: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.658367-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.445: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.664992-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.446: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.670300-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.447: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.676364-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.448: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.681656-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.449: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.688410-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.450: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.693800-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.451: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.700301-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.452: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', '[ns_server:info,2022-03-06T06:13:43.913-08:00,babysitter_of_ns_1@cb.local:<0.133.0>:ns_port_server:log:226]memcached<0.133.0>: 2022-03-06T06:13:43.706847-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.453: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.713887-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.454: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.719544-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.455: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.726588-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.456: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.732799-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.457: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.739935-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.458: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.745761-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.459: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.752760-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.460: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.758685-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.461: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.767042-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.462: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.773140-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.463: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.780897-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.464: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.787883-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.465: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.795308-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.466: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.805175-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.467: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.814565-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.468: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.821159-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.469: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.827940-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.470: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.835900-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.471: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.842434-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.472: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.851089-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.473: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.859450-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.474: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.866649-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.475: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.874603-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.476: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.883100-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.477: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.890335-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.478: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.899504-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.479: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.906737-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.480: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', '[ns_server:info,2022-03-06T06:13:44.121-08:00,babysitter_of_ns_1@cb.local:<0.133.0>:ns_port_server:log:226]memcached<0.133.0>: 2022-03-06T06:13:43.921160-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.481: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.935420-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.482: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.944200-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.483: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.955200-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.484: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.965265-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.485: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.973013-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.486: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.980881-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.487: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.990483-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.488: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.998592-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.489: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.008054-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.490: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.017942-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.491: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.025646-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.492: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.035500-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.493: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.044812-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.494: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.054337-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.495: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.063176-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.496: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.072466-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.497: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.081756-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.498: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.090172-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.499: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.099195-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.500: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.108679-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.501: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', '[ns_server:info,2022-03-06T06:13:44.324-08:00,babysitter_of_ns_1@cb.local:<0.133.0>:ns_port_server:log:226]memcached<0.133.0>: 2022-03-06T06:13:44.120185-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.502: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.130972-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.503: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.141308-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.504: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.155645-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.505: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.165309-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.506: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.175096-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.507: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', '[ns_server:info,2022-03-06T06:13:46.136-08:00,babysitter_of_ns_1@cb.local:<0.133.0>:ns_port_server:log:226]memcached<0.133.0>: 2022-03-06T06:13:45.936324-08:00 CRITICAL [(default) magma_0/kvstore-0/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-0/rev-000000001/seqIndex/state.24: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.939554-08:00 CRITICAL [(default) magma_0/kvstore-8/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-8/rev-000000001/seqIndex/state.20: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.943831-08:00 CRITICAL [(default) magma_0/kvstore-16/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-16/rev-000000001/seqIndex/state.42: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.948058-08:00 CRITICAL [(default) magma_0/kvstore-24/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-24/rev-000000001/seqIndex/state.90: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.951758-08:00 CRITICAL [(default) magma_0/kvstore-32/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-32/rev-000000001/seqIndex/state.129: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.955160-08:00 CRITICAL [(default) magma_0/kvstore-40/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-40/rev-000000001/seqIndex/state.169: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.960724-08:00 CRITICAL [(default) magma_0/kvstore-48/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-48/rev-000000001/seqIndex/state.223: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.966278-08:00 CRITICAL [(default) magma_0/kvstore-56/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-56/rev-000000001/seqIndex/state.240: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.973914-08:00 CRITICAL [(default) magma_0/kvstore-64/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-64/rev-000000001/seqIndex/state.266: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.980890-08:00 CRITICAL [(default) magma_0/kvstore-72/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-72/rev-000000001/seqIndex/state.280: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.990612-08:00 CRITICAL [(default) magma_0/kvstore-80/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-80/rev-000000001/seqIndex/state.294: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.000113-08:00 CRITICAL [(default) magma_0/kvstore-88/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-88/rev-000000001/seqIndex/state.20: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.003500-08:00 CRITICAL [(default) magma_0/kvstore-96/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-96/rev-000000001/seqIndex/state.44: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.007835-08:00 CRITICAL [(default) magma_0/kvstore-104/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-104/rev-000000001/seqIndex/state.71: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.010950-08:00 CRITICAL [(default) magma_0/kvstore-112/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-112/rev-000000001/seqIndex/state.109: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.014610-08:00 CRITICAL [(default) magma_0/kvstore-120/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-120/rev-000000001/seqIndex/state.140: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.019200-08:00 CRITICAL [(default) magma_0/kvstore-128/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-128/rev-000000001/seqIndex/state.195: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.024190-08:00 CRITICAL [(default) magma_0/kvstore-136/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-136/rev-000000001/seqIndex/state.248: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.031532-08:00 CRITICAL [(default) magma_0/kvstore-144/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-144/rev-000000001/seqIndex/state.779: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.036417-08:00 CRITICAL [(default) magma_0/kvstore-152/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-152/rev-000000001/seqIndex/state.793: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.040147-08:00 CRITICAL [(default) magma_0/kvstore-160/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-160/rev-000000001/seqIndex/state.822: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.041968-08:00 CRITICAL [(default) magma_0/kvstore-168/rev-000000001/localIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-168/rev-000000001/localIndex/sstable.1325.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default).\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.044559-08:00 CRITICAL [(default) magma_0/kvstore-176/rev-000000001/localIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-176/rev-000000001/localIndex/sstable.616.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default).\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.046805-08:00 CRITICAL [(default) magma_0/kvstore-184/rev-000000001/localIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-184/rev-000000001/localIndex/sstable.651.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default).\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.048428-08:00 CRITICAL [(default) magma_0/kvstore-192/rev-000000001/localIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-192/rev-000000001/localIndex/sstable.746.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default).\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.050537-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.508: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.061363-08:00 CRITICAL [(default) magma_0/kvstore-208/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-208/rev-000000001/seqIndex/state.183: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.066290-08:00 CRITICAL [(default) magma_0/kvstore-216/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-216/rev-000000001/seqIndex/state.205: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.071787-08:00 CRITICAL [(default) magma_0/kvstore-224/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-224/rev-000000001/seqIndex/state.234: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.078040-08:00 CRITICAL [(default) magma_0/kvstore-232/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-232/rev-000000001/seqIndex/state.263: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.085381-08:00 CRITICAL [(default) magma_0/kvstore-240/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-240/rev-000000001/seqIndex/state.286: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.094854-08:00 CRITICAL [(default) magma_0/kvstore-248/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-248/rev-000000001/seqIndex/state.301: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.103735-08:00 CRITICAL [(default) magma_0/kvstore-256/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-256/rev-000000001/seqIndex/state.23: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.106803-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.26: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.111564-08:00 CRITICAL [(default) magma_0/kvstore-272/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-272/rev-000000001/seqIndex/state.23: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.115170-08:00 CRITICAL [(default) magma_0/kvstore-280/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-280/rev-000000001/seqIndex/state.21: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.119823-08:00 CRITICAL [(default) magma_0/kvstore-288/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-288/rev-000000001/seqIndex/state.18: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.122953-08:00 CRITICAL [(default) magma_0/kvstore-296/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-296/rev-000000001/seqIndex/state.16: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.126354-08:00 CRITICAL [(default) magma_0/kvstore-304/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-304/rev-000000001/seqIndex/state.18: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.130308-08:00 CRITICAL [(default) magma_0/kvstore-312/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-312/rev-000000001/seqIndex/state.19: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.133563-08:00 CRITICAL [(default) magma_0/kvstore-320/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-320/rev-000000001/seqIndex/state.16: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.136620-08:00 CRITICAL [(default) magma_0/kvstore-328/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-328/rev-000000001/seqIndex/state.16: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', '[ns_server:info,2022-03-06T06:13:46.339-08:00,babysitter_of_ns_1@cb.local:<0.133.0>:ns_port_server:log:226]memcached<0.133.0>: 2022-03-06T06:13:46.139411-08:00 CRITICAL [(default) magma_0/kvstore-336/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-336/rev-000000001/seqIndex/state.18: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.142289-08:00 CRITICAL [(default) magma_0/kvstore-512/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-512/rev-000000001/seqIndex/state.23: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.145052-08:00 CRITICAL [(default) magma_0/kvstore-520/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-520/rev-000000001/seqIndex/state.22: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.149076-08:00 CRITICAL [(default) magma_0/kvstore-528/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-528/rev-000000001/seqIndex/state.18: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.152315-08:00 CRITICAL [(default) magma_0/kvstore-536/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-536/rev-000000001/seqIndex/state.15: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.155679-08:00 CRITICAL [(default) magma_0/kvstore-544/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-544/rev-000000001/seqIndex/state.19: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.158498-08:00 CRITICAL [(default) magma_0/kvstore-552/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-552/rev-000000001/seqIndex/state.16: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.161534-08:00 CRITICAL [(default) magma_0/kvstore-560/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-560/rev-000000001/seqIndex/state.19: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.164348-08:00 CRITICAL [(default) magma_0/kvstore-568/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-568/rev-000000001/seqIndex/state.17: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.167369-08:00 CRITICAL [(default) magma_0/kvstore-576/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-576/rev-000000001/seqIndex/state.19: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.170382-08:00 CRITICAL [(default) magma_0/kvstore-584/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-584/rev-000000001/seqIndex/state.21: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.173624-08:00 CRITICAL [(default) magma_0/kvstore-592/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-592/rev-000000001/seqIndex/state.17: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.176340-08:00 CRITICAL [(default) magma_0/kvstore-768/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-768/rev-000000001/seqIndex/state.17: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.179273-08:00 CRITICAL [(default) magma_0/kvstore-776/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-776/rev-000000001/seqIndex/state.24: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.182590-08:00 CRITICAL [(default) magma_0/kvstore-784/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-784/rev-000000001/seqIndex/state.23: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.186098-08:00 CRITICAL [(default) magma_0/kvstore-792/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-792/rev-000000001/seqIndex/state.549: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.188392-08:00 CRITICAL [(default) magma_0/kvstore-800/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-800/rev-000000001/seqIndex/state.551: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.190441-08:00 CRITICAL [(default) magma_0/kvstore-808/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-808/rev-000000001/seqIndex/state.25: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.193399-08:00 CRITICAL [(default) magma_0/kvstore-816/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-816/rev-000000001/seqIndex/state.25: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.197261-08:00 CRITICAL [(default) magma_0/kvstore-824/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-824/rev-000000001/seqIndex/state.28: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.199978-08:00 CRITICAL [(default) magma_0/kvstore-832/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-832/rev-000000001/seqIndex/state.21: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.202877-08:00 CRITICAL [(default) magma_0/kvstore-840/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-840/rev-000000001/seqIndex/state.18: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.206035-08:00 CRITICAL [(default) magma_0/kvstore-848/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-848/rev-000000001/seqIndex/state.195: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.211434-08:00 CRITICAL [(default) magma_1/kvstore-1/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-1/rev-000000001/seqIndex/state.23: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.214315-08:00 CRITICAL [(default) magma_1/kvstore-9/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-9/rev-000000001/seqIndex/state.29: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.217197-08:00 CRITICAL [(default) magma_1/kvstore-17/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-17/rev-000000001/seqIndex/state.59: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.220185-08:00 CRITICAL [(default) magma_1/kvstore-25/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-25/rev-000000001/seqIndex/state.96: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.224010-08:00 CRITICAL [(default) magma_1/kvstore-33/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-33/rev-000000001/seqIndex/state.125: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.228994-08:00 CRITICAL [(default) magma_1/kvstore-41/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-41/rev-000000001/seqIndex/state.169: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.234474-08:00 CRITICAL [(default) magma_1/kvstore-49/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-49/rev-000000001/seqIndex/state.223: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.239849-08:00 CRITICAL [(default) magma_1/kvstore-57/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-57/rev-000000001/seqIndex/state.246: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.246784-08:00 CRITICAL [(default) magma_1/kvstore-65/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-65/rev-000000001/seqIndex/state.270: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.255182-08:00 CRITICAL [(default) magma_1/kvstore-73/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-73/rev-000000001/seqIndex/state.282: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.263005-08:00 CRITICAL [(default) magma_1/kvstore-81/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-81/rev-000000001/seqIndex/state.297: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.272005-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.25: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.274564-08:00 CRITICAL [(default) magma_1/kvstore-97/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-97/rev-000000001/seqIndex/state.49: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.277217-08:00 CRITICAL [(default) magma_1/kvstore-105/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-105/rev-000000001/seqIndex/state.79: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.280632-08:00 CRITICAL [(default) magma_1/kvstore-113/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-113/rev-000000001/seqIndex/state.108: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.284588-08:00 CRITICAL [(default) magma_1/kvstore-121/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-121/rev-000000001/seqIndex/state.145: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.289144-08:00 CRITICAL [(default) magma_1/kvstore-129/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-129/rev-000000001/seqIndex/state.209: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.295424-08:00 CRITICAL [(default) magma_1/kvstore-137/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-137/rev-000000001/seqIndex/state.251: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.302092-08:00 CRITICAL [(default) magma_1/kvstore-145/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-145/rev-000000001/seqIndex/state.260: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.310236-08:00 CRITICAL [(default) magma_1/kvstore-153/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-153/rev-000000001/seqIndex/state.270: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.331438-08:00 CRITICAL [(default) magma_1/kvstore-161/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-161/rev-000000001/seqIndex/state.286: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', '[ns_server:info,2022-03-06T06:13:46.541-08:00,babysitter_of_ns_1@cb.local:<0.133.0>:ns_port_server:log:226]memcached<0.133.0>: 2022-03-06T06:13:46.340699-08:00 CRITICAL [(default) magma_1/kvstore-169/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-169/rev-000000001/seqIndex/state.293: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.348570-08:00 CRITICAL [(default) magma_1/kvstore-177/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-177/rev-000000001/seqIndex/state.57: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.350620-08:00 CRITICAL [(default) magma_1/kvstore-185/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-185/rev-000000001/seqIndex/state.79: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.353892-08:00 CRITICAL [(default) magma_1/kvstore-193/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-193/rev-000000001/seqIndex/state.115: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.357917-08:00 CRITICAL [(default) magma_1/kvstore-201/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-201/rev-000000001/seqIndex/state.148: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.361947-08:00 CRITICAL [(default) magma_1/kvstore-209/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-209/rev-000000001/seqIndex/state.183: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.366927-08:00 CRITICAL [(default) magma_1/kvstore-217/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-217/rev-000000001/seqIndex/state.207: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.372733-08:00 CRITICAL [(default) magma_1/kvstore-225/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-225/rev-000000001/seqIndex/state.238: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.379102-08:00 CRITICAL [(default) magma_1/kvstore-233/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-233/rev-000000001/seqIndex/state.265: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.386931-08:00 CRITICAL [(default) magma_1/kvstore-241/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-241/rev-000000001/seqIndex/state.813: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.389492-08:00 CRITICAL [(default) magma_1/kvstore-249/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-249/rev-000000001/seqIndex/state.821: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.409989-08:00 CRITICAL [(default) magma_1/kvstore-289/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-289/rev-000000001/seqIndex/state.18: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.412366-08:00 CRITICAL [(default) magma_1/kvstore-297/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-297/rev-000000001/seqIndex/state.15: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.414815-08:00 CRITICAL [(default) magma_1/kvstore-305/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-305/rev-000000001/seqIndex/state.19: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.417538-08:00 CRITICAL [(default) magma_1/kvstore-313/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-313/rev-000000001/seqIndex/state.18: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.420400-08:00 CRITICAL [(default) magma_1/kvstore-321/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-321/rev-000000001/seqIndex/state.19: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.423244-08:00 CRITICAL [(default) magma_1/kvstore-329/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-329/rev-000000001/seqIndex/state.545: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.425574-08:00 CRITICAL [(default) magma_1/kvstore-337/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-337/rev-000000001/seqIndex/state.19: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.439013-08:00 CRITICAL [(default) magma_1/kvstore-537/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-537/rev-000000001/seqIndex/state.545: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.440875-08:00 CRITICAL [(default) magma_1/kvstore-545/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-545/rev-000000001/seqIndex/state.544: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.442584-08:00 CRITICAL [(default) magma_1/kvstore-553/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-553/rev-000000001/seqIndex/state.545: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.444430-08:00 CRITICAL [(default) magma_1/kvstore-561/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-561/rev-000000001/seqIndex/state.547: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.445676-08:00 CRITICAL [(default) magma_1/kvstore-569/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-569/rev-000000001/seqIndex/state.546: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.447140-08:00 CRITICAL [(default) magma_1/kvstore-577/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-577/rev-000000001/seqIndex/state.20: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.450005-08:00 CRITICAL [(default) magma_1/kvstore-585/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-585/rev-000000001/seqIndex/state.20: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.452720-08:00 CRITICAL [(default) magma_1/kvstore-593/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-593/rev-000000001/seqIndex/state.90: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.455776-08:00 CRITICAL [(default) magma_1/kvstore-769/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-769/rev-000000001/seqIndex/state.22: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1/kvstore-769/rev-000000001\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.457979-08:00 CRITICAL [(default) magma_1/kvstore-777/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-777/rev-000000001/seqIndex/state.27: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.459961-08:00 CRITICAL [(default) magma_1/kvstore-785/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-785/rev-000000001/seqIndex/state.22: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.462586-08:00 CRITICAL [(default) magma_1/kvstore-793/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-793/rev-000000001/seqIndex/state.21: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.465633-08:00 CRITICAL [(default) magma_1/kvstore-801/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-801/rev-000000001/seqIndex/state.25: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.468668-08:00 CRITICAL [(default) magma_1/kvstore-809/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-809/rev-000000001/seqIndex/state.27: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.471188-08:00 CRITICAL [(default) magma_1/kvstore-817/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-817/rev-000000001/seqIndex/state.25: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.474298-08:00 CRITICAL [(default) magma_1/kvstore-825/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-825/rev-000000001/seqIndex/state.27: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.476752-08:00 CRITICAL [(default) magma_1/kvstore-833/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-833/rev-000000001/seqIndex/state.545: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.478377-08:00 CRITICAL [(default) magma_1/kvstore-841/rev-000000001/localIndex]Error occured during memtable flush (IOError: /data/default/magma.1/kvstore-841/rev-000000001/localIndex/sstable.563.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1).\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.479793-08:00 CRITICAL [(default) magma_1/kvstore-849/rev-000000001/localIndex]Error occured during memtable flush (IOError: /data/default/magma.1/kvstore-849/rev-000000001/localIndex/sstable.902.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1).\n'] 2022-03-06 09:10:21,201 | test | INFO | MainThread | [cluster_ready_functions:trigger_cb_collect_on_cluster:1130] Running cbcollect on node ns_1@172.23.100.35,ns_1@172.23.105.164,ns_1@172.23.105.206,ns_1@172.23.106.177 2022-03-06 09:10:31,223 | test | INFO | MainThread | [cluster_ready_functions:trigger_cb_collect_on_cluster:1133] ns_1@172.23.100.35,ns_1@172.23.105.164,ns_1@172.23.105.206,ns_1@172.23.106.177 - cbcollect status: True 2022-03-06 09:10:31,223 | test | INFO | MainThread | [cluster_ready_functions:wait_for_cb_collect_to_complete:1138] Polling active_tasks to check cbcollect status 2022-03-06 09:14:01,650 | test | INFO | MainThread | [cluster_ready_functions:copy_cb_collect_logs:1165] ns_1@172.23.100.35: Copying cbcollect ZIP file to Client 2022-03-06 09:14:02,516 | test | INFO | MainThread | [cluster_ready_functions:copy_cb_collect_logs:1165] ns_1@172.23.105.164: Copying cbcollect ZIP file to Client 2022-03-06 09:14:03,526 | test | INFO | MainThread | [cluster_ready_functions:copy_cb_collect_logs:1165] ns_1@172.23.105.206: Copying cbcollect ZIP file to Client 2022-03-06 09:14:04,311 | test | INFO | MainThread | [cluster_ready_functions:copy_cb_collect_logs:1165] ns_1@172.23.106.177: Copying cbcollect ZIP file to Client ERROR ====================================================================== ERROR: test_data_load_collections_with_graceful_failover_recovery (bucket_collections.collections_rebalance.CollectionsRebalance) ---------------------------------------------------------------------- Traceback (most recent call last): File "pytests/bucket_collections/collections_rebalance.py", line 92, in tearDown super(CollectionsRebalance, self).tearDown() File "pytests/bucket_collections/collections_base.py", line 101, in tearDown During the test, Remote Connections: 219, Disconnections: 219 SDK Connections: 30, Disconnections: 30 super(CollectionBase, self).tearDown() File "pytests/basetestcase.py", line 1109, in tearDown super(ClusterSetup, self).tearDown() File "pytests/basetestcase.py", line 552, in tearDown self.assertFalse(result, msg="Cb_log file validation failed") AssertionError: Cb_log file validation failed ---------------------------------------------------------------------- Ran 1 test in 11200.456s FAILED (errors=1) summary so far suite bucket_collections.collections_rebalance.CollectionsRebalance , pass 1 , fail 1 failures so far... bucket_collections.collections_rebalance.CollectionsRebalance.test_data_load_collections_with_graceful_failover_recovery testrunner logs, diags and results are available under /data/workspace/temp_rebalance_magma/logs/testrunner-22-Mar-06_03-39-55/test_2 Logs will be stored at /data/workspace/temp_rebalance_magma/logs/testrunner-22-Mar-06_03-39-55/test_3 guides/gradlew --refresh-dependencies testrunner -P jython=/opt/jython/bin/jython -P 'args=-i /tmp/win10-bucket-ops-temp_rebalance_magma.ini rerun=False,get-cbcollect-info=True -t bucket_collections.collections_rebalance.CollectionsRebalance.test_data_load_collections_with_hard_failover_recovery,nodes_init=4,nodes_failover=1,recovery_type=full,bucket_spec=magma_dgm.bloom_filter.4_node_1_replica_magma_1024,doc_size=30,randomize_value=True,data_load_stage=during,skip_validations=False,data_load_spec=volume_test_load_with_CRUD_on_collections,change_magma_quota=True,GROUP=failover' Test Input params: {'doc_size': '30', 'data_load_stage': 'during', 'cluster_name': 'win10-bucket-ops-temp_rebalance_magma', 'nodes_failover': '1', 'ini': '/tmp/win10-bucket-ops-temp_rebalance_magma.ini', 'get-cbcollect-info': 'True', 'recovery_type': 'full', 'conf_file': 'conf/magma/dgm_collections_bloom_filter.conf', 'change_magma_quota': 'True', 'skip_validations': 'False', 'spec': 'dgm_collections_bloom_filter', 'rerun': 'False', 'num_nodes': 7, 'logs_folder': '/data/workspace/temp_rebalance_magma/logs/testrunner-22-Mar-06_03-39-55/test_3', 'nodes_init': '4', 'GROUP': 'failover', 'bucket_spec': 'magma_dgm.bloom_filter.4_node_1_replica_magma_1024', 'case_number': 3, 'randomize_value': 'True', 'data_load_spec': 'volume_test_load_with_CRUD_on_collections'} test_data_load_collections_with_hard_failover_recovery (bucket_collections.collections_rebalance.CollectionsRebalance) ... 2022-03-06 09:14:05,490 | test | INFO | MainThread | [basetestcase:log_setup_status:647] ========= CollectionsRebalance setup started for test #3 test_data_load_collections_with_hard_failover_recovery ========= 2022-03-06 09:14:05,607 | test | INFO | MainThread | [basetestcase:log_setup_status:647] ========= BaseTestCase setup started for test #3 test_data_load_collections_with_hard_failover_recovery ========= 2022-03-06 09:14:42,385 | test | INFO | MainThread | [rest_client:monitorRebalance:1610] Rebalance done. Taken 11.0549998283 seconds to complete 2022-03-06 09:14:42,392 | test | INFO | MainThread | [common_lib:sleep:23] Sleep 5 seconds. Reason: Wait after rebalance complete 2022-03-06 09:14:47,493 | test | INFO | MainThread | [basetestcase:initialize_cluster:467] Initializing cluster : C1 Got this failure java.net.ConnectException: Connection refused: /172.23.100.37:8091 during connect (<_realsocket at 0x784554 type=client open_count=1 channel=[id: 0x9774b849, 0.0.0.0/0.0.0.0:48137] timeout=300.0>) 2022-03-06 09:14:50,075 | test | ERROR | pool-6-thread-21 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.100.37:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.105.164:8091 during connect (<_realsocket at 0x784555 type=client open_count=1 channel=[id: 0x8265944d, 0.0.0.0/0.0.0.0:48962] timeout=300.0>) 2022-03-06 09:14:50,381 | test | ERROR | pool-6-thread-15 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.105.164:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.100.34:8091 during connect (<_realsocket at 0x784556 type=client open_count=1 channel=[id: 0xe1dd3536, 0.0.0.0/0.0.0.0:46686] timeout=300.0>) 2022-03-06 09:14:50,631 | test | ERROR | pool-6-thread-19 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.100.34:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.105.206:8091 during connect (<_realsocket at 0x784557 type=client open_count=1 channel=[id: 0xfa19b3ae, 0.0.0.0/0.0.0.0:60601] timeout=300.0>) 2022-03-06 09:14:50,665 | test | ERROR | pool-6-thread-16 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.105.206:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.106.177:8091 during connect (<_realsocket at 0x784558 type=client open_count=1 channel=[id: 0x6c75dbf3, 0.0.0.0/0.0.0.0:37220] timeout=300.0>) 2022-03-06 09:14:50,684 | test | ERROR | pool-6-thread-17 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.106.177:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.100.35:8091 during connect (<_realsocket at 0x784559 type=client open_count=1 channel=[id: 0xc2b0ded6, 0.0.0.0/0.0.0.0:40926] timeout=300.0>) 2022-03-06 09:14:50,782 | test | ERROR | pool-6-thread-18 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.100.35:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.100.36:8091 during connect (<_realsocket at 0x78455a type=client open_count=1 channel=[id: 0xb033b82f, 0.0.0.0/0.0.0.0:44275] timeout=300.0>) 2022-03-06 09:14:50,953 | test | ERROR | pool-6-thread-20 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.100.36:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.100.37:8091 during connect (<_realsocket at 0x78455b type=client open_count=1 channel=[id: 0xfc1f8c38, 0.0.0.0/0.0.0.0:55940] timeout=300.0>) 2022-03-06 09:14:52,082 | test | ERROR | pool-6-thread-21 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.100.37:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.105.164:8091 during connect (<_realsocket at 0x78455c type=client open_count=1 channel=[id: 0x67980911, 0.0.0.0/0.0.0.0:49802] timeout=300.0>) 2022-03-06 09:14:52,388 | test | ERROR | pool-6-thread-15 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.105.164:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.100.34:8091 during connect (<_realsocket at 0x78455d type=client open_count=1 channel=[id: 0xba69733d, 0.0.0.0/0.0.0.0:58639] timeout=300.0>) 2022-03-06 09:14:52,638 | test | ERROR | pool-6-thread-19 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.100.34:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.105.206:8091 during connect (<_realsocket at 0x78455e type=client open_count=1 channel=[id: 0x34947fa3, 0.0.0.0/0.0.0.0:47685] timeout=300.0>) 2022-03-06 09:14:52,670 | test | ERROR | pool-6-thread-16 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.105.206:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.106.177:8091 during connect (<_realsocket at 0x78455f type=client open_count=1 channel=[id: 0x86739156, 0.0.0.0/0.0.0.0:45098] timeout=300.0>) 2022-03-06 09:14:52,688 | test | ERROR | pool-6-thread-17 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.106.177:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.100.35:8091 during connect (<_realsocket at 0x784560 type=client open_count=1 channel=[id: 0xb90da41a, 0.0.0.0/0.0.0.0:42184] timeout=300.0>) 2022-03-06 09:14:52,786 | test | ERROR | pool-6-thread-18 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.100.35:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.100.36:8091 during connect (<_realsocket at 0x784561 type=client open_count=1 channel=[id: 0x71f289b1, 0.0.0.0/0.0.0.0:39222] timeout=300.0>) 2022-03-06 09:14:52,957 | test | ERROR | pool-6-thread-20 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.100.36:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.100.37:8091 during connect (<_realsocket at 0x784562 type=client open_count=1 channel=[id: 0x8f4a1b11, 0.0.0.0/0.0.0.0:39290] timeout=300.0>) 2022-03-06 09:14:54,086 | test | ERROR | pool-6-thread-21 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.100.37:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.105.164:8091 during connect (<_realsocket at 0x784563 type=client open_count=1 channel=[id: 0x58a3f658, 0.0.0.0/0.0.0.0:49280] timeout=300.0>) 2022-03-06 09:14:54,391 | test | ERROR | pool-6-thread-15 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.105.164:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.100.34:8091 during connect (<_realsocket at 0x784564 type=client open_count=1 channel=[id: 0xaac1ed95, 0.0.0.0/0.0.0.0:46667] timeout=300.0>) 2022-03-06 09:14:54,641 | test | ERROR | pool-6-thread-19 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.100.34:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.105.206:8091 during connect (<_realsocket at 0x784565 type=client open_count=1 channel=[id: 0x99645d16, 0.0.0.0/0.0.0.0:46211] timeout=300.0>) 2022-03-06 09:14:54,674 | test | ERROR | pool-6-thread-16 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.105.206:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.106.177:8091 during connect (<_realsocket at 0x784566 type=client open_count=1 channel=[id: 0xa3a35b6c, 0.0.0.0/0.0.0.0:47156] timeout=300.0>) 2022-03-06 09:14:54,693 | test | ERROR | pool-6-thread-17 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.106.177:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.100.35:8091 during connect (<_realsocket at 0x784567 type=client open_count=1 channel=[id: 0x53c6ade4, 0.0.0.0/0.0.0.0:49413] timeout=300.0>) 2022-03-06 09:14:54,790 | test | ERROR | pool-6-thread-18 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.100.35:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.100.36:8091 during connect (<_realsocket at 0x784568 type=client open_count=1 channel=[id: 0x836933db, 0.0.0.0/0.0.0.0:48939] timeout=300.0>) 2022-03-06 09:14:54,961 | test | ERROR | pool-6-thread-20 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.100.36:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.100.37:8091 during connect (<_realsocket at 0x784569 type=client open_count=1 channel=[id: 0x06b3674a, 0.0.0.0/0.0.0.0:57839] timeout=300.0>) 2022-03-06 09:14:56,091 | test | ERROR | pool-6-thread-21 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.100.37:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.105.164:8091 during connect (<_realsocket at 0x78456a type=client open_count=1 channel=[id: 0x56816dfa, 0.0.0.0/0.0.0.0:38167] timeout=300.0>) 2022-03-06 09:14:56,397 | test | ERROR | pool-6-thread-15 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.105.164:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.100.34:8091 during connect (<_realsocket at 0x78456b type=client open_count=1 channel=[id: 0x4db42536, 0.0.0.0/0.0.0.0:47123] timeout=300.0>) 2022-03-06 09:14:56,647 | test | ERROR | pool-6-thread-19 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.100.34:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.105.206:8091 during connect (<_realsocket at 0x78456c type=client open_count=1 channel=[id: 0xd209c602, 0.0.0.0/0.0.0.0:41077] timeout=300.0>) 2022-03-06 09:14:56,678 | test | ERROR | pool-6-thread-16 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.105.206:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.106.177:8091 during connect (<_realsocket at 0x78456d type=client open_count=1 channel=[id: 0x12ac77a4, 0.0.0.0/0.0.0.0:42538] timeout=300.0>) 2022-03-06 09:14:56,697 | test | ERROR | pool-6-thread-17 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.106.177:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.100.35:8091 during connect (<_realsocket at 0x78456e type=client open_count=1 channel=[id: 0x3a30c91d, 0.0.0.0/0.0.0.0:44849] timeout=300.0>) 2022-03-06 09:14:56,796 | test | ERROR | pool-6-thread-18 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.100.35:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.100.36:8091 during connect (<_realsocket at 0x78456f type=client open_count=1 channel=[id: 0xd3fc018b, 0.0.0.0/0.0.0.0:43430] timeout=300.0>) 2022-03-06 09:14:56,966 | test | ERROR | pool-6-thread-20 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.100.36:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.100.37:8091 during connect (<_realsocket at 0x784570 type=client open_count=1 channel=[id: 0x886d6f9b, 0.0.0.0/0.0.0.0:41376] timeout=300.0>) 2022-03-06 09:14:58,095 | test | ERROR | pool-6-thread-21 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.100.37:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.100.34:8091 during connect (<_realsocket at 0x784571 type=client open_count=1 channel=[id: 0xe46ea28c, 0.0.0.0/0.0.0.0:50093] timeout=300.0>) 2022-03-06 09:14:58,651 | test | ERROR | pool-6-thread-19 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.100.34:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.100.35:8091 during connect (<_realsocket at 0x784572 type=client open_count=1 channel=[id: 0x3d4654cc, 0.0.0.0/0.0.0.0:38381] timeout=300.0>) 2022-03-06 09:14:58,799 | test | ERROR | pool-6-thread-18 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.100.35:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.100.36:8091 during connect (<_realsocket at 0x784573 type=client open_count=1 channel=[id: 0x7094f0ca, 0.0.0.0/0.0.0.0:56990] timeout=300.0>) 2022-03-06 09:14:58,970 | test | ERROR | pool-6-thread-20 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.100.36:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.100.37:8091 during connect (<_realsocket at 0x784574 type=client open_count=1 channel=[id: 0xba63d6b3, 0.0.0.0/0.0.0.0:46121] timeout=300.0>) 2022-03-06 09:15:00,099 | test | ERROR | pool-6-thread-21 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.100.37:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.100.35:8091 during connect (<_realsocket at 0x784575 type=client open_count=1 channel=[id: 0x527b2fbf, 0.0.0.0/0.0.0.0:39868] timeout=300.0>) 2022-03-06 09:15:00,805 | test | ERROR | pool-6-thread-18 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.100.35:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.100.36:8091 during connect (<_realsocket at 0x784576 type=client open_count=1 channel=[id: 0x86abd4bd, 0.0.0.0/0.0.0.0:35383] timeout=300.0>) 2022-03-06 09:15:00,976 | test | ERROR | pool-6-thread-20 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.100.36:8091/nodes/self error [Errno 111] Connection refused 2022-03-06 09:15:03,605 | infra | CRITICAL | pool-6-thread-22 | [task:call:6181] mcdMemoryReserved reported in nodes/self is: 9379 2022-03-06 09:15:03,607 | test | CRITICAL | pool-6-thread-22 | [table_view:display:72] Memory quota allocated: +---------------------+---------+ | Service | RAM MiB | +---------------------+---------+ | eventingMemoryQuota | 256 | | cbasMemoryQuota | 1024 | | indexMemoryQuota | 256 | | ftsMemoryQuota | 256 | | memoryQuota | 9279 | +---------------------+---------+ 2022-03-06 09:15:03,789 | test | WARNING | MainThread | [basetestcase:_initialize_nodes:702] RAM quota was defined less than 100 MB: 2022-03-06 09:15:15,678 | test | INFO | MainThread | [basetestcase:initialize_cluster:493] Cluster C1 initialized 2022-03-06 09:15:15,678 | test | INFO | MainThread | [basetestcase:log_setup_status:647] ========= BaseTestCase setup finished for test #3 test_data_load_collections_with_hard_failover_recovery ========= 2022-03-06 09:15:15,785 | test | INFO | MainThread | [basetestcase:log_setup_status:647] ========= ClusterSetup setup started for test #3 test_data_load_collections_with_hard_failover_recovery ========= 2022-03-06 09:15:28,269 | test | INFO | pool-6-thread-23 | [table_view:display:72] Rebalance Overview +----------------+----------+-----------------------+----------------+--------------+-----------------------+ | Nodes | Services | Version | CPU | Status | Membership / Recovery | +----------------+----------+-----------------------+----------------+--------------+-----------------------+ | 172.23.105.164 | kv | 7.1.0-2440-enterprise | 0.225563909774 | Cluster node | active / none | | 172.23.105.206 | None | | | <--- IN --- | | | 172.23.106.177 | None | | | <--- IN --- | | | 172.23.100.35 | None | | | <--- IN --- | | +----------------+----------+-----------------------+----------------+--------------+-----------------------+ 2022-03-06 09:15:28,283 | test | INFO | pool-6-thread-23 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 09:15:43,315 | test | INFO | pool-6-thread-23 | [task:check:415] Rebalance - status: none, progress: 100 2022-03-06 09:15:43,326 | test | INFO | pool-6-thread-23 | [task:check:474] Rebalance completed with progress: 100% in 15.0590000153 sec 2022-03-06 09:15:43,436 | test | INFO | MainThread | [table_view:display:72] Cluster statistics +----------------+----------+-----------------+-----------+-----------+----------------------+-------------------+-----------------------+ | Node | Services | CPU_utilization | Mem_total | Mem_free | Swap_mem_used | Active / Replica | Version | +----------------+----------+-----------------+-----------+-----------+----------------------+-------------------+-----------------------+ | 172.23.105.206 | kv | 1.06489601604 | 11.45 GiB | 10.48 GiB | 23.59 MiB / 3.50 GiB | 0 / 0 | 7.1.0-2440-enterprise | | 172.23.106.177 | kv | 0.488660568851 | 11.45 GiB | 10.43 GiB | 23.15 MiB / 3.50 GiB | 0 / 0 | 7.1.0-2440-enterprise | | 172.23.100.35 | kv | 0 | 0.0 Byte | 0.0 Byte | 0.0 Byte / 0.0 Byte | 0 / 0 | 7.1.0-2440-enterprise | | 172.23.105.164 | kv | 1.06343050169 | 11.45 GiB | 10.38 GiB | 0.0 Byte / 3.50 GiB | 0 / 0 | 7.1.0-2440-enterprise | +----------------+----------+-----------------+-----------+-----------+----------------------+-------------------+-----------------------+ 2022-03-06 09:15:43,437 | test | INFO | MainThread | [basetestcase:log_setup_status:647] ========= ClusterSetup setup complete for test #3 test_data_load_collections_with_hard_failover_recovery ========= 2022-03-06 09:15:43,437 | test | INFO | MainThread | [basetestcase:log_setup_status:647] ========= CollectionBase setup started for test #3 test_data_load_collections_with_hard_failover_recovery ========= 2022-03-06 09:15:43,571 | test | INFO | MainThread | [bucket_ready_functions:create_buckets_using_json_data:1913] Creating required buckets from template 2022-03-06 09:15:43,826 | test | INFO | MainThread | [bucket_ready_functions:wait_for_collection_creation_to_complete:2527] Waiting for all collections to be created 2022-03-06 09:15:44,111 | test | INFO | MainThread | [table_view:display:72] Bucket statistics +---------+-----------+-----------------+----------+------------+-----+-------+-----------+----------+-----------+-----+ | Bucket | Type | Storage Backend | Replicas | Durability | TTL | Items | RAM Quota | RAM Used | Disk Used | ARR | +---------+-----------+-----------------+----------+------------+-----+-------+-----------+----------+-----------+-----+ | bucket1 | couchbase | couchstore | 1 | none | 0 | 0 | 0.0 Byte | 0.0 Byte | 0.0 Byte | 100 | | bucket2 | couchbase | magma | 1 | none | 0 | 0 | 4.00 GiB | 0.0 Byte | 0.0 Byte | 100 | | default | couchbase | magma | 1 | none | 0 | 0 | 0.0 Byte | 0.0 Byte | 0.0 Byte | 100 | +---------+-----------+-----------------+----------+------------+-----+-------+-----------+----------+-----------+-----+ 2022-03-06 09:15:44,861 | test | INFO | MainThread | [common_lib:sleep:23] Sleep 30 seconds. Reason: Sleep while magma storage quota setting is taking effect 2022-03-06 09:16:14,862 | test | INFO | MainThread | [collections_base:collection_setup:174] Creating required SDK clients for client_pool 2022-03-06 09:16:20,572 | test | INFO | MainThread | [bucket_ready_functions:perform_tasks_from_spec:4993] Performing scope/collection specific operations 2022-03-06 09:16:20,578 | test | INFO | MainThread | [bucket_ready_functions:perform_tasks_from_spec:5083] Done Performing scope/collection specific operations 2022-03-06 10:13:50,388 | test | INFO | pool-6-thread-28 | [table_view:display:72] Ops trend for bucket 'bucket1' +-------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------+ | Min | Trend | Max | +-------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------+ | 0.000 | *....................................................................................................*******X | 5500.000 | | 0.000 | *********************************************************************************************************************************************************************************************************************************************..............********...........*******X | 1400.000 | | 0.000 | *********************************************************************************************************************************************************************************************************************************************************************.....********....*******X | 500.000 | | 0.000 | ***************************************************************************************************************************************************************************************************************************************************************************.*******......*******X | 400.000 | | 0.000 | **********************************************************************************************************************************************************************************************************************************************************************************************.********..*******X | 200.000 | | 0.000 | *******************************************************************************************************************************************************************************************************************************************************************************************************************************************.**************X | 100.000 | | 0.000 | *************************************************************************************************************************************************************************************************************************************************************************************************************************.*******X | 100.000 | | 0.000 | ****************************************************************************************************************************************************************************************************************************************************.....*******X | 300.000 | | 0.000 | *********************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************.*******X | 100.000 | | 0.000 | ***********************************************************************************************************************************************************X | 0.000 | +-------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------+ 2022-03-06 10:13:50,815 | test | INFO | pool-6-thread-29 | [table_view:display:72] Ops trend for bucket 'bucket2' +---------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------+ | Min | Trend | Max | +---------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------+ | 0.000 | *....................................................................................................*******X | 800.000 | | 0.000 | *********************************************************************************************************************************************************************************************************************************************............********...............................................................*******X | 600.000 | | 0.000 | *************************************************************************************************************************************************************************************************************************************************************.........................***************............*******X | 300.000 | | 0.000 | ***********************************************************************************************************************************************************************************************************************************************************************************.....................................*******X | 300.000 | | 200.000 | ........................*******X | 200.000 | | 0.000 | **************************************************************************************************************************************************************************************************************************************************************************************............********..................................................*******X | 500.000 | | 0.000 | ***********************************************************************************************************************************************************************************************************************************************************************************************************.........................*******X | 200.000 | | 100.000 | ...........********.............******X | 200.000 | | 0.000 | ********.....................................*******X | 300.000 | | 200.000 | ........................*******X | 200.000 | | 0.000 | **************************************************************************************************************************************************************************************************************************************************************************************************..................................................*******X | 400.000 | | 100.000 | ...........********.............********.........................******X | 400.000 | | 0.000 | ************************************************************************************************************************************************************************************************************************************............********......................................***************X | 400.000 | | 0.000 | ********............******X | 100.000 | | 0.000 | *********************************************************************************************************************************************************************************************************************.........................********..............................................................*******X | 700.000 | | 570.000 | ......................................................................*******X | 570.000 | | 385.300 | ...............................................X | 385.300 | | 330.000 | ........................................******X | 330.000 | | 200.000 | ........................*******X | 200.000 | | 0.000 | *************************************************************************************************************************************************************************************************************....................................................................................................*******X | 800.000 | | 300.000 | ....................................********.........................*******X | 500.000 | | 0.000 | ***********************************************************************************************************************************************************X | 0.000 | +---------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------+ 2022-03-06 10:13:50,953 | test | INFO | pool-6-thread-30 | [table_view:display:72] Ops trend for bucket 'default' +-----------+------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------+ | Min | Trend | Max | +-----------+------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------+ | 0.000 | *........********.......................................................................................****************X | 83744.800 | | 82757.300 | .............................................................................................********....********.********.******X | 87261.500 | | 85360.000 | ................................................................................................*******X | 85360.000 | | 82891.500 | .............................................................................................********....*******X | 86001.200 | | 84089.200 | ...............................................................................................*******X | 84089.200 | | 83385.000 | ..............................................................................................*******X | 83385.000 | | 80720.900 | ...........................................................................................********...********.*******X | 84211.400 | | 83303.000 | ..............................................................................................*******X | 83303.000 | | 82659.100 | .............................................................................................*******....*******X | 86162.600 | | 85463.600 | ................................................................................................********..*******X | 86452.200 | | 82781.600 | .............................................................................................******X | 82781.600 | | 82551.400 | .............................................................................................*******X | 82551.400 | | 79572.100 | ..........................................................................................*******X | 79572.100 | | 77710.200 | ........................................................................................*******X | 77710.200 | | 77317.500 | .......................................................................................********.....*******..********..*******X | 84998.600 | | 84573.700 | ...............................................................................................*******X | 84573.700 | | 83785.100 | ...............................................................................................******X | 83785.100 | | 82015.300 | ............................................................................................*******X | 82015.300 | | 78650.800 | .........................................................................................********.....*******X | 83455.000 | | 81965.300 | ............................................................................................********..******X | 83284.500 | | 79808.100 | ..........................................................................................*******X | 79808.100 | | 78536.600 | .........................................................................................***************X | 78802.200 | | 77066.500 | .......................................................................................*******X | 77066.500 | | 71900.000 | .................................................................................******X | 71900.000 | | 65869.100 | ..........................................................................********..........********...*******X | 77525.800 | | 75276.200 | .....................................................................................*******X | 75276.200 | | 75247.400 | .....................................................................................******X | 75247.400 | | 69222.100 | ..............................................................................********....*******X | 72916.200 | | 68419.600 | .............................................................................********.*******....*******X | 72946.100 | | 66847.000 | ...........................................................................********........*******...*******X | 76175.900 | | 72491.100 | ..................................................................................*******X | 72491.100 | | 69369.200 | ..............................................................................********.*******....********......********..*******X | 80683.300 | | 75792.000 | .....................................................................................*******X | 75792.000 | | 71108.600 | ................................................................................******X | 71108.600 | | 70109.500 | ...............................................................................********..********.*******.......********..*******X | 81087.300 | | 72483.300 | ..................................................................................*******...*....******X | 78588.600 | | 76479.000 | ......................................................................................*******X | 76479.000 | | 73325.000 | ...................................................................................********.******X | 74925.500 | | 69236.400 | ..............................................................................********...*******X | 71963.800 | | 66414.700 | ...........................................................................********...*******.....********..*******X | 75909.200 | | 70554.600 | ...............................................................................*******X | 70554.600 | | 65625.700 | ..........................................................................***************...*******......***************X | 73836.300 | | 68387.600 | .............................................................................********.********............******X | 79607.600 | | 72931.500 | ..................................................................................********.*******X | 73751.400 | | 71898.000 | .................................................................................******X | 71898.000 | | 69320.600 | ..............................................................................X | 69320.600 | | 66747.700 | ...........................................................................*******...********........*******X | 76592.500 | | 72767.800 | ..................................................................................********.******X | 74048.200 | | 65084.600 | .........................................................................********......******X | 70328.200 | | 69608.200 | ..............................................................................********.*******X | 70047.000 | | 67088.200 | ...........................................................................********.*******.*******X | 68884.700 | | 64352.200 | ........................................................................********.....********....*******....*******X | 75496.100 | | 71999.600 | .................................................................................*******...*******X | 74256.400 | | 66996.300 | ...........................................................................********...**************X | 69605.400 | | 68174.200 | .............................................................................*******X | 68174.200 | | 63490.900 | .......................................................................*******....*******X | 66896.100 | | 63387.400 | .......................................................................********...********....******X | 69754.000 | | 64276.200 | ........................................................................********....********.***************..*******X | 70335.400 | | 69916.800 | ...............................................................................******X | 69916.800 | | 64703.400 | .........................................................................********....*******X | 68289.600 | | 63476.800 | .......................................................................********....******X | 66760.900 | | 65756.500 | ..........................................................................********...*******X | 68491.400 | | 63361.200 | .......................................................................******X | 63361.200 | | 61535.400 | .....................................................................*******X | 61535.400 | | 55238.100 | ..............................................................*******.......********........********......******X | 73327.000 | | 72457.500 | ..................................................................................*******X | 72457.500 | | 59701.400 | ...................................................................******X | 59701.400 | | 58066.900 | .................................................................********............********........******X | 75481.500 | | 70388.900 | ...............................................................................********........********.*******X | 77886.700 | | 70128.600 | ...............................................................................******X | 70128.600 | | 63064.400 | .......................................................................*******X | 63064.400 | | 55146.400 | ..............................................................*******...*........*******..........*******X | 73383.200 | | 68894.900 | .............................................................................*******X | 68894.900 | | 68093.200 | .............................................................................******X | 68093.200 | | 64909.200 | .........................................................................******X | 64909.200 | | 56786.100 | ................................................................********........*******X | 64196.500 | | 62027.300 | ......................................................................*******.*******X | 63156.500 | | 61230.300 | .....................................................................********..******X | 63262.300 | | 56873.500 | ................................................................*******X | 56873.500 | | 56003.500 | ...............................................................*******...********.........*******X | 66819.500 | | 65871.100 | ..........................................................................*******.*******X | 66387.300 | | 53787.900 | ............................................................*******.********..*******........*******X | 63695.400 | | 60007.600 | ...................................................................*******X | 60007.600 | | 58914.300 | ..................................................................******X | 58914.300 | | 52437.100 | ...........................................................********........*******.*******X | 60605.400 | | 55645.300 | ..............................................................*******.....*******X | 60059.600 | | 59932.800 | ...................................................................******X | 59932.800 | | 56144.000 | ...............................................................********......******X | 61710.300 | | 56317.100 | ...............................................................********...*******.......*******X | 64839.600 | | 60780.800 | ....................................................................*******X | 60780.800 | | 60263.400 | ....................................................................*******.******X | 61777.000 | | 52536.600 | ...........................................................********........******X | 60201.800 | | 60146.400 | ...................................................................*******X | 60146.400 | | 51368.900 | .........................................................********..................*******.....*******X | 71120.100 | | 65293.200 | .........................................................................*******X | 65293.200 | | 62271.100 | ......................................................................******X | 62271.100 | | 49560.500 | .......................................................*******X | 49560.500 | | 45539.200 | ...................................................*******.................********......******X | 65469.400 | | 62038.100 | ......................................................................*******X | 62038.100 | | 60198.200 | ...................................................................******X | 60198.200 | | 56100.700 | ...............................................................*******X | 56100.700 | | 52145.200 | ..........................................................*******.......*.......******X | 63925.600 | | 56181.000 | ...............................................................*******X | 56181.000 | | 54073.500 | ............................................................*******......********.*******X | 59874.800 | | 58063.400 | .................................................................******X | 58063.400 | | 53372.400 | ............................................................*******X | 53372.400 | | 50738.200 | .........................................................*******.............********.******X | 63241.300 | | 60223.100 | ....................................................................*******X | 60223.100 | | 55000.900 | ..............................................................*******.*******X | 55997.300 | | 55952.700 | ...............................................................*******...*******.*...*******.....********.******X | 67944.600 | | 62395.200 | ......................................................................******X | 62395.200 | | 60702.500 | ....................................................................X | 60702.500 | | 55597.400 | ..............................................................*******...*******X | 58011.000 | | 52128.800 | ..........................................................*******..********....********..******X | 59167.800 | | 55730.700 | ..............................................................*******X | 55730.700 | | 49560.900 | .......................................................******X | 49560.900 | | 47843.600 | .....................................................********.....................********...*******.*******X | 68962.800 | | 67039.600 | ...........................................................................******X | 67039.600 | | 61897.000 | .....................................................................********......*******.********........********...*******.....********.*******X | 82797.700 | | 80750.100 | ...........................................................................................********..*******X | 82763.700 | | 82554.200 | .............................................................................................**************X | 82772.900 | | 82212.200 | .............................................................................................********.*******X | 83745.100 | | 82808.500 | .............................................................................................********.********.*******X | 84375.700 | | 82353.100 | .............................................................................................*******X | 82353.100 | | 82024.500 | ............................................................................................********.*******X | 82648.600 | | 81893.800 | ............................................................................................********.******X | 82530.200 | | 81219.400 | ............................................................................................****************.********.********.*******.*******X | 84654.700 | | 84103.400 | ...............................................................................................*******X | 84103.400 | | 82202.600 | .............................................................................................*******X | 82202.600 | | 80743.700 | ...........................................................................................******X | 80743.700 | | 80522.000 | ...........................................................................................*******X | 80522.000 | | 79695.500 | ..........................................................................................********.***************X | 81066.700 | | 80189.000 | ..........................................................................................********...****************..******X | 84521.400 | | 82396.700 | .............................................................................................*******X | 82396.700 | | 82324.900 | .............................................................................................********.*******X | 83515.000 | | 82103.400 | .............................................................................................*******X | 82103.400 | | 81596.400 | ............................................................................................***************X | 81745.200 | | 81455.000 | ............................................................................................********..******X | 83358.100 | | 81950.400 | ............................................................................................X | 81950.400 | | 81489.000 | ............................................................................................*******..***********************X | 83314.400 | | 82967.600 | ..............................................................................................*******X | 82967.600 | | 82865.900 | .............................................................................................*******X | 82865.900 | | 82833.000 | .............................................................................................******X | 82833.000 | | 82427.400 | .............................................................................................*******X | 82427.400 | | 81689.400 | ............................................................................................*******X | 81689.400 | | 80460.900 | ...........................................................................................*******X | 80460.900 | | 80156.200 | ..........................................................................................********.********.***************.*******X | 82142.700 | | 79728.200 | ..........................................................................................*******X | 79728.200 | | 78727.100 | .........................................................................................*******X | 78727.100 | | 78329.500 | ........................................................................................****************...*******.**************************************X | 81877.000 | | 80473.900 | ...........................................................................................********.*******X | 81692.600 | | 81559.100 | ............................................................................................********.***************...*******X | 85291.800 | | 83519.700 | ..............................................................................................*******X | 83519.700 | | 83222.000 | ..............................................................................................*******X | 83222.000 | | 82070.500 | .............................................................................................******X | 82070.500 | | 80720.200 | ...........................................................................................********...*******X | 82972.000 | | 80462.600 | ...........................................................................................********..*******X | 82071.300 | | 80951.500 | ...........................................................................................********.********..******X | 83680.300 | | 81476.100 | ............................................................................................*******************************X | 81971.000 | | 81545.400 | ............................................................................................***************X | 81672.800 | | 78095.400 | ........................................................................................********..***************.********...*******X | 82976.000 | | 78997.500 | .........................................................................................********..*******X | 80628.400 | | 79826.800 | ..........................................................................................********.**************X | 80943.800 | | 79323.800 | .........................................................................................********..*******X | 81128.000 | | 80725.400 | ...........................................................................................*******X | 80725.400 | | 77057.200 | .......................................................................................*******....*******X | 80829.600 | | 77502.200 | .......................................................................................*******X | 77502.200 | | 75224.900 | .....................................................................................********..*******X | 77230.700 | | 75516.600 | .....................................................................................********..******X | 77071.400 | | 76094.200 | ......................................................................................*************X | 76711.000 | +-----------+------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------+ 2022-03-06 10:13:52,214 | test | INFO | pool-6-thread-5 | [table_view:display:72] Ops trend for bucket 'default' +-----------+------------------------------------------------------------------------------------------------------+-----------+ | Min | Trend | Max | +-----------+------------------------------------------------------------------------------------------------------+-----------+ | 76711.000 | ...................................................................................................X | 76711.000 | +-----------+------------------------------------------------------------------------------------------------------+-----------+ 2022-03-06 10:13:52,321 | test | INFO | MainThread | [table_view:display:72] Cluster statistics +----------------+----------+-----------------+-----------+----------+----------------------+---------------------+-----------------------+ | Node | Services | CPU_utilization | Mem_total | Mem_free | Swap_mem_used | Active / Replica | Version | +----------------+----------+-----------------+-----------+----------+----------------------+---------------------+-----------------------+ | 172.23.105.206 | kv | 86.7527437056 | 11.45 GiB | 8.80 GiB | 23.59 MiB / 3.50 GiB | 61769333 / 61727218 | 7.1.0-2440-enterprise | | 172.23.106.177 | kv | 82.7778498249 | 11.45 GiB | 8.69 GiB | 23.15 MiB / 3.50 GiB | 62260762 / 62233701 | 7.1.0-2440-enterprise | | 172.23.100.35 | kv | 82.4902723735 | 11.45 GiB | 8.75 GiB | 0.0 Byte / 3.50 GiB | 62289017 / 62300359 | 7.1.0-2440-enterprise | | 172.23.105.164 | kv | 84.0859385051 | 11.45 GiB | 8.66 GiB | 0.0 Byte / 3.50 GiB | 61932847 / 61986971 | 7.1.0-2440-enterprise | +----------------+----------+-----------------+-----------+----------+----------------------+---------------------+-----------------------+ 2022-03-06 10:14:00,747 | test | INFO | MainThread | [bucket_ready_functions:validate_docs_per_collections_all_buckets:4742] Validating collection stats and item counts 2022-03-06 10:14:08,463 | test | ERROR | MainThread | [bucket_ready_functions:validate_manifest_uid:4605] Bucket UID mismatch. Expected: 0, Actual: 1 2022-03-06 10:14:08,463 | test | WARNING | MainThread | [bucket_ready_functions:validate_docs_per_collections_all_buckets:4759] Bucket manifest UID mismatch! 2022-03-06 10:14:12,421 | test | ERROR | MainThread | [bucket_ready_functions:validate_manifest_uid:4605] Bucket UID mismatch. Expected: 0, Actual: 1 2022-03-06 10:14:12,421 | test | WARNING | MainThread | [bucket_ready_functions:validate_docs_per_collections_all_buckets:4759] Bucket manifest UID mismatch! 2022-03-06 10:14:16,322 | test | ERROR | MainThread | [bucket_ready_functions:validate_manifest_uid:4605] Bucket UID mismatch. Expected: 0, Actual: 1 2022-03-06 10:14:16,322 | test | WARNING | MainThread | [bucket_ready_functions:validate_docs_per_collections_all_buckets:4759] Bucket manifest UID mismatch! 2022-03-06 10:14:26,315 | test | INFO | MainThread | [table_view:display:72] Bucket statistics +---------+-----------+-----------------+----------+------------+-----+-----------+-----------+------------+------------+-----------+ | Bucket | Type | Storage Backend | Replicas | Durability | TTL | Items | RAM Quota | RAM Used | Disk Used | ARR | +---------+-----------+-----------------+----------+------------+-----+-----------+-----------+------------+------------+-----------+ | bucket1 | couchbase | couchstore | 1 | none | 0 | 100000 | 7.81 GiB | 111.53 MiB | 199.71 MiB | 100 | | bucket2 | couchbase | magma | 1 | none | 0 | 100000 | 4.00 GiB | 299.52 MiB | 400.35 MiB | 100 | | default | couchbase | magma | 1 | none | 0 | 250000000 | 4.00 GiB | 2.33 GiB | 42.09 GiB | 3.9235464 | +---------+-----------+-----------------+----------+------------+-----+-----------+-----------+------------+------------+-----------+ 2022-03-06 10:14:26,315 | test | INFO | MainThread | [basetestcase:log_setup_status:647] ========= CollectionBase setup complete for test #3 test_data_load_collections_with_hard_failover_recovery ========= 2022-03-06 10:14:29,345 | test | INFO | MainThread | [collections_rebalance:load_collections_with_rebalance:1003] Doing collection data load during hard_failover_recovery 2022-03-06 10:14:29,345 | test | INFO | MainThread | [collections_rebalance:rebalance_operation:361] Starting rebalance operation of type : hard_failover_recovery 2022-03-06 10:14:29,345 | test | INFO | MainThread | [collections_rebalance:rebalance_operation:725] failing over nodes [ip:172.23.100.35 port:8091 ssh_username:root] 2022-03-06 10:14:38,262 | test | INFO | pool-6-thread-2 | [rest_client:monitorRebalance:1610] Rebalance done. Taken 8.03400015831 seconds to complete 2022-03-06 10:14:38,266 | test | INFO | pool-6-thread-2 | [common_lib:sleep:23] Sleep 8.03400015831 seconds. Reason: Wait after rebalance complete 2022-03-06 10:19:46,302 | test | INFO | MainThread | [collections_rebalance:data_load_after_failover:330] Starting a sync data load after failover 2022-03-06 10:19:46,302 | test | INFO | MainThread | [bucket_ready_functions:perform_tasks_from_spec:4993] Performing scope/collection specific operations 2022-03-06 10:19:50,259 | test | INFO | MainThread | [bucket_ready_functions:perform_tasks_from_spec:5083] Done Performing scope/collection specific operations 2022-03-06 10:59:36,128 | test | INFO | pool-6-thread-26 | [table_view:display:72] Ops trend for bucket 'bucket1' +--------+--------------------------------------------------------------------------------------------------------------------+---------+ | Min | Trend | Max | +--------+--------------------------------------------------------------------------------------------------------------------+---------+ | 0.000 | *******....................................................................................................******X | 940.000 | | 0.000 | *******........********..******X | 96.000 | | 0.000 | ***************.***************.........******X | 100.000 | | 0.000 | ***************.......******X | 68.000 | | 0.000 | *******************************...*******.................*******X | 192.000 | | 0.000 | **************.................*....................................******X | 500.000 | | 0.000 | ********.**************X | 16.000 | | 0.000 | *******...........*******X | 104.000 | | 48.000 | ....******X | 48.000 | | 16.000 | ********.******X | 20.000 | | 0.000 | *******.*******X | 16.000 | | 0.000 | *******.********.********....******X | 64.000 | | 0.000 | *******......*******X | 64.000 | | 32.000 | ..******X | 32.000 | | 0.000 | ***************.....********.........******X | 136.000 | | 52.000 | ....*******X | 52.000 | | 32.000 | ..**************X | 32.000 | | 16.000 | **************X | 16.000 | | 0.000 | *******...**************X | 32.000 | | 0.000 | *****************************.*******X | 16.000 | | 0.000 | *******...................................********......................................................******X | 844.600 | | 16.000 | ********.........................................................******X | 548.000 | | 0.000 | ********..******X | 20.000 | | 0.000 | ****************.....******X | 48.000 | | 0.000 | ***************...*******X | 36.000 | | 0.000 | **********************.*******X | 16.000 | | 0.000 | *******...********.....**************X | 80.000 | | 32.000 | ..******X | 32.000 | | 0.000 | ***************.***************.******X | 20.000 | | 16.000 | *******X | 16.000 | | 0.000 | ****************.....********.................................................*******X | 516.000 | | 48.000 | ....*******X | 48.000 | | 0.000 | *******.......*******X | 72.000 | | 16.000 | ********..*******X | 36.000 | | 16.000 | *******X | 16.000 | | 0.000 | ********............*******X | 120.000 | | 0.000 | ********...*******X | 32.000 | | 0.000 | ********..*******X | 20.000 | | 18.300 | X | 18.300 | | 16.000 | ********.********...******X | 52.000 | | 32.000 | ..*******X | 32.000 | | 16.000 | ********......*******X | 68.000 | | 22.500 | .X | 22.500 | | 0.000 | *******...*******X | 36.000 | | 0.000 | ****************.....*******X | 52.000 | | 0.000 | ********.********..*******X | 36.000 | | 0.000 | ********.......*******X | 68.000 | | 0.000 | ****************.....*******X | 52.000 | | 48.000 | ....*******X | 48.000 | | 32.000 | ..*******X | 32.000 | | 0.000 | ********...*******X | 36.000 | | 0.000 | ********.....******X | 48.000 | | 0.000 | ****************.....***************X | 56.000 | | 28.500 | ..X | 28.500 | | 16.000 | ******X | 16.000 | | 0.000 | ************************...********..*******X | 56.000 | | 0.000 | ****************...*******X | 36.000 | | 0.000 | ****************........******X | 76.000 | | 0.000 | ********...********..*******X | 48.000 | | 36.000 | ..*******X | 36.000 | | 0.000 | ********.....***************X | 52.100 | | 0.000 | ****************.....*******X | 52.000 | | 20.000 | .*******.***************X | 36.000 | | 16.000 | *******X | 16.000 | | 0.000 | ********.....*******X | 52.000 | | 32.000 | ..*******X | 32.000 | | 0.000 | ********.*******X | 16.000 | | 0.000 | ********.....*******X | 52.000 | | 40.000 | ...*******X | 40.000 | | 16.000 | ********...*******X | 40.000 | | 20.000 | .********...*******X | 52.000 | | 0.000 | ****************.....*******X | 48.000 | | 40.000 | ...*******X | 40.000 | | 16.000 | ***************X | 16.000 | | 0.000 | ********..***************...*******X | 56.000 | | 0.000 | ********..******X | 20.000 | | 0.000 | ********.*******X | 16.000 | | 0.000 | ********..********.********..*******X | 56.000 | | 20.000 | .************************.*******X | 36.000 | | 0.000 | ****************.********..*******X | 36.000 | | 16.000 | **************X | 16.000 | | 0.000 | ********.....*******X | 52.000 | | 0.000 | ********.********....*******X | 52.000 | | 0.000 | ****************....*******X | 40.000 | | 20.000 | .********.*******X | 36.000 | | 0.000 | ********.......*******X | 72.000 | | 32.000 | ..**************X | 36.000 | | 0.000 | ************************...*******X | 32.000 | | 31.400 | ..*******X | 31.400 | | 20.600 | .*******X | 20.600 | | 16.000 | ********..*******X | 32.000 | | 16.000 | ******X | 16.000 | | 0.000 | ********...********.....*******X | 84.000 | | 64.000 | .....*******X | 64.000 | | 0.000 | ************************...*******X | 36.000 | | 32.000 | ..**************X | 32.000 | | 16.000 | ***************X | 16.000 | | 0.000 | ********........******X | 80.000 | | 16.000 | ***********************X | 16.000 | | 0.000 | *************************X | 0.000 | +--------+--------------------------------------------------------------------------------------------------------------------+---------+ 2022-03-06 10:59:36,943 | test | INFO | pool-6-thread-21 | [table_view:display:72] Ops trend for bucket 'bucket2' +---------+-------------------------------------------------------------------------------------------------------------------------------------+---------+ | Min | Trend | Max | +---------+-------------------------------------------------------------------------------------------------------------------------------------+---------+ | 0.000 | *******.........................................................................******X | 392.000 | | 0.000 | *******........********...******X | 64.000 | | 0.000 | **********************...................................................................................................*******X | 532.000 | | 112.000 | ...................******X | 112.000 | | 0.000 | ***************...........******X | 64.000 | | 0.000 | **************************************....................*******X | 112.000 | | 16.000 | .******X | 16.000 | | 0.000 | ********......******X | 36.000 | | 0.000 | ********.............................................................................................******X | 500.000 | | 334.100 | .............................................................X | 334.100 | | 0.000 | *******..*******..................................................................................................*******X | 536.000 | | 64.000 | ..........******X | 64.000 | | 0.000 | ********..******X | 16.000 | | 0.000 | ********..******X | 16.000 | | 0.000 | ********..*******........********...............................................................................******X | 478.400 | | 0.000 | ********..*******......*******X | 48.000 | | 0.000 | **********************.................******X | 96.000 | | 0.000 | ***********************.....*******........................................................................................*******X | 500.000 | | 0.000 | *****************************......*******X | 36.000 | | 16.000 | .******X | 16.000 | | 0.000 | **********************........*******X | 48.000 | | 0.000 | *******.....********...........................................................................................******X | 516.000 | | 0.000 | ********..*******......*******X | 48.000 | | 0.000 | *******...................................................................................................******X | 532.000 | | 353.700 | ................................................................X | 353.700 | | 0.000 | **********************..******X | 16.000 | | 5.400 | X | 5.400 | | 0.000 | *******.....******X | 32.000 | | 0.000 | ********..*******......********....******X | 68.000 | | 64.000 | ..........*******X | 64.000 | | 0.000 | **********************........******X | 48.000 | | 32.000 | ....*******X | 32.000 | | 0.000 | ***********************...********.......*******X | 56.000 | | 40.000 | ......*******X | 40.000 | | 0.000 | ****************....................*******X | 108.000 | | 36.000 | .....*******X | 36.000 | | 20.000 | ..*******X | 20.000 | | 0.000 | ********......********............*******X | 100.000 | | 16.000 | .*******X | 16.000 | | 0.000 | ************************..********.*******X | 20.000 | | 16.000 | .****************....*******.......*******X | 72.000 | | 16.000 | .*******X | 16.000 | | 0.000 | ****************....................................................................................................*******X | 536.000 | | 20.000 | ..********X | 20.000 | | 16.000 | .*******X | 16.000 | | 0.000 | *******......*******X | 36.000 | | 0.000 | ********************************......*******X | 36.000 | | 0.000 | ************************..........*******X | 56.000 | | 32.000 | ....*******...*.******X | 52.000 | | 32.000 | ....*******X | 32.000 | | 20.000 | ..********.............*******X | 88.000 | | 0.000 | ****************************************...********..........*******X | 72.000 | | 16.000 | .*******X | 16.000 | | 0.000 | ****************.........*******X | 52.000 | | 16.000 | .********.......******X | 52.000 | | 0.000 | ********......*******X | 36.000 | | 0.000 | ********...*******X | 20.000 | | 16.000 | .***********************X | 16.000 | | 0.000 | ****************..*******X | 16.000 | | 0.000 | *******..........*******X | 56.000 | | 40.000 | ......*******X | 40.000 | | 20.000 | ..*******X | 20.000 | | 0.000 | ********.......*******X | 40.000 | | 16.000 | .********....*******..*******X | 48.000 | | 0.000 | ********...********..............*******X | 92.000 | | 20.000 | ..********..*******X | 32.000 | | 0.000 | ********..............*******X | 76.000 | | 0.000 | ************************.......*******X | 40.000 | | 20.000 | ..***************X | 20.000 | | 0.000 | ********..********.**************X | 20.000 | | 0.000 | ********..***************....*******X | 36.000 | | 0.000 | ********...********.....*******X | 48.000 | | 20.000 | ..*******X | 20.000 | | 16.000 | .*******X | 16.000 | | 0.000 | ********..****************.*******X | 20.000 | | 16.000 | .*******X | 16.000 | | 0.000 | ********........********.*******X | 52.000 | | 0.000 | ****************************************..*******X | 16.000 | | 0.000 | ********************************.....*******X | 32.000 | | 0.000 | ********..********.********..******X | 32.000 | | 0.000 | ********.....*******X | 32.000 | | 0.000 | ********..***************X | 16.000 | | 0.000 | ********.........*******X | 52.000 | | 0.000 | ***********************..********............*******X | 80.000 | | 20.000 | ..*******X | 20.000 | | 0.000 | ************************..********.......*******X | 52.000 | | 0.000 | *******........*******X | 48.000 | | 16.000 | .*******X | 16.000 | | 0.000 | ****************..************************...*******X | 32.000 | | 0.000 | *****************X | 0.000 | +---------+-------------------------------------------------------------------------------------------------------------------------------------+---------+ 2022-03-06 10:59:38,118 | test | INFO | pool-6-thread-17 | [table_view:display:72] Ops trend for bucket 'default' +------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------+------------+ | Min | Trend | Max | +------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------+------------+ | 0.000 | *******.................................................................................*******...................******X | 101047.200 | | 95640.600 | .............................................................................................*******X | 95640.600 | | 64554.000 | ..............................................................******X | 64554.000 | | 64189.500 | ..............................................................******X | 64189.500 | | 58314.700 | ........................................................******X | 58314.700 | | 47687.900 | ..............................................*X | 47687.900 | | 47687.900 | ..............................................*****X | 47687.900 | | 44489.900 | ...........................................******X | 44489.900 | | 42305.600 | ........................................********.......******X | 48924.000 | | 48737.500 | ...............................................********....*******......*******X | 58682.100 | | 52679.800 | ...................................................*******.*******..********......******X | 62646.300 | | 56079.900 | ......................................................*******X | 56079.900 | | 50348.700 | ................................................******X | 50348.700 | | 47856.800 | ..............................................****************....*******.*******X | 52820.200 | | 49910.200 | ................................................******X | 49910.200 | | 47659.400 | ..............................................******X | 47659.400 | | 43407.400 | .........................................********..*X | 45170.600 | | 45170.600 | ...........................................*****.*..*******.********..X | 50613.600 | | 50613.600 | .................................................*X | 50613.600 | | 50613.600 | .................................................*X | 50613.600 | | 50613.600 | .................................................X | 50613.600 | | 50613.600 | .................................................*..********..*******..********.*******..********....*******...*******X | 67667.500 | | 63066.600 | .............................................................*****X | 63066.600 | | 63066.600 | .............................................................X | 63066.600 | | 58739.000 | .........................................................**************X | 59377.100 | | 56160.000 | ......................................................******X | 56160.000 | | 54316.500 | ....................................................********.....X | 59286.800 | | 59286.800 | .........................................................****X | 59286.800 | | 59286.800 | .........................................................X | 59286.800 | | 55970.600 | ......................................................X | 55970.600 | | 55970.600 | ......................................................******X | 55970.600 | | 52686.100 | ...................................................******X | 52686.100 | | 49515.800 | ................................................*******.*..*X | 53070.900 | | 53070.900 | ...................................................**X | 53070.900 | | 53070.900 | ...................................................*X | 53070.900 | | 50876.700 | .................................................******X | 50876.700 | | 47116.700 | .............................................**************X | 47277.900 | | 46119.500 | ............................................X | 46119.500 | | 45689.900 | ............................................*******....*******..********.......***X | 58731.400 | | 58731.400 | .........................................................**X | 58731.400 | | 57411.800 | .......................................................*******.********..******X | 59750.700 | | 57838.900 | ........................................................*******X | 57838.900 | | 57222.100 | .......................................................*X | 57222.100 | | 57222.100 | .......................................................****X | 57222.100 | | 52956.700 | ...................................................*******X | 52956.700 | | 52755.900 | ...................................................****X | 52755.900 | | 52755.900 | ...................................................*X | 52755.900 | | 52159.400 | ..................................................*X | 52159.400 | | 52159.400 | ..................................................**X | 52159.400 | | 52159.400 | ..................................................**.*******X | 52606.400 | | 50277.600 | ................................................******X | 50277.600 | | 48690.800 | ...............................................******X | 48690.800 | | 45804.600 | ............................................***X | 45804.600 | | 45804.600 | ............................................***X | 45804.600 | | 45563.300 | ............................................******X | 45563.300 | | 45412.900 | ...........................................********.....******X | 50087.700 | | 48373.000 | ..............................................*******X | 48373.000 | | 47147.400 | .............................................****X | 47147.400 | | 47147.400 | .............................................**......*******.********..*X | 56032.900 | | 56032.900 | ......................................................*****.X | 57448.100 | | 57448.100 | .......................................................***X | 57448.100 | | 57448.100 | .......................................................*X | 57448.100 | | 57448.100 | .......................................................*..*******.....******X | 63768.700 | | 60279.700 | ..........................................................********.*******.*******....********...***************....................*******...........*******X | 100432.700 | | 100080.800 | ..................................................................................................*******X | 100080.800 | | 98033.100 | ................................................................................................*******X | 98033.100 | | 96478.300 | ..............................................................................................*****X | 96478.300 | | 96478.300 | ..............................................................................................*********X | 96992.000 | | 95917.500 | .............................................................................................*******X | 95917.500 | | 89770.700 | .......................................................................................****X | 89770.700 | | 89770.700 | .......................................................................................*X | 89770.700 | | 81976.800 | ................................................................................*******X | 81976.800 | | 79028.900 | .............................................................................********..*****X | 81010.100 | | 81010.100 | ...............................................................................**.*******X | 82681.500 | | 78531.900 | ............................................................................***X | 78531.900 | | 78531.900 | ............................................................................**X | 78531.900 | | 78531.900 | ............................................................................*.X | 79754.300 | | 79754.300 | .............................................................................*******.....*******X | 84227.600 | | 81123.600 | ...............................................................................*X | 81123.600 | | 81123.600 | ...............................................................................*****X | 81123.600 | | 81016.600 | ...............................................................................*******X | 81016.600 | | 80887.500 | ...............................................................................********......******X | 87446.600 | | 78491.400 | ............................................................................*X | 78491.400 | | 78491.400 | ............................................................................*X | 78491.400 | | 78491.400 | ............................................................................****..********..*******X | 82234.700 | | 79349.100 | .............................................................................*******X | 79349.100 | | 75135.000 | .........................................................................********.*******X | 76784.900 | | 73885.200 | ........................................................................*******X | 73885.200 | | 73156.900 | .......................................................................********.***X | 74194.800 | | 74194.800 | ........................................................................***X | 74194.800 | | 71757.100 | ......................................................................********..*******X | 74380.300 | | 68071.300 | ..................................................................********..********.********..*******X | 72816.500 | | 71932.800 | ......................................................................******X | 71932.800 | | 71477.700 | .....................................................................*.******X | 72591.600 | | 68970.700 | ...................................................................*******X | 68970.700 | | 68365.000 | ..................................................................********.....********X | 73275.200 | | 73275.200 | .......................................................................**X | 73275.200 | | 73275.200 | .......................................................................***X | 73275.200 | | 72892.600 | .......................................................................*******X | 72892.600 | | 72678.000 | ......................................................................*******X | 72678.000 | | 72575.900 | ......................................................................********....**X | 75912.600 | | 75912.600 | ..........................................................................**X | 75912.600 | | 75912.600 | ..........................................................................*X | 75912.600 | | 71333.200 | .....................................................................********.*******X | 72256.300 | | 69888.900 | ....................................................................********......*******X | 75878.700 | | 72013.600 | ......................................................................*******X | 72013.600 | | 71385.400 | .....................................................................********..********.*******X | 74234.500 | | 74100.900 | ........................................................................*******X | 74100.900 | | 72721.500 | ......................................................................*******X | 72721.500 | | 72037.100 | ......................................................................*******X | 72037.100 | | 67885.300 | ..................................................................********....******X | 71819.500 | | 68721.300 | ...................................................................***************X | 69075.000 | | 68274.500 | ..................................................................*******X | 68274.500 | | 67644.100 | .................................................................*******X | 67644.100 | | 65659.200 | ...............................................................********...*******.*******X | 68884.300 | | 64256.800 | ..............................................................********...*******X | 67497.800 | | 67269.800 | .................................................................********.********...*******X | 71544.100 | | 62740.900 | .............................................................********..********.*******X | 66557.400 | | 64106.500 | ..............................................................**************X | 64615.300 | | 64043.100 | ..............................................................*******X | 64043.100 | | 63388.300 | .............................................................********....*******X | 67055.700 | | 62551.300 | ............................................................********..*******X | 64053.400 | | 61350.100 | ...........................................................********.********..*******X | 64569.100 | | 62904.000 | .............................................................*******X | 62904.000 | | 62494.900 | ............................................................******X | 62494.900 | | 62070.400 | ............................................................X | 62070.400 | | 61793.600 | ............................................................*******.********X | 63363.700 | | 61535.400 | ...........................................................******X | 61535.400 | | 59854.900 | ..........................................................*******X | 59854.900 | | 58504.900 | ........................................................********....*******X | 61809.200 | | 57748.400 | ........................................................*******X | 57748.400 | | 50986.900 | .................................................********.....********....********.......******X | 67428.100 | | 67301.400 | .................................................................*******X | 67301.400 | | 66956.000 | .................................................................*******X | 66956.000 | | 65983.700 | ................................................................********.***************X | 66922.300 | | 66859.400 | .................................................................********.*******X | 67919.900 | | 67758.600 | ..................................................................******X | 67758.600 | | 64388.400 | ..............................................................********...*******X | 67031.500 | | 64454.800 | ..............................................................********..*******X | 66469.400 | | 63780.700 | ..............................................................********......*******X | 69921.600 | | 67465.500 | .................................................................*******..*******X | 68906.600 | | 67582.500 | .................................................................********.*******X | 68253.200 | | 66042.200 | ................................................................********.*******X | 67359.500 | | 66452.200 | ................................................................********.***************X | 67607.200 | | 65898.900 | ................................................................*******.*******X | 67315.000 | | 65520.600 | ...............................................................********..*******X | 67124.500 | | 64290.600 | ..............................................................*******X | 64290.600 | | 62597.400 | ............................................................********...*******X | 64908.800 | | 58352.800 | ........................................................********.....************************......*******X | 69301.200 | | 64675.500 | ...............................................................********.*******X | 65967.500 | | 64416.600 | ..............................................................******X | 64416.600 | | 61684.700 | ............................................................********....*******X | 66206.800 | | 65692.800 | ................................................................*******X | 65692.800 | | 62480.600 | ............................................................********......*******X | 68312.100 | | 65468.700 | ...............................................................*******..*******X | 66742.500 | | 65759.300 | ................................................................********..*******X | 68661.900 | | 62964.000 | .............................................................********....***************.**********************X | 68481.600 | | 65348.700 | ...............................................................********..*******X | 67663.900 | | 65431.200 | ...............................................................********.********..*******X | 68092.900 | | 66635.500 | ................................................................******X | 66635.500 | | 64233.200 | ..............................................................***X | 64233.200 | +------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------+------------+ 2022-03-06 10:59:39,414 | test | INFO | pool-6-thread-9 | [table_view:display:72] Ops trend for bucket 'default' +-----------+------------------------------------------------------------------------------------------------------+-----------+ | Min | Trend | Max | +-----------+------------------------------------------------------------------------------------------------------+-----------+ | 64233.200 | ...................................................................................................X | 64233.200 | +-----------+------------------------------------------------------------------------------------------------------+-----------+ 2022-03-06 10:59:39,502 | test | WARNING | MainThread | [rest_client:get_nodes:1896] 172.23.100.35 - Node not part of cluster inactiveFailed 2022-03-06 10:59:53,641 | test | INFO | MainThread | [bucket_ready_functions:perform_tasks_from_spec:4993] Performing scope/collection specific operations 2022-03-06 10:59:53,740 | test | INFO | pool-6-thread-8 | [table_view:display:72] Rebalance Overview +----------------+----------+-----------------------+----------------+--------------+-----------------------+ | Nodes | Services | Version | CPU | Status | Membership / Recovery | +----------------+----------+-----------------------+----------------+--------------+-----------------------+ | 172.23.105.206 | kv | 7.1.0-2440-enterprise | 72.1512583787 | Cluster node | active / none | | 172.23.106.177 | kv | 7.1.0-2440-enterprise | 54.5074172689 | Cluster node | active / none | | 172.23.100.35 | kv | 7.1.0-2440-enterprise | 0.362772079059 | Cluster node | inactiveAdded / full | | 172.23.105.164 | kv | 7.1.0-2440-enterprise | 71.0474177784 | Cluster node | active / none | +----------------+----------+-----------------------+----------------+--------------+-----------------------+ 2022-03-06 10:59:53,757 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 10:59:57,578 | test | INFO | MainThread | [bucket_ready_functions:perform_tasks_from_spec:5083] Done Performing scope/collection specific operations 2022-03-06 11:00:03,796 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 11:00:13,851 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 11:00:23,885 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 11:00:33,951 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 11:00:44,016 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 11:00:54,045 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 11:01:04,082 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 11:01:14,122 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 11:01:24,150 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 11:01:34,177 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 11:01:44,210 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 11:01:54,250 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 11:02:04,286 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 11:02:14,325 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 11:02:24,369 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 11:02:34,407 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 11:02:44,437 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 11:02:54,467 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 11:03:04,516 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 11:03:14,568 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 11:03:24,618 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 11:03:34,673 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 11:03:44,706 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 11:03:54,740 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 11:04:04,792 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 11:04:14,838 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 11:04:24,880 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 11:04:34,911 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 11:04:44,940 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 11:04:55,002 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 11:05:05,042 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 11:05:15,091 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 11:05:25,122 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 11:05:35,161 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 11:05:45,197 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 11:05:55,243 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 11:06:05,286 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 11:06:15,335 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 0.07 2022-03-06 11:06:25,369 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 0.07 2022-03-06 11:06:35,407 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 0.07 2022-03-06 11:06:45,440 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 0.07 2022-03-06 11:06:55,493 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 0.07 2022-03-06 11:07:05,526 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 0.07 2022-03-06 11:07:15,572 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 0.07 2022-03-06 11:07:25,611 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 0.07 2022-03-06 11:07:35,641 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 0.13 2022-03-06 11:07:45,667 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 0.13 2022-03-06 11:07:55,700 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 0.13 2022-03-06 11:08:05,740 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 0.13 2022-03-06 11:08:15,798 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 0.13 2022-03-06 11:08:25,839 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 0.13 2022-03-06 11:08:35,882 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 0.13 2022-03-06 11:08:45,937 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 0.13 2022-03-06 11:08:55,976 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 0.13 2022-03-06 11:09:06,019 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 0.13 2022-03-06 11:09:16,049 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 0.13 2022-03-06 11:09:26,076 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 0.13 2022-03-06 11:09:36,108 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 0.13 2022-03-06 11:09:46,138 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 0.13 2022-03-06 11:09:56,200 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 0.13 2022-03-06 11:10:06,226 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 0.13 2022-03-06 11:10:16,252 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 0.13 2022-03-06 11:10:26,292 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 0.13 2022-03-06 11:10:36,339 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 0.13 2022-03-06 11:10:46,371 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 0.13 2022-03-06 11:10:56,411 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 0.13 2022-03-06 11:11:06,453 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 0.13 2022-03-06 11:11:16,499 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 0.13 2022-03-06 11:11:26,529 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 0.2 2022-03-06 11:11:36,578 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 0.2 2022-03-06 11:11:46,611 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 0.2 2022-03-06 11:11:56,648 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 0.2 2022-03-06 11:12:06,690 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 0.2 2022-03-06 11:12:16,737 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 0.2 2022-03-06 11:12:26,786 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 0.2 2022-03-06 11:12:36,835 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 0.52 2022-03-06 11:12:46,868 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 9.47 2022-03-06 11:12:56,894 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 9.47 2022-03-06 11:13:06,944 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 9.47 2022-03-06 11:13:16,977 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 15.09 2022-03-06 11:13:27,020 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 15.09 2022-03-06 11:13:37,052 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 31.11 2022-03-06 11:13:47,081 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 48.37 2022-03-06 11:13:57,118 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 66.67 2022-03-06 11:14:07,155 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 66.67 2022-03-06 11:14:17,200 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 66.93 2022-03-06 11:14:27,247 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 67.06 2022-03-06 11:14:37,282 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 67.32 2022-03-06 11:14:47,312 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 67.45 2022-03-06 11:14:57,351 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 67.71 2022-03-06 11:15:07,388 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 67.9 2022-03-06 11:15:17,427 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 68.16 2022-03-06 11:15:27,461 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 68.29 2022-03-06 11:15:37,530 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 68.55 2022-03-06 11:15:47,573 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 68.69 2022-03-06 11:15:57,625 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 68.88 2022-03-06 11:16:07,680 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 69.08 2022-03-06 11:16:17,729 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 69.27 2022-03-06 11:16:27,767 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 69.47 2022-03-06 11:16:37,815 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 69.66 2022-03-06 11:16:47,852 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 69.86 2022-03-06 11:16:57,894 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 70.05 2022-03-06 11:17:07,927 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 70.25 2022-03-06 11:17:17,973 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 70.51 2022-03-06 11:17:28,036 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 70.7 2022-03-06 11:17:38,068 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 70.83 2022-03-06 11:17:48,112 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 71.09 2022-03-06 11:17:58,171 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 71.22 2022-03-06 11:18:08,207 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 71.42 2022-03-06 11:18:18,256 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 71.62 2022-03-06 11:18:28,298 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 71.81 2022-03-06 11:18:38,332 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 71.94 2022-03-06 11:18:48,368 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 72.14 2022-03-06 11:18:58,414 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 72.33 2022-03-06 11:19:08,467 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 72.53 2022-03-06 11:19:18,517 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 72.79 2022-03-06 11:19:28,565 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 72.85 2022-03-06 11:19:38,612 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 73.11 2022-03-06 11:19:48,654 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 73.18 2022-03-06 11:19:58,688 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 73.44 2022-03-06 11:20:08,717 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 73.57 2022-03-06 11:20:18,763 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 73.83 2022-03-06 11:20:28,805 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 73.89 2022-03-06 11:20:38,851 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 74.16 2022-03-06 11:20:48,885 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 74.29 2022-03-06 11:20:58,914 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 74.48 2022-03-06 11:21:08,947 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 74.61 2022-03-06 11:21:18,977 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 74.81 2022-03-06 11:21:29,019 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 74.94 2022-03-06 11:21:39,048 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 75.13 2022-03-06 11:21:49,084 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 75.33 2022-03-06 11:21:59,131 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 75.46 2022-03-06 11:22:09,177 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 75.72 2022-03-06 11:22:19,221 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 75.85 2022-03-06 11:22:29,267 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 76.04 2022-03-06 11:22:39,298 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 76.17 2022-03-06 11:22:49,329 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 76.37 2022-03-06 11:22:59,382 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 76.63 2022-03-06 11:23:09,417 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 76.76 2022-03-06 11:23:19,450 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 76.96 2022-03-06 11:23:29,499 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 77.02 2022-03-06 11:23:39,536 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 77.28 2022-03-06 11:23:49,568 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 77.35 2022-03-06 11:23:59,621 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 77.61 2022-03-06 11:24:09,654 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 77.67 2022-03-06 11:24:19,697 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 77.93 2022-03-06 11:24:29,734 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 78.0 2022-03-06 11:24:39,770 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 78.19 2022-03-06 11:24:49,815 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 78.26 2022-03-06 11:24:59,855 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 78.45 2022-03-06 11:25:09,894 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 78.58 2022-03-06 11:25:19,937 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 78.84 2022-03-06 11:25:29,977 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 78.97 2022-03-06 11:25:40,026 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 79.1 2022-03-06 11:25:50,063 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 79.23 2022-03-06 11:26:00,109 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 79.49 2022-03-06 11:26:10,160 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 79.62 2022-03-06 11:26:20,196 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 79.76 2022-03-06 11:26:30,234 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 79.95 2022-03-06 11:26:40,265 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 80.08 2022-03-06 11:26:50,295 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 80.28 2022-03-06 11:27:00,328 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 80.41 2022-03-06 11:27:10,368 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 80.6 2022-03-06 11:27:20,407 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 80.73 2022-03-06 11:27:30,438 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 80.99 2022-03-06 11:27:40,476 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 80.99 2022-03-06 11:27:50,516 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 81.25 2022-03-06 11:28:00,559 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 81.45 2022-03-06 11:28:10,596 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 81.51 2022-03-06 11:28:20,631 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 81.71 2022-03-06 11:28:30,674 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 81.9 2022-03-06 11:28:40,709 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 82.03 2022-03-06 11:28:50,740 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 82.03 2022-03-06 11:29:00,776 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 82.29 2022-03-06 11:29:10,809 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 82.42 2022-03-06 11:29:20,848 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 82.55 2022-03-06 11:29:30,890 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 82.82 2022-03-06 11:29:40,930 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 82.88 2022-03-06 11:29:50,986 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 83.08 2022-03-06 11:30:01,030 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 83.21 2022-03-06 11:30:11,072 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 83.34 2022-03-06 11:30:21,108 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 83.47 2022-03-06 11:30:31,147 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 83.6 2022-03-06 11:30:41,180 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 83.73 2022-03-06 11:30:51,217 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 83.92 2022-03-06 11:31:01,249 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 84.05 2022-03-06 11:31:11,286 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 84.12 2022-03-06 11:31:21,331 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 84.31 2022-03-06 11:31:31,369 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 84.38 2022-03-06 11:31:41,401 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 84.64 2022-03-06 11:31:51,447 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 84.7 2022-03-06 11:32:01,483 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 84.9 2022-03-06 11:32:11,528 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 85.03 2022-03-06 11:32:21,566 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 85.1 2022-03-06 11:32:31,599 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 85.29 2022-03-06 11:32:41,644 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 85.36 2022-03-06 11:32:51,678 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 85.55 2022-03-06 11:33:01,710 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 85.68 2022-03-06 11:33:11,747 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 85.75 2022-03-06 11:33:21,785 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 85.94 2022-03-06 11:33:31,825 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 86.01 2022-03-06 11:33:41,865 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 86.2 2022-03-06 11:33:51,903 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 86.33 2022-03-06 11:34:01,937 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 86.46 2022-03-06 11:34:11,973 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 86.59 2022-03-06 11:34:22,009 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 86.72 2022-03-06 11:34:32,040 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 86.86 2022-03-06 11:34:42,078 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 87.05 2022-03-06 11:34:52,111 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 87.18 2022-03-06 11:35:02,141 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 87.31 2022-03-06 11:35:12,187 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 87.44 2022-03-06 11:35:22,226 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 87.44 2022-03-06 11:35:27,674 | test | ERROR | pool-6-thread-3 | [task:execute_tasks:3946] Failed to load 53 docs from 200000 to 392000 of thread_name LoadDocs_MutateDocsFromSpecTask_1646593200.75_default__default_collection-1176_ttl=0_delete__200000_392000_1646593201.45 2022-03-06 11:35:32,267 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 87.7 2022-03-06 11:35:41,569 | test | ERROR | pool-6-thread-3 | [task:execute_tasks:3946] Failed to load 50 docs from 200000 to 392000 of thread_name LoadDocs_MutateDocsFromSpecTask_1646593200.75_default_scope-113_collection-1334_ttl=0_delete__200000_392000_1646593201.45 2022-03-06 11:35:42,309 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 87.83 2022-03-06 11:35:52,364 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 87.96 2022-03-06 11:36:02,407 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 88.09 2022-03-06 11:36:03,124 | test | ERROR | pool-6-thread-3 | [task:execute_tasks:3946] Failed to load 50 docs from 200000 to 392000 of thread_name LoadDocs_MutateDocsFromSpecTask_1646593200.75_default_scope-108_collection-1210_ttl=0_delete__200000_392000_1646593201.45 2022-03-06 11:36:12,437 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 88.22 2022-03-06 11:36:19,444 | test | ERROR | pool-6-thread-3 | [task:execute_tasks:3946] Failed to load 100 docs from 200000 to 392000 of thread_name LoadDocs_MutateDocsFromSpecTask_1646593200.75_default_scope-112_collection-1307_ttl=0_delete__200000_392000_1646593201.46 2022-03-06 11:36:22,474 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 88.35 2022-03-06 11:36:26,566 | test | ERROR | pool-6-thread-3 | [task:execute_tasks:3946] Failed to load 50 docs from 200000 to 392000 of thread_name LoadDocs_MutateDocsFromSpecTask_1646593200.75_default_scope-116_collection-1409_ttl=0_delete__200000_392000_1646593201.46 2022-03-06 11:36:32,536 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 88.42 2022-03-06 11:36:42,223 | test | ERROR | pool-6-thread-3 | [task:execute_tasks:3946] Failed to load 73 docs from 200000 to 392000 of thread_name LoadDocs_MutateDocsFromSpecTask_1646593200.75_default_scope-111_collection-1276_ttl=0_delete__200000_392000_1646593201.46 2022-03-06 11:36:42,568 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 88.61 2022-03-06 11:36:52,608 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 88.74 2022-03-06 11:37:02,655 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 88.81 2022-03-06 11:37:12,684 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 89.01 2022-03-06 11:37:22,720 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 89.07 2022-03-06 11:37:32,759 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 89.2 2022-03-06 11:37:42,806 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 89.33 2022-03-06 11:37:52,848 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 89.46 2022-03-06 11:38:02,881 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 89.66 2022-03-06 11:38:12,913 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 89.72 2022-03-06 11:38:22,973 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 89.92 2022-03-06 11:38:27,378 | test | INFO | pool-6-thread-27 | [table_view:display:72] Ops trend for bucket 'bucket1' +---------+---------------------------------------------------------------------------------------------------------------------------------------------+---------+ | Min | Trend | Max | +---------+---------------------------------------------------------------------------------------------------------------------------------------------+---------+ | 0.000 | **........................................................................******X | 428.400 | | 15.300 | .********................*******X | 111.000 | | 15.300 | .******X | 15.300 | | 0.000 | ********.....*******X | 30.600 | | 0.000 | ************************..................................................................................................*******X | 580.400 | | 0.000 | ******************************.....*******...........................................................*..............................******X | 561.200 | | 0.000 | **************.................................................................................................*******X | 576.500 | | 0.000 | ********..*******...........*******X | 80.000 | | 53.700 | ........*******............******X | 125.900 | | 0.000 | ********.....******X | 30.600 | | 19.200 | ..********..*******...............................................................................*******X | 500.000 | | 0.000 | *******..********...........******X | 80.000 | | 45.900 | ......*******X | 45.900 | | 0.000 | ***************.......******X | 45.900 | | 0.000 | ********..........******X | 61.200 | | 0.000 | *******.......******X | 45.900 | | 0.000 | ********..**********************....******X | 38.400 | | 0.000 | *******..........******X | 61.200 | | 30.600 | ....*******...............................................................................******X | 500.000 | | 332.500 | .......................................................X | 332.500 | | 5.500 | *******.******X | 9.800 | | 0.000 | **********************..*******..........*******X | 76.500 | | 0.000 | *******..******X | 15.300 | | 0.000 | *******...********........******X | 69.000 | | 0.000 | ********..****************************X | 15.300 | | 0.000 | **********************.........******X | 53.700 | | 30.600 | ....*******X | 30.600 | | 0.000 | *******...*******.*....******X | 49.800 | | 16.300 | .X | 16.300 | | 0.000 | **************.......******X | 46.000 | | 15.300 | .*******X | 15.300 | | 0.000 | *********************..*******X | 15.700 | | 15.600 | .******X | 15.600 | | 15.300 | .****************...........................*******X | 175.300 | | 19.200 | ..*******...***************X | 38.400 | | 34.500 | ....*******....*******X | 53.700 | | 49.800 | .......*******X | 49.800 | | 0.000 | *******..********..................................................................................................*******X | 591.800 | | 34.500 | ....******X | 34.500 | | 30.600 | ....*******X | 30.600 | | 15.300 | .*******X | 15.300 | | 0.000 | ********..................******X | 110.600 | | 76.500 | ...........********..*******X | 88.200 | | 15.300 | .*******X | 15.300 | | 0.000 | ************************..*******X | 15.300 | | 0.000 | *******.*....*******......*******X | 65.100 | | 49.800 | .......*******X | 49.800 | | 0.000 | ********..................******X | 110.600 | | 15.300 | .***************X | 15.300 | | 0.000 | ********......********......*******X | 72.900 | | 0.000 | *******************************...**************X | 19.200 | | 0.000 | ********************************........******X | 49.800 | | 15.300 | .*******X | 15.300 | | 0.000 | ****************........******X | 49.800 | | 0.000 | ****************.........*******...............*******X | 145.100 | | 19.200 | ..*******X | 19.200 | | 15.300 | .********...*******X | 30.600 | | 19.200 | ..*******X | 19.200 | | 0.000 | ***************************************.....******X | 34.500 | | 17.300 | .X | 17.300 | | 0.000 | *******...................*******X | 115.300 | | 0.000 | ********.........*******X | 53.700 | | 19.200 | ..*******X | 19.200 | | 0.000 | ************************...........******X | 69.000 | | 0.000 | ********...........*******X | 65.100 | | 0.000 | ********.....*******X | 34.500 | | 0.000 | *******..*******X | 15.300 | | 0.000 | ****************.........*******X | 53.700 | | 19.200 | ..******X | 19.200 | | 0.000 | ************************.........******X | 53.700 | | 0.000 | ********.....*******X | 34.500 | | 15.300 | .******X | 15.300 | | 0.000 | ********************************...........*******X | 65.100 | | 0.000 | ********.....*******X | 30.600 | | 0.000 | ********.....******X | 30.600 | | 0.000 | ********...............................*******X | 188.200 | | 65.100 | ..........*******X | 65.100 | | 0.000 | ********.....********..*******X | 45.900 | | 0.000 | ***********************.............*******X | 80.000 | | 0.000 | ************************.........................******X | 149.800 | | 130.600 | .....................*******X | 130.600 | | 0.000 | ********.....*******X | 30.600 | | 19.200 | ..*******X | 19.200 | | 0.000 | ***********************......*******X | 38.400 | | 15.300 | .******X | 15.300 | | 0.000 | ********.....********.............******X | 110.600 | | 0.000 | ****************................******X | 95.700 | | 0.000 | ********.....*******X | 30.600 | | 15.300 | .*******X | 15.300 | | 0.000 | ***********************************************.............******X | 80.000 | | 0.000 | *******************************................*******X | 95.300 | | 45.900 | ......*******X | 45.900 | | 0.000 | *************************X | 0.000 | +---------+---------------------------------------------------------------------------------------------------------------------------------------------+---------+ 2022-03-06 11:38:27,474 | test | INFO | pool-6-thread-5 | [table_view:display:72] Ops trend for bucket 'bucket2' +--------+------------------------------------------------------------------------------------------------------------------------------------------------------+---------+ | Min | Trend | Max | +--------+------------------------------------------------------------------------------------------------------------------------------------------------------+---------+ | 0.000 | **......................*******........................................................................*******X | 515.300 | | 45.900 | .......******X | 45.900 | | 45.900 | .......*******X | 45.900 | | 0.000 | ****************..*******X | 15.300 | | 0.000 | ***********************************************..............................................................................................******X | 515.300 | | 0.000 | ********..******X | 15.300 | | 0.000 | ***************......*******X | 34.500 | | 0.000 | *******.....******X | 30.600 | | 0.000 | ***************..*******X | 15.300 | | 0.000 | *******.....*******X | 30.600 | | 15.300 | .******X | 15.300 | | 0.000 | ***********************.......................******X | 125.900 | | 15.300 | .********...*******X | 30.600 | | 0.000 | *************************************....................******X | 110.600 | | 80.000 | .............*******X | 80.000 | | 0.000 | ***************..******X | 15.300 | | 0.000 | ********..............******X | 76.500 | | 0.000 | ********....................................................................................................******X | 545.900 | | 0.000 | ***************.......******X | 42.300 | | 7.500 | *******X | 7.500 | | 0.000 | *******..********...............................................................................................******X | 530.600 | | 30.600 | ....*******............X | 95.700 | | 95.700 | ................***X | 95.700 | | 95.700 | ................**X | 95.700 | | 15.300 | .*******...******X | 30.600 | | 0.000 | ***************.........******X | 49.800 | | 0.000 | ***********************.................*******..........................................................................******X | 500.000 | | 30.600 | ....*******X | 30.600 | | 0.000 | *******...********...******X | 34.500 | | 0.000 | ***************...........*******X | 65.100 | | 19.200 | ..******X | 19.200 | | 0.000 | ****************************************************...********.................*******X | 110.600 | | 15.400 | .*******X | 15.400 | | 15.300 | .********...********....*******................*******X | 141.200 | | 34.500 | .....*******X | 34.500 | | 4.900 | *******..********......*******X | 45.900 | | 19.200 | ..******X | 19.200 | | 0.000 | ********.....**************X | 30.600 | | 0.000 | ***********************.....*................******X | 118.400 | | 30.600 | ....********..................********...*******X | 145.100 | | 0.000 | ***********************...........................................................................................*******X | 500.000 | | 0.000 | ********...........*******X | 65.100 | | 34.500 | .....*******X | 34.500 | | 15.300 | .*******X | 15.300 | | 0.000 | ********..**************X | 15.300 | | 0.000 | ***********************...........*******X | 65.100 | | 0.000 | ***********************.....********....********..*******X | 65.100 | | 15.300 | .********...............******X | 95.300 | | 19.200 | ..*******X | 19.200 | | 0.000 | ********..................******X | 100.000 | | 74.900 | ............X | 74.900 | | 0.000 | *******......*******X | 34.500 | | 17.900 | ..********.............................................******X | 265.600 | | 0.000 | ********...*******X | 19.200 | | 0.000 | ********.........*******X | 53.700 | | 49.800 | ........******X | 49.800 | | 0.000 | ********..............*******X | 80.000 | | 0.000 | ********.....*******X | 30.600 | | 0.000 | ***********************...***************X | 19.200 | | 0.000 | *******************************..................*******X | 100.000 | | 0.000 | ****************..********...******X | 30.600 | | 0.000 | ************************..********...................******X | 118.400 | | 0.000 | ****************...********........*******X | 61.200 | | 34.500 | .....******X | 34.500 | | 15.300 | .*******X | 15.300 | | 0.000 | ********..................********...******X | 115.300 | | 0.000 | ********......*******X | 34.500 | | 15.300 | .*******X | 15.300 | | 0.000 | ********..******X | 15.300 | | 0.000 | ****************...****************...*******X | 34.500 | | 0.000 | ********...*******X | 19.200 | | 0.000 | ********.....******X | 30.600 | | 0.000 | ****************.........********.*******X | 57.600 | | 0.000 | ****************......********.....******X | 65.100 | | 15.300 | .*******X | 15.300 | | 0.000 | ***************............********...............*******X | 149.800 | | 95.300 | ................*******X | 95.300 | | 0.000 | ****************..................******X | 99.200 | | 49.600 | ........X | 49.600 | | 0.000 | *******....................*******X | 110.600 | | 19.200 | ..********..******X | 30.600 | | 0.000 | ********......*******X | 34.100 | | 15.700 | .*******............*******X | 80.000 | | 0.000 | ********..******X | 15.300 | | 0.000 | ********..*******X | 15.300 | | 0.000 | *******************************..*******X | 15.300 | | 0.000 | ****************..**********************X | 15.300 | | 0.000 | ***************.....*******X | 30.600 | | 15.300 | .********...............*******X | 95.300 | | 0.000 | *****************X | 0.000 | +--------+------------------------------------------------------------------------------------------------------------------------------------------------------+---------+ 2022-03-06 11:38:27,676 | test | INFO | pool-6-thread-28 | [table_view:display:72] Ops trend for bucket 'default' +-----------+-----------------------------------------------------------------------------------------------------------------------------+-----------+ | Min | Trend | Max | +-----------+-----------------------------------------------------------------------------------------------------------------------------+-----------+ | 0.000 | **..............*******......................................................................................*******X | 95754.200 | | 92327.000 | ...............................................................................................******X | 92327.000 | | 72061.700 | ..........................................................................*******X | 72061.700 | | 71965.600 | ..........................................................................*X | 71965.600 | | 71965.600 | ..........................................................................*****X | 71965.600 | | 58745.700 | ............................................................*******X | 58745.700 | | 41702.400 | ..........................................*******X | 41702.400 | | 30958.000 | ...............................********.*******..................X | 49592.000 | | 49592.000 | ..................................................****X | 49592.000 | | 49592.000 | ..................................................**.................********......*******X | 71106.600 | | 66984.200 | ....................................................................*****X | 66984.200 | | 66984.200 | ....................................................................X | 66984.200 | | 61100.100 | ..............................................................*******X | 61100.100 | | 40770.900 | .........................................***X | 40770.900 | | 40770.900 | .........................................***.......********..........X | 56864.100 | | 56864.100 | ..........................................................**X | 56864.100 | | 56864.100 | ..........................................................**X | 56864.100 | | 53814.200 | .......................................................*******....*X | 57532.000 | | 57532.000 | ...........................................................*****X | 57532.000 | | 52714.100 | ......................................................*******..******X | 55206.800 | | 44829.600 | .............................................********.....*******...*******.....****X | 57236.200 | | 57236.200 | ..........................................................*X | 57236.200 | | 56281.100 | .........................................................*******X | 56281.100 | | 44645.900 | .............................................******X | 44645.900 | | 39459.300 | ........................................********....*******..*******X | 45711.600 | | 41163.400 | .........................................*******...********..*******....*******X | 49108.500 | | 42684.900 | ...........................................*******..********...............**********X | 58919.800 | | 58919.800 | ............................................................***X | 58919.800 | | 56801.400 | ..........................................................*****X | 56801.400 | | 56801.400 | ..........................................................X | 56801.400 | | 40753.400 | .........................................*******..........*******X | 49988.700 | | 45460.000 | ..............................................*******..*******.....*******X | 52201.300 | | 51324.800 | ....................................................******X | 51324.800 | | 50539.700 | ...................................................******X | 50539.700 | | 48202.700 | .................................................******X | 48202.700 | | 47602.400 | ................................................******X | 47602.400 | | 44308.100 | .............................................**X | 44308.100 | | 44308.100 | .............................................*X | 44308.100 | | 44308.100 | .............................................**.*******X | 45803.600 | | 41556.700 | ..........................................******X | 41556.700 | | 40542.600 | .........................................****X | 40542.600 | | 40542.600 | .........................................*X | 40542.600 | | 38917.100 | .......................................********...*X | 41984.400 | | 41984.400 | ..........................................*X | 41984.400 | | 41984.400 | ..........................................*X | 41984.400 | | 41984.400 | ..........................................*....*******X | 45045.000 | | 44655.500 | .............................................******X | 44655.500 | | 42971.600 | ...........................................*******..*******....*******X | 48432.500 | | 46597.300 | ...............................................******X | 46597.300 | | 43417.000 | ............................................********.*******..******X | 46479.300 | | 42892.900 | ...........................................********..**************.*******....*******X | 49236.900 | | 47326.600 | ................................................******X | 47326.600 | | 44825.900 | .............................................*******..***************X | 46365.000 | | 45556.100 | ..............................................*******.......********..*X | 54371.600 | | 54371.600 | .......................................................***X | 54371.600 | | 54371.600 | .......................................................X | 54371.600 | | 50097.500 | ...................................................******X | 50097.500 | | 49202.800 | ..................................................********.***X | 50678.400 | | 50678.400 | ...................................................**X | 50678.400 | | 50678.400 | ...................................................*...*******.........*X | 61737.700 | | 61737.700 | ...............................................................*****...******X | 64285.300 | | 56564.800 | ..........................................................********....*******..*......*X | 68477.900 | | 68477.900 | ......................................................................*****...*******X | 71396.300 | | 69663.900 | .......................................................................********.******X | 70129.100 | | 68874.900 | ......................................................................********........****************...*******.......***X | 85756.600 | | 85756.600 | ........................................................................................*X | 85756.600 | | 85756.600 | ........................................................................................**.********...*******X | 89839.000 | | 85868.500 | ........................................................................................******X | 85868.500 | | 83085.600 | .....................................................................................****X | 83085.600 | | 83085.600 | .....................................................................................**X | 83085.600 | | 82463.200 | .....................................................................................****X | 82463.200 | | 82463.200 | .....................................................................................**X | 82463.200 | | 80259.500 | ..................................................................................*******X | 80259.500 | | 78654.600 | .................................................................................******X | 78654.600 | | 74586.500 | ............................................................................*******X | 74586.500 | | 61290.200 | ...............................................................*******X | 61290.200 | | 60933.600 | ..............................................................*X | 60933.600 | | 60933.600 | ..............................................................*X | 60933.600 | | 60933.600 | ..............................................................****.***************...*******X | 65078.900 | | 63099.600 | ................................................................X | 63099.600 | | 63099.600 | ................................................................*******....*******X | 66325.300 | | 60172.200 | .............................................................********.********....*X | 64847.900 | | 64847.900 | ..................................................................*X | 64847.900 | | 64847.900 | ..................................................................**X | 64847.900 | | 60347.500 | ..............................................................********...*******X | 63549.200 | | 62864.200 | ................................................................*******..*******X | 64948.200 | | 61799.100 | ...............................................................**X | 61799.100 | | 61799.100 | ...............................................................*****.*X | 63105.500 | | 63105.500 | ................................................................*X | 63105.500 | | 63105.500 | ................................................................***..*******X | 64313.200 | | 62901.300 | ................................................................*******X | 62901.300 | | 57270.900 | ..........................................................*******.....*******X | 61966.900 | | 57451.200 | ..........................................................********....*******....*....******X | 68760.600 | | 59772.900 | .............................................................********.....X | 64632.000 | | 64632.000 | ..................................................................**X | 64632.000 | | 64632.000 | ..................................................................**X | 64632.000 | | 64632.000 | ..................................................................X | 64632.000 | | 62527.100 | ................................................................*X | 62527.100 | | 62527.100 | ................................................................****X | 62527.100 | | 59452.700 | .............................................................********........****X | 67765.700 | | 67765.700 | .....................................................................*X | 67765.700 | | 67765.700 | .....................................................................X | 67765.700 | | 61852.900 | ...............................................................*X | 61852.900 | | 61852.900 | ...............................................................X | 61852.900 | | 61852.900 | ...............................................................***X | 61852.900 | | 58772.500 | ............................................................X | 58772.500 | | 57844.600 | ...........................................................******X | 57844.600 | | 56652.200 | ..........................................................*******X | 56652.200 | | 53764.800 | .......................................................********...............******X | 68283.100 | | 61222.900 | ..............................................................*******X | 61222.900 | | 57590.000 | ...........................................................********.*******...*******X | 61710.200 | | 56965.200 | ..........................................................********...*******X | 59840.000 | | 59147.900 | ............................................................*******..********..*******X | 62666.500 | | 59917.500 | .............................................................*******..********...*******X | 64262.000 | | 58400.700 | ...........................................................*******......*******X | 63305.900 | | 60544.100 | ..............................................................*******X | 60544.100 | | 56266.200 | .........................................................********...*******.....*******X | 63795.600 | | 57017.200 | ..........................................................********.....*******X | 61778.500 | | 61237.500 | ..............................................................*******..*******X | 62747.600 | | 59544.600 | .............................................................********..********....*******X | 65544.100 | | 55670.100 | .........................................................********...*******.********.*******X | 60422.700 | | 59057.600 | ............................................................*******X | 59057.600 | | 58778.600 | ............................................................*******X | 58778.600 | | 57338.100 | ..........................................................*******.....********.*******X | 62573.700 | | 60518.600 | ..............................................................********..******X | 62785.700 | | 61156.300 | ..............................................................*******X | 61156.300 | | 59851.000 | .............................................................********...*******X | 62904.200 | | 62047.100 | ...............................................................*******.........*******X | 70667.100 | | 55437.900 | ........................................................********...........*******X | 65270.400 | | 61736.300 | ...............................................................******X | 61736.300 | | 60245.800 | .............................................................********...********.*******X | 63571.000 | | 59711.900 | .............................................................*******.....*******X | 64588.200 | | 62017.500 | ...............................................................*******X | 62017.500 | | 60338.600 | ..............................................................*******X | 60338.600 | | 57737.600 | ...........................................................********..******X | 60133.700 | | 56762.500 | ..........................................................********..********...*******X | 62005.100 | | 53242.300 | ......................................................********.......*******X | 59913.800 | | 55959.000 | .........................................................***************..*******X | 57541.800 | | 56114.300 | .........................................................*******..********..*******X | 59438.100 | | 58787.300 | ............................................................******X | 58787.300 | | 56926.900 | ..........................................................********....***************X | 61155.400 | | 61004.200 | ..............................................................*******..*******X | 63174.600 | | 58396.400 | ...........................................................******X | 58396.400 | | 57595.600 | ...........................................................X | 57595.600 | | 56368.600 | .........................................................*******..*******X | 57620.000 | | 54233.700 | .......................................................********.......******X | 60823.700 | | 54194.000 | .......................................................********...*******X | 56852.200 | | 56326.200 | .........................................................*******.....*******X | 60542.900 | | 51981.400 | .....................................................********.......******X | 59197.600 | | 58213.800 | ...........................................................*******X | 58213.800 | | 53496.600 | ......................................................********........*******X | 60928.200 | | 51730.700 | .....................................................*******......*******X | 57640.100 | | 57271.500 | ..........................................................**************X | 57288.100 | | 56391.800 | .........................................................********......*******X | 62018.200 | | 56598.700 | ..........................................................**************X | 56903.900 | | 56856.800 | ..........................................................********....*******X | 61233.200 | | 52298.800 | .....................................................*******.....*******X | 56900.100 | | 55973.100 | .........................................................*******X | 55973.100 | | 51727.500 | .....................................................**X | 51727.500 | +-----------+-----------------------------------------------------------------------------------------------------------------------------+-----------+ 2022-03-06 11:38:29,085 | test | INFO | pool-6-thread-12 | [table_view:display:72] Ops trend for bucket 'default' +-----------+------------------------------------------------------------------------------------------------------+-----------+ | Min | Trend | Max | +-----------+------------------------------------------------------------------------------------------------------+-----------+ | 51727.500 | ...................................................................................................X | 51727.500 | +-----------+------------------------------------------------------------------------------------------------------+-----------+ 2022-03-06 11:38:33,003 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 90.11 2022-03-06 11:38:43,030 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 90.31 2022-03-06 11:38:53,055 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 90.57 2022-03-06 11:39:03,082 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 90.89 2022-03-06 11:39:13,108 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 91.15 2022-03-06 11:39:23,134 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 91.42 2022-03-06 11:39:33,161 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 91.68 2022-03-06 11:39:43,187 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 91.94 2022-03-06 11:39:53,213 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 92.26 2022-03-06 11:40:03,239 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 92.52 2022-03-06 11:40:13,263 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 92.72 2022-03-06 11:40:23,289 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 93.04 2022-03-06 11:40:33,315 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 93.37 2022-03-06 11:40:43,341 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 93.56 2022-03-06 11:40:53,369 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 93.82 2022-03-06 11:41:03,397 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 94.15 2022-03-06 11:41:13,424 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 94.34 2022-03-06 11:41:23,450 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 94.61 2022-03-06 11:41:33,479 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 94.87 2022-03-06 11:41:43,506 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 95.19 2022-03-06 11:41:53,533 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 95.58 2022-03-06 11:42:03,561 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 95.78 2022-03-06 11:42:13,589 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 95.97 2022-03-06 11:42:23,618 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 96.36 2022-03-06 11:42:33,644 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 96.62 2022-03-06 11:42:43,671 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 96.89 2022-03-06 11:42:53,697 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 97.15 2022-03-06 11:43:03,721 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 97.34 2022-03-06 11:43:13,752 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 97.6 2022-03-06 11:43:23,779 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 97.8 2022-03-06 11:43:33,805 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 97.99 2022-03-06 11:43:43,832 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 98.19 2022-03-06 11:43:53,858 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 98.38 2022-03-06 11:44:03,884 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 98.58 2022-03-06 11:44:13,908 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 98.77 2022-03-06 11:44:23,936 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 98.9 2022-03-06 11:44:33,963 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 98.97 2022-03-06 11:44:43,992 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 99.16 2022-03-06 11:44:54,019 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 99.16 2022-03-06 11:45:04,046 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 99.22 2022-03-06 11:45:14,072 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 99.42 2022-03-06 11:45:24,098 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 99.48 2022-03-06 11:45:34,125 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 99.55 2022-03-06 11:45:44,150 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 99.68 2022-03-06 11:45:54,174 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 99.74 2022-03-06 11:46:04,203 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: running, progress: 99.94 2022-03-06 11:46:19,236 | test | INFO | pool-6-thread-8 | [task:check:415] Rebalance - status: none, progress: 100 2022-03-06 11:46:19,250 | test | INFO | pool-6-thread-8 | [task:check:474] Rebalance completed with progress: 100% in 2785.50999999 sec 2022-03-06 11:46:19,496 | test | INFO | MainThread | [collections_rebalance:get_zone_info:969] nodes in zone inside get_zone_info():{'Group 1': ['172.23.105.164', '172.23.100.35', '172.23.106.177', '172.23.105.206']} 2022-03-06 11:46:22,719 | test | INFO | MainThread | [bucket_ready_functions:validate_docs_per_collections_all_buckets:4742] Validating collection stats and item counts 2022-03-06 11:46:30,355 | test | ERROR | MainThread | [bucket_ready_functions:validate_manifest_uid:4605] Bucket UID mismatch. Expected: 51, Actual: 34 2022-03-06 11:46:30,357 | test | WARNING | MainThread | [bucket_ready_functions:validate_docs_per_collections_all_buckets:4759] Bucket manifest UID mismatch! 2022-03-06 11:46:33,494 | test | ERROR | MainThread | [bucket_ready_functions:validate_manifest_uid:4605] Bucket UID mismatch. Expected: 40, Actual: 29 2022-03-06 11:46:33,496 | test | WARNING | MainThread | [bucket_ready_functions:validate_docs_per_collections_all_buckets:4759] Bucket manifest UID mismatch! 2022-03-06 11:46:36,894 | test | ERROR | MainThread | [bucket_ready_functions:validate_manifest_uid:4605] Bucket UID mismatch. Expected: 71, Actual: 48 2022-03-06 11:46:36,895 | test | WARNING | MainThread | [bucket_ready_functions:validate_docs_per_collections_all_buckets:4759] Bucket manifest UID mismatch! 2022-03-06 11:46:45,055 | test | INFO | MainThread | [table_view:display:72] Bucket statistics +---------+-----------+-----------------+----------+------------+-----+-----------+-----------+------------+------------+--------------+ | Bucket | Type | Storage Backend | Replicas | Durability | TTL | Items | RAM Quota | RAM Used | Disk Used | ARR | +---------+-----------+-----------------+----------+------------+-----+-----------+-----------+------------+------------+--------------+ | bucket1 | couchbase | couchstore | 1 | none | 0 | 114591 | 7.81 GiB | 102.80 MiB | 178.88 MiB | 100 | | bucket2 | couchbase | magma | 1 | none | 0 | 109455 | 4.00 GiB | 255.96 MiB | 418.49 MiB | 100 | | default | couchbase | magma | 1 | none | 0 | 184384000 | 4.00 GiB | 2.62 GiB | 50.95 GiB | 4.9128856083 | +---------+-----------+-----------------+----------+------------+-----+-----------+-----------+------------+------------+--------------+ 2022-03-06 11:46:45,443 | test | INFO | MainThread | [collections_base:tearDown:84] Bucket: bucket2, Active Resident ratio(DGM): 100% 2022-03-06 11:46:45,443 | test | INFO | MainThread | [collections_base:tearDown:87] Bucket: bucket2, Replica Resident ratio(DGM): 100% 2022-03-06 11:46:50,038 | test | INFO | MainThread | [collections_base:tearDown:84] Bucket: bucket1, Active Resident ratio(DGM): 100% 2022-03-06 11:46:50,038 | test | INFO | MainThread | [collections_base:tearDown:87] Bucket: bucket1, Replica Resident ratio(DGM): 100% 2022-03-06 11:46:55,497 | test | INFO | MainThread | [collections_base:tearDown:84] Bucket: default, Active Resident ratio(DGM): 2% 2022-03-06 11:46:55,499 | test | INFO | MainThread | [collections_base:tearDown:87] Bucket: default, Replica Resident ratio(DGM): 1% 2022-03-06 11:47:13,719 | test | INFO | MainThread | [basetestcase:check_logs:824] Comparing timestamps: Log's latest timestamp: 2022-03-06 06:13:45, Test's start timestamp is 2022-03-06 09:14:05.490000 2022-03-06 11:47:13,720 | test | INFO | MainThread | [basetestcase:check_coredump_exist:921] unwanted messages in /opt/couchbase/var/lib/couchbase/logs/memcached.log.000015.txt 2022-03-06 11:47:13,720 | test | INFO | MainThread | [basetestcase:check_logs:824] Comparing timestamps: Log's latest timestamp: 2022-03-06 06:13:45, Test's start timestamp is 2022-03-06 09:14:05.490000 2022-03-06 11:47:14,921 | test | INFO | MainThread | [basetestcase:check_logs:824] Comparing timestamps: Log's latest timestamp: 2022-03-06 06:13:45, Test's start timestamp is 2022-03-06 09:14:05.490000 2022-03-06 11:47:14,921 | test | INFO | MainThread | [basetestcase:check_logs:824] Comparing timestamps: Log's latest timestamp: 2022-03-06 06:13:45, Test's start timestamp is 2022-03-06 09:14:05.490000 2022-03-06 11:47:15,316 | test | INFO | MainThread | [basetestcase:check_logs:824] Comparing timestamps: Log's latest timestamp: 2022-03-06 06:13:45, Test's start timestamp is 2022-03-06 09:14:05.490000 2022-03-06 11:47:15,318 | test | INFO | MainThread | [basetestcase:check_coredump_exist:921] unwanted messages in /opt/couchbase/var/lib/couchbase/logs/memcached.log.000001.txt 2022-03-06 11:47:15,318 | test | INFO | MainThread | [basetestcase:check_logs:824] Comparing timestamps: Log's latest timestamp: 2022-03-06 06:13:45, Test's start timestamp is 2022-03-06 09:14:05.490000 2022-03-06 11:47:15,989 | test | INFO | MainThread | [basetestcase:check_logs:824] Comparing timestamps: Log's latest timestamp: 2022-03-06 06:13:45, Test's start timestamp is 2022-03-06 09:14:05.490000 2022-03-06 11:47:15,990 | test | INFO | MainThread | [basetestcase:check_logs:824] Comparing timestamps: Log's latest timestamp: 2022-03-06 06:13:45, Test's start timestamp is 2022-03-06 09:14:05.490000 2022-03-06 11:47:16,390 | test | INFO | MainThread | [basetestcase:check_logs:824] Comparing timestamps: Log's latest timestamp: 2022-03-06 06:13:45, Test's start timestamp is 2022-03-06 09:14:05.490000 2022-03-06 11:47:16,391 | test | INFO | MainThread | [basetestcase:check_coredump_exist:921] unwanted messages in /opt/couchbase/var/lib/couchbase/logs/memcached.log.000001.txt 2022-03-06 11:47:16,391 | test | INFO | MainThread | [basetestcase:check_logs:824] Comparing timestamps: Log's latest timestamp: 2022-03-06 06:13:45, Test's start timestamp is 2022-03-06 09:14:05.490000 2022-03-06 11:47:16,448 | test | INFO | MainThread | [basetestcase:check_logs:824] Comparing timestamps: Log's latest timestamp: 2022-03-06 06:13:46, Test's start timestamp is 2022-03-06 09:14:05.490000 2022-03-06 11:47:16,448 | test | INFO | MainThread | [basetestcase:check_coredump_exist:921] unwanted messages in /opt/couchbase/var/lib/couchbase/logs/memcached.log.000002.txt 2022-03-06 11:47:16,450 | test | INFO | MainThread | [basetestcase:check_logs:824] Comparing timestamps: Log's latest timestamp: 2022-03-06 06:13:46, Test's start timestamp is 2022-03-06 09:14:05.490000 2022-03-06 11:47:16,974 | test | INFO | MainThread | [basetestcase:check_logs:824] Comparing timestamps: Log's latest timestamp: 2022-03-06 06:13:46, Test's start timestamp is 2022-03-06 09:14:05.490000 2022-03-06 11:47:16,976 | test | INFO | MainThread | [basetestcase:check_logs:824] Comparing timestamps: Log's latest timestamp: 2022-03-06 06:13:46, Test's start timestamp is 2022-03-06 09:14:05.490000 2022-03-06 11:47:17,374 | test | INFO | MainThread | [basetestcase:check_logs:824] Comparing timestamps: Log's latest timestamp: 2022-03-06 06:13:43, Test's start timestamp is 2022-03-06 09:14:05.490000 2022-03-06 11:47:17,374 | test | INFO | MainThread | [basetestcase:check_coredump_exist:921] unwanted messages in /opt/couchbase/var/lib/couchbase/logs/memcached.log.000001.txt 2022-03-06 11:47:17,375 | test | INFO | MainThread | [basetestcase:check_logs:824] Comparing timestamps: Log's latest timestamp: 2022-03-06 06:13:43, Test's start timestamp is 2022-03-06 09:14:05.490000 2022-03-06 11:47:17,434 | test | INFO | MainThread | [basetestcase:check_logs:824] Comparing timestamps: Log's latest timestamp: 2022-03-06 06:13:46, Test's start timestamp is 2022-03-06 09:14:05.490000 2022-03-06 11:47:17,434 | test | INFO | MainThread | [basetestcase:check_coredump_exist:921] unwanted messages in /opt/couchbase/var/lib/couchbase/logs/memcached.log.000002.txt 2022-03-06 11:47:17,436 | test | INFO | MainThread | [basetestcase:check_logs:824] Comparing timestamps: Log's latest timestamp: 2022-03-06 06:13:46, Test's start timestamp is 2022-03-06 09:14:05.490000 2022-03-06 11:47:18,384 | test | INFO | MainThread | [basetestcase:check_logs:824] Comparing timestamps: Log's latest timestamp: 2022-03-06 06:13:46, Test's start timestamp is 2022-03-06 09:14:05.490000 2022-03-06 11:47:18,385 | test | INFO | MainThread | [basetestcase:check_logs:824] Comparing timestamps: Log's latest timestamp: 2022-03-06 06:13:46, Test's start timestamp is 2022-03-06 09:14:05.490000 ok ---------------------------------------------------------------------- Ran 1 test in 9194.439s OK During the test, Remote Connections: 208, Disconnections: 209 SDK Connections: 30, Disconnections: 30 !!!!!! CRITICAL :: Shell disconnection mismatch !!!!! summary so far suite bucket_collections.collections_rebalance.CollectionsRebalance , pass 2 , fail 1 failures so far... bucket_collections.collections_rebalance.CollectionsRebalance.test_data_load_collections_with_graceful_failover_recovery testrunner logs, diags and results are available under /data/workspace/temp_rebalance_magma/logs/testrunner-22-Mar-06_03-39-55/test_3 Logs will be stored at /data/workspace/temp_rebalance_magma/logs/testrunner-22-Mar-06_03-39-55/test_4 guides/gradlew --refresh-dependencies testrunner -P jython=/opt/jython/bin/jython -P 'args=-i /tmp/win10-bucket-ops-temp_rebalance_magma.ini rerun=False,get-cbcollect-info=True -t bucket_collections.collections_rebalance.CollectionsRebalance.test_data_load_collections_with_hard_failover_recovery,nodes_init=4,nodes_failover=1,recovery_type=delta,bucket_spec=magma_dgm.bloom_filter.4_node_1_replica_magma_1024,doc_size=30,randomize_value=True,data_load_stage=during,skip_validations=False,data_load_spec=volume_test_load_with_CRUD_on_collections,change_magma_quota=True,GROUP=failover' Test Input params: {'doc_size': '30', 'data_load_stage': 'during', 'cluster_name': 'win10-bucket-ops-temp_rebalance_magma', 'nodes_failover': '1', 'ini': '/tmp/win10-bucket-ops-temp_rebalance_magma.ini', 'get-cbcollect-info': 'True', 'recovery_type': 'delta', 'conf_file': 'conf/magma/dgm_collections_bloom_filter.conf', 'change_magma_quota': 'True', 'skip_validations': 'False', 'spec': 'dgm_collections_bloom_filter', 'rerun': 'False', 'num_nodes': 7, 'logs_folder': '/data/workspace/temp_rebalance_magma/logs/testrunner-22-Mar-06_03-39-55/test_4', 'nodes_init': '4', 'GROUP': 'failover', 'bucket_spec': 'magma_dgm.bloom_filter.4_node_1_replica_magma_1024', 'case_number': 4, 'randomize_value': 'True', 'data_load_spec': 'volume_test_load_with_CRUD_on_collections'} test_data_load_collections_with_hard_failover_recovery (bucket_collections.collections_rebalance.CollectionsRebalance) ... 2022-03-06 11:47:20,042 | test | INFO | MainThread | [basetestcase:log_setup_status:647] ========= CollectionsRebalance setup started for test #4 test_data_load_collections_with_hard_failover_recovery ========= 2022-03-06 11:47:20,157 | test | INFO | MainThread | [basetestcase:log_setup_status:647] ========= BaseTestCase setup started for test #4 test_data_load_collections_with_hard_failover_recovery ========= 2022-03-06 11:47:51,709 | infra | ERROR | MainThread | [Rest_Connection:_http_request:286] DELETE http://172.23.105.164:8091/pools/default/buckets/bucket2 body: headers: {'Accept': '*/*', 'Connection': 'close', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==', 'Content-Type': 'application/x-www-form-urlencoded'} error: 500 reason: unknown {"_":"Bucket deletion not yet complete, but will continue.\r\n"} auth: Administrator:password 2022-03-06 11:47:52,095 | infra | ERROR | MainThread | [remote_util:log_command_output:3012] wc: /data: Is a directory 2022-03-06 11:47:52,096 | test | ERROR | MainThread | [bucket_ready_functions:delete_bucket:1617] Unable to get timings for bucket: global name 'StatsCommon' is not defined 2022-03-06 11:48:22,256 | infra | ERROR | MainThread | [Rest_Connection:_http_request:286] DELETE http://172.23.105.164:8091/pools/default/buckets/default body: headers: {'Accept': '*/*', 'Connection': 'close', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==', 'Content-Type': 'application/x-www-form-urlencoded'} error: 500 reason: unknown {"_":"Bucket deletion not yet complete, but will continue.\r\n"} auth: Administrator:password 2022-03-06 11:48:22,579 | infra | ERROR | MainThread | [remote_util:log_command_output:3012] wc: /data: Is a directory 2022-03-06 11:48:22,579 | test | ERROR | MainThread | [bucket_ready_functions:delete_bucket:1617] Unable to get timings for bucket: global name 'StatsCommon' is not defined 2022-03-06 11:50:23,647 | test | ERROR | MainThread | [rest_client:_rebalance_status_and_progress:1677] {u'errorMessage': u'Rebalance failed. See logs for detailed reason. You can try again.', u'type': u'rebalance', u'masterRequestTimedOut': False, u'statusId': u'20469f133a05785027a75984fae8c602', u'subtype': u'rebalance', u'statusIsStale': False, u'lastReportURI': u'/logs/rebalanceReport?reportID=8c57e0a7e0295ee61bfc41efa718f8ca', u'status': u'notRunning'} - rebalance failed 2022-03-06 11:50:23,671 | test | INFO | MainThread | [rest_client:print_UI_logs:2842] Latest logs from UI on 172.23.105.164: 2022-03-06 11:50:23,671 | test | ERROR | MainThread | [rest_client:print_UI_logs:2844] {u'code': 0, u'module': u'ns_orchestrator', u'type': u'critical', u'node': u'ns_1@172.23.105.164', u'tstamp': 1646596222819L, u'shortText': u'message', u'serverTime': u'2022-03-06T11:50:22.819Z', u'text': u'Rebalance exited with reason {buckets_shutdown_wait_failed,\n [{\'ns_1@172.23.105.164\',\n {\'EXIT\',\n {old_buckets_shutdown_wait_failed,\n ["default","bucket2"]}}}]}.\nRebalance Operation Id = 67e177841c31ff159e319178d902a096'} 2022-03-06 11:50:23,671 | test | ERROR | MainThread | [rest_client:print_UI_logs:2844] {u'code': 0, u'module': u'ns_rebalancer', u'type': u'critical', u'node': u'ns_1@172.23.105.164', u'tstamp': 1646596222815L, u'shortText': u'message', u'serverTime': u'2022-03-06T11:50:22.815Z', u'text': u'Failed to wait deletion of some buckets on some nodes: [{\'ns_1@172.23.105.164\',\n {\'EXIT\',\n {old_buckets_shutdown_wait_failed,\n ["default",\n "bucket2"]}}}]\n'} 2022-03-06 11:50:23,671 | test | ERROR | MainThread | [rest_client:print_UI_logs:2844] {u'code': 0, u'module': u'ns_orchestrator', u'type': u'info', u'node': u'ns_1@172.23.105.164', u'tstamp': 1646596102803L, u'shortText': u'message', u'serverTime': u'2022-03-06T11:48:22.803Z', u'text': u"Starting rebalance, KeepNodes = ['ns_1@172.23.105.164'], EjectNodes = ['ns_1@172.23.105.206',\n 'ns_1@172.23.106.177',\n 'ns_1@172.23.100.35'], Failed over and being ejected nodes = []; no delta recovery nodes; Operation Id = 67e177841c31ff159e319178d902a096"} 2022-03-06 11:50:23,671 | test | ERROR | MainThread | [rest_client:print_UI_logs:2844] {u'code': 102, u'module': u'menelaus_web', u'type': u'warning', u'node': u'ns_1@172.23.105.164', u'tstamp': 1646596102794L, u'shortText': u'client-side error report', u'serverTime': u'2022-03-06T11:48:22.794Z', u'text': u'Client-side error-report for user "Administrator" on node \'ns_1@172.23.105.164\':\nUser-Agent:Python-httplib2/$Rev: 259 $\nStarting rebalance from test, ejected nodes [u\'ns_1@172.23.105.206\', u\'ns_1@172.23.106.177\', u\'ns_1@172.23.100.35\']'} 2022-03-06 11:50:23,671 | test | ERROR | MainThread | [rest_client:print_UI_logs:2844] {u'code': 0, u'module': u'ns_memcached', u'type': u'info', u'node': u'ns_1@172.23.100.35', u'tstamp': 1646596041808L, u'shortText': u'message', u'serverTime': u'2022-03-06T11:47:21.808Z', u'text': u'Shutting down bucket "bucket2" on \'ns_1@172.23.100.35\' for deletion'} 2022-03-06 11:50:23,673 | test | ERROR | MainThread | [rest_client:print_UI_logs:2844] {u'code': 0, u'module': u'ns_memcached', u'type': u'info', u'node': u'ns_1@172.23.105.164', u'tstamp': 1646596041752L, u'shortText': u'message', u'serverTime': u'2022-03-06T11:47:21.752Z', u'text': u'Shutting down bucket "bucket2" on \'ns_1@172.23.105.164\' for deletion'} 2022-03-06 11:50:23,673 | test | ERROR | MainThread | [rest_client:print_UI_logs:2844] {u'code': 0, u'module': u'ns_memcached', u'type': u'info', u'node': u'ns_1@172.23.105.206', u'tstamp': 1646596041752L, u'shortText': u'message', u'serverTime': u'2022-03-06T11:47:21.752Z', u'text': u'Shutting down bucket "bucket2" on \'ns_1@172.23.105.206\' for deletion'} 2022-03-06 11:50:23,673 | test | ERROR | MainThread | [rest_client:print_UI_logs:2844] {u'code': 0, u'module': u'ns_memcached', u'type': u'info', u'node': u'ns_1@172.23.106.177', u'tstamp': 1646596041750L, u'shortText': u'message', u'serverTime': u'2022-03-06T11:47:21.750Z', u'text': u'Shutting down bucket "bucket2" on \'ns_1@172.23.106.177\' for deletion'} 2022-03-06 11:50:23,673 | test | ERROR | MainThread | [rest_client:print_UI_logs:2844] {u'code': 11, u'module': u'menelaus_web', u'type': u'info', u'node': u'ns_1@172.23.105.164', u'tstamp': 1646596041584L, u'shortText': u'message', u'serverTime': u'2022-03-06T11:47:21.584Z', u'text': u'Deleted bucket "bucket1"\n'} 2022-03-06 11:50:23,673 | test | ERROR | MainThread | [rest_client:print_UI_logs:2844] {u'code': 0, u'module': u'ns_memcached', u'type': u'info', u'node': u'ns_1@172.23.100.35', u'tstamp': 1646596040462L, u'shortText': u'message', u'serverTime': u'2022-03-06T11:47:20.462Z', u'text': u'Shutting down bucket "bucket1" on \'ns_1@172.23.100.35\' for deletion'} Traceback (most recent call last): File "pytests/basetestcase.py", line 363, in setUp self.cluster_util.cluster_cleanup(cluster, File "couchbase_utils/cluster_utils/cluster_ready_functions.py", line 169, in cluster_cleanup self.cleanup_cluster(cluster, master=cluster.master) File "couchbase_utils/cluster_utils/cluster_ready_functions.py", line 222, in cleanup_cluster ejectedNodes=[node.id for node in nodes File "lib/membase/api/rest_client.py", line 143, in remove_nodes return self.rest.monitorRebalance() File "lib/membase/api/rest_client.py", line 1580, in monitorRebalance progress = self._rebalance_progress() File "lib/membase/api/rest_client.py", line 1717, in _rebalance_progress return self._rebalance_status_and_progress()[1] File "lib/membase/api/rest_client.py", line 1679, in _rebalance_status_and_progress raise RebalanceFailedException(msg) RebalanceFailedException: Rebalance Failed: {u'errorMessage': u'Rebalance failed. See logs for detailed reason. You can try again.', u'type': u'rebalance', u'masterRequestTimedOut': False, u'statusId': u'20469f133a05785027a75984fae8c602', u'subtype': u'rebalance', u'statusIsStale': False, u'lastReportURI': u'/logs/rebalanceReport?reportID=8c57e0a7e0295ee61bfc41efa718f8ca', u'status': u'notRunning'} - rebalance failed 2022-03-06 11:50:23,697 | test | INFO | MainThread | [cluster_ready_functions:trigger_cb_collect_on_cluster:1130] Running cbcollect on node ns_1@172.23.100.35,ns_1@172.23.105.164,ns_1@172.23.105.206,ns_1@172.23.106.177 2022-03-06 11:50:33,723 | test | INFO | MainThread | [cluster_ready_functions:trigger_cb_collect_on_cluster:1133] ns_1@172.23.100.35,ns_1@172.23.105.164,ns_1@172.23.105.206,ns_1@172.23.106.177 - cbcollect status: True 2022-03-06 11:50:33,723 | test | INFO | MainThread | [cluster_ready_functions:wait_for_cb_collect_to_complete:1138] Polling active_tasks to check cbcollect status 2022-03-06 11:54:04,144 | test | INFO | MainThread | [cluster_ready_functions:copy_cb_collect_logs:1165] ns_1@172.23.100.35: Copying cbcollect ZIP file to Client 2022-03-06 11:54:05,058 | test | INFO | MainThread | [cluster_ready_functions:copy_cb_collect_logs:1165] ns_1@172.23.105.164: Copying cbcollect ZIP file to Client 2022-03-06 11:54:06,289 | test | INFO | MainThread | [cluster_ready_functions:copy_cb_collect_logs:1165] ns_1@172.23.105.206: Copying cbcollect ZIP file to Client 2022-03-06 11:54:07,098 | test | INFO | MainThread | [cluster_ready_functions:copy_cb_collect_logs:1165] ns_1@172.23.106.177: Copying cbcollect ZIP file to Client 2022-03-06 11:54:07,872 | test | ERROR | MainThread | [basetestcase:setUp:368] Rebalance Failed: {u'errorMessage': u'Rebalance failed. See logs for detailed reason. You can try again.', u'type': u'rebalance', u'masterRequestTimedOut': False, u'statusId': u'20469f133a05785027a75984fae8c602', u'subtype': u'rebalance', u'statusIsStale': False, u'lastReportURI': u'/logs/rebalanceReport?reportID=8c57e0a7e0295ee61bfc41efa718f8ca', u'status': u'notRunning'} - rebalance failed 2022-03-06 11:54:07,874 | test | INFO | MainThread | [basetestcase:initialize_cluster:467] Initializing cluster : C1 Got this failure java.net.ConnectException: Connection refused: /172.23.100.36:8091 during connect (<_realsocket at 0xba7f3c type=client open_count=1 channel=[id: 0x082ae115, 0.0.0.0/0.0.0.0:59030] timeout=300.0>) 2022-03-06 11:54:10,641 | test | ERROR | pool-7-thread-13 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.100.36:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.100.37:8091 during connect (<_realsocket at 0xba7f3d type=client open_count=1 channel=[id: 0x4af3906e, 0.0.0.0/0.0.0.0:47859] timeout=300.0>) 2022-03-06 11:54:11,016 | test | ERROR | pool-7-thread-14 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.100.37:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.100.34:8091 during connect (<_realsocket at 0xba7f3e type=client open_count=1 channel=[id: 0x97dfac50, 0.0.0.0/0.0.0.0:52833] timeout=300.0>) 2022-03-06 11:54:11,150 | test | ERROR | pool-7-thread-12 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.100.34:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.100.36:8091 during connect (<_realsocket at 0xba7f3f type=client open_count=1 channel=[id: 0x9cc09af2, 0.0.0.0/0.0.0.0:36094] timeout=300.0>) 2022-03-06 11:54:12,648 | test | ERROR | pool-7-thread-13 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.100.36:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.100.37:8091 during connect (<_realsocket at 0xba7f40 type=client open_count=1 channel=[id: 0xeb03cf73, 0.0.0.0/0.0.0.0:43084] timeout=300.0>) 2022-03-06 11:54:13,020 | test | ERROR | pool-7-thread-14 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.100.37:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.100.34:8091 during connect (<_realsocket at 0xba7f41 type=client open_count=1 channel=[id: 0xd07e0a80, 0.0.0.0/0.0.0.0:35810] timeout=300.0>) 2022-03-06 11:54:13,155 | test | ERROR | pool-7-thread-12 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.100.34:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.100.36:8091 during connect (<_realsocket at 0xba7f42 type=client open_count=1 channel=[id: 0xa94d713a, 0.0.0.0/0.0.0.0:48849] timeout=300.0>) 2022-03-06 11:54:14,654 | test | ERROR | pool-7-thread-13 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.100.36:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.100.37:8091 during connect (<_realsocket at 0xba7f43 type=client open_count=1 channel=[id: 0xb66966d1, 0.0.0.0/0.0.0.0:58348] timeout=300.0>) 2022-03-06 11:54:15,025 | test | ERROR | pool-7-thread-14 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.100.37:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.100.34:8091 during connect (<_realsocket at 0xba7f44 type=client open_count=1 channel=[id: 0xa84a011b, 0.0.0.0/0.0.0.0:34349] timeout=300.0>) 2022-03-06 11:54:15,160 | test | ERROR | pool-7-thread-12 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.100.34:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.100.36:8091 during connect (<_realsocket at 0xba7f45 type=client open_count=1 channel=[id: 0xe6975332, 0.0.0.0/0.0.0.0:56727] timeout=300.0>) 2022-03-06 11:54:16,657 | test | ERROR | pool-7-thread-13 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.100.36:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.100.37:8091 during connect (<_realsocket at 0xba7f46 type=client open_count=1 channel=[id: 0x690eca05, 0.0.0.0/0.0.0.0:53252] timeout=300.0>) 2022-03-06 11:54:17,029 | test | ERROR | pool-7-thread-14 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.100.37:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.100.34:8091 during connect (<_realsocket at 0xba7f47 type=client open_count=1 channel=[id: 0x212497ac, 0.0.0.0/0.0.0.0:39566] timeout=300.0>) 2022-03-06 11:54:17,164 | test | ERROR | pool-7-thread-12 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.100.34:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.100.36:8091 during connect (<_realsocket at 0xba7f48 type=client open_count=1 channel=[id: 0x7b74bc40, 0.0.0.0/0.0.0.0:33877] timeout=300.0>) 2022-03-06 11:54:18,663 | test | ERROR | pool-7-thread-13 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.100.36:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.100.37:8091 during connect (<_realsocket at 0xba7f49 type=client open_count=1 channel=[id: 0x37fd28c7, 0.0.0.0/0.0.0.0:50104] timeout=300.0>) 2022-03-06 11:54:19,033 | test | ERROR | pool-7-thread-14 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.100.37:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.100.34:8091 during connect (<_realsocket at 0xba7f4a type=client open_count=1 channel=[id: 0x51c202c4, 0.0.0.0/0.0.0.0:33955] timeout=300.0>) 2022-03-06 11:54:19,168 | test | ERROR | pool-7-thread-12 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.100.34:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.100.36:8091 during connect (<_realsocket at 0xba7f4b type=client open_count=1 channel=[id: 0x5bd6768d, 0.0.0.0/0.0.0.0:44926] timeout=300.0>) 2022-03-06 11:54:20,667 | test | ERROR | pool-7-thread-13 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.100.36:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.100.37:8091 during connect (<_realsocket at 0xba7f4c type=client open_count=1 channel=[id: 0x2854ba61, 0.0.0.0/0.0.0.0:41710] timeout=300.0>) 2022-03-06 11:54:21,039 | test | ERROR | pool-7-thread-14 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.100.37:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.100.35:8091 during connect (<_realsocket at 0xba7f4d type=client open_count=1 channel=[id: 0x9f20d0d7, 0.0.0.0/0.0.0.0:40896] timeout=300.0>) 2022-03-06 11:54:22,990 | test | ERROR | pool-7-thread-11 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.100.35:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.105.206:8091 during connect (<_realsocket at 0xba7f4e type=client open_count=1 channel=[id: 0x0e9f6b03, 0.0.0.0/0.0.0.0:48110] timeout=300.0>) 2022-03-06 11:54:23,740 | test | ERROR | pool-7-thread-9 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.105.206:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.106.177:8091 during connect (<_realsocket at 0xba7f4f type=client open_count=1 channel=[id: 0xadbb4ded, 0.0.0.0/0.0.0.0:46448] timeout=300.0>) 2022-03-06 11:54:23,897 | test | ERROR | pool-7-thread-10 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.106.177:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.105.164:8091 during connect (<_realsocket at 0xba7f50 type=client open_count=1 channel=[id: 0x4d5d27b5, 0.0.0.0/0.0.0.0:51212] timeout=300.0>) 2022-03-06 11:54:23,989 | test | ERROR | pool-7-thread-8 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.105.164:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.100.35:8091 during connect (<_realsocket at 0xba7f51 type=client open_count=1 channel=[id: 0xd086ad13, 0.0.0.0/0.0.0.0:46863] timeout=300.0>) 2022-03-06 11:54:24,994 | test | ERROR | pool-7-thread-11 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.100.35:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.105.206:8091 during connect (<_realsocket at 0xba7f52 type=client open_count=1 channel=[id: 0x4a4639a7, 0.0.0.0/0.0.0.0:35297] timeout=300.0>) 2022-03-06 11:54:25,746 | test | ERROR | pool-7-thread-9 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.105.206:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.106.177:8091 during connect (<_realsocket at 0xba7f53 type=client open_count=1 channel=[id: 0xa98c8e7a, 0.0.0.0/0.0.0.0:38565] timeout=300.0>) 2022-03-06 11:54:25,901 | test | ERROR | pool-7-thread-10 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.106.177:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.105.164:8091 during connect (<_realsocket at 0xba7f54 type=client open_count=1 channel=[id: 0x73a5224e, 0.0.0.0/0.0.0.0:42923] timeout=300.0>) 2022-03-06 11:54:25,993 | test | ERROR | pool-7-thread-8 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.105.164:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.100.35:8091 during connect (<_realsocket at 0xba7f55 type=client open_count=1 channel=[id: 0xa72cecd5, 0.0.0.0/0.0.0.0:48888] timeout=300.0>) 2022-03-06 11:54:27,000 | test | ERROR | pool-7-thread-11 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.100.35:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.105.206:8091 during connect (<_realsocket at 0xba7f56 type=client open_count=1 channel=[id: 0x442933d2, 0.0.0.0/0.0.0.0:36550] timeout=300.0>) 2022-03-06 11:54:27,750 | test | ERROR | pool-7-thread-9 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.105.206:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.106.177:8091 during connect (<_realsocket at 0xba7f57 type=client open_count=1 channel=[id: 0xf59bf3e2, 0.0.0.0/0.0.0.0:47513] timeout=300.0>) 2022-03-06 11:54:27,905 | test | ERROR | pool-7-thread-10 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.106.177:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.105.164:8091 during connect (<_realsocket at 0xba7f58 type=client open_count=1 channel=[id: 0x577a9ce6, 0.0.0.0/0.0.0.0:60796] timeout=300.0>) 2022-03-06 11:54:27,997 | test | ERROR | pool-7-thread-8 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.105.164:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.100.35:8091 during connect (<_realsocket at 0xba7f59 type=client open_count=1 channel=[id: 0xe2824813, 0.0.0.0/0.0.0.0:48252] timeout=300.0>) 2022-03-06 11:54:29,006 | test | ERROR | pool-7-thread-11 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.100.35:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.105.206:8091 during connect (<_realsocket at 0xba7f5a type=client open_count=1 channel=[id: 0x39f19d73, 0.0.0.0/0.0.0.0:42743] timeout=300.0>) 2022-03-06 11:54:29,753 | test | ERROR | pool-7-thread-9 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.105.206:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.106.177:8091 during connect (<_realsocket at 0xba7f5b type=client open_count=1 channel=[id: 0xbaa16e25, 0.0.0.0/0.0.0.0:37095] timeout=300.0>) 2022-03-06 11:54:29,910 | test | ERROR | pool-7-thread-10 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.106.177:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.105.164:8091 during connect (<_realsocket at 0xba7f5c type=client open_count=1 channel=[id: 0x78cf4c1d, 0.0.0.0/0.0.0.0:56163] timeout=300.0>) 2022-03-06 11:54:30,002 | test | ERROR | pool-7-thread-8 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.105.164:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.100.35:8091 during connect (<_realsocket at 0xba7f5d type=client open_count=1 channel=[id: 0xc93cc88f, 0.0.0.0/0.0.0.0:51270] timeout=300.0>) 2022-03-06 11:54:31,010 | test | ERROR | pool-7-thread-11 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.100.35:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.105.164:8091 during connect (<_realsocket at 0xba7f5e type=client open_count=1 channel=[id: 0x2415af69, 0.0.0.0/0.0.0.0:42915] timeout=300.0>) 2022-03-06 11:54:32,006 | test | ERROR | pool-7-thread-8 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.105.164:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.100.35:8091 during connect (<_realsocket at 0xba7f5f type=client open_count=1 channel=[id: 0x2ca677ae, 0.0.0.0/0.0.0.0:48776] timeout=300.0>) 2022-03-06 11:54:33,015 | test | ERROR | pool-7-thread-11 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.100.35:8091/nodes/self error [Errno 111] Connection refused Got this failure java.net.ConnectException: Connection refused: /172.23.100.35:8091 during connect (<_realsocket at 0xba7f60 type=client open_count=1 channel=[id: 0x26770352, 0.0.0.0/0.0.0.0:52701] timeout=300.0>) 2022-03-06 11:54:35,019 | test | ERROR | pool-7-thread-11 | [rest_client:_http_request:838] Socket error while connecting to http://172.23.100.35:8091/nodes/self error [Errno 111] Connection refused 2022-03-06 11:54:37,638 | infra | CRITICAL | pool-7-thread-15 | [task:call:6181] mcdMemoryReserved reported in nodes/self is: 9379 2022-03-06 11:54:37,638 | test | CRITICAL | pool-7-thread-15 | [table_view:display:72] Memory quota allocated: +---------------------+---------+ | Service | RAM MiB | +---------------------+---------+ | eventingMemoryQuota | 256 | | cbasMemoryQuota | 1024 | | indexMemoryQuota | 256 | | ftsMemoryQuota | 256 | | memoryQuota | 9279 | +---------------------+---------+ 2022-03-06 11:54:37,819 | test | WARNING | MainThread | [basetestcase:_initialize_nodes:702] RAM quota was defined less than 100 MB: 2022-03-06 11:54:49,658 | test | INFO | MainThread | [basetestcase:initialize_cluster:493] Cluster C1 initialized 2022-03-06 11:54:49,660 | test | INFO | MainThread | [basetestcase:log_setup_status:647] ========= BaseTestCase setup finished for test #4 test_data_load_collections_with_hard_failover_recovery ========= 2022-03-06 11:54:49,676 | test | INFO | MainThread | [basetestcase:log_setup_status:647] ========= ClusterSetup setup started for test #4 test_data_load_collections_with_hard_failover_recovery ========= 2022-03-06 11:55:06,880 | test | INFO | pool-7-thread-16 | [table_view:display:72] Rebalance Overview +----------------+----------+-----------------------+---------------+--------------+-----------------------+ | Nodes | Services | Version | CPU | Status | Membership / Recovery | +----------------+----------+-----------------------+---------------+--------------+-----------------------+ | 172.23.105.164 | kv | 7.1.0-2440-enterprise | 81.6334259025 | Cluster node | active / none | | 172.23.105.206 | None | | | <--- IN --- | | | 172.23.106.177 | None | | | <--- IN --- | | | 172.23.100.35 | None | | | <--- IN --- | | +----------------+----------+-----------------------+---------------+--------------+-----------------------+ 2022-03-06 11:55:06,894 | test | INFO | pool-7-thread-16 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 11:55:21,926 | test | INFO | pool-7-thread-16 | [task:check:415] Rebalance - status: none, progress: 100 2022-03-06 11:55:21,937 | test | INFO | pool-7-thread-16 | [task:check:474] Rebalance completed with progress: 100% in 15.0590000153 sec 2022-03-06 11:55:22,048 | test | INFO | MainThread | [table_view:display:72] Cluster statistics +----------------+----------+-----------------+-----------+-----------+----------------------+-------------------+-----------------------+ | Node | Services | CPU_utilization | Mem_total | Mem_free | Swap_mem_used | Active / Replica | Version | +----------------+----------+-----------------+-----------+-----------+----------------------+-------------------+-----------------------+ | 172.23.105.206 | kv | 0.992462311558 | 11.45 GiB | 10.44 GiB | 23.59 MiB / 3.50 GiB | 0 / 0 | 7.1.0-2440-enterprise | | 172.23.106.177 | kv | 0.689914701455 | 11.45 GiB | 10.33 GiB | 23.15 MiB / 3.50 GiB | 0 / 0 | 7.1.0-2440-enterprise | | 172.23.100.35 | kv | 0.162846047852 | 11.45 GiB | 10.49 GiB | 0.0 Byte / 3.50 GiB | 0 / 0 | 7.1.0-2440-enterprise | | 172.23.105.164 | kv | 0.539387857501 | 11.45 GiB | 10.39 GiB | 0.0 Byte / 3.50 GiB | 0 / 0 | 7.1.0-2440-enterprise | +----------------+----------+-----------------+-----------+-----------+----------------------+-------------------+-----------------------+ 2022-03-06 11:55:22,049 | test | INFO | MainThread | [basetestcase:log_setup_status:647] ========= ClusterSetup setup complete for test #4 test_data_load_collections_with_hard_failover_recovery ========= 2022-03-06 11:55:22,049 | test | INFO | MainThread | [basetestcase:log_setup_status:647] ========= CollectionBase setup started for test #4 test_data_load_collections_with_hard_failover_recovery ========= 2022-03-06 11:55:22,184 | test | INFO | MainThread | [bucket_ready_functions:create_buckets_using_json_data:1913] Creating required buckets from template 2022-03-06 11:55:22,451 | test | INFO | MainThread | [bucket_ready_functions:wait_for_collection_creation_to_complete:2527] Waiting for all collections to be created 2022-03-06 11:55:22,729 | test | INFO | MainThread | [table_view:display:72] Bucket statistics +---------+-----------+-----------------+----------+------------+-----+-------+-----------+-----------+-----------+-----+ | Bucket | Type | Storage Backend | Replicas | Durability | TTL | Items | RAM Quota | RAM Used | Disk Used | ARR | +---------+-----------+-----------------+----------+------------+-----+-------+-----------+-----------+-----------+-----+ | bucket1 | couchbase | couchstore | 1 | none | 0 | 0 | 0.0 Byte | 0.0 Byte | 0.0 Byte | 100 | | bucket2 | couchbase | magma | 1 | none | 0 | 25738 | 4.00 GiB | 99.16 MiB | 0.0 Byte | 100 | | default | couchbase | magma | 1 | none | 0 | 0 | 0.0 Byte | 0.0 Byte | 0.0 Byte | 100 | +---------+-----------+-----------------+----------+------------+-----+-------+-----------+-----------+-----------+-----+ 2022-03-06 11:55:23,471 | test | INFO | MainThread | [common_lib:sleep:23] Sleep 30 seconds. Reason: Sleep while magma storage quota setting is taking effect 2022-03-06 11:55:53,471 | test | INFO | MainThread | [collections_base:collection_setup:174] Creating required SDK clients for client_pool 2022-03-06 11:55:59,221 | test | INFO | MainThread | [bucket_ready_functions:perform_tasks_from_spec:4993] Performing scope/collection specific operations 2022-03-06 11:55:59,227 | test | INFO | MainThread | [bucket_ready_functions:perform_tasks_from_spec:5083] Done Performing scope/collection specific operations 2022-03-06 12:55:33,526 | test | INFO | pool-7-thread-21 | [table_view:display:72] Ops trend for bucket 'bucket2' +---------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------+ | Min | Trend | Max | +---------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------+ | 0.000 | ***............................................................................*******X | 1800.000 | | 0.000 | *******************************************************************************************************************************************************************************************************************************************************..............................********......................................................................*******X | 2367.500 | | 0.000 | ********************************************************************************************************************************************************************************************************************************************************************.............................*******X | 700.000 | | 400.000 | ...............*******X | 400.000 | | 100.000 | ...*******X | 100.000 | | 0.000 | ***************************************************************************************************************************************************************************************************************************************************************************........********........********.........*******X | 600.000 | | 300.000 | ...........*******X | 300.000 | | 0.000 | **************************************************************************************************************************************************************************************************************************************************************************************....*******X | 100.000 | | 0.000 | ********................*******X | 400.000 | | 100.000 | ...******X | 100.000 | | 0.000 | ****************************************************************************************************************************************************************************************************************************************************************************************************....**************X | 100.000 | | 0.000 | ********........**************X | 200.000 | | 100.000 | ...*******X | 100.000 | | 0.000 | ***************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************..........*******.*******X | 261.500 | | 0.000 | *****************************************************************************************************************************************************************************************************************************************************........*******X | 200.000 | | 100.000 | ...*******X | 100.000 | | 0.000 | *****************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************............*******X | 300.000 | | 0.000 | **********************************************************************************************************************************************X | 0.000 | +---------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------+ 2022-03-06 12:55:34,063 | test | INFO | pool-7-thread-22 | [table_view:display:72] Ops trend for bucket 'bucket1' +---------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------+ | Min | Trend | Max | +---------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------+ | 0.000 | *************************************************************************************************************************************************************************************************************************************************************************......................................................*******X | 600.000 | | 0.000 | *********************************************************************************************************************************************************************************************************************************************************************..................********...........................*******X | 500.000 | | 200.000 | .................*******X | 200.000 | | 0.000 | **********************************************************************************************************************************************************************************************************************************************************************************.........********....................................*******X | 500.000 | | 400.000 | ...................................*******X | 400.000 | | 0.000 | ***************************************************************************************************************************************************************************************************************************************************************************************...............................................................******X | 700.000 | | 0.000 | ********.................................................................................*******X | 900.000 | | 100.000 | ........*******X | 100.000 | | 0.000 | ***********************************************************************************************************************************************************************************************************************************************************************************************************..................***************......................................................*******X | 802.300 | | 197.700 | ................******X | 197.700 | | 0.000 | **************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************.........********...........................................................................................******X | 1100.000 | | 600.000 | .....................................................*******X | 600.000 | | 0.000 | **************************************************************************************************************************************************************************************************************************************..................***************...........................*******X | 500.000 | | 0.000 | ***************************************************************************************************************************************************************************************************************************************.........*******.............................................................*....................******X | 1000.000 | | 300.000 | ..........................*******X | 300.000 | | 0.000 | ***************************************************************************************************************************************************************************************************************************************.........********.........*******X | 200.000 | | 0.000 | **********************************************************************************************************************************************X | 0.000 | +---------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------+ 2022-03-06 12:55:35,170 | test | INFO | pool-7-thread-23 | [table_view:display:72] Ops trend for bucket 'default' +-----------+--------------------------------------------------------------------------------------------------------------------------------------------------------+-----------+ | Min | Trend | Max | +-----------+--------------------------------------------------------------------------------------------------------------------------------------------------------+-----------+ | 0.000 | **........................********..........................................................................*******X | 86328.100 | | 81751.100 | ............................................................................................*******X | 81751.100 | | 79131.900 | .........................................................................................********.......*******X | 85393.500 | | 84695.700 | ................................................................................................********..***************X | 87202.600 | | 79510.200 | ..........................................................................................*******X | 79510.200 | | 73611.300 | ...................................................................................********.....********......********.....*******X | 87252.400 | | 86086.000 | .................................................................................................******X | 86086.000 | | 83616.000 | ..............................................................................................*******X | 83616.000 | | 78053.900 | ........................................................................................*******X | 78053.900 | | 75466.200 | .....................................................................................********......********...*******X | 82940.100 | | 82714.500 | .............................................................................................********...*******X | 85225.000 | | 84681.700 | ................................................................................................**************X | 84711.900 | | 81533.000 | ............................................................................................*******X | 81533.000 | | 78614.600 | .........................................................................................*******X | 78614.600 | | 75386.400 | .....................................................................................********..*******..*******X | 78799.000 | | 78200.200 | ........................................................................................*******X | 78200.200 | | 75117.800 | .....................................................................................*******X | 75117.800 | | 72846.300 | ..................................................................................*******X | 72846.300 | | 64059.100 | ........................................................................******X | 64059.100 | | 61830.600 | .....................................................................********....********...********.......*******...*******X | 76285.100 | | 73455.600 | ...................................................................................*******X | 73455.600 | | 64070.800 | ........................................................................*******X | 64070.800 | | 60809.800 | ....................................................................********.................******X | 75293.900 | | 73132.400 | ..................................................................................********.*******X | 74058.700 | | 71113.600 | ................................................................................*******X | 71113.600 | | 65867.800 | ..........................................................................********..............********.******X | 78930.700 | | 77068.000 | .......................................................................................********....*******X | 80988.600 | | 76274.200 | ......................................................................................*******X | 76274.200 | | 75252.400 | .....................................................................................********..*******.***************X | 78169.600 | | 77967.700 | ........................................................................................********....*******X | 81166.300 | | 80704.600 | ...........................................................................................*******X | 80704.600 | | 75201.100 | .....................................................................................*******..********.*******X | 77788.500 | | 72905.100 | ..................................................................................********....******X | 76212.600 | | 72545.200 | ..................................................................................********....*******X | 76450.200 | | 71456.500 | ................................................................................********.....******X | 75050.200 | | 72274.600 | .................................................................................********.*******X | 72539.800 | | 70052.500 | ...............................................................................********.......******X | 76020.100 | | 75175.700 | .....................................................................................*******X | 75175.700 | | 69708.000 | ..............................................................................*******X | 69708.000 | | 58225.200 | .................................................................*******......********.............********.******X | 75719.200 | | 74351.000 | ....................................................................................*******X | 74351.000 | | 62975.400 | .......................................................................*******X | 62975.400 | | 59969.900 | ...................................................................********..........*******..............*******X | 80741.700 | | 75163.400 | .....................................................................................*******X | 75163.400 | | 66692.700 | ...........................................................................*******X | 66692.700 | | 65891.400 | ..........................................................................******X | 65891.400 | | 62498.100 | ......................................................................X | 62498.100 | | 61387.300 | .....................................................................*******...........********.....*******X | 75715.300 | | 66739.400 | ...........................................................................********...******X | 69553.500 | | 69472.300 | ..............................................................................*******X | 69472.300 | | 69169.100 | ..............................................................................**************X | 69620.900 | | 65780.300 | ..........................................................................*******X | 65780.300 | | 65286.100 | .........................................................................*******X | 65286.100 | | 65268.600 | .........................................................................******X | 65268.600 | | 61975.400 | ......................................................................********....*******X | 66187.100 | | 58486.800 | ..................................................................*******...........*******X | 68718.400 | | 67370.400 | ............................................................................********......********.******X | 74002.900 | | 68179.000 | .............................................................................*******X | 68179.000 | | 47246.200 | .....................................................********...................********.........*******....********.*******X | 76316.600 | | 73617.600 | ...................................................................................******X | 73617.600 | | 69735.600 | ..............................................................................*******X | 69735.600 | | 64760.400 | .........................................................................*******........*******X | 71600.600 | | 67182.000 | ...........................................................................********......*******X | 71997.400 | | 69340.100 | ..............................................................................******X | 69340.100 | | 63196.000 | .......................................................................****************..******X | 65300.800 | | 57867.200 | .................................................................*******X | 57867.200 | | 48492.900 | ......................................................********..********....*******.......*..******X | 61425.300 | | 57910.400 | .................................................................********.........********......**************X | 71157.300 | | 68058.400 | .............................................................................*******X | 68058.400 | | 63232.100 | .......................................................................******X | 63232.100 | | 58631.500 | ..................................................................X | 58631.500 | | 57071.400 | ................................................................*******........********..................*******X | 80034.400 | | 70273.200 | ...............................................................................*******X | 70273.200 | | 56769.000 | ................................................................******X | 56769.000 | | 56364.400 | ...............................................................********.......********........********........******X | 76222.000 | | 72029.200 | .................................................................................*******X | 72029.200 | | 69357.800 | ..............................................................................*******X | 69357.800 | | 63478.500 | .......................................................................******X | 63478.500 | | 58750.900 | ..................................................................********........********.*******X | 66943.600 | | 63284.100 | .......................................................................******X | 63284.100 | | 62207.300 | ......................................................................********...........******X | 72232.700 | | 71358.100 | ................................................................................*******X | 71358.100 | | 67382.400 | ............................................................................********..*******X | 69063.500 | | 62875.700 | .......................................................................******X | 62875.700 | | 61780.000 | .....................................................................********......********......******X | 72231.800 | | 69395.900 | ..............................................................................*******X | 69395.900 | | 62877.800 | .......................................................................*******X | 62877.800 | | 57446.500 | ................................................................********..*******..........********...*******X | 70235.600 | | 62521.200 | ......................................................................******X | 62521.200 | | 57380.100 | ................................................................********.*******.********....*******X | 62237.500 | | 59799.500 | ...................................................................******X | 59799.500 | | 44890.300 | ..................................................********.........********...............*******..*******X | 67924.300 | | 64456.000 | ........................................................................******X | 64456.000 | | 62125.500 | ......................................................................******X | 62125.500 | | 58230.500 | .................................................................********.********...........******X | 68361.000 | | 59730.900 | ...................................................................*******X | 59730.900 | | 50453.700 | ........................................................******X | 50453.700 | | 49795.900 | ........................................................********.*******.............*******X | 62758.500 | | 58808.100 | ..................................................................*******X | 58808.100 | | 42520.900 | ...............................................*******............*******X | 52364.400 | | 51045.900 | .........................................................*******X | 51045.900 | | 42056.600 | ...............................................*******..............********..............*******..*******X | 68576.800 | | 60898.900 | ....................................................................*******X | 60898.900 | | 56651.100 | ...............................................................*******X | 56651.100 | | 50636.300 | .........................................................******X | 50636.300 | | 42201.000 | ...............................................********.....********....................******X | 64303.000 | | 48723.100 | ......................................................*******X | 48723.100 | | 45624.800 | ...................................................********..*******X | 47268.200 | | 43618.500 | ................................................*******........********..******X | 51590.100 | | 45224.700 | ..................................................********...********..********....******X | 52980.800 | | 51520.300 | ..........................................................********.....******X | 56042.400 | | 44824.600 | ..................................................*******X | 44824.600 | | 38684.800 | ...........................................*******X | 38684.800 | | 36037.100 | ........................................*******.........*...*******.............*******X | 57645.100 | | 53646.100 | ............................................................******X | 53646.100 | | 47657.400 | .....................................................********........********................*******X | 68300.700 | | 51202.800 | .........................................................*******.********..******X | 53817.900 | | 49464.800 | .......................................................*******X | 49464.800 | | 38731.400 | ...........................................********.....*******...........********........******X | 59853.700 | | 43259.000 | ................................................*******X | 43259.000 | | 33889.700 | .....................................*******...*******X | 36642.700 | | 34025.200 | .....................................********.........********.....******X | 45754.800 | | 45511.200 | ...................................................********...........******X | 55143.800 | | 39159.100 | ...........................................********.........*******X | 46589.500 | | 43519.100 | ................................................*******......*******X | 48399.800 | | 35869.100 | ........................................********..********..........*******.............*******X | 58253.200 | | 52875.600 | ...........................................................******X | 52875.600 | | 42492.100 | ...............................................********........********.................********.......********..*******X | 72120.700 | | 69917.000 | ...............................................................................******X | 69917.000 | | 63127.500 | .......................................................................********.........********.....********......********..*******.********.*******X | 84251.800 | | 83656.600 | ..............................................................................................********..***************X | 85479.100 | | 85162.000 | ................................................................................................******X | 85162.000 | | 83993.500 | ...............................................................................................*******X | 83993.500 | | 82531.400 | .............................................................................................********.*******X | 83216.000 | | 82621.800 | .............................................................................................********.****************..*******.*******X | 85881.100 | | 83441.000 | ..............................................................................................*******X | 83441.000 | | 82996.100 | ..............................................................................................****************.*******X | 84152.000 | | 82613.700 | .............................................................................................********....*******X | 85797.200 | | 83907.300 | ...............................................................................................*******.*******X | 85126.600 | | 83802.100 | ...............................................................................................*******X | 83802.100 | | 83221.800 | ..............................................................................................********..*******X | 84917.300 | | 84404.900 | ...............................................................................................*******X | 84404.900 | | 81952.200 | ............................................................................................********..*******..***************X | 84871.200 | | 83420.300 | ..............................................................................................*******X | 83420.300 | | 82835.300 | .............................................................................................*******X | 82835.300 | | 82587.800 | .............................................................................................********.******X | 82999.800 | | 82569.100 | .............................................................................................X | 82569.100 | | 82127.100 | .............................................................................................*******...*******X | 84961.900 | | 84325.200 | ...............................................................................................*******X | 84325.200 | | 82530.500 | .............................................................................................********..******X | 84265.900 | | 81922.400 | ............................................................................................********.*******X | 82175.400 | | 82069.900 | .............................................................................................***********************.*******X | 83297.600 | | 80823.500 | ...........................................................................................********...*******X | 83582.700 | | 83179.200 | ..............................................................................................********.******X | 84056.800 | | 83214.700 | ..............................................................................................*******X | 83214.700 | | 82694.900 | .............................................................................................*******X | 82694.900 | | 81140.000 | ...........................................................................................********....***************X | 84494.500 | | 82995.100 | ..............................................................................................*******X | 82995.100 | | 82865.800 | .............................................................................................*******..*******X | 84476.500 | | 82555.900 | .............................................................................................********.**************X | 83393.300 | | 83197.500 | ..............................................................................................X | 83197.500 | | 83051.000 | ..............................................................................................******X | 83051.000 | | 81235.600 | ............................................................................................********...*******X | 84311.000 | | 81079.500 | ...........................................................................................*******...********.*******X | 84418.300 | | 82085.700 | .............................................................................................*******X | 82085.700 | | 81745.100 | ............................................................................................*******X | 81745.100 | | 79404.000 | ..........................................................................................**************X | 80038.800 | | 79195.800 | .........................................................................................********..*******X | 80680.900 | | 78926.900 | .........................................................................................********.*******X | 79687.600 | | 78598.100 | .........................................................................................***************.********.*******X | 80425.500 | | 78119.300 | ........................................................................................********..*******X | 79663.600 | | 77917.200 | ........................................................................................********..********.*******.*******X | 81723.900 | | 81213.400 | ............................................................................................********.*******X | 82138.100 | | 78845.500 | .........................................................................................*******X | 78845.500 | | 77420.000 | .......................................................................................*******...********....***************X | 83533.700 | | 81043.300 | ...........................................................................................********.********.*******.*******X | 83297.700 | | 80423.900 | ...........................................................................................****************...******X | 82930.100 | | 82445.600 | .............................................................................................****************..*******X | 84315.800 | | 83196.300 | ..............................................................................................*******X | 83196.300 | | 82652.800 | .............................................................................................*******.********.*******X | 84118.900 | | 81943.700 | ............................................................................................*******X | 81943.700 | | 76437.600 | ......................................................................................X | 76437.600 | +-----------+--------------------------------------------------------------------------------------------------------------------------------------------------------+-----------+ 2022-03-06 12:55:37,651 | test | INFO | pool-7-thread-18 | [table_view:display:72] Ops trend for bucket 'default' +-----------+-------------------------------------------------------------------------------------------------------+-----------+ | Min | Trend | Max | +-----------+-------------------------------------------------------------------------------------------------------+-----------+ | 76437.600 | ...................................................................................................*X | 76437.600 | +-----------+-------------------------------------------------------------------------------------------------------+-----------+ 2022-03-06 12:55:37,750 | test | INFO | MainThread | [table_view:display:72] Cluster statistics +----------------+----------+-----------------+-----------+----------+----------------------+---------------------+-----------------------+ | Node | Services | CPU_utilization | Mem_total | Mem_free | Swap_mem_used | Active / Replica | Version | +----------------+----------+-----------------+-----------+----------+----------------------+---------------------+-----------------------+ | 172.23.105.206 | kv | 82.8116780642 | 11.45 GiB | 8.74 GiB | 23.59 MiB / 3.50 GiB | 62036893 / 61979706 | 7.1.0-2440-enterprise | | 172.23.106.177 | kv | 83.8364293086 | 11.45 GiB | 8.62 GiB | 23.15 MiB / 3.50 GiB | 62138442 / 62082846 | 7.1.0-2440-enterprise | | 172.23.100.35 | kv | 80.4232117357 | 11.45 GiB | 8.69 GiB | 0.0 Byte / 3.50 GiB | 62124469 / 62172799 | 7.1.0-2440-enterprise | | 172.23.105.164 | kv | 85.2918287938 | 11.45 GiB | 8.65 GiB | 0.0 Byte / 3.50 GiB | 62107364 / 62119033 | 7.1.0-2440-enterprise | +----------------+----------+-----------------+-----------+----------+----------------------+---------------------+-----------------------+ 2022-03-06 12:55:45,453 | test | INFO | MainThread | [bucket_ready_functions:validate_docs_per_collections_all_buckets:4742] Validating collection stats and item counts 2022-03-06 12:55:53,096 | test | ERROR | MainThread | [bucket_ready_functions:validate_manifest_uid:4605] Bucket UID mismatch. Expected: 0, Actual: 1 2022-03-06 12:55:53,096 | test | WARNING | MainThread | [bucket_ready_functions:validate_docs_per_collections_all_buckets:4759] Bucket manifest UID mismatch! 2022-03-06 12:55:56,954 | test | ERROR | MainThread | [bucket_ready_functions:validate_manifest_uid:4605] Bucket UID mismatch. Expected: 0, Actual: 1 2022-03-06 12:55:56,954 | test | WARNING | MainThread | [bucket_ready_functions:validate_docs_per_collections_all_buckets:4759] Bucket manifest UID mismatch! 2022-03-06 12:56:00,798 | test | ERROR | MainThread | [bucket_ready_functions:validate_manifest_uid:4605] Bucket UID mismatch. Expected: 0, Actual: 1 2022-03-06 12:56:00,798 | test | WARNING | MainThread | [bucket_ready_functions:validate_docs_per_collections_all_buckets:4759] Bucket manifest UID mismatch! 2022-03-06 12:56:10,845 | test | INFO | MainThread | [table_view:display:72] Bucket statistics +---------+-----------+-----------------+----------+------------+-----+-----------+-----------+------------+------------+-----------+ | Bucket | Type | Storage Backend | Replicas | Durability | TTL | Items | RAM Quota | RAM Used | Disk Used | ARR | +---------+-----------+-----------------+----------+------------+-----+-----------+-----------+------------+------------+-----------+ | bucket1 | couchbase | couchstore | 1 | none | 0 | 100000 | 7.81 GiB | 112.39 MiB | 94.71 MiB | 100 | | bucket2 | couchbase | magma | 1 | none | 0 | 100000 | 4.00 GiB | 285.35 MiB | 320.83 MiB | 100 | | default | couchbase | magma | 1 | none | 0 | 250000000 | 4.00 GiB | 2.12 GiB | 40.37 GiB | 3.1689192 | +---------+-----------+-----------------+----------+------------+-----+-----------+-----------+------------+------------+-----------+ 2022-03-06 12:56:10,845 | test | INFO | MainThread | [basetestcase:log_setup_status:647] ========= CollectionBase setup complete for test #4 test_data_load_collections_with_hard_failover_recovery ========= 2022-03-06 12:56:13,923 | test | INFO | MainThread | [collections_rebalance:load_collections_with_rebalance:1003] Doing collection data load during hard_failover_recovery 2022-03-06 12:56:13,923 | test | INFO | MainThread | [collections_rebalance:rebalance_operation:361] Starting rebalance operation of type : hard_failover_recovery 2022-03-06 12:56:13,923 | test | INFO | MainThread | [collections_rebalance:rebalance_operation:725] failing over nodes [ip:172.23.100.35 port:8091 ssh_username:root] 2022-03-06 12:56:22,845 | test | INFO | pool-7-thread-12 | [rest_client:monitorRebalance:1610] Rebalance done. Taken 8.03200006485 seconds to complete 2022-03-06 12:56:22,848 | test | INFO | pool-7-thread-12 | [common_lib:sleep:23] Sleep 8.03200006485 seconds. Reason: Wait after rebalance complete 2022-03-06 13:01:30,881 | test | INFO | MainThread | [collections_rebalance:data_load_after_failover:330] Starting a sync data load after failover 2022-03-06 13:01:30,881 | test | INFO | MainThread | [bucket_ready_functions:perform_tasks_from_spec:4993] Performing scope/collection specific operations 2022-03-06 13:01:34,427 | test | INFO | MainThread | [bucket_ready_functions:perform_tasks_from_spec:5083] Done Performing scope/collection specific operations 2022-03-06 13:42:31,061 | test | INFO | pool-7-thread-28 | [table_view:display:72] Ops trend for bucket 'bucket2' +---------+------------------------------------------------------------------------------------------------------------------------+---------+ | Min | Trend | Max | +---------+------------------------------------------------------------------------------------------------------------------------+---------+ | 0.000 | ****....................................................................................................******X | 724.000 | | 0.000 | ****************................................*******X | 232.000 | | 0.000 | *******....********............................................................................................******X | 700.000 | | 0.000 | ********......******X | 48.000 | | 0.000 | *******.............................................................................*******X | 564.000 | | 16.000 | .******X | 16.000 | | 0.000 | ********..*******.......................................................................*******X | 532.000 | | 164.000 | .....................******X | 164.000 | | 0.000 | ********.......******X | 52.000 | | 0.000 | ***********************.......******X | 52.000 | | 0.000 | ***************...............******X | 112.000 | | 68.000 | ........*******X | 68.000 | | 52.000 | ......******X | 52.000 | | 16.000 | .*******X | 16.000 | | 0.000 | *******.....................................................................******X | 500.000 | | 0.000 | ********..******X | 16.000 | | 0.000 | ********....*******..........................................................................*******X | 568.000 | | 52.000 | ......******X | 52.000 | | 0.000 | ***************.......................................................................******X | 520.000 | | 16.000 | .*******X | 16.000 | | 0.000 | **********************....*************X | 32.000 | | 16.000 | .******X | 16.000 | | 0.000 | ********..*******............................................................................******X | 568.000 | | 20.000 | .******X | 20.000 | | 13.400 | X | 13.400 | | 0.000 | *******............******X | 88.000 | | 0.000 | *******......*******X | 48.000 | | 0.000 | *******.....................................................................******X | 500.000 | | 0.000 | *******....*******X | 32.000 | | 20.000 | .*******X | 20.000 | | 0.000 | *******..*******X | 16.000 | | 0.000 | *******..*******X | 16.000 | | 0.000 | *************************************.........********.......*******X | 120.000 | | 0.000 | ****************.........*******X | 68.000 | | 0.000 | ***************....*******X | 36.000 | | 0.000 | ********..***************X | 20.000 | | 16.000 | .*******X | 16.000 | | 0.000 | ********.........*******X | 72.000 | | 0.000 | ********................*******X | 120.000 | | 16.000 | .*******....*******X | 48.000 | | 32.000 | ...********...***************X | 52.000 | | 16.000 | .****************............*******X | 108.000 | | 16.000 | .*******X | 16.000 | | 0.000 | ****************...........*******..*******X | 96.000 | | 76.000 | .........*******X | 76.000 | | 0.000 | ********....*******X | 36.000 | | 0.000 | ********..********.....*******X | 56.000 | | 0.000 | ********..............*******X | 104.000 | | 0.000 | ********..****************..*******X | 36.000 | | 30.900 | ...X | 30.900 | | 20.000 | .******X | 20.000 | | 0.000 | ************************........*******X | 64.000 | | 36.000 | ...*******X | 36.000 | | 0.000 | ****************..***************X | 16.000 | | 0.000 | ********..*******X | 16.000 | | 0.000 | ****************..***************X | 16.000 | | 0.000 | ********......*******X | 48.000 | | 16.000 | .***************X | 16.000 | | 0.000 | ************************.......*******X | 56.000 | | 0.000 | ********************************..***************X | 16.000 | | 0.000 | ************************..*******X | 16.000 | | 0.000 | ****************************************....*******X | 32.000 | | 0.000 | ********.....*******X | 40.000 | | 0.000 | ****************..*******X | 20.000 | | 0.000 | ****************....*******X | 36.000 | | 0.000 | *******....*******X | 36.000 | | 0.000 | ********..****************X | 20.000 | | 0.000 | ********..********..*******X | 36.000 | | 16.000 | .*******X | 16.000 | | 0.000 | ***************......*******X | 48.000 | | 0.000 | ********..*******X | 16.000 | | 0.000 | ********************************....*******X | 32.000 | | 16.000 | .********...*******X | 40.000 | | 0.000 | ****************..*******X | 16.000 | | 0.000 | ****************..........*******X | 76.000 | | 0.000 | ***********************..****************...*******X | 40.000 | | 20.000 | .*******X | 20.000 | | 0.000 | ****************.*******X | 10.200 | | 5.800 | ********..********..*******X | 36.000 | | 16.000 | .********..*******X | 36.000 | | 0.000 | ************************....*******X | 32.000 | | 0.000 | ********..*******X | 16.000 | | 0.000 | *******************************..*******X | 16.000 | | 0.000 | ************************..*******X | 16.000 | | 0.000 | ****************..***************X | 20.000 | | 0.000 | ****************************************..*******X | 16.000 | | 0.000 | *******....*******X | 32.000 | | 0.000 | ********..*******X | 16.000 | | 0.000 | ********......*******X | 48.000 | | 0.000 | ********....*******X | 36.000 | | 0.000 | ********..********....*******X | 48.000 | | 0.000 | ****************.......*******X | 52.000 | | 0.000 | ***************....********..*******X | 48.000 | | 0.000 | *********************X | 0.000 | +---------+------------------------------------------------------------------------------------------------------------------------+---------+ 2022-03-06 13:42:31,546 | test | INFO | pool-7-thread-16 | [table_view:display:72] Ops trend for bucket 'bucket1' +--------+----------------------------------------------------------------------------------------------------------------------------+---------+ | Min | Trend | Max | +--------+----------------------------------------------------------------------------------------------------------------------------+---------+ | 0.000 | ****...................******X | 116.000 | | 0.000 | ****************..............******X | 84.000 | | 0.000 | ****************..............................................................................................******X | 564.000 | | 0.000 | ********........******X | 48.000 | | 0.000 | ***************..******X | 16.000 | | 0.000 | ***************....................................................................................................******X | 596.000 | | 16.000 | .********......*******..*******X | 64.000 | | 0.000 | **********************.............*******X | 80.000 | | 0.000 | ***************........*******.................................................................................******X | 532.000 | | 16.000 | .***************X | 16.000 | | 0.000 | **************........*******X | 48.000 | | 0.000 | **********************................*******X | 100.000 | | 0.000 | **********************..........*******X | 64.000 | | 16.000 | .*************X | 16.000 | | 0.000 | *******..*******X | 16.000 | | 0.000 | *******.....******X | 32.000 | | 0.000 | ***************.........................................................................................******X | 532.000 | | 16.000 | .*******X | 16.000 | | 0.000 | *******......********........******X | 88.000 | | 0.000 | *******..*******X | 16.000 | | 0.000 | *******...******X | 20.000 | | 0.000 | *******..*******X | 16.000 | | 0.000 | ***************..*******.................................................................................*******X | 500.000 | | 16.000 | .******X | 16.000 | | 0.000 | ********..*******.................................................................................******X | 500.000 | | 0.000 | ****************........*******X | 48.000 | | 32.000 | ....*******X | 32.000 | | 0.000 | *******...********..........*******X | 80.000 | | 0.000 | ****************.....*******X | 32.000 | | 0.000 | ********.....*******X | 32.000 | | 16.000 | .**************X | 16.000 | | 0.000 | ****************........*******X | 48.000 | | 16.000 | .*******X | 16.000 | | 0.000 | ********......********....................*******X | 156.700 | | 63.300 | .........*******X | 63.300 | | 36.000 | .....*******X | 36.000 | | 0.000 | ****************........******X | 48.000 | | 16.000 | .*******X | 16.000 | | 0.000 | ********.....********...............*******X | 124.000 | | 72.000 | ...........*******X | 72.000 | | 0.000 | ****************...........*******X | 68.000 | | 0.000 | ****************............*******X | 72.000 | | 0.000 | ********.....*******X | 32.000 | | 16.000 | .*******X | 16.000 | | 0.000 | ****************.............*******X | 80.000 | | 36.000 | .....*******X | 36.000 | | 0.000 | ************************.......********...*******X | 60.900 | | 0.000 | ********..*******X | 16.000 | | 0.000 | ************************......*******X | 36.000 | | 16.000 | .****************....*******X | 40.000 | | 20.000 | ..*******X | 20.000 | | 0.000 | ****************........********....*******X | 72.000 | | 36.000 | .....*******X | 36.000 | | 0.000 | ****************..*******X | 16.000 | | 5.500 | X | 5.500 | | 0.000 | ***************..********...*******X | 32.000 | | 0.000 | ********...*******X | 20.000 | | 0.000 | ****************..*******X | 16.000 | | 0.000 | ********..********........*******X | 64.000 | | 0.000 | ********************************...***************X | 20.000 | | 0.000 | ********.*..*******......*******X | 56.000 | | 0.000 | ********......********.......*******X | 80.000 | | 20.000 | ..*******X | 20.000 | | 16.000 | .***************X | 16.000 | | 0.000 | ****************************************........*******X | 52.000 | | 0.000 | ****************............*******X | 72.000 | | 0.000 | ********......*******X | 36.000 | | 32.000 | ....*******X | 32.000 | | 0.000 | ***********************......********......*******X | 76.000 | | 0.000 | ************************......*******X | 40.000 | | 36.000 | .....*******X | 36.000 | | 16.000 | .*******X | 16.000 | | 0.000 | ************************...********..****************.********..*******X | 48.000 | | 16.000 | .*******X | 16.000 | | 0.000 | ************************.........*******X | 56.000 | | 16.000 | .*******X | 16.000 | | 0.000 | ********..******X | 16.000 | | 0.000 | ****************************************........*******X | 48.000 | | 0.000 | ********************************.............*******X | 80.000 | | 0.000 | ************************..********....*******X | 36.000 | | 20.000 | ..********..***************X | 32.000 | | 0.000 | ****************........******X | 48.000 | | 32.000 | ....*******X | 32.000 | | 16.000 | .****************...*******X | 32.000 | | 0.000 | ****************...*******X | 20.000 | | 0.000 | ********...*******X | 20.000 | | 0.000 | ********.....*******X | 32.000 | | 16.000 | .***************..............*******X | 100.000 | | 16.000 | .*******X | 16.000 | | 0.000 | ********..*******X | 16.000 | | 0.000 | ****************........*******X | 52.000 | | 48.000 | .......*******X | 48.000 | | 0.000 | *********************X | 0.000 | +--------+----------------------------------------------------------------------------------------------------------------------------+---------+ 2022-03-06 13:42:32,694 | test | INFO | pool-7-thread-3 | [table_view:display:72] Ops trend for bucket 'default' +------------+-------------------------------------------------------------------------------------------------------------------------+------------+ | Min | Trend | Max | +------------+-------------------------------------------------------------------------------------------------------------------------+------------+ | 0.000 | ****.........................................*******...........................................................*******X | 127308.900 | | 122572.500 | ...............................................................................................******X | 122572.500 | | 109858.900 | .....................................................................................*******X | 109858.900 | | 105146.000 | .................................................................................*******X | 105146.000 | | 100844.600 | ..............................................................................******X | 100844.600 | | 65196.700 | ..................................................*******X | 65196.700 | | 48706.200 | .....................................******X | 48706.200 | | 47983.400 | ....................................****X | 47983.400 | | 47983.400 | ....................................*X | 47983.400 | | 47554.300 | ....................................*******X | 47554.300 | | 46983.300 | ...................................*******.********.....*******X | 54493.700 | | 51816.600 | .......................................******X | 51816.600 | | 49800.600 | ......................................X | 49800.600 | | 49800.600 | ......................................**X | 49800.600 | | 49800.600 | ......................................**********X | 50098.800 | | 44722.500 | ..................................*************X | 45605.200 | | 44002.800 | .................................*********************X | 44452.200 | | 39231.600 | .............................******X | 39231.600 | | 37682.500 | ............................********..*******..*******..*******....********.......*******..*X | 61490.500 | | 61490.500 | ...............................................*****X | 61490.500 | | 55627.800 | ..........................................*X | 55627.800 | | 55627.800 | ..........................................**X | 55627.800 | | 55627.800 | ..........................................*X | 55627.800 | | 52196.200 | .......................................*******.*******X | 52893.800 | | 50754.300 | ......................................******X | 50754.300 | | 47617.100 | ....................................*******X | 47617.100 | | 41309.500 | ...............................*******..........***X | 53936.400 | | 53936.400 | .........................................***X | 53936.400 | | 50836.700 | ......................................*******.******X | 51498.900 | | 40091.000 | ..............................********....***X | 45434.100 | | 45434.100 | ..................................***..*******.******X | 48814.700 | | 48447.300 | .....................................*******X | 48447.300 | | 46152.200 | ...................................*******...********..**X | 52376.100 | | 52376.100 | ........................................*******************..*******.....********..******X | 64643.900 | | 61650.200 | ...............................................******X | 61650.200 | | 55295.500 | ..........................................**************X | 55572.800 | | 54264.900 | .........................................******X | 54264.900 | | 50202.500 | ......................................*******X | 50202.500 | | 48693.700 | .....................................******X | 48693.700 | | 46512.400 | ...................................*******X | 46512.400 | | 45222.300 | ..................................*******....******X | 50731.500 | | 46587.000 | ...................................********.*******......****X | 54956.400 | | 54956.400 | ..........................................**X | 54956.400 | | 54006.200 | .........................................******X | 54006.200 | | 50489.000 | ......................................*******..*******...*X | 56719.100 | | 56719.100 | ...........................................*****X | 56719.100 | | 54326.300 | .........................................********....*******X | 58971.200 | | 58503.500 | ............................................******X | 58503.500 | | 56257.900 | ...........................................*X | 56257.900 | | 56257.900 | ...........................................*X | 56257.900 | | 56257.900 | ...........................................**X | 56257.900 | | 56257.900 | ...........................................*********....****X | 61145.000 | | 61145.000 | ...............................................***********...........X | 76348.200 | | 76348.200 | ..........................................................*****X | 76348.200 | | 76348.200 | ..........................................................*..................*******.*******X | 100488.200 | | 89183.000 | .....................................................................*******X | 89183.000 | | 86755.600 | ...................................................................****X | 86755.600 | | 86755.600 | ...................................................................**X | 86755.600 | | 84716.000 | .................................................................*******X | 84716.000 | | 70580.700 | ......................................................********.***************....********.............*******X | 92965.300 | | 92083.300 | .......................................................................********.******X | 93381.700 | | 91703.100 | .......................................................................*******X | 91703.100 | | 89357.200 | .....................................................................********...*******X | 93454.900 | | 90828.700 | ......................................................................*******X | 90828.700 | | 89806.600 | .....................................................................*******....********..*******X | 97676.400 | | 89737.400 | .....................................................................X | 89737.400 | | 89737.400 | .....................................................................****X | 89737.400 | | 89737.400 | .....................................................................**...******X | 93686.500 | | 92476.800 | .......................................................................*X | 92476.800 | | 92476.800 | .......................................................................******.********....*******X | 98104.000 | | 81596.100 | ...............................................................X | 81596.100 | | 81596.100 | ...............................................................**X | 81596.100 | | 81596.100 | ...............................................................***X | 81596.100 | | 74613.500 | .........................................................********....*******X | 79542.400 | | 72696.000 | ........................................................*******...******X | 76389.600 | | 76389.600 | ...........................................................*.********.*******X | 80017.800 | | 73665.300 | ........................................................********......*******X | 80207.300 | | 75994.400 | ..........................................................***X | 75994.400 | | 75994.400 | ..........................................................***X | 75994.400 | | 68377.700 | ....................................................********........*******X | 77779.100 | | 73507.000 | ........................................................*******.********.*******X | 75324.300 | | 74743.600 | .........................................................*******X | 74743.600 | | 74077.800 | .........................................................********..********...*******X | 81296.000 | | 77378.000 | ...........................................................********.*******X | 78624.900 | | 75345.400 | ..........................................................*******X | 75345.400 | | 74655.300 | .........................................................*******X | 74655.300 | | 73555.700 | ........................................................*******X | 73555.700 | | 72254.000 | .......................................................********.*******X | 73471.400 | | 70604.400 | ......................................................***************X | 70852.900 | | 66546.900 | ...................................................********.*******X | 68400.000 | | 66715.600 | ...................................................********.*******X | 67991.000 | | 67406.600 | ...................................................*******X | 67406.600 | | 64517.400 | .................................................********..********.******X | 67723.700 | | 66926.600 | ...................................................X | 66926.600 | | 66377.400 | ...................................................******X | 66377.400 | | 64752.000 | .................................................********.....*******X | 70418.200 | | 66467.400 | ...................................................*******X | 66467.400 | | 62821.400 | ................................................*******...*******X | 66881.600 | | 66825.000 | ...................................................***************X | 66842.800 | | 66254.300 | ...................................................***************X | 66913.400 | | 65077.100 | ..................................................********..*******X | 67932.800 | | 67393.700 | ...................................................*******X | 67393.700 | | 62899.900 | ................................................********..*******X | 65513.000 | | 62866.300 | ................................................********..********..*******X | 68303.800 | | 64858.200 | .................................................*******X | 64858.200 | | 59070.500 | .............................................********......********.******X | 68208.700 | | 67795.700 | ....................................................********...*******X | 72010.500 | | 68555.700 | ....................................................********.*******X | 69671.400 | | 67101.400 | ...................................................*******X | 67101.400 | | 63408.400 | ................................................*******X | 63408.400 | | 62678.400 | ................................................*******X | 62678.400 | | 61767.700 | ...............................................*******X | 61767.700 | | 61442.900 | ...............................................*******X | 61442.900 | | 59231.800 | .............................................*******..*******X | 62192.100 | | 61942.700 | ...............................................*******X | 61942.700 | | 59387.700 | .............................................********.********...*******X | 64123.700 | | 60883.100 | ..............................................********...******X | 64018.100 | | 57789.200 | ............................................*******X | 57789.200 | | 56071.200 | ...........................................***************X | 56655.600 | | 56432.600 | ...........................................*******X | 56432.600 | | 54369.600 | .........................................*******X | 54369.600 | | 54046.900 | .........................................********.******X | 55612.100 | | 53745.300 | .........................................****************...*******X | 57609.100 | | 51605.000 | .......................................********......**************X | 59680.900 | | 56180.000 | ...........................................********..********.*******X | 61086.200 | | 60493.600 | ..............................................********..*******X | 62513.800 | | 59750.600 | .............................................********....******X | 64696.800 | | 63790.000 | .................................................*******X | 63790.000 | | 63057.900 | ................................................*******X | 63057.900 | | 61501.800 | ...............................................****************..******X | 64550.400 | | 64033.300 | .................................................********.*******X | 66092.300 | | 61809.600 | ...............................................*******X | 61809.600 | | 61461.500 | ...............................................********.***************.********.*******X | 66136.500 | | 63389.700 | ................................................*******X | 63389.700 | | 62379.900 | ...............................................*******X | 62379.900 | | 60855.100 | ..............................................********......*******X | 67712.100 | | 65459.900 | ..................................................*******X | 65459.900 | | 65227.200 | ..................................................*******..*******X | 68140.400 | | 62074.000 | ...............................................********.*******X | 63620.000 | | 62627.000 | ................................................********..******X | 65499.100 | | 62278.200 | ...............................................********...*******X | 65631.800 | | 62964.000 | ................................................********.*******X | 63968.700 | | 62639.300 | ................................................**********************X | 62990.400 | | 62867.900 | ................................................***************X | 63164.500 | | 59017.800 | .............................................*******..********.*******X | 63330.900 | | 59437.800 | .............................................*******X | 59437.800 | | 58109.400 | ............................................*******X | 58109.400 | | 57736.600 | ............................................*******.*.*******..*******X | 62681.500 | | 60083.200 | ..............................................*******X | 60083.200 | | 55589.600 | ..........................................********......********..******X | 66156.300 | | 64422.600 | .................................................********....********..********...*******X | 76169.200 | | 68828.900 | .....................................................*******X | 68828.900 | | 64053.100 | .................................................******X | 64053.100 | +------------+-------------------------------------------------------------------------------------------------------------------------+------------+ 2022-03-06 13:42:35,190 | test | INFO | pool-7-thread-25 | [table_view:display:72] Ops trend for bucket 'default' +-----------+------------------------------------------------------------------------------------------------------+-----------+ | Min | Trend | Max | +-----------+------------------------------------------------------------------------------------------------------+-----------+ | 64053.100 | ...................................................................................................X | 64053.100 | | 42243.500 | ................................................................X | 42243.500 | +-----------+------------------------------------------------------------------------------------------------------+-----------+ 2022-03-06 13:42:35,282 | test | WARNING | MainThread | [rest_client:get_nodes:1896] 172.23.100.35 - Node not part of cluster inactiveFailed 2022-03-06 13:42:49,671 | test | INFO | MainThread | [bucket_ready_functions:perform_tasks_from_spec:4993] Performing scope/collection specific operations 2022-03-06 13:42:49,821 | test | INFO | pool-7-thread-11 | [table_view:display:72] Rebalance Overview +----------------+----------+-----------------------+---------------+--------------+-----------------------+ | Nodes | Services | Version | CPU | Status | Membership / Recovery | +----------------+----------+-----------------------+---------------+--------------+-----------------------+ | 172.23.105.206 | kv | 7.1.0-2440-enterprise | 69.1447452027 | Cluster node | active / none | | 172.23.106.177 | kv | 7.1.0-2440-enterprise | 7.62467026755 | Cluster node | active / none | | 172.23.100.35 | kv | 7.1.0-2440-enterprise | 48.4732336783 | Cluster node | inactiveAdded / delta | | 172.23.105.164 | kv | 7.1.0-2440-enterprise | 12.7375707992 | Cluster node | active / none | +----------------+----------+-----------------------+---------------+--------------+-----------------------+ 2022-03-06 13:42:49,838 | test | INFO | pool-7-thread-11 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 13:42:53,165 | test | INFO | MainThread | [bucket_ready_functions:perform_tasks_from_spec:5083] Done Performing scope/collection specific operations 2022-03-06 13:42:59,871 | test | INFO | pool-7-thread-11 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 13:43:09,914 | test | INFO | pool-7-thread-11 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 13:43:19,957 | test | INFO | pool-7-thread-11 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 13:43:29,993 | test | INFO | pool-7-thread-11 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 13:43:40,026 | test | INFO | pool-7-thread-11 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 13:43:50,055 | test | INFO | pool-7-thread-11 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 13:44:00,078 | test | INFO | pool-7-thread-11 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 13:44:10,117 | test | INFO | pool-7-thread-11 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 13:44:20,153 | test | INFO | pool-7-thread-11 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 13:44:30,200 | test | INFO | pool-7-thread-11 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 13:44:40,252 | test | INFO | pool-7-thread-11 | [task:check:415] Rebalance - status: running, progress: 0.0 2022-03-06 13:44:50,283 | test | ERROR | pool-7-thread-11 | [rest_client:_rebalance_status_and_progress:1677] {u'errorMessage': u'Rebalance failed. See logs for detailed reason. You can try again.', u'type': u'rebalance', u'masterRequestTimedOut': False, u'statusId': u'246e388668684e5dab36c9e59f068280', u'subtype': u'rebalance', u'statusIsStale': False, u'lastReportURI': u'/logs/rebalanceReport?reportID=ff47d35308f52a99f8c4d1fb8a1d7a15', u'status': u'notRunning'} - rebalance failed 2022-03-06 13:44:50,322 | test | INFO | pool-7-thread-11 | [rest_client:print_UI_logs:2842] Latest logs from UI on 172.23.105.164: 2022-03-06 13:44:50,322 | test | ERROR | pool-7-thread-11 | [rest_client:print_UI_logs:2844] {u'code': 0, u'module': u'ns_orchestrator', u'type': u'critical', u'node': u'ns_1@172.23.105.164', u'tstamp': 1646603089791L, u'shortText': u'message', u'serverTime': u'2022-03-06T13:44:49.791Z', u'text': u'Rebalance exited with reason {buckets_shutdown_wait_failed,\n [{\'ns_1@172.23.100.35\',\n {\'EXIT\',\n {old_buckets_shutdown_wait_failed,\n ["default","bucket2"]}}}]}.\nRebalance Operation Id = e3e3a85f0e65f39a1b9eae473faeeed4'} 2022-03-06 13:44:50,322 | test | ERROR | pool-7-thread-11 | [rest_client:print_UI_logs:2844] {u'code': 0, u'module': u'ns_rebalancer', u'type': u'critical', u'node': u'ns_1@172.23.105.164', u'tstamp': 1646603089789L, u'shortText': u'message', u'serverTime': u'2022-03-06T13:44:49.789Z', u'text': u'Failed to wait deletion of some buckets on some nodes: [{\'ns_1@172.23.100.35\',\n {\'EXIT\',\n {old_buckets_shutdown_wait_failed,\n ["default",\n "bucket2"]}}}]\n'} 2022-03-06 13:44:50,323 | test | ERROR | pool-7-thread-11 | [rest_client:print_UI_logs:2844] {u'code': 0, u'module': u'ns_orchestrator', u'type': u'info', u'node': u'ns_1@172.23.105.164', u'tstamp': 1646602969784L, u'shortText': u'message', u'serverTime': u'2022-03-06T13:42:49.784Z', u'text': u"Starting rebalance, KeepNodes = ['ns_1@172.23.105.206','ns_1@172.23.106.177',\n 'ns_1@172.23.100.35','ns_1@172.23.105.164'], EjectNodes = [], Failed over and being ejected nodes = [], Delta recovery nodes = ['ns_1@172.23.100.35'], Delta recovery buckets = all; Operation Id = e3e3a85f0e65f39a1b9eae473faeeed4"} 2022-03-06 13:44:50,323 | test | ERROR | pool-7-thread-11 | [rest_client:print_UI_logs:2844] {u'code': 0, u'module': u'ns_orchestrator', u'type': u'info', u'node': u'ns_1@172.23.105.164', u'tstamp': 1646600174762L, u'shortText': u'message', u'serverTime': u'2022-03-06T12:56:14.762Z', u'text': u'Failover completed successfully.\nRebalance Operation Id = 29f343f24f75cfd315eb971c4e193d8d'} 2022-03-06 13:44:50,323 | test | ERROR | pool-7-thread-11 | [rest_client:print_UI_logs:2844] {u'code': 0, u'module': u'failover', u'type': u'info', u'node': u'ns_1@172.23.105.164', u'tstamp': 1646600174596L, u'shortText': u'message', u'serverTime': u'2022-03-06T12:56:14.596Z', u'text': u"Deactivating failed over nodes ['ns_1@172.23.100.35']"} 2022-03-06 13:44:50,323 | test | ERROR | pool-7-thread-11 | [rest_client:print_UI_logs:2844] {u'code': 0, u'module': u'failover', u'type': u'info', u'node': u'ns_1@172.23.105.164', u'tstamp': 1646600174590L, u'shortText': u'message', u'serverTime': u'2022-03-06T12:56:14.590Z', u'text': u"Failed over ['ns_1@172.23.100.35']: ok"} 2022-03-06 13:44:50,323 | test | ERROR | pool-7-thread-11 | [rest_client:print_UI_logs:2844] {u'code': 0, u'module': u'ns_memcached', u'type': u'info', u'node': u'ns_1@172.23.100.35', u'tstamp': 1646600174455L, u'shortText': u'message', u'serverTime': u'2022-03-06T12:56:14.455Z', u'text': u'Shutting down bucket "bucket2" on \'ns_1@172.23.100.35\' for deletion'} 2022-03-06 13:44:50,325 | test | ERROR | pool-7-thread-11 | [rest_client:print_UI_logs:2844] {u'code': 0, u'module': u'ns_memcached', u'type': u'info', u'node': u'ns_1@172.23.100.35', u'tstamp': 1646600174311L, u'shortText': u'message', u'serverTime': u'2022-03-06T12:56:14.311Z', u'text': u'Shutting down bucket "bucket1" on \'ns_1@172.23.100.35\' for deletion'} 2022-03-06 13:44:50,325 | test | ERROR | pool-7-thread-11 | [rest_client:print_UI_logs:2844] {u'code': 0, u'module': u'ns_orchestrator', u'type': u'info', u'node': u'ns_1@172.23.105.164', u'tstamp': 1646600173974L, u'shortText': u'message', u'serverTime': u'2022-03-06T12:56:13.974Z', u'text': u"Starting failover of nodes ['ns_1@172.23.100.35']. Operation Id = 29f343f24f75cfd315eb971c4e193d8d"} 2022-03-06 13:44:50,325 | test | ERROR | pool-7-thread-11 | [rest_client:print_UI_logs:2844] {u'code': 0, u'module': u'failover', u'type': u'info', u'node': u'ns_1@172.23.105.164', u'tstamp': 1646600173974L, u'shortText': u'message', u'serverTime': u'2022-03-06T12:56:13.974Z', u'text': u"Starting failing over ['ns_1@172.23.100.35']"} 2022-03-06 13:44:50,325 | test | ERROR | pool-7-thread-11 | [task:call:326] Rebalance Failed: {u'errorMessage': u'Rebalance failed. See logs for detailed reason. You can try again.', u'type': u'rebalance', u'masterRequestTimedOut': False, u'statusId': u'246e388668684e5dab36c9e59f068280', u'subtype': u'rebalance', u'statusIsStale': False, u'lastReportURI': u'/logs/rebalanceReport?reportID=ff47d35308f52a99f8c4d1fb8a1d7a15', u'status': u'notRunning'} - rebalance failed FAIL 2022-03-06 13:44:51,661 | test | INFO | MainThread | [table_view:display:72] Bucket statistics +---------+-----------+-----------------+----------+------------+-----+-----------+-----------+------------+------------+---------------+ | Bucket | Type | Storage Backend | Replicas | Durability | TTL | Items | RAM Quota | RAM Used | Disk Used | ARR | +---------+-----------+-----------------+----------+------------+-----+-----------+-----------+------------+------------+---------------+ | bucket1 | couchbase | couchstore | 1 | none | 0 | 82254 | 5.86 GiB | 70.63 MiB | 91.12 MiB | 100 | | bucket2 | couchbase | magma | 1 | none | 0 | 86268 | 3.00 GiB | 202.51 MiB | 299.41 MiB | 100 | | default | couchbase | magma | 1 | none | 0 | 217487702 | 3.00 GiB | 2.31 GiB | 41.89 GiB | 5.46960443768 | +---------+-----------+-----------------+----------+------------+-----+-----------+-----------+------------+------------+---------------+ 2022-03-06 13:44:52,213 | test | INFO | MainThread | [collections_base:tearDown:84] Bucket: bucket2, Active Resident ratio(DGM): 100% 2022-03-06 13:44:52,214 | test | INFO | MainThread | [collections_base:tearDown:87] Bucket: bucket2, Replica Resident ratio(DGM): 100% 2022-03-06 13:45:01,444 | test | INFO | MainThread | [collections_base:tearDown:84] Bucket: bucket1, Active Resident ratio(DGM): 100% 2022-03-06 13:45:01,444 | test | INFO | MainThread | [collections_base:tearDown:87] Bucket: bucket1, Replica Resident ratio(DGM): 100% 2022-03-06 13:45:10,276 | test | INFO | MainThread | [collections_base:tearDown:84] Bucket: default, Active Resident ratio(DGM): 5% 2022-03-06 13:45:10,278 | test | INFO | MainThread | [collections_base:tearDown:87] Bucket: default, Replica Resident ratio(DGM): 0% 2022-03-06 13:45:32,029 | test | ERROR | pool-7-thread-3 | [task:call:515] 'default' 2022-03-06 13:45:32,032 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,032 | test | ERROR | pool-7-thread-3 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,033 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,035 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,036 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,036 | test | ERROR | pool-7-thread-3 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,036 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,038 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,039 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,039 | test | ERROR | pool-7-thread-3 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,040 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,040 | test | ERROR | pool-7-thread-3 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,042 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,042 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,043 | test | ERROR | pool-7-thread-3 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,043 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,045 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,046 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,046 | test | ERROR | pool-7-thread-3 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,046 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,048 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,049 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,049 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,051 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,052 | test | ERROR | pool-7-thread-3 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,052 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,052 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,053 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,055 | test | ERROR | pool-7-thread-3 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,055 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,055 | test | ERROR | pool-7-thread-3 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,055 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,056 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,058 | test | ERROR | pool-7-thread-3 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,059 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,059 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,059 | test | ERROR | pool-7-thread-3 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,061 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,062 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,062 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,063 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,065 | test | ERROR | pool-7-thread-3 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,065 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,065 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,066 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,068 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,069 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,069 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,071 | test | ERROR | pool-7-thread-3 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,072 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,072 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,073 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,075 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,075 | test | ERROR | pool-7-thread-3 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,075 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,076 | test | ERROR | pool-7-thread-3 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,076 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,078 | test | ERROR | pool-7-thread-3 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,078 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,078 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,079 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,081 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,082 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,082 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,084 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,084 | test | ERROR | pool-7-thread-3 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,085 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,085 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,086 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,088 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,088 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,089 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,091 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,092 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,092 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,094 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,095 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,095 | test | ERROR | pool-7-thread-3 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,095 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,096 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,096 | test | ERROR | pool-7-thread-3 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,098 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,098 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,098 | test | ERROR | pool-7-thread-3 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,099 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,099 | test | ERROR | pool-7-thread-3 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,101 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,101 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,105 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,107 | test | ERROR | pool-7-thread-3 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,108 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,108 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,109 | test | ERROR | pool-7-thread-3 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,109 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,111 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,111 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,112 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,117 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,117 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,119 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,121 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,121 | test | ERROR | pool-7-thread-30 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,121 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,121 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,122 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,122 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,124 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,124 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,124 | test | ERROR | pool-7-thread-27 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,125 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,125 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,125 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,125 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,125 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,127 | test | ERROR | pool-7-thread-30 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,127 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,127 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,128 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,128 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,128 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,128 | test | ERROR | pool-7-thread-3 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,128 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,128 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,130 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,130 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,131 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,131 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,131 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,131 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,131 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,132 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,132 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,132 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,134 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,134 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,134 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,134 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,134 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,135 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,135 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,137 | test | ERROR | pool-7-thread-3 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,137 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,137 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,138 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,138 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,138 | test | ERROR | pool-7-thread-30 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,138 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,138 | test | ERROR | pool-7-thread-27 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,138 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,138 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,140 | test | ERROR | pool-7-thread-30 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,140 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,140 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,140 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,141 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,141 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,141 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,141 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,141 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,141 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,142 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,142 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,144 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,144 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,144 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,144 | test | ERROR | pool-7-thread-30 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,145 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,147 | test | ERROR | pool-7-thread-27 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,147 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,148 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,148 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,148 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,148 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,148 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,150 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,151 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,151 | test | ERROR | pool-7-thread-30 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,151 | test | ERROR | pool-7-thread-3 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,151 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,151 | test | ERROR | pool-7-thread-2 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,151 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,151 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,151 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,151 | test | ERROR | pool-7-thread-30 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,151 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,153 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,153 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,153 | test | ERROR | pool-7-thread-27 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,153 | test | ERROR | pool-7-thread-30 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,153 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,154 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,154 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,154 | test | ERROR | pool-7-thread-2 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,154 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,154 | test | ERROR | pool-7-thread-27 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,154 | test | ERROR | pool-7-thread-3 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,154 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,154 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,154 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,155 | test | ERROR | pool-7-thread-2 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,155 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,155 | test | ERROR | pool-7-thread-27 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,155 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,155 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,155 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,157 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,157 | test | ERROR | pool-7-thread-27 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,157 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,157 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,157 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,157 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,160 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,160 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,160 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,160 | test | ERROR | pool-7-thread-2 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,161 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,161 | test | ERROR | pool-7-thread-3 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,161 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,161 | test | ERROR | pool-7-thread-27 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,161 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,161 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,161 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,163 | test | ERROR | pool-7-thread-3 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,163 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,163 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,163 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,164 | test | ERROR | pool-7-thread-2 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,164 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,164 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,164 | test | ERROR | pool-7-thread-30 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,164 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,165 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,165 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,167 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,167 | test | ERROR | pool-7-thread-27 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,167 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,167 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,167 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,168 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,168 | test | ERROR | pool-7-thread-2 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,168 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,168 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,168 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,170 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,170 | test | ERROR | pool-7-thread-30 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,170 | test | ERROR | pool-7-thread-27 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,170 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,170 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,170 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,171 | test | ERROR | pool-7-thread-3 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,171 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,171 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,171 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,171 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,171 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,173 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,173 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,173 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,173 | test | ERROR | pool-7-thread-3 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,174 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,174 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,174 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,174 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,174 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,174 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,176 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,176 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,176 | test | ERROR | pool-7-thread-30 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,176 | test | ERROR | pool-7-thread-3 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,177 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,177 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,177 | test | ERROR | pool-7-thread-2 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,177 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,177 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,177 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,177 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,177 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,178 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,178 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,178 | test | ERROR | pool-7-thread-3 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,178 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,180 | test | ERROR | pool-7-thread-2 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,180 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,180 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,180 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,180 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,180 | test | ERROR | pool-7-thread-2 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,180 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,181 | test | ERROR | pool-7-thread-27 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,181 | test | ERROR | pool-7-thread-30 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,181 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,181 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,181 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,183 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,183 | test | ERROR | pool-7-thread-3 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,183 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,183 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,184 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,184 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,184 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,184 | test | ERROR | pool-7-thread-30 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,184 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,184 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,186 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,186 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,186 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,186 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,187 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,187 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,187 | test | ERROR | pool-7-thread-3 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,187 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,187 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,188 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,188 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,188 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,190 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,190 | test | ERROR | pool-7-thread-2 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,190 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,190 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,190 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,190 | test | ERROR | pool-7-thread-3 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,190 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,190 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,191 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,191 | test | ERROR | pool-7-thread-30 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,191 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,191 | test | ERROR | pool-7-thread-3 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,191 | test | ERROR | pool-7-thread-27 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,191 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,193 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,193 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,193 | test | ERROR | pool-7-thread-27 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,193 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,194 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,194 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,194 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,194 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,194 | test | ERROR | pool-7-thread-27 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,194 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,194 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,196 | test | ERROR | pool-7-thread-30 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,196 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,196 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,196 | test | ERROR | pool-7-thread-27 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,196 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,197 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,197 | test | ERROR | pool-7-thread-30 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,197 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,197 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,197 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,197 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,197 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,198 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,198 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,198 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,198 | test | ERROR | pool-7-thread-30 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,200 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,200 | test | ERROR | pool-7-thread-27 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,200 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,200 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,200 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,200 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,200 | test | ERROR | pool-7-thread-3 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,200 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,201 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,201 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,201 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,201 | test | ERROR | pool-7-thread-3 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,203 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,203 | test | ERROR | pool-7-thread-2 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,203 | test | ERROR | pool-7-thread-30 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,203 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,203 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,203 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,203 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,203 | test | ERROR | pool-7-thread-27 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,203 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,203 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,204 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,204 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,204 | test | ERROR | pool-7-thread-27 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,206 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,206 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,206 | test | ERROR | pool-7-thread-30 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,206 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,206 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,207 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,207 | test | ERROR | pool-7-thread-2 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,207 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,207 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,207 | test | ERROR | pool-7-thread-27 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,207 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,207 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,207 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,209 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,209 | test | ERROR | pool-7-thread-30 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,209 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,209 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,210 | test | ERROR | pool-7-thread-2 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,210 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,210 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,210 | test | ERROR | pool-7-thread-30 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,210 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,210 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,210 | test | ERROR | pool-7-thread-2 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,210 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,210 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,210 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,210 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,211 | test | ERROR | pool-7-thread-27 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,211 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,213 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,213 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,213 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,229 | test | ERROR | pool-7-thread-30 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,229 | test | ERROR | pool-7-thread-2 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,230 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,230 | test | ERROR | pool-7-thread-27 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,230 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,230 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,230 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,232 | test | ERROR | pool-7-thread-2 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,233 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,233 | test | ERROR | pool-7-thread-3 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,233 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,233 | test | ERROR | pool-7-thread-27 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,233 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,233 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,236 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,236 | test | ERROR | pool-7-thread-2 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,236 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,236 | test | ERROR | pool-7-thread-27 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,236 | test | ERROR | pool-7-thread-3 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,236 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,236 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,237 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,239 | test | ERROR | pool-7-thread-2 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,239 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,240 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,240 | test | ERROR | pool-7-thread-27 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,240 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,240 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,240 | test | ERROR | pool-7-thread-2 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,240 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,242 | test | ERROR | pool-7-thread-3 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,242 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,242 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,242 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,242 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,242 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,243 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,243 | test | ERROR | pool-7-thread-13 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,243 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,243 | test | ERROR | pool-7-thread-25 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,243 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,243 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,244 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,243 | test | ERROR | pool-7-thread-3 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,243 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,244 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,246 | test | ERROR | pool-7-thread-13 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,246 | test | ERROR | pool-7-thread-2 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,246 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,246 | test | ERROR | pool-7-thread-25 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,246 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,246 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,246 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,246 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,246 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,246 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,247 | test | ERROR | pool-7-thread-13 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,247 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,249 | test | ERROR | pool-7-thread-27 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,249 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,249 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,249 | test | ERROR | pool-7-thread-25 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,249 | test | ERROR | pool-7-thread-30 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,249 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,249 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,249 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,249 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,250 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,250 | test | ERROR | pool-7-thread-25 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,250 | test | ERROR | pool-7-thread-27 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,250 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,250 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,250 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,250 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,250 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,252 | test | ERROR | pool-7-thread-2 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,252 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,252 | test | ERROR | pool-7-thread-25 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,252 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,253 | test | ERROR | pool-7-thread-27 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,253 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,253 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,253 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,253 | test | ERROR | pool-7-thread-2 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,253 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,253 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,253 | test | ERROR | pool-7-thread-27 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,253 | test | ERROR | pool-7-thread-25 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,255 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,255 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,255 | test | ERROR | pool-7-thread-30 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,255 | test | ERROR | pool-7-thread-3 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,255 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,255 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,255 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,255 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,256 | test | ERROR | pool-7-thread-25 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,256 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,256 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,256 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,256 | test | ERROR | pool-7-thread-2 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,256 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,256 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,256 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,257 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,257 | test | ERROR | pool-7-thread-25 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,257 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,259 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,259 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,259 | test | ERROR | pool-7-thread-13 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,259 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,259 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,259 | test | ERROR | pool-7-thread-3 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,259 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,260 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,260 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,260 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,260 | test | ERROR | pool-7-thread-13 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,262 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,262 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,262 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,262 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,262 | test | ERROR | pool-7-thread-2 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,263 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,263 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,263 | test | ERROR | pool-7-thread-13 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,263 | test | ERROR | pool-7-thread-3 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,263 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,263 | test | ERROR | pool-7-thread-30 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,263 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,263 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,263 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,263 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,265 | test | ERROR | pool-7-thread-27 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,265 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,265 | test | ERROR | pool-7-thread-13 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,265 | test | ERROR | pool-7-thread-3 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,266 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,266 | test | ERROR | pool-7-thread-30 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,266 | test | ERROR | pool-7-thread-2 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,266 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,266 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,266 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,266 | test | ERROR | pool-7-thread-27 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,266 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,266 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,266 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,266 | test | ERROR | pool-7-thread-30 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,267 | test | ERROR | pool-7-thread-3 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,267 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,267 | test | ERROR | pool-7-thread-2 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,267 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,267 | test | ERROR | pool-7-thread-27 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,267 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,267 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,267 | test | ERROR | pool-7-thread-13 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,269 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,269 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,269 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,269 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,269 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,269 | test | ERROR | pool-7-thread-25 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,269 | test | ERROR | pool-7-thread-27 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,269 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,269 | test | ERROR | pool-7-thread-13 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,269 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,270 | test | ERROR | pool-7-thread-30 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,270 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,270 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,270 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,270 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,272 | test | ERROR | pool-7-thread-25 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,272 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,272 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,272 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,273 | test | ERROR | pool-7-thread-2 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,273 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,273 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,273 | test | ERROR | pool-7-thread-30 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,273 | test | ERROR | pool-7-thread-27 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,273 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,273 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,273 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,273 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,273 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,273 | test | ERROR | pool-7-thread-2 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,275 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,275 | test | ERROR | pool-7-thread-25 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,275 | test | ERROR | pool-7-thread-27 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,275 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,275 | test | ERROR | pool-7-thread-13 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,275 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,275 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,276 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,276 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,276 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,276 | test | ERROR | pool-7-thread-25 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,276 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,276 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,278 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,278 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,278 | test | ERROR | pool-7-thread-2 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,278 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,278 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,278 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,278 | test | ERROR | pool-7-thread-30 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,279 | test | ERROR | pool-7-thread-3 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,279 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,279 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,279 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,279 | test | ERROR | pool-7-thread-2 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,279 | test | ERROR | pool-7-thread-13 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,279 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,279 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,279 | test | ERROR | pool-7-thread-30 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,279 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,279 | test | ERROR | pool-7-thread-27 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,279 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,279 | test | ERROR | pool-7-thread-3 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,279 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,280 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,280 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,280 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,282 | test | ERROR | pool-7-thread-25 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,282 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,282 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,282 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,282 | test | ERROR | pool-7-thread-2 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,282 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,282 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,282 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,282 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,283 | test | ERROR | pool-7-thread-3 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,283 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,283 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,285 | test | ERROR | pool-7-thread-30 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,285 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,285 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,285 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,285 | test | ERROR | pool-7-thread-25 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,286 | test | ERROR | pool-7-thread-3 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,286 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,286 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,286 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,286 | test | ERROR | pool-7-thread-30 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,286 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,286 | test | ERROR | pool-7-thread-13 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,286 | test | ERROR | pool-7-thread-2 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,286 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,286 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,286 | test | ERROR | pool-7-thread-27 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,286 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,288 | test | ERROR | pool-7-thread-3 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,288 | test | ERROR | pool-7-thread-25 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,288 | test | ERROR | pool-7-thread-30 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,288 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,288 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,288 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,288 | test | ERROR | pool-7-thread-13 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,288 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,289 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,289 | test | ERROR | pool-7-thread-27 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,289 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,289 | test | ERROR | pool-7-thread-25 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,289 | test | ERROR | pool-7-thread-3 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,289 | test | ERROR | pool-7-thread-13 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,289 | test | ERROR | pool-7-thread-30 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,289 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,289 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,289 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,289 | test | ERROR | pool-7-thread-2 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,289 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,289 | test | ERROR | pool-7-thread-27 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,290 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,290 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,292 | test | ERROR | pool-7-thread-3 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,292 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,292 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,292 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,292 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,292 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,292 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,293 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,293 | test | ERROR | pool-7-thread-2 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,293 | test | ERROR | pool-7-thread-3 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,293 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,293 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,293 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,293 | test | ERROR | pool-7-thread-25 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,293 | test | ERROR | pool-7-thread-27 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,295 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,295 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,295 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,295 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,296 | test | ERROR | pool-7-thread-3 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,296 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,296 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,296 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,296 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,296 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,296 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,298 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,298 | test | ERROR | pool-7-thread-27 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,298 | test | ERROR | pool-7-thread-30 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,298 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,298 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,298 | test | ERROR | pool-7-thread-13 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,298 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,298 | test | ERROR | pool-7-thread-2 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,299 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,299 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,299 | test | ERROR | pool-7-thread-3 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,299 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,299 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,299 | test | ERROR | pool-7-thread-25 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,299 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,299 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,299 | test | ERROR | pool-7-thread-2 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,299 | test | ERROR | pool-7-thread-13 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,299 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,301 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,301 | test | ERROR | pool-7-thread-3 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,301 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,301 | test | ERROR | pool-7-thread-30 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,301 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,301 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,301 | test | ERROR | pool-7-thread-25 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,302 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,302 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,302 | test | ERROR | pool-7-thread-2 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,302 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,302 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,302 | test | ERROR | pool-7-thread-3 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,302 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,302 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,303 | test | ERROR | pool-7-thread-25 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,303 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,303 | test | ERROR | pool-7-thread-13 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,305 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,305 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,305 | test | ERROR | pool-7-thread-2 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,305 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,305 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,305 | test | ERROR | pool-7-thread-3 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,305 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,306 | test | ERROR | pool-7-thread-25 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,306 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,306 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,306 | test | ERROR | pool-7-thread-27 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,306 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,306 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,306 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,306 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,308 | test | ERROR | pool-7-thread-25 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,308 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,309 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,309 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,309 | test | ERROR | pool-7-thread-30 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,309 | test | ERROR | pool-7-thread-27 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,309 | test | ERROR | pool-7-thread-3 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,309 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,309 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,309 | test | ERROR | pool-7-thread-2 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,309 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,309 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,311 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,309 | test | ERROR | pool-7-thread-28 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,309 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,311 | test | ERROR | pool-7-thread-27 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,312 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,312 | test | ERROR | pool-7-thread-30 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,312 | test | ERROR | pool-7-thread-13 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,312 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,312 | test | ERROR | pool-7-thread-3 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,312 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,312 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,312 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,312 | test | ERROR | pool-7-thread-25 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,312 | test | ERROR | pool-7-thread-28 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,312 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,312 | test | ERROR | pool-7-thread-28 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,312 | test | ERROR | pool-7-thread-27 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,313 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,313 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,313 | test | ERROR | pool-7-thread-13 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,315 | test | ERROR | pool-7-thread-3 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,315 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,315 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,315 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,315 | test | ERROR | pool-7-thread-25 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,315 | test | ERROR | pool-7-thread-28 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,315 | test | ERROR | pool-7-thread-27 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,315 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,315 | test | ERROR | pool-7-thread-28 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,315 | test | ERROR | pool-7-thread-2 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,315 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,315 | test | ERROR | pool-7-thread-30 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,315 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,315 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,315 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,316 | test | ERROR | pool-7-thread-3 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,316 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,316 | test | ERROR | pool-7-thread-25 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,316 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,316 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,316 | test | ERROR | pool-7-thread-28 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,318 | test | ERROR | pool-7-thread-2 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,318 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,318 | test | ERROR | pool-7-thread-13 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,318 | test | ERROR | pool-7-thread-3 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,318 | test | ERROR | pool-7-thread-30 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,318 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,319 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,319 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,319 | test | ERROR | pool-7-thread-25 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,319 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,319 | test | ERROR | pool-7-thread-27 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,319 | test | ERROR | pool-7-thread-28 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,319 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,319 | test | ERROR | pool-7-thread-28 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,319 | test | ERROR | pool-7-thread-2 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,319 | test | ERROR | pool-7-thread-13 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,319 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,319 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,321 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,321 | test | ERROR | pool-7-thread-25 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,321 | test | ERROR | pool-7-thread-30 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,321 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,321 | test | ERROR | pool-7-thread-27 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,321 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,321 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,322 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,322 | test | ERROR | pool-7-thread-28 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,322 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,322 | test | ERROR | pool-7-thread-28 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,322 | test | ERROR | pool-7-thread-3 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,322 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,322 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,322 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,322 | test | ERROR | pool-7-thread-27 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,322 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,323 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,323 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,323 | test | ERROR | pool-7-thread-28 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,325 | test | ERROR | pool-7-thread-25 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,325 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,325 | test | ERROR | pool-7-thread-28 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,325 | test | ERROR | pool-7-thread-3 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,325 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,325 | test | ERROR | pool-7-thread-13 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,325 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,325 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,325 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,325 | test | ERROR | pool-7-thread-30 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,325 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,325 | test | ERROR | pool-7-thread-28 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,326 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,326 | test | ERROR | pool-7-thread-13 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,326 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,326 | test | ERROR | pool-7-thread-3 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,326 | test | ERROR | pool-7-thread-25 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,326 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,328 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,328 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,328 | test | ERROR | pool-7-thread-30 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,328 | test | ERROR | pool-7-thread-2 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,328 | test | ERROR | pool-7-thread-28 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,328 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,328 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,328 | test | ERROR | pool-7-thread-27 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,328 | test | ERROR | pool-7-thread-28 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,328 | test | ERROR | pool-7-thread-13 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,328 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,328 | test | ERROR | pool-7-thread-3 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,328 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,328 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,329 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,329 | test | ERROR | pool-7-thread-2 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,329 | test | ERROR | pool-7-thread-30 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,331 | test | ERROR | pool-7-thread-28 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,331 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,331 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,331 | test | ERROR | pool-7-thread-28 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,331 | test | ERROR | pool-7-thread-27 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,331 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,331 | test | ERROR | pool-7-thread-13 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,331 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,331 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,332 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,332 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,332 | test | ERROR | pool-7-thread-28 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,332 | test | ERROR | pool-7-thread-28 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,332 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,332 | test | ERROR | pool-7-thread-13 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,332 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,332 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,332 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,334 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,334 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,334 | test | ERROR | pool-7-thread-28 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,334 | test | ERROR | pool-7-thread-28 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,334 | test | ERROR | pool-7-thread-27 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,335 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,335 | test | ERROR | pool-7-thread-3 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,335 | test | ERROR | pool-7-thread-30 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,335 | test | ERROR | pool-7-thread-13 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,335 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,335 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,335 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,335 | test | ERROR | pool-7-thread-2 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,335 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,335 | test | ERROR | pool-7-thread-28 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,335 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,335 | test | ERROR | pool-7-thread-27 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,335 | test | ERROR | pool-7-thread-28 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,336 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,336 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,336 | test | ERROR | pool-7-thread-3 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,338 | test | ERROR | pool-7-thread-13 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,338 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,338 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,338 | test | ERROR | pool-7-thread-2 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,338 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,338 | test | ERROR | pool-7-thread-27 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,338 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,338 | test | ERROR | pool-7-thread-28 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,338 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,338 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,338 | test | ERROR | pool-7-thread-28 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,338 | test | ERROR | pool-7-thread-3 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,338 | test | ERROR | pool-7-thread-13 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,338 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,338 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,339 | test | ERROR | pool-7-thread-2 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,339 | test | ERROR | pool-7-thread-25 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,339 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,339 | test | ERROR | pool-7-thread-28 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,339 | test | ERROR | pool-7-thread-27 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,339 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,341 | test | ERROR | pool-7-thread-28 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,339 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,341 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,341 | test | ERROR | pool-7-thread-13 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,341 | test | ERROR | pool-7-thread-3 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,341 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,341 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,342 | test | ERROR | pool-7-thread-2 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,342 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,342 | test | ERROR | pool-7-thread-25 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,342 | test | ERROR | pool-7-thread-30 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,342 | test | ERROR | pool-7-thread-28 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,342 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,342 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,342 | test | ERROR | pool-7-thread-28 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,342 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,342 | test | ERROR | pool-7-thread-27 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,342 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,342 | test | ERROR | pool-7-thread-2 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,342 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,344 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,344 | test | ERROR | pool-7-thread-30 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,344 | test | ERROR | pool-7-thread-25 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,344 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,344 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,344 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,345 | test | ERROR | pool-7-thread-13 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,345 | test | ERROR | pool-7-thread-27 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,345 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,345 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,345 | test | ERROR | pool-7-thread-28 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,345 | test | ERROR | pool-7-thread-3 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,345 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,345 | test | ERROR | pool-7-thread-30 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,345 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,345 | test | ERROR | pool-7-thread-2 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,345 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,346 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,346 | test | ERROR | pool-7-thread-13 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,346 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,346 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,348 | test | ERROR | pool-7-thread-13 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,348 | test | ERROR | pool-7-thread-2 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,348 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,348 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,348 | test | ERROR | pool-7-thread-25 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,348 | test | ERROR | pool-7-thread-28 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,348 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,348 | test | ERROR | pool-7-thread-28 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,348 | test | ERROR | pool-7-thread-13 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,348 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,348 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,349 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,349 | test | ERROR | pool-7-thread-28 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,349 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,349 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,346 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,351 | test | ERROR | pool-7-thread-3 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,351 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,351 | test | ERROR | pool-7-thread-25 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,351 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,351 | test | ERROR | pool-7-thread-27 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,351 | test | ERROR | pool-7-thread-13 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,351 | test | ERROR | pool-7-thread-2 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,351 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,352 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,352 | test | ERROR | pool-7-thread-25 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,351 | test | ERROR | pool-7-thread-28 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,351 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,351 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,351 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,352 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,352 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,352 | test | ERROR | pool-7-thread-28 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,352 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,355 | test | ERROR | pool-7-thread-27 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,355 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,355 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,355 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,355 | test | ERROR | pool-7-thread-13 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,355 | test | ERROR | pool-7-thread-28 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,355 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,355 | test | ERROR | pool-7-thread-25 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,355 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,355 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,357 | test | ERROR | pool-7-thread-30 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,357 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,357 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,357 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,357 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,357 | test | ERROR | pool-7-thread-28 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,357 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,358 | test | ERROR | pool-7-thread-25 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,358 | test | ERROR | pool-7-thread-28 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,358 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,358 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,358 | test | ERROR | pool-7-thread-27 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,358 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,358 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,358 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,358 | test | ERROR | pool-7-thread-3 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,359 | test | ERROR | pool-7-thread-28 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,359 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,359 | test | ERROR | pool-7-thread-25 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,359 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,359 | test | ERROR | pool-7-thread-28 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,359 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,359 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,359 | test | ERROR | pool-7-thread-13 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,361 | test | ERROR | pool-7-thread-30 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,361 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,361 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,361 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,361 | test | ERROR | pool-7-thread-2 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,361 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,361 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,361 | test | ERROR | pool-7-thread-28 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,362 | test | ERROR | pool-7-thread-27 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,362 | test | ERROR | pool-7-thread-25 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,362 | test | ERROR | pool-7-thread-28 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,362 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,362 | test | ERROR | pool-7-thread-3 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,362 | test | ERROR | pool-7-thread-30 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,362 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,362 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,362 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,362 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,364 | test | ERROR | pool-7-thread-13 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,364 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,364 | test | ERROR | pool-7-thread-28 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,364 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,365 | test | ERROR | pool-7-thread-28 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,365 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,365 | test | ERROR | pool-7-thread-25 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,365 | test | ERROR | pool-7-thread-3 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,365 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,365 | test | ERROR | pool-7-thread-13 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,365 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,365 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,365 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,365 | test | ERROR | pool-7-thread-28 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,365 | test | ERROR | pool-7-thread-2 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,365 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,365 | test | ERROR | pool-7-thread-28 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,365 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,367 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,367 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,367 | test | ERROR | pool-7-thread-30 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,367 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,368 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,368 | test | ERROR | pool-7-thread-28 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,368 | test | ERROR | pool-7-thread-2 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,368 | test | ERROR | pool-7-thread-28 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,368 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,368 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,368 | test | ERROR | pool-7-thread-13 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,368 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,368 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,369 | test | ERROR | pool-7-thread-27 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,369 | test | ERROR | pool-7-thread-13 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,369 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,369 | test | ERROR | pool-7-thread-25 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,371 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,369 | test | ERROR | pool-7-thread-28 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,371 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,371 | test | ERROR | pool-7-thread-30 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,371 | test | ERROR | pool-7-thread-2 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,371 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,371 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,371 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,371 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,371 | test | ERROR | pool-7-thread-28 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,371 | test | ERROR | pool-7-thread-28 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,371 | test | ERROR | pool-7-thread-13 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,372 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,372 | test | ERROR | pool-7-thread-3 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,372 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,372 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,372 | test | ERROR | pool-7-thread-30 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,372 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,374 | test | ERROR | pool-7-thread-25 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,374 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,374 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,374 | test | ERROR | pool-7-thread-28 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,374 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,375 | test | ERROR | pool-7-thread-2 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,375 | test | ERROR | pool-7-thread-27 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,375 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,375 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,375 | test | ERROR | pool-7-thread-30 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,375 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,375 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,375 | test | ERROR | pool-7-thread-28 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,375 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,375 | test | ERROR | pool-7-thread-13 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,377 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,377 | test | ERROR | pool-7-thread-2 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,377 | test | ERROR | pool-7-thread-27 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,377 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,377 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,378 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,378 | test | ERROR | pool-7-thread-30 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,378 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,378 | test | ERROR | pool-7-thread-2 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,378 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,378 | test | ERROR | pool-7-thread-25 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,378 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,378 | test | ERROR | pool-7-thread-28 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,378 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,378 | test | ERROR | pool-7-thread-3 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,378 | test | ERROR | pool-7-thread-27 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,380 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,380 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,380 | test | ERROR | pool-7-thread-30 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,380 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,381 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,381 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,381 | test | ERROR | pool-7-thread-25 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,381 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,381 | test | ERROR | pool-7-thread-28 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,381 | test | ERROR | pool-7-thread-27 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,381 | test | ERROR | pool-7-thread-3 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,381 | test | ERROR | pool-7-thread-28 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,381 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,381 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,381 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,382 | test | ERROR | pool-7-thread-2 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,382 | test | ERROR | pool-7-thread-30 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,382 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,382 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,382 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,384 | test | ERROR | pool-7-thread-13 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,384 | test | ERROR | pool-7-thread-28 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,384 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,384 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,384 | test | ERROR | pool-7-thread-2 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,384 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,384 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,384 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,384 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,385 | test | ERROR | pool-7-thread-28 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,385 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,385 | test | ERROR | pool-7-thread-27 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,385 | test | ERROR | pool-7-thread-2 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,385 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,385 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,387 | test | ERROR | pool-7-thread-3 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,387 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,387 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,388 | test | ERROR | pool-7-thread-28 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,388 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,388 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,388 | test | ERROR | pool-7-thread-2 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,388 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,388 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,388 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,388 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,390 | test | ERROR | pool-7-thread-28 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,390 | test | ERROR | pool-7-thread-13 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,390 | test | ERROR | pool-7-thread-25 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,390 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,390 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,390 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,390 | test | ERROR | pool-7-thread-2 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,391 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,391 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,391 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,391 | test | ERROR | pool-7-thread-28 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,391 | test | ERROR | pool-7-thread-13 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,391 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,392 | test | ERROR | pool-7-thread-2 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,392 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,392 | test | ERROR | pool-7-thread-13 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,392 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,392 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,394 | test | ERROR | pool-7-thread-3 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,394 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,394 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,394 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,395 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,394 | test | ERROR | pool-7-thread-28 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,395 | test | ERROR | pool-7-thread-27 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,397 | test | ERROR | pool-7-thread-30 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,397 | test | ERROR | pool-7-thread-28 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,397 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,397 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,398 | test | ERROR | pool-7-thread-13 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,398 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,398 | test | ERROR | pool-7-thread-27 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,400 | test | ERROR | pool-7-thread-2 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,398 | test | ERROR | pool-7-thread-30 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,400 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,400 | test | ERROR | pool-7-thread-25 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,400 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,400 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,401 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,401 | test | ERROR | pool-7-thread-28 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,401 | test | ERROR | pool-7-thread-13 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,401 | test | ERROR | pool-7-thread-3 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,401 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,401 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,401 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,401 | test | ERROR | pool-7-thread-28 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,403 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,404 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,404 | test | ERROR | pool-7-thread-30 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,404 | test | ERROR | pool-7-thread-27 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,404 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,404 | test | ERROR | pool-7-thread-25 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,404 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,404 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,404 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,404 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,405 | test | ERROR | pool-7-thread-2 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,405 | test | ERROR | pool-7-thread-28 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,407 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,407 | test | ERROR | pool-7-thread-28 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,407 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,407 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,407 | test | ERROR | pool-7-thread-25 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,407 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,407 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,407 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,408 | test | ERROR | pool-7-thread-3 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,410 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,410 | test | ERROR | pool-7-thread-28 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,410 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,410 | test | ERROR | pool-7-thread-2 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,411 | test | ERROR | pool-7-thread-28 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,411 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,411 | test | ERROR | pool-7-thread-27 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,411 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,411 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,411 | test | ERROR | pool-7-thread-25 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,411 | test | ERROR | pool-7-thread-30 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,411 | test | ERROR | pool-7-thread-2 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,411 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,411 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,413 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,413 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,413 | test | ERROR | pool-7-thread-3 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,413 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,414 | test | ERROR | pool-7-thread-15 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,414 | test | ERROR | pool-7-thread-22 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,414 | test | ERROR | pool-7-thread-28 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,414 | test | ERROR | pool-7-thread-15 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,414 | test | ERROR | pool-7-thread-28 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,414 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,414 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,414 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,414 | test | ERROR | pool-7-thread-27 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,415 | test | ERROR | pool-7-thread-15 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,415 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,415 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,415 | test | ERROR | pool-7-thread-28 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,415 | test | ERROR | pool-7-thread-25 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,415 | test | ERROR | pool-7-thread-15 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,415 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,415 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,417 | test | ERROR | pool-7-thread-22 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,417 | test | ERROR | pool-7-thread-13 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,417 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,417 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,417 | test | ERROR | pool-7-thread-30 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,418 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,418 | test | ERROR | pool-7-thread-15 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,418 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,418 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,418 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,418 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,418 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,418 | test | ERROR | pool-7-thread-28 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,418 | test | ERROR | pool-7-thread-15 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,420 | test | ERROR | pool-7-thread-22 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,420 | test | ERROR | pool-7-thread-22 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,421 | test | ERROR | pool-7-thread-13 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,421 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,421 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,421 | test | ERROR | pool-7-thread-2 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,421 | test | ERROR | pool-7-thread-27 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,421 | test | ERROR | pool-7-thread-28 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,421 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,421 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,421 | test | ERROR | pool-7-thread-28 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,421 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,423 | test | ERROR | pool-7-thread-22 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,423 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,423 | test | ERROR | pool-7-thread-15 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,423 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,424 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,424 | test | ERROR | pool-7-thread-22 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,424 | test | ERROR | pool-7-thread-27 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,424 | test | ERROR | pool-7-thread-28 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,424 | test | ERROR | pool-7-thread-2 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,424 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,424 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,424 | test | ERROR | pool-7-thread-30 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,426 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,426 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,426 | test | ERROR | pool-7-thread-15 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,426 | test | ERROR | pool-7-thread-3 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,427 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,427 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,427 | test | ERROR | pool-7-thread-2 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,427 | test | ERROR | pool-7-thread-30 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,427 | test | ERROR | pool-7-thread-22 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,427 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,427 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,427 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,427 | test | ERROR | pool-7-thread-28 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,427 | test | ERROR | pool-7-thread-27 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,427 | test | ERROR | pool-7-thread-15 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,427 | test | ERROR | pool-7-thread-28 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,427 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,427 | test | ERROR | pool-7-thread-22 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,428 | test | ERROR | pool-7-thread-25 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,430 | test | ERROR | pool-7-thread-2 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,430 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,430 | test | ERROR | pool-7-thread-3 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,430 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,430 | test | ERROR | pool-7-thread-15 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,430 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,430 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,430 | test | ERROR | pool-7-thread-30 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,430 | test | ERROR | pool-7-thread-15 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,430 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,430 | test | ERROR | pool-7-thread-28 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,430 | test | ERROR | pool-7-thread-22 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,430 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,430 | test | ERROR | pool-7-thread-22 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,431 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,433 | test | ERROR | pool-7-thread-22 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,434 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,434 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,434 | test | ERROR | pool-7-thread-3 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,434 | test | ERROR | pool-7-thread-15 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,434 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,434 | test | ERROR | pool-7-thread-15 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,436 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,436 | test | ERROR | pool-7-thread-15 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,437 | test | ERROR | pool-7-thread-3 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,437 | test | ERROR | pool-7-thread-15 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,437 | test | ERROR | pool-7-thread-28 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,437 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,437 | test | ERROR | pool-7-thread-13 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,437 | test | ERROR | pool-7-thread-22 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,437 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,437 | test | ERROR | pool-7-thread-30 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,437 | test | ERROR | pool-7-thread-27 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,438 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,438 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,438 | test | ERROR | pool-7-thread-15 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,438 | test | ERROR | pool-7-thread-25 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,440 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,440 | test | ERROR | pool-7-thread-2 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,440 | test | ERROR | pool-7-thread-22 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,440 | test | ERROR | pool-7-thread-28 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,440 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,440 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,440 | test | ERROR | pool-7-thread-22 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,440 | test | ERROR | pool-7-thread-13 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,441 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,441 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,441 | test | ERROR | pool-7-thread-15 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,443 | test | ERROR | pool-7-thread-15 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,443 | test | ERROR | pool-7-thread-25 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,444 | test | ERROR | pool-7-thread-22 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,444 | test | ERROR | pool-7-thread-28 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,444 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,444 | test | ERROR | pool-7-thread-22 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,443 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,444 | test | ERROR | pool-7-thread-28 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,444 | test | ERROR | pool-7-thread-28 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,444 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,446 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,444 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,446 | test | ERROR | pool-7-thread-2 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,446 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,446 | test | ERROR | pool-7-thread-22 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,447 | test | ERROR | pool-7-thread-3 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,447 | test | ERROR | pool-7-thread-15 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,447 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,447 | test | ERROR | pool-7-thread-2 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,447 | test | ERROR | pool-7-thread-16 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,447 | test | ERROR | pool-7-thread-27 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,447 | test | ERROR | pool-7-thread-22 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,447 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,447 | test | ERROR | pool-7-thread-28 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,447 | test | ERROR | pool-7-thread-13 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,448 | test | ERROR | pool-7-thread-28 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,448 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,448 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,448 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,448 | test | ERROR | pool-7-thread-15 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,450 | test | ERROR | pool-7-thread-25 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,450 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,450 | test | ERROR | pool-7-thread-3 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,450 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,450 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,450 | test | ERROR | pool-7-thread-16 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,450 | test | ERROR | pool-7-thread-22 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,450 | test | ERROR | pool-7-thread-23 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,450 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,450 | test | ERROR | pool-7-thread-16 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,450 | test | ERROR | pool-7-thread-22 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,451 | test | ERROR | pool-7-thread-28 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,451 | test | ERROR | pool-7-thread-13 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,453 | test | ERROR | pool-7-thread-15 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,454 | test | ERROR | pool-7-thread-15 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,454 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,454 | test | ERROR | pool-7-thread-27 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,457 | test | ERROR | pool-7-thread-30 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,454 | test | ERROR | pool-7-thread-3 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,454 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,457 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,457 | test | ERROR | pool-7-thread-25 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,457 | test | ERROR | pool-7-thread-23 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,457 | test | ERROR | pool-7-thread-29 | [task:execute_tasks:3946] Failed to load 200 docs from 392000 to 545600 of thread_name LoadDocs_MutateDocsFromSpecTask_1646602976.36_default_scope-156_collection-1769_ttl=0_read__392000_545600_1646602976.41 2022-03-06 13:45:32,457 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,459 | test | ERROR | pool-7-thread-23 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,457 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,457 | test | ERROR | pool-7-thread-16 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,459 | test | ERROR | pool-7-thread-28 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,457 | test | ERROR | pool-7-thread-22 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,457 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,459 | test | ERROR | pool-7-thread-15 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,459 | test | ERROR | pool-7-thread-2 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,459 | test | ERROR | pool-7-thread-16 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,459 | test | ERROR | pool-7-thread-22 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,459 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,459 | test | ERROR | pool-7-thread-28 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,460 | test | INFO | pool-7-thread-9 | [table_view:display:72] Ops trend for bucket 'bucket2' +-------+----------------------------------------------------------------------------------------------------------------------------+--------+ | Min | Trend | Max | +-------+----------------------------------------------------------------------------------------------------------------------------+--------+ | 0.000 | *****....................*******X | 15.300 | | 0.000 | **********************************************....................*******........................................******X | 45.900 | | 0.000 | *******....................********................................................................................******X | 76.500 | | 0.000 | *********************X | 0.000 | +-------+----------------------------------------------------------------------------------------------------------------------------+--------+ 2022-03-06 13:45:32,460 | test | ERROR | pool-7-thread-23 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,460 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,460 | test | ERROR | pool-7-thread-23 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,460 | test | ERROR | pool-7-thread-15 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,460 | test | ERROR | pool-7-thread-9 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,460 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,461 | test | ERROR | pool-7-thread-13 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,461 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,463 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,463 | test | ERROR | pool-7-thread-9 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,463 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,463 | test | ERROR | pool-7-thread-23 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,463 | test | ERROR | pool-7-thread-9 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,463 | test | ERROR | pool-7-thread-15 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,463 | test | ERROR | pool-7-thread-23 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,463 | test | ERROR | pool-7-thread-15 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,463 | test | ERROR | pool-7-thread-22 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,463 | test | ERROR | pool-7-thread-16 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,463 | test | ERROR | pool-7-thread-28 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,463 | test | ERROR | pool-7-thread-30 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,463 | test | ERROR | pool-7-thread-25 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,463 | test | ERROR | pool-7-thread-28 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,464 | test | ERROR | pool-7-thread-2 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,464 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,464 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,464 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,464 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,466 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,466 | test | ERROR | pool-7-thread-9 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,466 | test | ERROR | pool-7-thread-15 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,466 | test | ERROR | pool-7-thread-9 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,466 | test | ERROR | pool-7-thread-15 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,467 | test | ERROR | pool-7-thread-16 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,467 | test | ERROR | pool-7-thread-27 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,467 | test | ERROR | pool-7-thread-16 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,467 | test | ERROR | pool-7-thread-22 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,467 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,467 | test | ERROR | pool-7-thread-2 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,467 | test | ERROR | pool-7-thread-23 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,467 | test | ERROR | pool-7-thread-22 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,467 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,467 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,467 | test | ERROR | pool-7-thread-28 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,467 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,467 | test | ERROR | pool-7-thread-9 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,467 | test | ERROR | pool-7-thread-25 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,469 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,469 | test | ERROR | pool-7-thread-27 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,469 | test | ERROR | pool-7-thread-22 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,469 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,470 | test | ERROR | pool-7-thread-2 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,470 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,470 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,470 | test | ERROR | pool-7-thread-15 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,470 | test | ERROR | pool-7-thread-16 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,470 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,470 | test | ERROR | pool-7-thread-22 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,470 | test | ERROR | pool-7-thread-23 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,470 | test | ERROR | pool-7-thread-16 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,470 | test | ERROR | pool-7-thread-9 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,470 | test | ERROR | pool-7-thread-3 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,471 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,471 | test | ERROR | pool-7-thread-2 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,473 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,473 | test | ERROR | pool-7-thread-22 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,473 | test | ERROR | pool-7-thread-9 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,473 | test | ERROR | pool-7-thread-22 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,473 | test | ERROR | pool-7-thread-9 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,471 | test | ERROR | pool-7-thread-28 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,473 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,473 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,474 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,474 | test | ERROR | pool-7-thread-27 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,474 | test | ERROR | pool-7-thread-16 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,474 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,476 | test | ERROR | pool-7-thread-15 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,476 | test | ERROR | pool-7-thread-2 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,476 | test | ERROR | pool-7-thread-15 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,476 | test | ERROR | pool-7-thread-16 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,476 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,476 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,476 | test | ERROR | pool-7-thread-23 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,476 | test | ERROR | pool-7-thread-25 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,476 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,476 | test | ERROR | pool-7-thread-22 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,476 | test | ERROR | pool-7-thread-23 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,476 | test | ERROR | pool-7-thread-9 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,476 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,479 | test | ERROR | pool-7-thread-28 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,479 | test | ERROR | pool-7-thread-27 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,477 | test | ERROR | pool-7-thread-2 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,479 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,479 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,479 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,479 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,479 | test | ERROR | pool-7-thread-1 | [task:call:515] 'NoneType' object has no attribute 'get_multi' 2022-03-06 13:45:32,480 | test | ERROR | pool-7-thread-15 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,480 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,480 | test | ERROR | pool-7-thread-15 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,480 | test | ERROR | pool-7-thread-25 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,480 | test | ERROR | pool-7-thread-23 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,480 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,480 | test | ERROR | pool-7-thread-22 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,480 | test | ERROR | pool-7-thread-23 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,480 | test | ERROR | pool-7-thread-22 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,480 | test | ERROR | pool-7-thread-9 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,482 | test | ERROR | pool-7-thread-28 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,482 | test | ERROR | pool-7-thread-30 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,482 | test | ERROR | pool-7-thread-13 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,482 | test | ERROR | pool-7-thread-28 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,482 | test | ERROR | pool-7-thread-9 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,482 | test | ERROR | pool-7-thread-1 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,482 | test | ERROR | pool-7-thread-16 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,482 | test | ERROR | pool-7-thread-2 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,482 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,482 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,483 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,483 | test | ERROR | pool-7-thread-1 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,483 | test | ERROR | pool-7-thread-27 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,483 | test | ERROR | pool-7-thread-15 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,483 | test | ERROR | pool-7-thread-16 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,483 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,483 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,483 | test | ERROR | pool-7-thread-22 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,483 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,483 | test | ERROR | pool-7-thread-22 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,483 | test | ERROR | pool-7-thread-23 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,483 | test | ERROR | pool-7-thread-28 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,483 | test | ERROR | pool-7-thread-23 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,484 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,484 | test | ERROR | pool-7-thread-9 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,486 | test | ERROR | pool-7-thread-9 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,486 | test | ERROR | pool-7-thread-25 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,486 | test | ERROR | pool-7-thread-13 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,486 | test | ERROR | pool-7-thread-23 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,486 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,486 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,486 | test | ERROR | pool-7-thread-23 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,486 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,486 | test | ERROR | pool-7-thread-2 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,486 | test | ERROR | pool-7-thread-15 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,486 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,486 | test | ERROR | pool-7-thread-15 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,486 | test | ERROR | pool-7-thread-27 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,486 | test | ERROR | pool-7-thread-1 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,487 | test | ERROR | pool-7-thread-22 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,487 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,487 | test | ERROR | pool-7-thread-16 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,487 | test | ERROR | pool-7-thread-9 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,487 | test | ERROR | pool-7-thread-16 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,487 | test | ERROR | pool-7-thread-9 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,489 | test | ERROR | pool-7-thread-30 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,487 | test | ERROR | pool-7-thread-22 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,489 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,489 | test | ERROR | pool-7-thread-13 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,489 | test | ERROR | pool-7-thread-28 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,489 | test | ERROR | pool-7-thread-1 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,489 | test | ERROR | pool-7-thread-3 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,490 | test | ERROR | pool-7-thread-28 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,490 | test | ERROR | pool-7-thread-15 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,490 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,490 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,490 | test | ERROR | pool-7-thread-25 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,490 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,490 | test | ERROR | pool-7-thread-30 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,492 | test | ERROR | pool-7-thread-22 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,492 | test | ERROR | pool-7-thread-16 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,492 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,492 | test | ERROR | pool-7-thread-23 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,492 | test | ERROR | pool-7-thread-1 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,492 | test | ERROR | pool-7-thread-22 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,492 | test | ERROR | pool-7-thread-2 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,492 | test | ERROR | pool-7-thread-28 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,492 | test | ERROR | pool-7-thread-27 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,492 | test | ERROR | pool-7-thread-3 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,492 | test | ERROR | pool-7-thread-16 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,493 | test | ERROR | pool-7-thread-13 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,493 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,493 | test | ERROR | pool-7-thread-28 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,493 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,493 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,493 | test | ERROR | pool-7-thread-23 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,493 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,493 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,493 | test | ERROR | pool-7-thread-22 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,493 | test | ERROR | pool-7-thread-1 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,493 | test | ERROR | pool-7-thread-22 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,494 | test | ERROR | pool-7-thread-15 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,494 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,494 | test | ERROR | pool-7-thread-15 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,494 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,494 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,496 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,496 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,496 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,496 | test | ERROR | pool-7-thread-9 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,496 | test | ERROR | pool-7-thread-15 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,496 | test | ERROR | pool-7-thread-16 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,496 | test | ERROR | pool-7-thread-2 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,496 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,497 | test | ERROR | pool-7-thread-13 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,497 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,497 | test | ERROR | pool-7-thread-16 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,497 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,497 | test | ERROR | pool-7-thread-16 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,499 | test | ERROR | pool-7-thread-25 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,499 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,499 | test | ERROR | pool-7-thread-13 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,500 | test | ERROR | pool-7-thread-28 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,500 | test | ERROR | pool-7-thread-23 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,500 | test | ERROR | pool-7-thread-15 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,500 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,500 | test | ERROR | pool-7-thread-27 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,500 | test | ERROR | pool-7-thread-1 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,500 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,500 | test | ERROR | pool-7-thread-28 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,500 | test | ERROR | pool-7-thread-16 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,500 | test | ERROR | pool-7-thread-23 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,500 | test | ERROR | pool-7-thread-22 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,500 | test | ERROR | pool-7-thread-16 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,500 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,500 | test | ERROR | pool-7-thread-9 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,503 | test | ERROR | pool-7-thread-13 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,503 | test | ERROR | pool-7-thread-2 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,500 | test | ERROR | pool-7-thread-1 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,500 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,500 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,503 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,500 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,503 | test | ERROR | pool-7-thread-23 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,503 | test | ERROR | pool-7-thread-28 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,503 | test | ERROR | pool-7-thread-9 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,503 | test | ERROR | pool-7-thread-23 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,503 | test | ERROR | pool-7-thread-28 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,503 | test | ERROR | pool-7-thread-15 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,503 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,503 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,503 | test | ERROR | pool-7-thread-16 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,503 | test | ERROR | pool-7-thread-22 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,505 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,505 | test | ERROR | pool-7-thread-30 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,505 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,506 | test | ERROR | pool-7-thread-25 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,506 | test | ERROR | pool-7-thread-2 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,506 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,506 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,506 | test | ERROR | pool-7-thread-1 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,506 | test | ERROR | pool-7-thread-23 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,506 | test | ERROR | pool-7-thread-13 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,506 | test | ERROR | pool-7-thread-16 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,507 | test | ERROR | pool-7-thread-9 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,506 | test | ERROR | pool-7-thread-15 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,507 | test | ERROR | pool-7-thread-22 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,507 | test | ERROR | pool-7-thread-30 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,507 | test | ERROR | pool-7-thread-23 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,507 | test | ERROR | pool-7-thread-28 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,507 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,507 | test | ERROR | pool-7-thread-16 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,507 | test | ERROR | pool-7-thread-27 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,507 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,507 | test | ERROR | pool-7-thread-9 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,507 | test | ERROR | pool-7-thread-3 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,509 | test | ERROR | pool-7-thread-28 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,509 | test | ERROR | pool-7-thread-1 | [task:batch_update:626] 'NoneType' object has no attribute 'upsert_multi' 2022-03-06 13:45:32,509 | test | ERROR | pool-7-thread-2 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,509 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,509 | test | ERROR | pool-7-thread-25 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,509 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,509 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,509 | test | ERROR | pool-7-thread-1 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,509 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,509 | test | ERROR | pool-7-thread-15 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,509 | test | ERROR | pool-7-thread-16 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,509 | test | ERROR | pool-7-thread-15 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,509 | test | ERROR | pool-7-thread-23 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,510 | test | ERROR | pool-7-thread-22 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,510 | test | ERROR | pool-7-thread-30 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,510 | test | ERROR | pool-7-thread-9 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,510 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,510 | test | ERROR | pool-7-thread-9 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,512 | test | ERROR | pool-7-thread-16 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,512 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,512 | test | ERROR | pool-7-thread-13 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,512 | test | ERROR | pool-7-thread-28 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,512 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,512 | test | ERROR | pool-7-thread-28 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,513 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,513 | test | ERROR | pool-7-thread-22 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,513 | test | ERROR | pool-7-thread-25 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,513 | test | ERROR | pool-7-thread-2 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,513 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,513 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,513 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,515 | test | ERROR | pool-7-thread-16 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,515 | test | ERROR | pool-7-thread-15 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,515 | test | ERROR | pool-7-thread-16 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,515 | test | ERROR | pool-7-thread-15 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,515 | test | ERROR | pool-7-thread-9 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,515 | test | ERROR | pool-7-thread-23 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,515 | test | ERROR | pool-7-thread-1 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,516 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,516 | test | ERROR | pool-7-thread-23 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,516 | test | ERROR | pool-7-thread-28 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,516 | test | ERROR | pool-7-thread-13 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,516 | test | ERROR | pool-7-thread-9 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,516 | test | ERROR | pool-7-thread-1 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,516 | test | ERROR | pool-7-thread-28 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,516 | test | ERROR | pool-7-thread-3 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,516 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,516 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,517 | test | ERROR | pool-7-thread-27 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,517 | test | ERROR | pool-7-thread-25 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,517 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,517 | test | ERROR | pool-7-thread-22 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,517 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,517 | test | ERROR | pool-7-thread-2 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,517 | test | ERROR | pool-7-thread-16 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,519 | test | ERROR | pool-7-thread-30 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,519 | test | ERROR | pool-7-thread-23 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,519 | test | ERROR | pool-7-thread-28 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,519 | test | ERROR | pool-7-thread-16 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,519 | test | ERROR | pool-7-thread-22 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,519 | test | ERROR | pool-7-thread-15 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,519 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,519 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,519 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,519 | test | ERROR | pool-7-thread-15 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,519 | test | ERROR | pool-7-thread-9 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,519 | test | ERROR | pool-7-thread-23 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,519 | test | ERROR | pool-7-thread-1 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,519 | test | ERROR | pool-7-thread-3 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,519 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,519 | test | ERROR | pool-7-thread-1 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,520 | test | ERROR | pool-7-thread-25 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,520 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,520 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,522 | test | ERROR | pool-7-thread-15 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,522 | test | ERROR | pool-7-thread-28 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,523 | test | ERROR | pool-7-thread-22 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,523 | test | ERROR | pool-7-thread-28 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,522 | test | ERROR | pool-7-thread-15 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,523 | test | ERROR | pool-7-thread-16 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,522 | test | ERROR | pool-7-thread-13 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,523 | test | ERROR | pool-7-thread-25 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,523 | test | ERROR | pool-7-thread-9 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,523 | test | ERROR | pool-7-thread-16 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,523 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,523 | test | ERROR | pool-7-thread-30 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,523 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,523 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,523 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,523 | test | ERROR | pool-7-thread-3 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,523 | test | ERROR | pool-7-thread-1 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,525 | test | ERROR | pool-7-thread-2 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,523 | test | ERROR | pool-7-thread-1 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,525 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,525 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,525 | test | ERROR | pool-7-thread-23 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,526 | test | ERROR | pool-7-thread-23 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,526 | test | ERROR | pool-7-thread-28 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,526 | test | ERROR | pool-7-thread-22 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,526 | test | ERROR | pool-7-thread-13 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,526 | test | ERROR | pool-7-thread-28 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,526 | test | ERROR | pool-7-thread-16 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,526 | test | ERROR | pool-7-thread-15 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,526 | test | ERROR | pool-7-thread-27 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,526 | test | ERROR | pool-7-thread-22 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,526 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,526 | test | ERROR | pool-7-thread-16 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,526 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,526 | test | ERROR | pool-7-thread-3 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,528 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,528 | test | ERROR | pool-7-thread-30 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,528 | test | ERROR | pool-7-thread-1 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,529 | test | ERROR | pool-7-thread-15 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,529 | test | ERROR | pool-7-thread-23 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,529 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,529 | test | ERROR | pool-7-thread-1 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,529 | test | ERROR | pool-7-thread-25 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,529 | test | ERROR | pool-7-thread-23 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,529 | test | ERROR | pool-7-thread-15 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,529 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,529 | test | ERROR | pool-7-thread-22 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,529 | test | ERROR | pool-7-thread-27 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,529 | test | ERROR | pool-7-thread-28 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,529 | test | ERROR | pool-7-thread-2 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,529 | test | ERROR | pool-7-thread-9 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,529 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,529 | test | ERROR | pool-7-thread-28 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,529 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,529 | test | ERROR | pool-7-thread-3 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,529 | test | ERROR | pool-7-thread-22 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,530 | test | ERROR | pool-7-thread-9 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,530 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,530 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,530 | test | ERROR | pool-7-thread-16 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,530 | test | ERROR | pool-7-thread-30 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,530 | test | ERROR | pool-7-thread-16 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,532 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,532 | test | ERROR | pool-7-thread-25 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,532 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,532 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,532 | test | ERROR | pool-7-thread-22 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,532 | test | ERROR | pool-7-thread-1 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,533 | test | ERROR | pool-7-thread-15 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,533 | test | ERROR | pool-7-thread-23 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,533 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,533 | test | ERROR | pool-7-thread-2 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,533 | test | ERROR | pool-7-thread-13 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,535 | test | ERROR | pool-7-thread-23 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,535 | test | ERROR | pool-7-thread-15 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,535 | test | ERROR | pool-7-thread-9 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,535 | test | ERROR | pool-7-thread-16 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,535 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,535 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,535 | test | ERROR | pool-7-thread-28 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,535 | test | ERROR | pool-7-thread-30 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,536 | test | ERROR | pool-7-thread-22 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,536 | test | ERROR | pool-7-thread-25 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,536 | test | ERROR | pool-7-thread-28 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,536 | test | ERROR | pool-7-thread-22 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,536 | test | ERROR | pool-7-thread-3 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,536 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,536 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,536 | test | ERROR | pool-7-thread-27 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,536 | test | ERROR | pool-7-thread-1 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,536 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,536 | test | ERROR | pool-7-thread-9 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,536 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,536 | test | ERROR | pool-7-thread-1 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,538 | test | ERROR | pool-7-thread-23 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,538 | test | ERROR | pool-7-thread-23 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,538 | test | ERROR | pool-7-thread-16 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,538 | test | ERROR | pool-7-thread-13 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,538 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,538 | test | ERROR | pool-7-thread-2 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,539 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,539 | test | ERROR | pool-7-thread-15 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,539 | test | ERROR | pool-7-thread-22 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,539 | test | ERROR | pool-7-thread-22 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,539 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,539 | test | ERROR | pool-7-thread-28 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,539 | test | ERROR | pool-7-thread-3 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,539 | test | ERROR | pool-7-thread-28 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,539 | test | ERROR | pool-7-thread-30 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,540 | test | ERROR | pool-7-thread-16 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,540 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,540 | test | ERROR | pool-7-thread-16 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,540 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,539 | test | ERROR | pool-7-thread-27 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,540 | test | ERROR | pool-7-thread-2 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,540 | test | ERROR | pool-7-thread-1 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,540 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,542 | test | ERROR | pool-7-thread-13 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,542 | test | ERROR | pool-7-thread-23 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,542 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,542 | test | ERROR | pool-7-thread-1 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,542 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,542 | test | ERROR | pool-7-thread-23 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,542 | test | ERROR | pool-7-thread-15 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,542 | test | ERROR | pool-7-thread-9 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,542 | test | ERROR | pool-7-thread-28 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,542 | test | ERROR | pool-7-thread-15 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,542 | test | ERROR | pool-7-thread-9 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,542 | test | ERROR | pool-7-thread-28 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,542 | test | ERROR | pool-7-thread-22 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,542 | test | ERROR | pool-7-thread-1 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,542 | test | ERROR | pool-7-thread-16 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,543 | test | ERROR | pool-7-thread-1 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,543 | test | ERROR | pool-7-thread-22 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,543 | test | ERROR | pool-7-thread-30 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,543 | test | ERROR | pool-7-thread-23 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,543 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,543 | test | ERROR | pool-7-thread-16 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,545 | test | ERROR | pool-7-thread-3 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,545 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,546 | test | ERROR | pool-7-thread-25 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,546 | test | ERROR | pool-7-thread-27 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,546 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,546 | test | ERROR | pool-7-thread-1 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,546 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,546 | test | ERROR | pool-7-thread-13 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,546 | test | ERROR | pool-7-thread-1 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,546 | test | ERROR | pool-7-thread-15 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,546 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,546 | test | ERROR | pool-7-thread-28 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,546 | test | ERROR | pool-7-thread-15 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,546 | test | ERROR | pool-7-thread-22 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,546 | test | ERROR | pool-7-thread-28 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,546 | test | ERROR | pool-7-thread-2 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,546 | test | ERROR | pool-7-thread-22 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,546 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,546 | test | ERROR | pool-7-thread-9 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,548 | test | ERROR | pool-7-thread-1 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,548 | test | ERROR | pool-7-thread-3 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,548 | test | ERROR | pool-7-thread-1 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,548 | test | ERROR | pool-7-thread-16 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,548 | test | ERROR | pool-7-thread-30 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,548 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,549 | test | ERROR | pool-7-thread-16 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,549 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,549 | test | ERROR | pool-7-thread-23 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,549 | test | ERROR | pool-7-thread-13 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,549 | test | ERROR | pool-7-thread-23 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,549 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,549 | test | ERROR | pool-7-thread-16 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,549 | test | ERROR | pool-7-thread-30 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,549 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,551 | test | ERROR | pool-7-thread-23 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,551 | test | ERROR | pool-7-thread-13 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,551 | test | ERROR | pool-7-thread-23 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,551 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,551 | test | ERROR | pool-7-thread-16 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,551 | test | ERROR | pool-7-thread-16 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,552 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'delete_multi' 2022-03-06 13:45:32,552 | test | ERROR | pool-7-thread-27 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,552 | test | ERROR | pool-7-thread-22 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,552 | test | ERROR | pool-7-thread-28 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,552 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,552 | test | ERROR | pool-7-thread-2 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,552 | test | ERROR | pool-7-thread-16 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,552 | test | ERROR | pool-7-thread-13 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,552 | test | ERROR | pool-7-thread-3 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,552 | test | ERROR | pool-7-thread-27 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,552 | test | ERROR | pool-7-thread-15 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,553 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,553 | test | ERROR | pool-7-thread-9 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,553 | test | ERROR | pool-7-thread-22 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,553 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,553 | test | ERROR | pool-7-thread-13 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,553 | test | ERROR | pool-7-thread-30 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,553 | test | ERROR | pool-7-thread-15 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,553 | test | ERROR | pool-7-thread-25 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,553 | test | ERROR | pool-7-thread-1 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,553 | test | ERROR | pool-7-thread-16 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,555 | test | ERROR | pool-7-thread-25 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,553 | test | ERROR | pool-7-thread-9 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,553 | test | ERROR | pool-7-thread-28 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,553 | test | ERROR | pool-7-thread-27 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,555 | test | ERROR | pool-7-thread-30 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,555 | test | ERROR | pool-7-thread-1 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,555 | test | ERROR | pool-7-thread-22 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,555 | test | ERROR | pool-7-thread-23 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,555 | test | ERROR | pool-7-thread-2 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,555 | test | ERROR | pool-7-thread-3 | [task:batch_create:577] 'NoneType' object has no attribute 'set_multi' 2022-03-06 13:45:32,555 | test | ERROR | pool-7-thread-23 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,555 | test | ERROR | pool-7-thread-2 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,555 | test | ERROR | pool-7-thread-22 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,555 | test | ERROR | pool-7-thread-3 | [task:call:515] 'NoneType' object has no attribute 'bucket' 2022-03-06 13:45:32,608 | test | INFO | MainThread | [basetestcase:check_logs:824] Comparing timestamps: Log's latest timestamp: 2022-03-06 06:13:45, Test's start timestamp is 2022-03-06 11:47:20.041000 2022-03-06 13:45:32,608 | test | INFO | MainThread | [basetestcase:check_coredump_exist:921] unwanted messages in /opt/couchbase/var/lib/couchbase/logs/memcached.log.000015.txt 2022-03-06 13:45:32,608 | test | INFO | MainThread | [basetestcase:check_logs:824] Comparing timestamps: Log's latest timestamp: 2022-03-06 06:13:45, Test's start timestamp is 2022-03-06 11:47:20.041000 2022-03-06 13:45:33,608 | test | INFO | pool-7-thread-24 | [table_view:display:72] Ops trend for bucket 'bucket1' +-------+----------------------------------------------------------------------------------------------------------------------------------+---------+ | Min | Trend | Max | +-------+----------------------------------------------------------------------------------------------------------------------------------+---------+ | 0.000 | *****........*******X | 15.300 | | 0.000 | ********........*******X | 15.300 | | 0.000 | *******************************.................*******..................................................................******X | 145.500 | | 0.000 | **************.................................*...................................................................******X | 175.300 | | 0.000 | **********************X | 0.000 | +-------+----------------------------------------------------------------------------------------------------------------------------------+---------+ 2022-03-06 13:45:33,937 | test | INFO | MainThread | [basetestcase:check_logs:824] Comparing timestamps: Log's latest timestamp: 2022-03-06 11:54:11, Test's start timestamp is 2022-03-06 11:47:20.041000 2022-03-06 13:45:33,938 | test | INFO | MainThread | [basetestcase:check_coredump_exist:921] unwanted messages in /opt/couchbase/var/lib/couchbase/logs/memcached.log.000031.txt 2022-03-06 13:45:33,940 | test | INFO | MainThread | [basetestcase:check_logs:824] Comparing timestamps: Log's latest timestamp: 2022-03-06 11:54:11, Test's start timestamp is 2022-03-06 11:47:20.041000 2022-03-06 13:45:33,940 | test | CRITICAL | MainThread | [basetestcase:check_coredump_exist:924] 172.23.105.164: Found ' CRITICAL ' logs - 2022-03-06T11:54:11.508696-08:00 CRITICAL [(default) magma_0/kvstore-368/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-368/rev-000000001/seqIndex/state.867: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0/kvstore-368/rev-000000001 2022-03-06T11:54:11.517598-08:00 CRITICAL [(default) magma_0/kvstore-376/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-376/rev-000000001/seqIndex/state.868: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0 2022-03-06T11:54:11.526927-08:00 CRITICAL [(default) magma_0/kvstore-384/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-384/rev-000000001/seqIndex/state.855: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0 2022-03-06T11:54:11.534417-08:00 CRITICAL [(default) magma_0/kvstore-392/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-392/rev-000000001/seqIndex/state.865: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0 2022-03-06T11:54:11.538679-08:00 CRITICAL [(default) magma_0/kvstore-400/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-400/rev-000000001/seqIndex/state.878: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0 2022-03-06T11:54:11.545538-08:00 CRITICAL [(default) magma_0/kvstore-408/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-408/rev-000000001/seqIndex/state.866: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0 2022-03-06T11:54:11.552745-08:00 CRITICAL [(default) magma_0/kvstore-416/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-416/rev-000000001/seqIndex/state.860: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0 2022-03-06T11:54:11.561831-08:00 CRITICAL [(default) magma_0/kvstore-424/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-424/rev-000000001/seqIndex/state.869: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0 2022-03-06T11:54:11.569554-08:00 CRITICAL [(default) magma_0/kvstore-432/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-432/rev-000000001/seqIndex/state.871: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0 2022-03-06T11:54:11.575419-08:00 CRITICAL [(default) magma_0/kvstore-440/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-440/rev-000000001/seqIndex/state.869: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0 2022-03-06T11:54:11.581252-08:00 CRITICAL [(default) magma_0/kvstore-448/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-448/rev-000000001/seqIndex/state.861: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0 2022-03-06T11:54:11.587071-08:00 CRITICAL [(default) magma_0/kvstore-456/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-456/rev-000000001/seqIndex/state.869: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0 2022-03-06T11:54:11.594446-08:00 CRITICAL [(default) magma_0/kvstore-464/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-464/rev-000000001/seqIndex/state.863: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0 2022-03-06T11:54:11.604146-08:00 CRITICAL [(default) magma_0/kvstore-472/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-472/rev-000000001/seqIndex/state.873: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0 2022-03-06T11:54:11.614020-08:00 CRITICAL [(default) magma_0/kvstore-480/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-480/rev-000000001/seqIndex/state.871: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0 2022-03-06T11:54:11.621019-08:00 CRITICAL [(default) magma_0/kvstore-488/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-488/rev-000000001/seqIndex/state.863: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0 2022-03-06T11:54:11.627487-08:00 CRITICAL [(default) magma_0/kvstore-496/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-496/rev-000000001/seqIndex/state.863: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0 2022-03-06T11:54:11.635685-08:00 CRITICAL [(default) magma_0/kvstore-504/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-504/rev-000000001/seqIndex/state.860: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0 2022-03-06T11:54:11.643493-08:00 CRITICAL [(default) magma_0/kvstore-600/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-600/rev-000000001/seqIndex/state.869: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0 2022-03-06T11:54:11.652237-08:00 CRITICAL [(default) magma_0/kvstore-608/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-608/rev-000000001/seqIndex/state.873: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0 2022-03-06T11:54:11.660247-08:00 CRITICAL [(default) magma_0/kvstore-616/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-616/rev-000000001/seqIndex/state.854: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0 2022-03-06T11:54:11.665953-08:00 CRITICAL [(default) magma_0/kvstore-624/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-624/rev-000000001/seqIndex/state.861: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0 2022-03-06T11:54:11.674513-08:00 CRITICAL [(default) magma_0/kvstore-632/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-632/rev-000000001/seqIndex/state.859: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0 2022-03-06T11:54:11.681789-08:00 CRITICAL [(default) magma_0/kvstore-640/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-640/rev-000000001/seqIndex/state.859: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0 2022-03-06T11:54:11.688905-08:00 CRITICAL [(default) magma_0/kvstore-648/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-648/rev-000000001/seqIndex/state.877: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0 2022-03-06T11:54:11.695865-08:00 CRITICAL [(default) magma_0/kvstore-656/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-656/rev-000000001/seqIndex/state.848: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0 2022-03-06T11:54:11.702939-08:00 CRITICAL [(default) magma_0/kvstore-664/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-664/rev-000000001/seqIndex/state.865: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0 2022-03-06T11:54:11.709354-08:00 CRITICAL [(default) magma_0/kvstore-672/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-672/rev-000000001/seqIndex/state.869: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0 2022-03-06T11:54:11.717191-08:00 CRITICAL [(default) magma_0/kvstore-680/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-680/rev-000000001/seqIndex/state.870: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0 2022-03-06T11:54:11.724954-08:00 CRITICAL [(default) magma_0/kvstore-856/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-856/rev-000000001/seqIndex/state.877: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0 2022-03-06T11:54:11.732708-08:00 CRITICAL [(default) magma_0/kvstore-864/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-864/rev-000000001/seqIndex/state.881: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0 2022-03-06T11:54:11.739073-08:00 CRITICAL [(default) magma_0/kvstore-872/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-872/rev-000000001/seqIndex/state.886: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0 2022-03-06T11:54:11.745366-08:00 CRITICAL [(default) magma_0/kvstore-880/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-880/rev-000000001/seqIndex/state.867: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0 2022-03-06T11:54:11.751263-08:00 CRITICAL [(default) magma_0/kvstore-888/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-888/rev-000000001/seqIndex/state.871: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0 2022-03-06T11:54:11.758602-08:00 CRITICAL [(default) magma_0/kvstore-896/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-896/rev-000000001/seqIndex/state.884: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0 2022-03-06T11:54:11.765495-08:00 CRITICAL [(default) magma_0/kvstore-904/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-904/rev-000000001/seqIndex/state.879: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0 2022-03-06T11:54:11.772826-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.859: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0 2022-03-06T11:54:11.781849-08:00 CRITICAL [(default) magma_0/kvstore-920/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-920/rev-000000001/seqIndex/state.872: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0 2022-03-06T11:54:11.789381-08:00 CRITICAL [(default) magma_0/kvstore-928/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-928/rev-000000001/seqIndex/state.871: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0 2022-03-06 13:45:33,997 | test | INFO | MainThread | [basetestcase:check_logs:824] Comparing timestamps: Log's latest timestamp: 2022-03-06 11:54:11, Test's start timestamp is 2022-03-06 11:47:20.041000 2022-03-06 13:45:34,000 | test | INFO | MainThread | [basetestcase:check_logs:824] Comparing timestamps: Log's latest timestamp: 2022-03-06 11:54:11, Test's start timestamp is 2022-03-06 11:47:20.041000 2022-03-06 13:45:34,000 | test | INFO | MainThread | [basetestcase:check_coredump_exist:933] unwanted messages in /opt/couchbase/var/lib/couchbase/logs/babysitter.log 2022-03-06 13:45:34,002 | test | CRITICAL | MainThread | [basetestcase:check_coredump_exist:935] 172.23.105.164: Found ' CRITICAL ' logs - ['[ns_server:info,2022-03-06T06:13:43.800-08:00,babysitter_of_ns_1@cb.local:<0.133.0>:ns_port_server:log:226]memcached<0.133.0>: 2022-03-06T06:13:43.591107-08:00 CRITICAL [(default) magma_0/kvstore-864/rev-000000001/localIndex]Failed to write tree state /data/default/magma.0/kvstore-864/rev-000000001/localIndex/state.1095: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.591474-08:00 CRITICAL [(default) magma_0/kvstore-864/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-864/rev-000000001/seqIndex/state.1096: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.591782-08:00 CRITICAL [(default) magma_0/kvstore-864/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-864/rev-000000001/seqIndex/state.1097: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.592119-08:00 CRITICAL [(default) magma_0/kvstore-864/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-864/rev-000000001/seqIndex/state.1098: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.592507-08:00 CRITICAL [(default) magma_0/kvstore-864/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-864/rev-000000001/seqIndex/state.1099: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.592973-08:00 CRITICAL [(default) magma_0/kvstore-864/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-864/rev-000000001/seqIndex/state.1100: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.593463-08:00 CRITICAL [(default) magma_0/kvstore-864/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-864/rev-000000001/seqIndex/state.1101: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.594024-08:00 CRITICAL [(default) magma_0/kvstore-864/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-864/rev-000000001/seqIndex/state.1102: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.594643-08:00 CRITICAL [(default) magma_0/kvstore-864/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-864/rev-000000001/seqIndex/state.1103: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.595344-08:00 CRITICAL [(default) magma_0/kvstore-864/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-864/rev-000000001/seqIndex/state.1104: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.596049-08:00 CRITICAL [(default) magma_0/kvstore-864/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-864/rev-000000001/seqIndex/state.1105: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.596793-08:00 CRITICAL [(default) magma_0/kvstore-864/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-864/rev-000000001/seqIndex/state.1106: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.597558-08:00 CRITICAL [(default) magma_0/kvstore-864/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-864/rev-000000001/seqIndex/state.1107: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.598424-08:00 CRITICAL [(default) magma_0/kvstore-864/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-864/rev-000000001/seqIndex/state.1108: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.599804-08:00 CRITICAL [(default) magma_0/kvstore-864/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-864/rev-000000001/seqIndex/state.1109: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.600894-08:00 CRITICAL [(default) magma_0/kvstore-864/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-864/rev-000000001/seqIndex/state.1110: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.602235-08:00 CRITICAL [(default) magma_0/kvstore-864/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-864/rev-000000001/seqIndex/state.1111: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.603541-08:00 CRITICAL [(default) magma_0/kvstore-864/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-864/rev-000000001/seqIndex/state.1112: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.605085-08:00 CRITICAL [(default) magma_0/kvstore-864/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-864/rev-000000001/seqIndex/state.1113: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.606921-08:00 CRITICAL [(default) magma_0/kvstore-864/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-864/rev-000000001/seqIndex/state.1114: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.608644-08:00 CRITICAL [(default) magma_0/kvstore-864/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-864/rev-000000001/seqIndex/state.1115: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', '[ns_server:info,2022-03-06T06:13:45.263-08:00,babysitter_of_ns_1@cb.local:<0.133.0>:ns_port_server:log:226]memcached<0.133.0>: 2022-03-06T06:13:45.062993-08:00 CRITICAL [(default) magma_0/kvstore-48/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-48/rev-000000001/seqIndex/state.834: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0/kvstore-48/rev-000000001\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.068642-08:00 CRITICAL [(default) magma_0/kvstore-56/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-56/rev-000000001/seqIndex/state.837: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.073736-08:00 CRITICAL [(default) magma_0/kvstore-64/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-64/rev-000000001/seqIndex/state.827: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.080689-08:00 CRITICAL [(default) magma_0/kvstore-72/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-72/rev-000000001/seqIndex/state.832: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.084886-08:00 CRITICAL [(default) magma_0/kvstore-80/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-80/rev-000000001/seqIndex/state.840: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.089529-08:00 CRITICAL [(default) magma_0/kvstore-256/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-256/rev-000000001/seqIndex/state.839: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.096475-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.839: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.102642-08:00 CRITICAL [(default) magma_0/kvstore-272/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-272/rev-000000001/seqIndex/state.826: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.110251-08:00 CRITICAL [(default) magma_0/kvstore-280/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-280/rev-000000001/seqIndex/state.851: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.116847-08:00 CRITICAL [(default) magma_0/kvstore-288/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-288/rev-000000001/seqIndex/state.837: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.123701-08:00 CRITICAL [(default) magma_0/kvstore-296/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-296/rev-000000001/seqIndex/state.834: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.130909-08:00 CRITICAL [(default) magma_0/kvstore-304/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-304/rev-000000001/seqIndex/state.844: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.139625-08:00 CRITICAL [(default) magma_0/kvstore-312/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-312/rev-000000001/seqIndex/state.839: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.147167-08:00 CRITICAL [(default) magma_0/kvstore-320/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-320/rev-000000001/seqIndex/state.834: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.153396-08:00 CRITICAL [(default) magma_0/kvstore-328/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-328/rev-000000001/seqIndex/state.845: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.159833-08:00 CRITICAL [(default) magma_0/kvstore-336/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-336/rev-000000001/seqIndex/state.830: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.168409-08:00 CRITICAL [(default) magma_0/kvstore-344/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-344/rev-000000001/seqIndex/state.838: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.173526-08:00 CRITICAL [(default) magma_0/kvstore-352/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-352/rev-000000001/seqIndex/state.837: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.177540-08:00 CRITICAL [(default) magma_0/kvstore-360/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-360/rev-000000001/seqIndex/state.843: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.181248-08:00 CRITICAL [(default) magma_0/kvstore-368/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-368/rev-000000001/seqIndex/state.844: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.184947-08:00 CRITICAL [(default) magma_0/kvstore-376/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-376/rev-000000001/seqIndex/state.834: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.189012-08:00 CRITICAL [(default) magma_0/kvstore-384/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-384/rev-000000001/seqIndex/state.836: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.192843-08:00 CRITICAL [(default) magma_0/kvstore-392/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-392/rev-000000001/seqIndex/state.829: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.196629-08:00 CRITICAL [(default) magma_0/kvstore-400/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-400/rev-000000001/seqIndex/state.827: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.200173-08:00 CRITICAL [(default) magma_0/kvstore-408/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-408/rev-000000001/seqIndex/state.833: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.203700-08:00 CRITICAL [(default) magma_0/kvstore-416/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-416/rev-000000001/seqIndex/state.1136: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.205825-08:00 CRITICAL [(default) magma_0/kvstore-424/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-424/rev-000000001/seqIndex/state.1143: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.208286-08:00 CRITICAL [(default) magma_0/kvstore-432/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-432/rev-000000001/seqIndex/state.1140: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.210111-08:00 CRITICAL [(default) magma_0/kvstore-440/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-440/rev-000000001/seqIndex/state.1136: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.212001-08:00 CRITICAL [(default) magma_0/kvstore-448/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-448/rev-000000001/seqIndex/state.1134: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.213931-08:00 CRITICAL [(default) magma_0/kvstore-456/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-456/rev-000000001/seqIndex/state.1138: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.215743-08:00 CRITICAL [(default) magma_0/kvstore-464/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-464/rev-000000001/seqIndex/state.1131: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.217268-08:00 CRITICAL [(default) magma_0/kvstore-472/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-472/rev-000000001/seqIndex/state.1136: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.218897-08:00 CRITICAL [(default) magma_0/kvstore-480/rev-000000001/localIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-480/rev-000000001/localIndex/sstable.1542.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.220118-08:00 CRITICAL [(default) magma_0/kvstore-488/rev-000000001/localIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-488/rev-000000001/localIndex/sstable.1569.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.221443-08:00 CRITICAL [(default) magma_0/kvstore-496/rev-000000001/localIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-496/rev-000000001/localIndex/sstable.1559.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.222704-08:00 CRITICAL [(default) magma_0/kvstore-504/rev-000000001/localIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-504/rev-000000001/localIndex/sstable.1563.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.224323-08:00 CRITICAL [(default) magma_0/kvstore-600/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-600/rev-000000001/seqIndex/state.828: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.228554-08:00 CRITICAL [(default) magma_0/kvstore-608/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-608/rev-000000001/seqIndex/state.833: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.231852-08:00 CRITICAL [(default) magma_0/kvstore-616/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-616/rev-000000001/seqIndex/state.837: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.237264-08:00 CRITICAL [(default) magma_0/kvstore-624/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-624/rev-000000001/seqIndex/state.829: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.241691-08:00 CRITICAL [(default) magma_0/kvstore-632/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-632/rev-000000001/seqIndex/state.820: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.247240-08:00 CRITICAL [(default) magma_0/kvstore-640/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-640/rev-000000001/seqIndex/state.841: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.251089-08:00 CRITICAL [(default) magma_0/kvstore-648/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-648/rev-000000001/seqIndex/state.837: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.254835-08:00 CRITICAL [(default) magma_0/kvstore-656/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-656/rev-000000001/seqIndex/state.824: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.259840-08:00 CRITICAL [(default) magma_0/kvstore-664/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-664/rev-000000001/seqIndex/state.832: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', '[ns_server:info,2022-03-06T06:13:45.466-08:00,babysitter_of_ns_1@cb.local:<0.133.0>:ns_port_server:log:226]memcached<0.133.0>: 2022-03-06T06:13:45.265706-08:00 CRITICAL [(default) magma_0/kvstore-672/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-672/rev-000000001/seqIndex/state.1127: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.269707-08:00 CRITICAL [(default) magma_0/kvstore-680/rev-000000001/localIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-680/rev-000000001/localIndex/sstable.1557.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.271151-08:00 CRITICAL [(default) magma_0/kvstore-856/rev-000000001/localIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-856/rev-000000001/localIndex/sstable.1566.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.272353-08:00 CRITICAL [(default) magma_0/kvstore-864/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-864/rev-000000001/seqIndex/state.1116: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.274461-08:00 CRITICAL [(default) magma_0/kvstore-872/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-872/rev-000000001/seqIndex/state.832: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.278921-08:00 CRITICAL [(default) magma_0/kvstore-880/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-880/rev-000000001/seqIndex/state.825: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.283659-08:00 CRITICAL [(default) magma_0/kvstore-888/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-888/rev-000000001/seqIndex/state.817: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.289457-08:00 CRITICAL [(default) magma_0/kvstore-896/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-896/rev-000000001/seqIndex/state.839: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.294132-08:00 CRITICAL [(default) magma_0/kvstore-904/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-904/rev-000000001/seqIndex/state.838: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.297494-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.825: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.302502-08:00 CRITICAL [(default) magma_0/kvstore-920/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-920/rev-000000001/seqIndex/state.836: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.305653-08:00 CRITICAL [(default) magma_0/kvstore-928/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-928/rev-000000001/seqIndex/state.827: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.309798-08:00 CRITICAL [(default) magma_0/kvstore-936/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-936/rev-000000001/seqIndex/state.826: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', '[ns_server:info,2022-03-06T11:54:11.709-08:00,babysitter_of_ns_1@cb.local:<0.133.0>:ns_port_server:log:226]memcached<0.133.0>: 2022-03-06T11:54:11.508696-08:00 CRITICAL [(default) magma_0/kvstore-368/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-368/rev-000000001/seqIndex/state.867: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0/kvstore-368/rev-000000001\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.517598-08:00 CRITICAL [(default) magma_0/kvstore-376/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-376/rev-000000001/seqIndex/state.868: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.526927-08:00 CRITICAL [(default) magma_0/kvstore-384/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-384/rev-000000001/seqIndex/state.855: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.534417-08:00 CRITICAL [(default) magma_0/kvstore-392/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-392/rev-000000001/seqIndex/state.865: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.538679-08:00 CRITICAL [(default) magma_0/kvstore-400/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-400/rev-000000001/seqIndex/state.878: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.545538-08:00 CRITICAL [(default) magma_0/kvstore-408/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-408/rev-000000001/seqIndex/state.866: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.552745-08:00 CRITICAL [(default) magma_0/kvstore-416/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-416/rev-000000001/seqIndex/state.860: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.561831-08:00 CRITICAL [(default) magma_0/kvstore-424/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-424/rev-000000001/seqIndex/state.869: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.569554-08:00 CRITICAL [(default) magma_0/kvstore-432/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-432/rev-000000001/seqIndex/state.871: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.575419-08:00 CRITICAL [(default) magma_0/kvstore-440/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-440/rev-000000001/seqIndex/state.869: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.581252-08:00 CRITICAL [(default) magma_0/kvstore-448/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-448/rev-000000001/seqIndex/state.861: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.587071-08:00 CRITICAL [(default) magma_0/kvstore-456/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-456/rev-000000001/seqIndex/state.869: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.594446-08:00 CRITICAL [(default) magma_0/kvstore-464/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-464/rev-000000001/seqIndex/state.863: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.604146-08:00 CRITICAL [(default) magma_0/kvstore-472/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-472/rev-000000001/seqIndex/state.873: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.614020-08:00 CRITICAL [(default) magma_0/kvstore-480/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-480/rev-000000001/seqIndex/state.871: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.621019-08:00 CRITICAL [(default) magma_0/kvstore-488/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-488/rev-000000001/seqIndex/state.863: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.627487-08:00 CRITICAL [(default) magma_0/kvstore-496/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-496/rev-000000001/seqIndex/state.863: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.635685-08:00 CRITICAL [(default) magma_0/kvstore-504/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-504/rev-000000001/seqIndex/state.860: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.643493-08:00 CRITICAL [(default) magma_0/kvstore-600/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-600/rev-000000001/seqIndex/state.869: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.652237-08:00 CRITICAL [(default) magma_0/kvstore-608/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-608/rev-000000001/seqIndex/state.873: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.660247-08:00 CRITICAL [(default) magma_0/kvstore-616/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-616/rev-000000001/seqIndex/state.854: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.665953-08:00 CRITICAL [(default) magma_0/kvstore-624/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-624/rev-000000001/seqIndex/state.861: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.674513-08:00 CRITICAL [(default) magma_0/kvstore-632/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-632/rev-000000001/seqIndex/state.859: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.681789-08:00 CRITICAL [(default) magma_0/kvstore-640/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-640/rev-000000001/seqIndex/state.859: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.688905-08:00 CRITICAL [(default) magma_0/kvstore-648/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-648/rev-000000001/seqIndex/state.877: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.695865-08:00 CRITICAL [(default) magma_0/kvstore-656/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-656/rev-000000001/seqIndex/state.848: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.702939-08:00 CRITICAL [(default) magma_0/kvstore-664/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-664/rev-000000001/seqIndex/state.865: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', '[ns_server:info,2022-03-06T11:54:11.910-08:00,babysitter_of_ns_1@cb.local:<0.133.0>:ns_port_server:log:226]memcached<0.133.0>: 2022-03-06T11:54:11.709354-08:00 CRITICAL [(default) magma_0/kvstore-672/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-672/rev-000000001/seqIndex/state.869: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.717191-08:00 CRITICAL [(default) magma_0/kvstore-680/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-680/rev-000000001/seqIndex/state.870: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.724954-08:00 CRITICAL [(default) magma_0/kvstore-856/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-856/rev-000000001/seqIndex/state.877: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.732708-08:00 CRITICAL [(default) magma_0/kvstore-864/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-864/rev-000000001/seqIndex/state.881: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.739073-08:00 CRITICAL [(default) magma_0/kvstore-872/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-872/rev-000000001/seqIndex/state.886: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.745366-08:00 CRITICAL [(default) magma_0/kvstore-880/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-880/rev-000000001/seqIndex/state.867: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.751263-08:00 CRITICAL [(default) magma_0/kvstore-888/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-888/rev-000000001/seqIndex/state.871: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.758602-08:00 CRITICAL [(default) magma_0/kvstore-896/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-896/rev-000000001/seqIndex/state.884: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.765495-08:00 CRITICAL [(default) magma_0/kvstore-904/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-904/rev-000000001/seqIndex/state.879: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.772826-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.859: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.781849-08:00 CRITICAL [(default) magma_0/kvstore-920/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-920/rev-000000001/seqIndex/state.872: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.789381-08:00 CRITICAL [(default) magma_0/kvstore-928/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-928/rev-000000001/seqIndex/state.871: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.797228-08:00 CRITICAL [(default) magma_0/kvstore-936/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-936/rev-000000001/seqIndex/state.871: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n'] 2022-03-06 13:45:34,413 | test | INFO | pool-7-thread-8 | [table_view:display:72] Ops trend for bucket 'default' +-----------+---------------------------------------------------------------------------------------------------------------------------+-----------+ | Min | Trend | Max | +-----------+---------------------------------------------------------------------------------------------------------------------------+-----------+ | 0.000 | *****................................................********....................................................*******X | 72528.700 | | 66800.600 | ...........................................................................................******X | 66800.600 | | 42708.200 | .........................................................*******X | 42708.200 | | 39597.300 | .....................................................*******X | 39597.300 | | 35787.000 | ................................................*******..........................*******X | 55002.300 | | 54263.100 | .........................................................................*******............*******............******X | 71203.300 | | 67540.200 | ............................................................................................******X | 67540.200 | | 43542.800 | ...........................................................********..........*******.....*******X | 55108.600 | | 12491.800 | ................******X | 12491.800 | | 1614.200 | .X | 1614.200 | +-----------+---------------------------------------------------------------------------------------------------------------------------+-----------+ 2022-03-06 13:45:34,463 | test | INFO | MainThread | [basetestcase:check_logs:824] Comparing timestamps: Log's latest timestamp: 2022-03-06 11:54:11, Test's start timestamp is 2022-03-06 11:47:20.041000 2022-03-06 13:45:34,466 | test | INFO | MainThread | [basetestcase:check_logs:824] Comparing timestamps: Log's latest timestamp: 2022-03-06 11:54:11, Test's start timestamp is 2022-03-06 11:47:20.041000 2022-03-06 13:45:34,467 | test | INFO | MainThread | [basetestcase:check_coredump_exist:933] unwanted messages in /opt/couchbase/var/lib/couchbase/logs/babysitter.log 2022-03-06 13:45:34,467 | test | CRITICAL | MainThread | [basetestcase:check_coredump_exist:935] 172.23.105.206: Found ' CRITICAL ' logs - ['[ns_server:info,2022-03-06T06:13:45.017-08:00,babysitter_of_ns_1@cb.local:<0.133.0>:ns_port_server:log:226]memcached<0.133.0>: 2022-03-06T06:13:44.816833-08:00 CRITICAL [(default) magma_0/kvstore-168/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-168/rev-000000001/seqIndex/state.993: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.822945-08:00 CRITICAL [(default) magma_0/kvstore-344/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-344/rev-000000001/seqIndex/state.977: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.829153-08:00 CRITICAL [(default) magma_0/kvstore-352/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-352/rev-000000001/seqIndex/state.977: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.832937-08:00 CRITICAL [(default) magma_0/kvstore-360/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-360/rev-000000001/seqIndex/state.986: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.838185-08:00 CRITICAL [(default) magma_0/kvstore-368/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-368/rev-000000001/seqIndex/state.980: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.842357-08:00 CRITICAL [(default) magma_0/kvstore-376/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-376/rev-000000001/seqIndex/state.984: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.845678-08:00 CRITICAL [(default) magma_0/kvstore-384/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-384/rev-000000001/seqIndex/state.986: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.851009-08:00 CRITICAL [(default) magma_0/kvstore-392/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-392/rev-000000001/seqIndex/state.984: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.855360-08:00 CRITICAL [(default) magma_0/kvstore-400/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-400/rev-000000001/seqIndex/state.1276: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.857938-08:00 CRITICAL [(default) magma_0/kvstore-408/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-408/rev-000000001/seqIndex/state.1269: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.859975-08:00 CRITICAL [(default) magma_0/kvstore-416/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-416/rev-000000001/seqIndex/state.1273: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.861788-08:00 CRITICAL [(default) magma_0/kvstore-424/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-424/rev-000000001/seqIndex/state.1273: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.864151-08:00 CRITICAL [(default) magma_0/kvstore-512/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-512/rev-000000001/seqIndex/state.979: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.871684-08:00 CRITICAL [(default) magma_0/kvstore-520/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-520/rev-000000001/seqIndex/state.974: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.877737-08:00 CRITICAL [(default) magma_0/kvstore-528/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-528/rev-000000001/seqIndex/state.974: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.884831-08:00 CRITICAL [(default) magma_0/kvstore-536/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-536/rev-000000001/seqIndex/state.970: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.891945-08:00 CRITICAL [(default) magma_0/kvstore-544/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-544/rev-000000001/seqIndex/state.977: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.899607-08:00 CRITICAL [(default) magma_0/kvstore-552/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-552/rev-000000001/seqIndex/state.974: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.906179-08:00 CRITICAL [(default) magma_0/kvstore-560/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-560/rev-000000001/seqIndex/state.977: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.913231-08:00 CRITICAL [(default) magma_0/kvstore-568/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-568/rev-000000001/seqIndex/state.974: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.921694-08:00 CRITICAL [(default) magma_0/kvstore-576/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-576/rev-000000001/seqIndex/state.978: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.928431-08:00 CRITICAL [(default) magma_0/kvstore-584/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-584/rev-000000001/seqIndex/state.976: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.936665-08:00 CRITICAL [(default) magma_0/kvstore-592/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-592/rev-000000001/seqIndex/state.968: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.944971-08:00 CRITICAL [(default) magma_0/kvstore-600/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-600/rev-000000001/seqIndex/state.962: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.949908-08:00 CRITICAL [(default) magma_0/kvstore-608/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-608/rev-000000001/seqIndex/state.970: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.955066-08:00 CRITICAL [(default) magma_0/kvstore-616/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-616/rev-000000001/seqIndex/state.980: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.959946-08:00 CRITICAL [(default) magma_0/kvstore-624/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-624/rev-000000001/seqIndex/state.974: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.965001-08:00 CRITICAL [(default) magma_0/kvstore-632/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-632/rev-000000001/seqIndex/state.971: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.971605-08:00 CRITICAL [(default) magma_0/kvstore-640/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-640/rev-000000001/seqIndex/state.971: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.975554-08:00 CRITICAL [(default) magma_0/kvstore-648/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-648/rev-000000001/seqIndex/state.978: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.980430-08:00 CRITICAL [(default) magma_0/kvstore-656/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-656/rev-000000001/seqIndex/state.969: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.983913-08:00 CRITICAL [(default) magma_0/kvstore-664/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-664/rev-000000001/seqIndex/state.975: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.988697-08:00 CRITICAL [(default) magma_0/kvstore-672/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-672/rev-000000001/seqIndex/state.976: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.995915-08:00 CRITICAL [(default) magma_0/kvstore-680/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-680/rev-000000001/seqIndex/state.1266: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.001603-08:00 CRITICAL [(default) magma_0/kvstore-688/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-688/rev-000000001/seqIndex/state.1288: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.003724-08:00 CRITICAL [(default) magma_0/kvstore-696/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-696/rev-000000001/seqIndex/state.1280: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.008196-08:00 CRITICAL [(default) magma_0/kvstore-704/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-704/rev-000000001/seqIndex/state.1283: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.009531-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/state.1282: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.011881-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1281: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.013717-08:00 CRITICAL [(default) magma_0/kvstore-728/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-728/rev-000000001/seqIndex/state.1274: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.015612-08:00 CRITICAL [(default) magma_0/kvstore-736/rev-000000001/localIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-736/rev-000000001/localIndex/sstable.1833.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', '[ns_server:info,2022-03-06T06:13:45.218-08:00,babysitter_of_ns_1@cb.local:<0.133.0>:ns_port_server:log:226]memcached<0.133.0>: 2022-03-06T06:13:45.017464-08:00 CRITICAL [(default) magma_0/kvstore-744/rev-000000001/localIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-744/rev-000000001/localIndex/sstable.1835.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.019455-08:00 CRITICAL [(default) magma_0/kvstore-752/rev-000000001/localIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-752/rev-000000001/localIndex/sstable.1846.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.021698-08:00 CRITICAL [(default) magma_0/kvstore-760/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-760/rev-000000001/seqIndex/state.1265: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.023506-08:00 CRITICAL [(default) magma_0/kvstore-944/rev-000000001/localIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-944/rev-000000001/localIndex/sstable.1850.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.025659-08:00 CRITICAL [(default) magma_0/kvstore-952/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-952/rev-000000001/seqIndex/state.962: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.030726-08:00 CRITICAL [(default) magma_0/kvstore-960/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-960/rev-000000001/seqIndex/state.972: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.035722-08:00 CRITICAL [(default) magma_0/kvstore-968/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-968/rev-000000001/seqIndex/state.968: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.040624-08:00 CRITICAL [(default) magma_0/kvstore-976/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-976/rev-000000001/seqIndex/state.969: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.045802-08:00 CRITICAL [(default) magma_0/kvstore-984/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-984/rev-000000001/seqIndex/state.977: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.051555-08:00 CRITICAL [(default) magma_0/kvstore-992/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-992/rev-000000001/seqIndex/state.972: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.057065-08:00 CRITICAL [(default) magma_0/kvstore-1000/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-1000/rev-000000001/seqIndex/state.977: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.060951-08:00 CRITICAL [(default) magma_0/kvstore-1008/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-1008/rev-000000001/seqIndex/state.981: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.065137-08:00 CRITICAL [(default) magma_0/kvstore-1016/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-1016/rev-000000001/seqIndex/state.975: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', '[ns_server:info,2022-03-06T11:54:11.399-08:00,babysitter_of_ns_1@cb.local:<0.133.0>:ns_port_server:log:226]memcached<0.133.0>: 2022-03-06T11:54:11.199066-08:00 CRITICAL [(default) magma_0/kvstore-360/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-360/rev-000000001/seqIndex/state.940: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0/kvstore-360/rev-000000001\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.205843-08:00 CRITICAL [(default) magma_0/kvstore-368/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-368/rev-000000001/seqIndex/state.939: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.211361-08:00 CRITICAL [(default) magma_0/kvstore-376/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-376/rev-000000001/seqIndex/state.1359: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.215422-08:00 CRITICAL [(default) magma_0/kvstore-384/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-384/rev-000000001/seqIndex/state.1361: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.218069-08:00 CRITICAL [(default) magma_0/kvstore-392/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-392/rev-000000001/seqIndex/state.1365: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.220452-08:00 CRITICAL [(default) magma_0/kvstore-400/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-400/rev-000000001/seqIndex/state.1353: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.222392-08:00 CRITICAL [(default) magma_0/kvstore-408/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-408/rev-000000001/seqIndex/state.1368: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.224845-08:00 CRITICAL [(default) magma_0/kvstore-416/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-416/rev-000000001/seqIndex/state.1357: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.227102-08:00 CRITICAL [(default) magma_0/kvstore-424/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-424/rev-000000001/seqIndex/state.1351: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.228908-08:00 CRITICAL [(default) magma_0/kvstore-512/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-512/rev-000000001/seqIndex/state.1367: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.230598-08:00 CRITICAL [(default) magma_0/kvstore-520/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-520/rev-000000001/seqIndex/state.1360: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.233392-08:00 CRITICAL [(default) magma_0/kvstore-528/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-528/rev-000000001/seqIndex/state.1356: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.236112-08:00 CRITICAL [(default) magma_0/kvstore-536/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-536/rev-000000001/seqIndex/state.1368: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.238197-08:00 CRITICAL [(default) magma_0/kvstore-544/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-544/rev-000000001/seqIndex/state.945: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.248538-08:00 CRITICAL [(default) magma_0/kvstore-552/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-552/rev-000000001/seqIndex/state.948: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.258743-08:00 CRITICAL [(default) magma_0/kvstore-560/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-560/rev-000000001/seqIndex/state.948: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.269619-08:00 CRITICAL [(default) magma_0/kvstore-568/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-568/rev-000000001/seqIndex/state.947: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.281590-08:00 CRITICAL [(default) magma_0/kvstore-576/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-576/rev-000000001/seqIndex/state.945: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.291426-08:00 CRITICAL [(default) magma_0/kvstore-584/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-584/rev-000000001/seqIndex/state.944: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.302238-08:00 CRITICAL [(default) magma_0/kvstore-592/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-592/rev-000000001/seqIndex/state.937: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.310399-08:00 CRITICAL [(default) magma_0/kvstore-600/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-600/rev-000000001/seqIndex/state.936: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.318228-08:00 CRITICAL [(default) magma_0/kvstore-608/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-608/rev-000000001/seqIndex/state.941: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.325420-08:00 CRITICAL [(default) magma_0/kvstore-616/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-616/rev-000000001/seqIndex/state.939: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.332126-08:00 CRITICAL [(default) magma_0/kvstore-624/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-624/rev-000000001/seqIndex/state.928: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.338177-08:00 CRITICAL [(default) magma_0/kvstore-632/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-632/rev-000000001/seqIndex/state.944: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.346008-08:00 CRITICAL [(default) magma_0/kvstore-640/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-640/rev-000000001/seqIndex/state.935: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.352894-08:00 CRITICAL [(default) magma_0/kvstore-648/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-648/rev-000000001/seqIndex/state.940: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.360519-08:00 CRITICAL [(default) magma_0/kvstore-656/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-656/rev-000000001/seqIndex/state.934: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.367190-08:00 CRITICAL [(default) magma_0/kvstore-664/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-664/rev-000000001/seqIndex/state.931: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.374701-08:00 CRITICAL [(default) magma_0/kvstore-672/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-672/rev-000000001/seqIndex/state.945: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.382805-08:00 CRITICAL [(default) magma_0/kvstore-680/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-680/rev-000000001/seqIndex/state.940: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.391298-08:00 CRITICAL [(default) magma_0/kvstore-688/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-688/rev-000000001/seqIndex/state.941: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.397528-08:00 CRITICAL [(default) magma_0/kvstore-696/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-696/rev-000000001/seqIndex/state.944: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', '[ns_server:info,2022-03-06T11:54:11.604-08:00,babysitter_of_ns_1@cb.local:<0.133.0>:ns_port_server:log:226]memcached<0.133.0>: 2022-03-06T11:54:11.403564-08:00 CRITICAL [(default) magma_0/kvstore-704/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-704/rev-000000001/seqIndex/state.942: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.410071-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/state.934: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.416158-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.952: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.422293-08:00 CRITICAL [(default) magma_0/kvstore-728/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-728/rev-000000001/seqIndex/state.946: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.428124-08:00 CRITICAL [(default) magma_0/kvstore-736/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-736/rev-000000001/seqIndex/state.929: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.433537-08:00 CRITICAL [(default) magma_0/kvstore-744/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-744/rev-000000001/seqIndex/state.936: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.440103-08:00 CRITICAL [(default) magma_0/kvstore-752/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-752/rev-000000001/seqIndex/state.934: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.446243-08:00 CRITICAL [(default) magma_0/kvstore-760/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-760/rev-000000001/seqIndex/state.946: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.454732-08:00 CRITICAL [(default) magma_0/kvstore-944/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-944/rev-000000001/seqIndex/state.1369: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.457007-08:00 CRITICAL [(default) magma_0/kvstore-952/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-952/rev-000000001/seqIndex/state.1365: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.459510-08:00 CRITICAL [(default) magma_0/kvstore-960/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-960/rev-000000001/seqIndex/state.1365: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.462064-08:00 CRITICAL [(default) magma_0/kvstore-968/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-968/rev-000000001/seqIndex/state.1373: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.464810-08:00 CRITICAL [(default) magma_0/kvstore-976/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-976/rev-000000001/seqIndex/state.1364: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.466990-08:00 CRITICAL [(default) magma_0/kvstore-984/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-984/rev-000000001/seqIndex/state.1369: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.469236-08:00 CRITICAL [(default) magma_0/kvstore-992/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-992/rev-000000001/seqIndex/state.1375: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.470957-08:00 CRITICAL [(default) magma_0/kvstore-1000/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-1000/rev-000000001/seqIndex/state.1368: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.473260-08:00 CRITICAL [(default) magma_0/kvstore-1008/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-1008/rev-000000001/seqIndex/state.1373: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.475245-08:00 CRITICAL [(default) magma_0/kvstore-1016/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-1016/rev-000000001/seqIndex/state.1369: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n'] 2022-03-06 13:45:34,882 | test | INFO | MainThread | [basetestcase:check_logs:824] Comparing timestamps: Log's latest timestamp: 2022-03-06 11:54:12, Test's start timestamp is 2022-03-06 11:47:20.041000 2022-03-06 13:45:34,891 | test | INFO | MainThread | [basetestcase:check_logs:824] Comparing timestamps: Log's latest timestamp: 2022-03-06 11:54:12, Test's start timestamp is 2022-03-06 11:47:20.041000 2022-03-06 13:45:34,891 | test | INFO | MainThread | [basetestcase:check_coredump_exist:933] unwanted messages in /opt/couchbase/var/lib/couchbase/logs/babysitter.log 2022-03-06 13:45:34,894 | test | CRITICAL | MainThread | [basetestcase:check_coredump_exist:935] 172.23.106.177: Found ' CRITICAL ' logs - ['[ns_server:info,2022-03-06T06:13:43.721-08:00,babysitter_of_ns_1@cb.local:<0.133.0>:ns_port_server:log:226]memcached<0.133.0>: 2022-03-06T06:13:43.520289-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during merge sstables (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1824.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0). \n', 'memcached<0.133.0>: 2022-03-06T06:13:43.550887-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/keyIndex]Error occured during merge sstables (IOError: /data/default/magma.0/kvstore-712/rev-000000001/keyIndex/sstable.3509.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0). \n', 'memcached<0.133.0>: 2022-03-06T06:13:43.551400-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1825.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.551675-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1826.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.551859-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1827.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.552066-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1828.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.552234-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1829.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.552451-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1830.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.552743-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1831.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.552956-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1832.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.553149-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1833.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.553354-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1834.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.553604-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1835.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.553800-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1836.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.553985-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1837.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.554185-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1838.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.554360-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1839.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.554533-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1840.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.554704-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1841.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.554873-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1842.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.555074-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1843.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.555241-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1844.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.555411-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1845.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.555583-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1846.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.555752-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1847.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.555922-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1848.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.556118-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1849.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.556288-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1850.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.556458-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1851.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.556622-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1852.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.556789-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1853.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.556957-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1854.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.557149-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1855.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.557317-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1856.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.557499-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1857.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.557669-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1858.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.557840-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1859.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.558009-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1860.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.558203-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1861.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.558369-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1862.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.558536-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1863.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.558705-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1864.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.558872-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1865.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.559064-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1866.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.559235-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1867.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.559403-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1868.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.559572-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1869.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.559747-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1870.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.559917-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1871.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.560106-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1872.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.560275-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1873.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.560445-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1874.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.560615-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1875.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.560780-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1876.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.560947-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1877.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.561134-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1878.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.561373-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1879.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.561675-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1880.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.562129-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1881.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.562423-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1882.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', '[ns_server:info,2022-03-06T06:13:44.805-08:00,babysitter_of_ns_1@cb.local:<0.133.0>:ns_port_server:log:226]memcached<0.133.0>: 2022-03-06T06:13:44.604108-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1078: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.605709-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1079: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.607275-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1080: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.608316-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1081: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.609569-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1082: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.610952-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1083: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.612477-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1084: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.613793-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1085: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.615223-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1086: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.616977-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1087: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.618630-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1088: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.620380-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1089: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.621833-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1090: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.623673-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1091: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.625437-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1092: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.627751-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1093: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.629589-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1094: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.631832-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1095: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.633942-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1096: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.636264-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1097: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.638721-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1098: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.641352-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1099: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.643643-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1100: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.646240-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1101: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.648966-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1102: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.651818-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1103: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.654207-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1104: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.656761-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1105: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.659698-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1106: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.662574-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1107: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.665451-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1108: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.668326-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1109: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.671541-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1110: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.674542-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1111: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.677588-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1112: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.681338-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1113: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.684760-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1114: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.688237-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1115: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.691798-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1116: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.695605-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1117: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.699291-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1118: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.703119-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1119: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.706991-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1120: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.711273-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1121: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.714891-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1122: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.719037-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1123: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.722784-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1124: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.727046-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1125: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.731283-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1126: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.735843-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1127: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.740229-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1128: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.744630-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1129: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.748947-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1130: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.753262-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1131: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.757824-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1132: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.762476-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1133: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.767118-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1134: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.771982-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1135: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.776850-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1136: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.782853-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1137: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', '[ns_server:info,2022-03-06T06:13:45.006-08:00,babysitter_of_ns_1@cb.local:<0.133.0>:ns_port_server:log:226]memcached<0.133.0>: 2022-03-06T06:13:44.806058-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1141: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.811948-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1142: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.817492-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1143: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.823125-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1144: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.828792-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1145: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.834292-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1146: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.840819-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1147: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.847571-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1148: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.853550-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1149: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.859525-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1150: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.865645-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1151: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.871873-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1152: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.877904-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1153: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.884091-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1154: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.890327-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1155: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.896693-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1156: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.903363-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1157: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.909858-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1158: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.916779-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1159: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.923488-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1160: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.929555-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1161: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.935346-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1162: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.941766-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1163: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.947986-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1164: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.954550-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1165: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.960693-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1166: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.967442-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1167: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.973800-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1168: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.980344-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1169: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.987179-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1170: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.993706-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1171: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.000709-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1172: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', '[ns_server:info,2022-03-06T06:13:45.208-08:00,babysitter_of_ns_1@cb.local:<0.133.0>:ns_port_server:log:226]memcached<0.133.0>: 2022-03-06T06:13:45.007855-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1173: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.015646-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1174: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.022666-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1175: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.030525-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1176: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.038351-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1177: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.045598-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1178: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.053203-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1179: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.060636-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1180: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.068199-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1181: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.075727-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1182: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.083501-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1183: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.091192-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1184: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.099627-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1185: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.108553-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1186: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.117571-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1187: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.125872-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1188: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.133837-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1189: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.142533-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1190: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.151791-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1191: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.161327-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1192: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.171049-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1193: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.180470-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1194: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.189057-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1195: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.197934-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1196: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.207684-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1197: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', '[ns_server:info,2022-03-06T06:13:45.419-08:00,babysitter_of_ns_1@cb.local:<0.133.0>:ns_port_server:log:226]memcached<0.133.0>: 2022-03-06T06:13:45.218036-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1198: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.227868-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1199: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.237864-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1200: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.248405-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1201: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.258483-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1202: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.266946-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1203: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.276857-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1204: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.287113-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1205: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.297550-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1206: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.306806-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1207: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.315073-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1208: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.322360-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1209: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.331792-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1210: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.341207-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1211: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.351110-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1212: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.361486-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1213: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.372088-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1214: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.382549-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1215: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.393068-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1216: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.403484-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1217: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.414167-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1218: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', '[ns_server:info,2022-03-06T06:13:45.625-08:00,babysitter_of_ns_1@cb.local:<0.133.0>:ns_port_server:log:226]memcached<0.133.0>: 2022-03-06T06:13:45.425378-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1219: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.436074-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1220: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.446796-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1221: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.457615-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1222: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.468607-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1223: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.479487-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1224: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.490579-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1225: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.501735-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1226: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.513119-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1227: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.524394-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1228: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', '[ns_server:info,2022-03-06T06:13:45.994-08:00,babysitter_of_ns_1@cb.local:<0.133.0>:ns_port_server:log:226]memcached<0.133.0>: 2022-03-06T06:13:45.793886-08:00 CRITICAL [(default) magma_0/kvstore-176/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-176/rev-000000001/seqIndex/state.1073: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.800207-08:00 CRITICAL [(default) magma_0/kvstore-184/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-184/rev-000000001/seqIndex/state.1079: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.806224-08:00 CRITICAL [(default) magma_0/kvstore-192/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-192/rev-000000001/seqIndex/state.1082: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.812545-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.1081: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.818656-08:00 CRITICAL [(default) magma_0/kvstore-208/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-208/rev-000000001/seqIndex/state.1089: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.824592-08:00 CRITICAL [(default) magma_0/kvstore-216/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-216/rev-000000001/seqIndex/state.1083: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.832464-08:00 CRITICAL [(default) magma_0/kvstore-224/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-224/rev-000000001/seqIndex/state.1086: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.841121-08:00 CRITICAL [(default) magma_0/kvstore-232/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-232/rev-000000001/seqIndex/state.1085: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.847768-08:00 CRITICAL [(default) magma_0/kvstore-240/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-240/rev-000000001/seqIndex/state.1077: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.853901-08:00 CRITICAL [(default) magma_0/kvstore-248/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-248/rev-000000001/seqIndex/state.1076: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.860891-08:00 CRITICAL [(default) magma_0/kvstore-432/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-432/rev-000000001/seqIndex/state.1083: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.865389-08:00 CRITICAL [(default) magma_0/kvstore-440/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-440/rev-000000001/seqIndex/state.1378: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.867251-08:00 CRITICAL [(default) magma_0/kvstore-448/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-448/rev-000000001/seqIndex/state.1384: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.868606-08:00 CRITICAL [(default) magma_0/kvstore-456/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-456/rev-000000001/seqIndex/state.1385: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.869859-08:00 CRITICAL [(default) magma_0/kvstore-464/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-464/rev-000000001/seqIndex/state.1372: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.872472-08:00 CRITICAL [(default) magma_0/kvstore-472/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-472/rev-000000001/seqIndex/state.1369: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.874793-08:00 CRITICAL [(default) magma_0/kvstore-480/rev-000000001/localIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-480/rev-000000001/localIndex/sstable.2024.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default).\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.877801-08:00 CRITICAL [(default) magma_0/kvstore-488/rev-000000001/localIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-488/rev-000000001/localIndex/sstable.2021.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default).\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.880316-08:00 CRITICAL [(default) magma_0/kvstore-496/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-496/rev-000000001/seqIndex/state.1378: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.883644-08:00 CRITICAL [(default) magma_0/kvstore-504/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-504/rev-000000001/seqIndex/state.1084: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.888527-08:00 CRITICAL [(default) magma_0/kvstore-688/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-688/rev-000000001/seqIndex/state.1368: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.890998-08:00 CRITICAL [(default) magma_0/kvstore-696/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-696/rev-000000001/seqIndex/state.1374: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.892699-08:00 CRITICAL [(default) magma_0/kvstore-704/rev-000000001/localIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-704/rev-000000001/localIndex/sstable.2004.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default).\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.894413-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/sstable.1931.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default).\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.899532-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.1229: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.907735-08:00 CRITICAL [(default) magma_0/kvstore-728/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-728/rev-000000001/seqIndex/state.1077: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.912357-08:00 CRITICAL [(default) magma_0/kvstore-736/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-736/rev-000000001/seqIndex/state.1071: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.916746-08:00 CRITICAL [(default) magma_0/kvstore-744/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-744/rev-000000001/seqIndex/state.1077: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.921750-08:00 CRITICAL [(default) magma_0/kvstore-752/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-752/rev-000000001/seqIndex/state.1080: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.925366-08:00 CRITICAL [(default) magma_0/kvstore-760/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-760/rev-000000001/seqIndex/state.1078: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.929761-08:00 CRITICAL [(default) magma_0/kvstore-768/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-768/rev-000000001/seqIndex/state.1076: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.938837-08:00 CRITICAL [(default) magma_0/kvstore-776/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-776/rev-000000001/seqIndex/state.1090: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.944984-08:00 CRITICAL [(default) magma_0/kvstore-784/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-784/rev-000000001/seqIndex/state.1082: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.951195-08:00 CRITICAL [(default) magma_0/kvstore-792/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-792/rev-000000001/seqIndex/state.1083: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.958047-08:00 CRITICAL [(default) magma_0/kvstore-800/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-800/rev-000000001/seqIndex/state.1077: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.964941-08:00 CRITICAL [(default) magma_0/kvstore-808/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-808/rev-000000001/seqIndex/state.1081: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.971776-08:00 CRITICAL [(default) magma_0/kvstore-816/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-816/rev-000000001/seqIndex/state.1084: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.978801-08:00 CRITICAL [(default) magma_0/kvstore-824/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-824/rev-000000001/seqIndex/state.1080: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.986628-08:00 CRITICAL [(default) magma_0/kvstore-832/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-832/rev-000000001/seqIndex/state.1083: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.993831-08:00 CRITICAL [(default) magma_0/kvstore-840/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-840/rev-000000001/seqIndex/state.1090: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', '[ns_server:info,2022-03-06T06:13:46.200-08:00,babysitter_of_ns_1@cb.local:<0.133.0>:ns_port_server:log:226]memcached<0.133.0>: 2022-03-06T06:13:46.000701-08:00 CRITICAL [(default) magma_0/kvstore-848/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-848/rev-000000001/seqIndex/state.1080: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.006794-08:00 CRITICAL [(default) magma_0/kvstore-856/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-856/rev-000000001/seqIndex/state.1085: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.010415-08:00 CRITICAL [(default) magma_0/kvstore-864/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-864/rev-000000001/seqIndex/state.1078: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.015158-08:00 CRITICAL [(default) magma_0/kvstore-872/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-872/rev-000000001/seqIndex/state.1087: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.019502-08:00 CRITICAL [(default) magma_0/kvstore-880/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-880/rev-000000001/seqIndex/state.1087: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.023407-08:00 CRITICAL [(default) magma_0/kvstore-888/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-888/rev-000000001/seqIndex/state.1082: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.027634-08:00 CRITICAL [(default) magma_0/kvstore-896/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-896/rev-000000001/seqIndex/state.1081: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.039501-08:00 CRITICAL [(default) magma_0/kvstore-904/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-904/rev-000000001/seqIndex/state.1088: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.043365-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1092: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.047976-08:00 CRITICAL [(default) magma_0/kvstore-920/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-920/rev-000000001/seqIndex/state.1085: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.051897-08:00 CRITICAL [(default) magma_0/kvstore-928/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-928/rev-000000001/seqIndex/state.1091: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.056095-08:00 CRITICAL [(default) magma_0/kvstore-936/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-936/rev-000000001/seqIndex/state.1085: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.060468-08:00 CRITICAL [(default) magma_0/kvstore-944/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-944/rev-000000001/seqIndex/state.1085: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.064519-08:00 CRITICAL [(default) magma_0/kvstore-952/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-952/rev-000000001/seqIndex/state.1082: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.068800-08:00 CRITICAL [(default) magma_0/kvstore-960/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-960/rev-000000001/seqIndex/state.1087: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.073069-08:00 CRITICAL [(default) magma_0/kvstore-968/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-968/rev-000000001/seqIndex/state.1082: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.077195-08:00 CRITICAL [(default) magma_0/kvstore-976/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-976/rev-000000001/seqIndex/state.1075: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.081689-08:00 CRITICAL [(default) magma_0/kvstore-984/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-984/rev-000000001/seqIndex/state.1072: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.086877-08:00 CRITICAL [(default) magma_0/kvstore-992/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-992/rev-000000001/seqIndex/state.1084: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.090954-08:00 CRITICAL [(default) magma_0/kvstore-1000/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-1000/rev-000000001/seqIndex/state.1071: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.094751-08:00 CRITICAL [(default) magma_0/kvstore-1008/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-1008/rev-000000001/seqIndex/state.1069: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.099226-08:00 CRITICAL [(default) magma_0/kvstore-1016/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-1016/rev-000000001/seqIndex/state.1088: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', '[ns_server:info,2022-03-06T06:13:46.449-08:00,babysitter_of_ns_1@cb.local:<0.133.0>:ns_port_server:log:226]memcached<0.133.0>: 2022-03-06T06:13:46.249255-08:00 CRITICAL [(default) magma_1/kvstore-833/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-833/rev-000000001/seqIndex/state.1081: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.256030-08:00 CRITICAL [(default) magma_1/kvstore-841/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-841/rev-000000001/seqIndex/state.1074: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.265182-08:00 CRITICAL [(default) magma_1/kvstore-849/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-849/rev-000000001/seqIndex/state.1078: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.271517-08:00 CRITICAL [(default) magma_1/kvstore-857/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-857/rev-000000001/seqIndex/state.1087: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.275989-08:00 CRITICAL [(default) magma_1/kvstore-865/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-865/rev-000000001/seqIndex/state.1090: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.280316-08:00 CRITICAL [(default) magma_1/kvstore-873/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-873/rev-000000001/seqIndex/state.1084: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.284433-08:00 CRITICAL [(default) magma_1/kvstore-881/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-881/rev-000000001/seqIndex/state.1077: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.294301-08:00 CRITICAL [(default) magma_1/kvstore-889/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-889/rev-000000001/seqIndex/state.1079: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.299549-08:00 CRITICAL [(default) magma_1/kvstore-897/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-897/rev-000000001/seqIndex/state.1096: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.302884-08:00 CRITICAL [(default) magma_1/kvstore-905/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-905/rev-000000001/seqIndex/state.1081: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.307271-08:00 CRITICAL [(default) magma_1/kvstore-913/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-913/rev-000000001/seqIndex/state.1091: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.311524-08:00 CRITICAL [(default) magma_1/kvstore-921/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-921/rev-000000001/seqIndex/state.1089: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.315214-08:00 CRITICAL [(default) magma_1/kvstore-929/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-929/rev-000000001/seqIndex/state.1085: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.319675-08:00 CRITICAL [(default) magma_1/kvstore-937/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-937/rev-000000001/seqIndex/state.1093: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.323803-08:00 CRITICAL [(default) magma_1/kvstore-945/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-945/rev-000000001/seqIndex/state.1080: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.327485-08:00 CRITICAL [(default) magma_1/kvstore-953/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-953/rev-000000001/seqIndex/state.1081: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.331163-08:00 CRITICAL [(default) magma_1/kvstore-961/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-961/rev-000000001/seqIndex/state.1374: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.332898-08:00 CRITICAL [(default) magma_1/kvstore-969/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-969/rev-000000001/seqIndex/state.1382: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.334841-08:00 CRITICAL [(default) magma_1/kvstore-977/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-977/rev-000000001/seqIndex/state.1375: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.336744-08:00 CRITICAL [(default) magma_1/kvstore-985/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-985/rev-000000001/seqIndex/state.1377: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.338622-08:00 CRITICAL [(default) magma_1/kvstore-993/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-993/rev-000000001/seqIndex/state.1389: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.340264-08:00 CRITICAL [(default) magma_1/kvstore-1001/rev-000000001/localIndex]Error occured during memtable flush (IOError: /data/default/magma.1/kvstore-1001/rev-000000001/localIndex/sstable.1997.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1).\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.342754-08:00 CRITICAL [(default) magma_1/kvstore-1009/rev-000000001/localIndex]Error occured during memtable flush (IOError: /data/default/magma.1/kvstore-1009/rev-000000001/localIndex/sstable.2039.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1).\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.344185-08:00 CRITICAL [(default) magma_1/kvstore-1017/rev-000000001/localIndex]Error occured during memtable flush (IOError: /data/default/magma.1/kvstore-1017/rev-000000001/localIndex/sstable.2027.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1).\n', '[ns_server:info,2022-03-06T11:54:09.876-08:00,babysitter_of_ns_1@cb.local:<0.133.0>:ns_port_server:log:226]memcached<0.133.0>: 2022-03-06T11:54:09.670616-08:00 CRITICAL [(default) magma_0/kvstore-904/rev-000000001/keyIndex]Error occured during merge sstables (IOError: /data/default/magma.0/kvstore-904/rev-000000001/keyIndex/sstable.2283.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0). \n', 'memcached<0.133.0>: 2022-03-06T11:54:09.671076-08:00 CRITICAL [(default) magma_0/kvstore-904/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-904/rev-000000001/seqIndex/sstable.1713.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.671358-08:00 CRITICAL [(default) magma_0/kvstore-904/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-904/rev-000000001/seqIndex/sstable.1714.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.671536-08:00 CRITICAL [(default) magma_0/kvstore-904/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-904/rev-000000001/seqIndex/sstable.1715.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.672054-08:00 CRITICAL [(default) magma_0/kvstore-904/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-904/rev-000000001/seqIndex/sstable.1716.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.672258-08:00 CRITICAL [(default) magma_0/kvstore-904/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-904/rev-000000001/seqIndex/sstable.1717.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.672458-08:00 CRITICAL [(default) magma_0/kvstore-904/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-904/rev-000000001/seqIndex/sstable.1718.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.672798-08:00 CRITICAL [(default) magma_0/kvstore-904/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-904/rev-000000001/seqIndex/sstable.1719.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.673096-08:00 CRITICAL [(default) magma_0/kvstore-904/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-904/rev-000000001/seqIndex/sstable.1720.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.673292-08:00 CRITICAL [(default) magma_0/kvstore-904/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-904/rev-000000001/seqIndex/sstable.1721.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.673486-08:00 CRITICAL [(default) magma_0/kvstore-904/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-904/rev-000000001/seqIndex/sstable.1722.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.673675-08:00 CRITICAL [(default) magma_0/kvstore-904/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-904/rev-000000001/seqIndex/sstable.1723.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.673853-08:00 CRITICAL [(default) magma_0/kvstore-904/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-904/rev-000000001/seqIndex/sstable.1724.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.674040-08:00 CRITICAL [(default) magma_0/kvstore-904/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-904/rev-000000001/seqIndex/sstable.1725.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.674212-08:00 CRITICAL [(default) magma_0/kvstore-904/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-904/rev-000000001/seqIndex/sstable.1726.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.674437-08:00 CRITICAL [(default) magma_0/kvstore-904/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-904/rev-000000001/seqIndex/sstable.1727.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.674659-08:00 CRITICAL [(default) magma_0/kvstore-904/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-904/rev-000000001/seqIndex/sstable.1728.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.674847-08:00 CRITICAL [(default) magma_0/kvstore-904/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-904/rev-000000001/seqIndex/sstable.1729.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.675065-08:00 CRITICAL [(default) magma_0/kvstore-904/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-904/rev-000000001/seqIndex/sstable.1730.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.675318-08:00 CRITICAL [(default) magma_0/kvstore-904/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-904/rev-000000001/seqIndex/sstable.1731.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.675511-08:00 CRITICAL [(default) magma_0/kvstore-904/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-904/rev-000000001/seqIndex/sstable.1732.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.675719-08:00 CRITICAL [(default) magma_0/kvstore-904/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-904/rev-000000001/seqIndex/sstable.1733.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.675897-08:00 CRITICAL [(default) magma_0/kvstore-904/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-904/rev-000000001/seqIndex/sstable.1734.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.676096-08:00 CRITICAL [(default) magma_0/kvstore-904/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-904/rev-000000001/seqIndex/sstable.1735.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.676280-08:00 CRITICAL [(default) magma_0/kvstore-904/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-904/rev-000000001/seqIndex/sstable.1736.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.676481-08:00 CRITICAL [(default) magma_0/kvstore-904/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-904/rev-000000001/seqIndex/sstable.1737.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.676646-08:00 CRITICAL [(default) magma_0/kvstore-904/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-904/rev-000000001/seqIndex/sstable.1738.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.676808-08:00 CRITICAL [(default) magma_0/kvstore-904/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-904/rev-000000001/seqIndex/sstable.1739.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.677051-08:00 CRITICAL [(default) magma_0/kvstore-904/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-904/rev-000000001/seqIndex/sstable.1740.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.677273-08:00 CRITICAL [(default) magma_0/kvstore-904/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-904/rev-000000001/seqIndex/sstable.1741.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.677484-08:00 CRITICAL [(default) magma_0/kvstore-904/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-904/rev-000000001/seqIndex/sstable.1742.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.677658-08:00 CRITICAL [(default) magma_0/kvstore-904/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-904/rev-000000001/seqIndex/sstable.1743.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.677834-08:00 CRITICAL [(default) magma_0/kvstore-904/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-904/rev-000000001/seqIndex/sstable.1744.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.678002-08:00 CRITICAL [(default) magma_0/kvstore-904/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-904/rev-000000001/seqIndex/sstable.1745.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.678190-08:00 CRITICAL [(default) magma_0/kvstore-904/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-904/rev-000000001/seqIndex/sstable.1746.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.678353-08:00 CRITICAL [(default) magma_0/kvstore-904/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-904/rev-000000001/seqIndex/sstable.1747.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.678517-08:00 CRITICAL [(default) magma_0/kvstore-904/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-904/rev-000000001/seqIndex/sstable.1748.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.678692-08:00 CRITICAL [(default) magma_0/kvstore-904/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-904/rev-000000001/seqIndex/sstable.1749.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.678907-08:00 CRITICAL [(default) magma_0/kvstore-904/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-904/rev-000000001/seqIndex/sstable.1750.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.679129-08:00 CRITICAL [(default) magma_0/kvstore-904/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-904/rev-000000001/seqIndex/sstable.1751.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.679338-08:00 CRITICAL [(default) magma_0/kvstore-904/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-904/rev-000000001/seqIndex/sstable.1752.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.679517-08:00 CRITICAL [(default) magma_0/kvstore-904/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-904/rev-000000001/seqIndex/sstable.1753.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.679823-08:00 CRITICAL [(default) magma_0/kvstore-904/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-904/rev-000000001/seqIndex/sstable.1754.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.680007-08:00 CRITICAL [(default) magma_0/kvstore-904/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-904/rev-000000001/seqIndex/sstable.1755.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.680219-08:00 CRITICAL [(default) magma_0/kvstore-904/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-904/rev-000000001/seqIndex/sstable.1756.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.680412-08:00 CRITICAL [(default) magma_0/kvstore-904/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-904/rev-000000001/seqIndex/sstable.1757.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.680580-08:00 CRITICAL [(default) magma_0/kvstore-904/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-904/rev-000000001/seqIndex/sstable.1758.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.680757-08:00 CRITICAL [(default) magma_0/kvstore-904/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-904/rev-000000001/seqIndex/sstable.1759.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.680920-08:00 CRITICAL [(default) magma_0/kvstore-904/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-904/rev-000000001/seqIndex/sstable.1760.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.681108-08:00 CRITICAL [(default) magma_0/kvstore-904/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-904/rev-000000001/seqIndex/sstable.1761.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.681274-08:00 CRITICAL [(default) magma_0/kvstore-904/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-904/rev-000000001/seqIndex/sstable.1762.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.681436-08:00 CRITICAL [(default) magma_0/kvstore-904/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-904/rev-000000001/seqIndex/sstable.1763.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.681598-08:00 CRITICAL [(default) magma_0/kvstore-904/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-904/rev-000000001/seqIndex/sstable.1764.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.681797-08:00 CRITICAL [(default) magma_0/kvstore-904/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-904/rev-000000001/seqIndex/sstable.1765.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.682076-08:00 CRITICAL [(default) magma_0/kvstore-904/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-904/rev-000000001/seqIndex/sstable.1766.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.682284-08:00 CRITICAL [(default) magma_0/kvstore-904/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-904/rev-000000001/seqIndex/sstable.1767.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.682483-08:00 CRITICAL [(default) magma_0/kvstore-904/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-904/rev-000000001/seqIndex/sstable.1768.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.682649-08:00 CRITICAL [(default) magma_0/kvstore-904/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-904/rev-000000001/seqIndex/sstable.1769.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.682827-08:00 CRITICAL [(default) magma_0/kvstore-904/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-904/rev-000000001/seqIndex/sstable.1770.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.682990-08:00 CRITICAL [(default) magma_0/kvstore-904/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-904/rev-000000001/seqIndex/sstable.1771.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', '[ns_server:info,2022-03-06T11:54:10.082-08:00,babysitter_of_ns_1@cb.local:<0.133.0>:ns_port_server:log:226]memcached<0.133.0>: 2022-03-06T11:54:09.875847-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.979: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.879551-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.980: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.884276-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.981: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.889654-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.982: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.895637-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.983: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.902337-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.984: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.908616-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.985: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.915131-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.986: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.921036-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.987: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.927176-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.988: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.933651-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.989: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.939971-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.990: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.946636-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.991: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.953268-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.992: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.959832-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.993: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.966591-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.994: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.972840-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.995: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.979226-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.996: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.985103-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.997: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.991194-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.998: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.998813-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.999: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.005171-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1000: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.011914-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1001: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.019820-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1002: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.027126-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1003: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.034036-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1004: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.041520-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1005: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.049062-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1006: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.056399-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1007: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.063531-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1008: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.071162-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1009: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.079081-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1010: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', '[ns_server:info,2022-03-06T11:54:10.287-08:00,babysitter_of_ns_1@cb.local:<0.133.0>:ns_port_server:log:226]memcached<0.133.0>: 2022-03-06T11:54:10.087041-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1011: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.094264-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1012: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.101855-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1013: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.109833-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1014: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.118380-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1015: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.126553-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1016: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.134212-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1017: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.142326-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1018: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.149460-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1019: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.158148-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1020: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.167791-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1021: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.176682-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1022: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.184551-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1023: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.194208-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1024: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.201432-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1025: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.207397-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1026: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.223774-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1027: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.231935-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1028: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.238280-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1029: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.245534-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1030: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.257877-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1031: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.264617-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1032: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.272360-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1033: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.280311-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1034: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', '[ns_server:info,2022-03-06T11:54:10.488-08:00,babysitter_of_ns_1@cb.local:<0.133.0>:ns_port_server:log:226]memcached<0.133.0>: 2022-03-06T11:54:10.287584-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1035: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.296189-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1036: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.304897-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1037: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.314381-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1038: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.323656-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1039: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.333543-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1040: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.342948-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1041: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.353130-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1042: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.363253-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1043: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.373633-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1044: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.382475-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1045: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.392081-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1046: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.402575-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1047: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.413399-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1048: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.423977-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1049: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.434578-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1050: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.444841-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1051: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.455521-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1052: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.466101-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1053: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.477182-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1054: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.487932-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1055: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', '[ns_server:info,2022-03-06T11:54:10.699-08:00,babysitter_of_ns_1@cb.local:<0.133.0>:ns_port_server:log:226]memcached<0.133.0>: 2022-03-06T11:54:10.498864-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1056: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.510108-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1057: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.521150-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1058: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.532425-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1059: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.543458-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1060: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.554104-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1061: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.564348-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1062: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.574461-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1063: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.585639-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1064: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.597264-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1065: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.608832-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1066: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.620109-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1067: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.630183-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1068: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.640496-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1069: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.648657-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1070: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.658364-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1071: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.668608-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1072: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.680261-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1073: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.692148-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1074: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', '[ns_server:info,2022-03-06T11:54:10.904-08:00,babysitter_of_ns_1@cb.local:<0.133.0>:ns_port_server:log:226]memcached<0.133.0>: 2022-03-06T11:54:10.704098-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1075: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.717263-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1076: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.728993-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1077: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.740564-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1078: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.751783-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1079: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.762933-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1080: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.777583-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1081: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.789455-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1082: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.801765-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1083: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.813509-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1084: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.825939-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1085: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.838877-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1086: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.852285-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1087: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.864140-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1088: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.876435-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1089: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.888721-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1090: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.901992-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1091: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', '[ns_server:info,2022-03-06T11:54:11.116-08:00,babysitter_of_ns_1@cb.local:<0.133.0>:ns_port_server:log:226]memcached<0.133.0>: 2022-03-06T11:54:10.916294-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1092: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.929629-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1093: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.943195-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1094: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.956525-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1095: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.970004-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1096: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.982537-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1097: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.996906-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1098: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.010803-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1099: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.025003-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1100: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.038421-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1101: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.053422-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1102: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.067650-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1103: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.081822-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1104: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.096131-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1105: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.110444-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1106: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', '[ns_server:info,2022-03-06T11:54:11.325-08:00,babysitter_of_ns_1@cb.local:<0.133.0>:ns_port_server:log:226]memcached<0.133.0>: 2022-03-06T11:54:11.125054-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1107: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.139763-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1108: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.154571-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1109: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.171879-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1110: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.187668-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1111: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.200362-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1112: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.213304-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1113: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.227224-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1114: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.241686-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1115: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.257000-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1116: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.274272-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1117: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.289407-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1118: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.304875-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1119: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.319819-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1120: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', '[ns_server:info,2022-03-06T11:54:11.535-08:00,babysitter_of_ns_1@cb.local:<0.133.0>:ns_port_server:log:226]memcached<0.133.0>: 2022-03-06T11:54:11.335168-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1121: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.350767-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1122: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.364270-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1123: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.376312-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1124: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.389177-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1125: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.404660-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1126: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.421413-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1127: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.437716-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1128: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.453849-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1129: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.469941-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1130: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.485867-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1131: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.502333-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1132: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.515993-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1133: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.529939-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1134: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', '[ns_server:info,2022-03-06T11:54:11.748-08:00,babysitter_of_ns_1@cb.local:<0.133.0>:ns_port_server:log:226]memcached<0.133.0>: 2022-03-06T11:54:11.547753-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1135: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', '[ns_server:info,2022-03-06T11:54:12.113-08:00,babysitter_of_ns_1@cb.local:<0.133.0>:ns_port_server:log:226]memcached<0.133.0>: 2022-03-06T11:54:11.912838-08:00 CRITICAL [(default) magma_0/kvstore-472/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-472/rev-000000001/seqIndex/state.936: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0/kvstore-472/rev-000000001\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.920130-08:00 CRITICAL [(default) magma_0/kvstore-480/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-480/rev-000000001/seqIndex/state.930: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.927203-08:00 CRITICAL [(default) magma_0/kvstore-488/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-488/rev-000000001/seqIndex/state.929: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.933423-08:00 CRITICAL [(default) magma_0/kvstore-496/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-496/rev-000000001/seqIndex/state.941: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.940796-08:00 CRITICAL [(default) magma_0/kvstore-504/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-504/rev-000000001/seqIndex/state.937: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.946100-08:00 CRITICAL [(default) magma_0/kvstore-688/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-688/rev-000000001/seqIndex/state.932: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.950998-08:00 CRITICAL [(default) magma_0/kvstore-696/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-696/rev-000000001/seqIndex/state.929: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.958049-08:00 CRITICAL [(default) magma_0/kvstore-704/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-704/rev-000000001/seqIndex/state.940: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.963100-08:00 CRITICAL [(default) magma_0/kvstore-712/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-712/rev-000000001/seqIndex/state.932: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.968957-08:00 CRITICAL [(default) magma_0/kvstore-720/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-720/rev-000000001/seqIndex/state.938: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.977574-08:00 CRITICAL [(default) magma_0/kvstore-728/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-728/rev-000000001/seqIndex/state.936: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.983999-08:00 CRITICAL [(default) magma_0/kvstore-736/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-736/rev-000000001/seqIndex/state.936: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.990427-08:00 CRITICAL [(default) magma_0/kvstore-744/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-744/rev-000000001/seqIndex/state.932: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.996730-08:00 CRITICAL [(default) magma_0/kvstore-752/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-752/rev-000000001/seqIndex/state.948: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.003123-08:00 CRITICAL [(default) magma_0/kvstore-760/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-760/rev-000000001/seqIndex/state.942: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.009112-08:00 CRITICAL [(default) magma_0/kvstore-768/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-768/rev-000000001/seqIndex/state.1359: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.010713-08:00 CRITICAL [(default) magma_0/kvstore-776/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-776/rev-000000001/seqIndex/state.1358: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.012880-08:00 CRITICAL [(default) magma_0/kvstore-784/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-784/rev-000000001/seqIndex/state.1349: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.015184-08:00 CRITICAL [(default) magma_0/kvstore-792/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-792/rev-000000001/seqIndex/state.1363: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.017524-08:00 CRITICAL [(default) magma_0/kvstore-800/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-800/rev-000000001/seqIndex/state.1362: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.019875-08:00 CRITICAL [(default) magma_0/kvstore-808/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-808/rev-000000001/seqIndex/state.1363: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.021602-08:00 CRITICAL [(default) magma_0/kvstore-816/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-816/rev-000000001/seqIndex/state.1356: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.023333-08:00 CRITICAL [(default) magma_0/kvstore-824/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-824/rev-000000001/seqIndex/state.1365: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.025110-08:00 CRITICAL [(default) magma_0/kvstore-832/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-832/rev-000000001/seqIndex/state.1372: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.027168-08:00 CRITICAL [(default) magma_0/kvstore-840/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-840/rev-000000001/seqIndex/state.1364: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.028854-08:00 CRITICAL [(default) magma_0/kvstore-848/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-848/rev-000000001/seqIndex/state.1352: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.030679-08:00 CRITICAL [(default) magma_0/kvstore-856/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-856/rev-000000001/seqIndex/state.1366: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.032385-08:00 CRITICAL [(default) magma_0/kvstore-864/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-864/rev-000000001/seqIndex/state.1356: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.034277-08:00 CRITICAL [(default) magma_0/kvstore-872/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-872/rev-000000001/seqIndex/state.1363: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.036098-08:00 CRITICAL [(default) magma_0/kvstore-880/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-880/rev-000000001/seqIndex/state.1359: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.038306-08:00 CRITICAL [(default) magma_0/kvstore-888/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-888/rev-000000001/seqIndex/state.1362: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.039718-08:00 CRITICAL [(default) magma_0/kvstore-896/rev-000000001/localIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-896/rev-000000001/localIndex/sstable.1971.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.041116-08:00 CRITICAL [(default) magma_0/kvstore-904/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-904/rev-000000001/seqIndex/sstable.1859.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0).\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.045051-08:00 CRITICAL [(default) magma_0/kvstore-912/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-912/rev-000000001/seqIndex/state.1136: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.060937-08:00 CRITICAL [(default) magma_0/kvstore-920/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-920/rev-000000001/seqIndex/state.935: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.066270-08:00 CRITICAL [(default) magma_0/kvstore-928/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-928/rev-000000001/seqIndex/state.933: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.073357-08:00 CRITICAL [(default) magma_0/kvstore-936/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-936/rev-000000001/seqIndex/state.938: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.080057-08:00 CRITICAL [(default) magma_0/kvstore-944/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-944/rev-000000001/seqIndex/state.931: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.086560-08:00 CRITICAL [(default) magma_0/kvstore-952/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-952/rev-000000001/seqIndex/state.940: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.093699-08:00 CRITICAL [(default) magma_0/kvstore-960/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-960/rev-000000001/seqIndex/state.924: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.099771-08:00 CRITICAL [(default) magma_0/kvstore-968/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-968/rev-000000001/seqIndex/state.934: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.105921-08:00 CRITICAL [(default) magma_0/kvstore-976/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-976/rev-000000001/seqIndex/state.925: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.112464-08:00 CRITICAL [(default) magma_0/kvstore-984/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-984/rev-000000001/seqIndex/state.939: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', '[ns_server:info,2022-03-06T11:54:12.320-08:00,babysitter_of_ns_1@cb.local:<0.133.0>:ns_port_server:log:226]memcached<0.133.0>: 2022-03-06T11:54:12.119766-08:00 CRITICAL [(default) magma_0/kvstore-992/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-992/rev-000000001/seqIndex/state.928: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.127926-08:00 CRITICAL [(default) magma_0/kvstore-1000/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-1000/rev-000000001/seqIndex/state.938: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.136383-08:00 CRITICAL [(default) magma_0/kvstore-1008/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-1008/rev-000000001/seqIndex/state.924: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.143782-08:00 CRITICAL [(default) magma_0/kvstore-1016/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-1016/rev-000000001/seqIndex/state.935: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n'] 2022-03-06 13:45:35,250 | test | INFO | MainThread | [basetestcase:check_logs:824] Comparing timestamps: Log's latest timestamp: 2022-03-06 06:13:46, Test's start timestamp is 2022-03-06 11:47:20.041000 2022-03-06 13:45:35,250 | test | INFO | MainThread | [basetestcase:check_coredump_exist:921] unwanted messages in /opt/couchbase/var/lib/couchbase/logs/memcached.log.000002.txt 2022-03-06 13:45:35,250 | test | INFO | MainThread | [basetestcase:check_logs:824] Comparing timestamps: Log's latest timestamp: 2022-03-06 06:13:46, Test's start timestamp is 2022-03-06 11:47:20.041000 2022-03-06 13:45:35,309 | test | INFO | MainThread | [basetestcase:check_logs:824] Comparing timestamps: Log's latest timestamp: 2022-03-06 11:54:13, Test's start timestamp is 2022-03-06 11:47:20.041000 2022-03-06 13:45:35,319 | test | INFO | MainThread | [basetestcase:check_logs:824] Comparing timestamps: Log's latest timestamp: 2022-03-06 11:54:13, Test's start timestamp is 2022-03-06 11:47:20.041000 2022-03-06 13:45:35,319 | test | INFO | MainThread | [basetestcase:check_coredump_exist:933] unwanted messages in /opt/couchbase/var/lib/couchbase/logs/babysitter.log 2022-03-06 13:45:35,325 | test | CRITICAL | MainThread | [basetestcase:check_coredump_exist:935] 172.23.100.35: Found ' CRITICAL ' logs - ['[ns_server:info,2022-03-06T06:13:43.498-08:00,babysitter_of_ns_1@cb.local:<0.133.0>:ns_port_server:log:226]memcached<0.133.0>: 2022-03-06T06:13:43.297341-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/keyIndex]Error occured during merge sstables (IOError: /data/default/magma.0/kvstore-200/rev-000000001/keyIndex/sstable.1297.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0). \n', 'memcached<0.133.0>: 2022-03-06T06:13:43.297795-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.345: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.298263-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.346: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.298672-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.347: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.299150-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.348: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.299701-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.349: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.300259-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.350: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.301022-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.351: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.301844-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.352: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.302641-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.353: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.303629-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.354: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.304684-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.355: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.305841-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.356: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.307070-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.357: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.308244-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.358: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.309408-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.359: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.310805-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.360: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.312134-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.361: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.313712-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.362: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.315291-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.363: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.317005-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.364: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.318876-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.365: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.320736-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.366: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.322638-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.367: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.324606-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.368: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.326879-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.369: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.328794-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.370: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.331899-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.371: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.334765-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.372: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.337533-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.373: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.340410-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.374: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.343296-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.375: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.346058-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.376: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.348965-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.377: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.351970-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.378: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.354606-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.379: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.357544-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.380: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.360618-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.381: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.363581-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.382: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.366845-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.383: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.370179-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.384: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.373766-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.385: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.377505-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.386: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.380104-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.387: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.382512-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.388: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.384938-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.389: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.387426-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.390: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.390441-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.391: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.393355-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.392: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.395946-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.393: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.399314-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.394: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.403451-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.395: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.407749-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.396: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.412009-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.397: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.416315-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.398: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.420779-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.399: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.425281-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.400: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.429785-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.401: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.434450-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.402: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.439210-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.403: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', '[ns_server:info,2022-03-06T06:13:43.702-08:00,babysitter_of_ns_1@cb.local:<0.133.0>:ns_port_server:log:226]memcached<0.133.0>: 2022-03-06T06:13:43.500526-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.416: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.505581-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.417: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.512220-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.418: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.516429-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.419: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.520435-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.420: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.524455-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.421: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.530328-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.422: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.534456-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.423: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.539952-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.424: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.544177-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.425: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.548441-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.426: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.554138-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.427: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.560506-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.428: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.565046-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.429: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.569649-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.430: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.576090-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.431: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.580722-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.432: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.586768-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.433: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.591444-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.434: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.597613-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.435: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.604292-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.436: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.611708-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.437: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.619347-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.438: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.624300-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.439: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.629161-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.440: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.635670-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.441: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.640726-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.442: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.647438-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.443: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.653205-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.444: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.658367-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.445: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.664992-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.446: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.670300-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.447: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.676364-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.448: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.681656-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.449: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.688410-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.450: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.693800-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.451: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.700301-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.452: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', '[ns_server:info,2022-03-06T06:13:43.913-08:00,babysitter_of_ns_1@cb.local:<0.133.0>:ns_port_server:log:226]memcached<0.133.0>: 2022-03-06T06:13:43.706847-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.453: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.713887-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.454: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.719544-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.455: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.726588-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.456: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.732799-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.457: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.739935-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.458: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.745761-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.459: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.752760-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.460: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.758685-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.461: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.767042-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.462: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.773140-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.463: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.780897-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.464: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.787883-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.465: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.795308-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.466: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.805175-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.467: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.814565-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.468: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.821159-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.469: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.827940-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.470: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.835900-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.471: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.842434-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.472: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.851089-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.473: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.859450-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.474: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.866649-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.475: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.874603-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.476: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.883100-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.477: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.890335-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.478: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.899504-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.479: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.906737-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.480: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', '[ns_server:info,2022-03-06T06:13:44.121-08:00,babysitter_of_ns_1@cb.local:<0.133.0>:ns_port_server:log:226]memcached<0.133.0>: 2022-03-06T06:13:43.921160-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.481: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.935420-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.482: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.944200-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.483: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.955200-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.484: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.965265-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.485: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.973013-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.486: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.980881-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.487: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.990483-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.488: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:43.998592-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.489: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.008054-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.490: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.017942-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.491: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.025646-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.492: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.035500-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.493: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.044812-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.494: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.054337-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.495: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.063176-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.496: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.072466-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.497: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.081756-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.498: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.090172-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.499: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.099195-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.500: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.108679-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.501: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', '[ns_server:info,2022-03-06T06:13:44.324-08:00,babysitter_of_ns_1@cb.local:<0.133.0>:ns_port_server:log:226]memcached<0.133.0>: 2022-03-06T06:13:44.120185-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.502: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.130972-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.503: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.141308-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.504: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.155645-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.505: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.165309-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.506: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:44.175096-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.507: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', '[ns_server:info,2022-03-06T06:13:46.136-08:00,babysitter_of_ns_1@cb.local:<0.133.0>:ns_port_server:log:226]memcached<0.133.0>: 2022-03-06T06:13:45.936324-08:00 CRITICAL [(default) magma_0/kvstore-0/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-0/rev-000000001/seqIndex/state.24: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.939554-08:00 CRITICAL [(default) magma_0/kvstore-8/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-8/rev-000000001/seqIndex/state.20: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.943831-08:00 CRITICAL [(default) magma_0/kvstore-16/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-16/rev-000000001/seqIndex/state.42: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.948058-08:00 CRITICAL [(default) magma_0/kvstore-24/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-24/rev-000000001/seqIndex/state.90: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.951758-08:00 CRITICAL [(default) magma_0/kvstore-32/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-32/rev-000000001/seqIndex/state.129: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.955160-08:00 CRITICAL [(default) magma_0/kvstore-40/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-40/rev-000000001/seqIndex/state.169: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.960724-08:00 CRITICAL [(default) magma_0/kvstore-48/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-48/rev-000000001/seqIndex/state.223: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.966278-08:00 CRITICAL [(default) magma_0/kvstore-56/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-56/rev-000000001/seqIndex/state.240: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.973914-08:00 CRITICAL [(default) magma_0/kvstore-64/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-64/rev-000000001/seqIndex/state.266: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.980890-08:00 CRITICAL [(default) magma_0/kvstore-72/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-72/rev-000000001/seqIndex/state.280: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:45.990612-08:00 CRITICAL [(default) magma_0/kvstore-80/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-80/rev-000000001/seqIndex/state.294: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.000113-08:00 CRITICAL [(default) magma_0/kvstore-88/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-88/rev-000000001/seqIndex/state.20: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.003500-08:00 CRITICAL [(default) magma_0/kvstore-96/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-96/rev-000000001/seqIndex/state.44: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.007835-08:00 CRITICAL [(default) magma_0/kvstore-104/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-104/rev-000000001/seqIndex/state.71: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.010950-08:00 CRITICAL [(default) magma_0/kvstore-112/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-112/rev-000000001/seqIndex/state.109: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.014610-08:00 CRITICAL [(default) magma_0/kvstore-120/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-120/rev-000000001/seqIndex/state.140: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.019200-08:00 CRITICAL [(default) magma_0/kvstore-128/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-128/rev-000000001/seqIndex/state.195: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.024190-08:00 CRITICAL [(default) magma_0/kvstore-136/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-136/rev-000000001/seqIndex/state.248: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.031532-08:00 CRITICAL [(default) magma_0/kvstore-144/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-144/rev-000000001/seqIndex/state.779: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.036417-08:00 CRITICAL [(default) magma_0/kvstore-152/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-152/rev-000000001/seqIndex/state.793: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.040147-08:00 CRITICAL [(default) magma_0/kvstore-160/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-160/rev-000000001/seqIndex/state.822: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.041968-08:00 CRITICAL [(default) magma_0/kvstore-168/rev-000000001/localIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-168/rev-000000001/localIndex/sstable.1325.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default).\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.044559-08:00 CRITICAL [(default) magma_0/kvstore-176/rev-000000001/localIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-176/rev-000000001/localIndex/sstable.616.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default).\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.046805-08:00 CRITICAL [(default) magma_0/kvstore-184/rev-000000001/localIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-184/rev-000000001/localIndex/sstable.651.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default).\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.048428-08:00 CRITICAL [(default) magma_0/kvstore-192/rev-000000001/localIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-192/rev-000000001/localIndex/sstable.746.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default).\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.050537-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.508: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.061363-08:00 CRITICAL [(default) magma_0/kvstore-208/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-208/rev-000000001/seqIndex/state.183: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.066290-08:00 CRITICAL [(default) magma_0/kvstore-216/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-216/rev-000000001/seqIndex/state.205: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.071787-08:00 CRITICAL [(default) magma_0/kvstore-224/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-224/rev-000000001/seqIndex/state.234: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.078040-08:00 CRITICAL [(default) magma_0/kvstore-232/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-232/rev-000000001/seqIndex/state.263: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.085381-08:00 CRITICAL [(default) magma_0/kvstore-240/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-240/rev-000000001/seqIndex/state.286: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.094854-08:00 CRITICAL [(default) magma_0/kvstore-248/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-248/rev-000000001/seqIndex/state.301: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.103735-08:00 CRITICAL [(default) magma_0/kvstore-256/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-256/rev-000000001/seqIndex/state.23: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.106803-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.26: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.111564-08:00 CRITICAL [(default) magma_0/kvstore-272/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-272/rev-000000001/seqIndex/state.23: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.115170-08:00 CRITICAL [(default) magma_0/kvstore-280/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-280/rev-000000001/seqIndex/state.21: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.119823-08:00 CRITICAL [(default) magma_0/kvstore-288/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-288/rev-000000001/seqIndex/state.18: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.122953-08:00 CRITICAL [(default) magma_0/kvstore-296/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-296/rev-000000001/seqIndex/state.16: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.126354-08:00 CRITICAL [(default) magma_0/kvstore-304/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-304/rev-000000001/seqIndex/state.18: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.130308-08:00 CRITICAL [(default) magma_0/kvstore-312/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-312/rev-000000001/seqIndex/state.19: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.133563-08:00 CRITICAL [(default) magma_0/kvstore-320/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-320/rev-000000001/seqIndex/state.16: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.136620-08:00 CRITICAL [(default) magma_0/kvstore-328/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-328/rev-000000001/seqIndex/state.16: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', '[ns_server:info,2022-03-06T06:13:46.339-08:00,babysitter_of_ns_1@cb.local:<0.133.0>:ns_port_server:log:226]memcached<0.133.0>: 2022-03-06T06:13:46.139411-08:00 CRITICAL [(default) magma_0/kvstore-336/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-336/rev-000000001/seqIndex/state.18: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.142289-08:00 CRITICAL [(default) magma_0/kvstore-512/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-512/rev-000000001/seqIndex/state.23: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.145052-08:00 CRITICAL [(default) magma_0/kvstore-520/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-520/rev-000000001/seqIndex/state.22: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.149076-08:00 CRITICAL [(default) magma_0/kvstore-528/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-528/rev-000000001/seqIndex/state.18: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.152315-08:00 CRITICAL [(default) magma_0/kvstore-536/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-536/rev-000000001/seqIndex/state.15: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.155679-08:00 CRITICAL [(default) magma_0/kvstore-544/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-544/rev-000000001/seqIndex/state.19: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.158498-08:00 CRITICAL [(default) magma_0/kvstore-552/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-552/rev-000000001/seqIndex/state.16: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.161534-08:00 CRITICAL [(default) magma_0/kvstore-560/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-560/rev-000000001/seqIndex/state.19: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.164348-08:00 CRITICAL [(default) magma_0/kvstore-568/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-568/rev-000000001/seqIndex/state.17: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.167369-08:00 CRITICAL [(default) magma_0/kvstore-576/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-576/rev-000000001/seqIndex/state.19: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.170382-08:00 CRITICAL [(default) magma_0/kvstore-584/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-584/rev-000000001/seqIndex/state.21: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.173624-08:00 CRITICAL [(default) magma_0/kvstore-592/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-592/rev-000000001/seqIndex/state.17: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.176340-08:00 CRITICAL [(default) magma_0/kvstore-768/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-768/rev-000000001/seqIndex/state.17: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.179273-08:00 CRITICAL [(default) magma_0/kvstore-776/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-776/rev-000000001/seqIndex/state.24: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.182590-08:00 CRITICAL [(default) magma_0/kvstore-784/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-784/rev-000000001/seqIndex/state.23: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.186098-08:00 CRITICAL [(default) magma_0/kvstore-792/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-792/rev-000000001/seqIndex/state.549: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.188392-08:00 CRITICAL [(default) magma_0/kvstore-800/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-800/rev-000000001/seqIndex/state.551: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.190441-08:00 CRITICAL [(default) magma_0/kvstore-808/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-808/rev-000000001/seqIndex/state.25: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.193399-08:00 CRITICAL [(default) magma_0/kvstore-816/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-816/rev-000000001/seqIndex/state.25: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.197261-08:00 CRITICAL [(default) magma_0/kvstore-824/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-824/rev-000000001/seqIndex/state.28: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.199978-08:00 CRITICAL [(default) magma_0/kvstore-832/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-832/rev-000000001/seqIndex/state.21: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.202877-08:00 CRITICAL [(default) magma_0/kvstore-840/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-840/rev-000000001/seqIndex/state.18: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.206035-08:00 CRITICAL [(default) magma_0/kvstore-848/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-848/rev-000000001/seqIndex/state.195: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.211434-08:00 CRITICAL [(default) magma_1/kvstore-1/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-1/rev-000000001/seqIndex/state.23: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.214315-08:00 CRITICAL [(default) magma_1/kvstore-9/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-9/rev-000000001/seqIndex/state.29: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.217197-08:00 CRITICAL [(default) magma_1/kvstore-17/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-17/rev-000000001/seqIndex/state.59: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.220185-08:00 CRITICAL [(default) magma_1/kvstore-25/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-25/rev-000000001/seqIndex/state.96: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.224010-08:00 CRITICAL [(default) magma_1/kvstore-33/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-33/rev-000000001/seqIndex/state.125: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.228994-08:00 CRITICAL [(default) magma_1/kvstore-41/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-41/rev-000000001/seqIndex/state.169: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.234474-08:00 CRITICAL [(default) magma_1/kvstore-49/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-49/rev-000000001/seqIndex/state.223: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.239849-08:00 CRITICAL [(default) magma_1/kvstore-57/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-57/rev-000000001/seqIndex/state.246: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.246784-08:00 CRITICAL [(default) magma_1/kvstore-65/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-65/rev-000000001/seqIndex/state.270: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.255182-08:00 CRITICAL [(default) magma_1/kvstore-73/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-73/rev-000000001/seqIndex/state.282: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.263005-08:00 CRITICAL [(default) magma_1/kvstore-81/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-81/rev-000000001/seqIndex/state.297: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.272005-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.25: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.274564-08:00 CRITICAL [(default) magma_1/kvstore-97/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-97/rev-000000001/seqIndex/state.49: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.277217-08:00 CRITICAL [(default) magma_1/kvstore-105/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-105/rev-000000001/seqIndex/state.79: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.280632-08:00 CRITICAL [(default) magma_1/kvstore-113/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-113/rev-000000001/seqIndex/state.108: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.284588-08:00 CRITICAL [(default) magma_1/kvstore-121/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-121/rev-000000001/seqIndex/state.145: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.289144-08:00 CRITICAL [(default) magma_1/kvstore-129/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-129/rev-000000001/seqIndex/state.209: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.295424-08:00 CRITICAL [(default) magma_1/kvstore-137/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-137/rev-000000001/seqIndex/state.251: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.302092-08:00 CRITICAL [(default) magma_1/kvstore-145/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-145/rev-000000001/seqIndex/state.260: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.310236-08:00 CRITICAL [(default) magma_1/kvstore-153/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-153/rev-000000001/seqIndex/state.270: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.331438-08:00 CRITICAL [(default) magma_1/kvstore-161/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-161/rev-000000001/seqIndex/state.286: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', '[ns_server:info,2022-03-06T06:13:46.541-08:00,babysitter_of_ns_1@cb.local:<0.133.0>:ns_port_server:log:226]memcached<0.133.0>: 2022-03-06T06:13:46.340699-08:00 CRITICAL [(default) magma_1/kvstore-169/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-169/rev-000000001/seqIndex/state.293: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.348570-08:00 CRITICAL [(default) magma_1/kvstore-177/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-177/rev-000000001/seqIndex/state.57: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.350620-08:00 CRITICAL [(default) magma_1/kvstore-185/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-185/rev-000000001/seqIndex/state.79: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.353892-08:00 CRITICAL [(default) magma_1/kvstore-193/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-193/rev-000000001/seqIndex/state.115: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.357917-08:00 CRITICAL [(default) magma_1/kvstore-201/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-201/rev-000000001/seqIndex/state.148: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.361947-08:00 CRITICAL [(default) magma_1/kvstore-209/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-209/rev-000000001/seqIndex/state.183: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.366927-08:00 CRITICAL [(default) magma_1/kvstore-217/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-217/rev-000000001/seqIndex/state.207: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.372733-08:00 CRITICAL [(default) magma_1/kvstore-225/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-225/rev-000000001/seqIndex/state.238: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.379102-08:00 CRITICAL [(default) magma_1/kvstore-233/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-233/rev-000000001/seqIndex/state.265: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.386931-08:00 CRITICAL [(default) magma_1/kvstore-241/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-241/rev-000000001/seqIndex/state.813: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.389492-08:00 CRITICAL [(default) magma_1/kvstore-249/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-249/rev-000000001/seqIndex/state.821: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.409989-08:00 CRITICAL [(default) magma_1/kvstore-289/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-289/rev-000000001/seqIndex/state.18: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.412366-08:00 CRITICAL [(default) magma_1/kvstore-297/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-297/rev-000000001/seqIndex/state.15: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.414815-08:00 CRITICAL [(default) magma_1/kvstore-305/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-305/rev-000000001/seqIndex/state.19: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.417538-08:00 CRITICAL [(default) magma_1/kvstore-313/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-313/rev-000000001/seqIndex/state.18: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.420400-08:00 CRITICAL [(default) magma_1/kvstore-321/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-321/rev-000000001/seqIndex/state.19: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.423244-08:00 CRITICAL [(default) magma_1/kvstore-329/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-329/rev-000000001/seqIndex/state.545: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.425574-08:00 CRITICAL [(default) magma_1/kvstore-337/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-337/rev-000000001/seqIndex/state.19: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.439013-08:00 CRITICAL [(default) magma_1/kvstore-537/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-537/rev-000000001/seqIndex/state.545: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.440875-08:00 CRITICAL [(default) magma_1/kvstore-545/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-545/rev-000000001/seqIndex/state.544: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.442584-08:00 CRITICAL [(default) magma_1/kvstore-553/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-553/rev-000000001/seqIndex/state.545: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.444430-08:00 CRITICAL [(default) magma_1/kvstore-561/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-561/rev-000000001/seqIndex/state.547: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.445676-08:00 CRITICAL [(default) magma_1/kvstore-569/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-569/rev-000000001/seqIndex/state.546: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.447140-08:00 CRITICAL [(default) magma_1/kvstore-577/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-577/rev-000000001/seqIndex/state.20: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.450005-08:00 CRITICAL [(default) magma_1/kvstore-585/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-585/rev-000000001/seqIndex/state.20: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.452720-08:00 CRITICAL [(default) magma_1/kvstore-593/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-593/rev-000000001/seqIndex/state.90: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.455776-08:00 CRITICAL [(default) magma_1/kvstore-769/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-769/rev-000000001/seqIndex/state.22: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1/kvstore-769/rev-000000001\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.457979-08:00 CRITICAL [(default) magma_1/kvstore-777/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-777/rev-000000001/seqIndex/state.27: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.459961-08:00 CRITICAL [(default) magma_1/kvstore-785/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-785/rev-000000001/seqIndex/state.22: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.462586-08:00 CRITICAL [(default) magma_1/kvstore-793/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-793/rev-000000001/seqIndex/state.21: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.465633-08:00 CRITICAL [(default) magma_1/kvstore-801/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-801/rev-000000001/seqIndex/state.25: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.468668-08:00 CRITICAL [(default) magma_1/kvstore-809/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-809/rev-000000001/seqIndex/state.27: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.471188-08:00 CRITICAL [(default) magma_1/kvstore-817/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-817/rev-000000001/seqIndex/state.25: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.474298-08:00 CRITICAL [(default) magma_1/kvstore-825/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-825/rev-000000001/seqIndex/state.27: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.476752-08:00 CRITICAL [(default) magma_1/kvstore-833/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-833/rev-000000001/seqIndex/state.545: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.478377-08:00 CRITICAL [(default) magma_1/kvstore-841/rev-000000001/localIndex]Error occured during memtable flush (IOError: /data/default/magma.1/kvstore-841/rev-000000001/localIndex/sstable.563.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1).\n', 'memcached<0.133.0>: 2022-03-06T06:13:46.479793-08:00 CRITICAL [(default) magma_1/kvstore-849/rev-000000001/localIndex]Error occured during memtable flush (IOError: /data/default/magma.1/kvstore-849/rev-000000001/localIndex/sstable.902.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1).\n', '[ns_server:info,2022-03-06T11:54:09.799-08:00,babysitter_of_ns_1@cb.local:<0.133.0>:ns_port_server:log:226]memcached<0.133.0>: 2022-03-06T11:54:09.588903-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/localIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/localIndex/state.181: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.589500-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.182: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.590310-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.183: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.591444-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.184: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.592084-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.185: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.593018-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.186: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.593817-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.187: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.0\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.595405-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.188: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.596196-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.189: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.597046-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.190: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.597956-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.191: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.598933-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.192: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.599903-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.193: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.600919-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.194: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.601966-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.195: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.603556-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.196: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.605198-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.197: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.606825-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.198: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.608580-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.199: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.610396-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.200: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.612288-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.201: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.615272-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.202: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.617208-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.203: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.619214-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.204: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.621468-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.205: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.624096-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.206: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.626310-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.207: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.628503-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.208: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.630877-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.209: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.633371-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.210: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.635404-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.211: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.637198-08:00 CRITICAL [(default) magma_1/kvstore-81/rev-000000001/keyIndex]Error occured during merge sstables (IOError: /data/default/magma.1/kvstore-81/rev-000000001/keyIndex/sstable.2414.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1). \n', 'memcached<0.133.0>: 2022-03-06T11:54:09.637685-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.212: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.638029-08:00 CRITICAL [(default) magma_1/kvstore-81/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.1/kvstore-81/rev-000000001/seqIndex/sstable.1997.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1).\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.638209-08:00 CRITICAL [(default) magma_1/kvstore-81/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.1/kvstore-81/rev-000000001/seqIndex/sstable.1998.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1).\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.638370-08:00 CRITICAL [(default) magma_1/kvstore-81/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.1/kvstore-81/rev-000000001/seqIndex/sstable.1999.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1).\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.638503-08:00 CRITICAL [(default) magma_1/kvstore-81/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.1/kvstore-81/rev-000000001/seqIndex/sstable.2000.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1).\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.638635-08:00 CRITICAL [(default) magma_1/kvstore-81/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.1/kvstore-81/rev-000000001/seqIndex/sstable.2001.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1).\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.638758-08:00 CRITICAL [(default) magma_1/kvstore-81/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.1/kvstore-81/rev-000000001/seqIndex/sstable.2002.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1).\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.638882-08:00 CRITICAL [(default) magma_1/kvstore-81/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.1/kvstore-81/rev-000000001/seqIndex/sstable.2003.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1).\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.639056-08:00 CRITICAL [(default) magma_1/kvstore-81/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.1/kvstore-81/rev-000000001/seqIndex/sstable.2004.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1).\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.639189-08:00 CRITICAL [(default) magma_1/kvstore-81/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.1/kvstore-81/rev-000000001/seqIndex/sstable.2005.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1).\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.639328-08:00 CRITICAL [(default) magma_1/kvstore-81/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.1/kvstore-81/rev-000000001/seqIndex/sstable.2006.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1).\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.639444-08:00 CRITICAL [(default) magma_1/kvstore-81/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.1/kvstore-81/rev-000000001/seqIndex/sstable.2007.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1).\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.639574-08:00 CRITICAL [(default) magma_1/kvstore-81/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.1/kvstore-81/rev-000000001/seqIndex/sstable.2008.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1).\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.639706-08:00 CRITICAL [(default) magma_1/kvstore-81/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.1/kvstore-81/rev-000000001/seqIndex/sstable.2009.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1).\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.639826-08:00 CRITICAL [(default) magma_1/kvstore-81/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.1/kvstore-81/rev-000000001/seqIndex/sstable.2010.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1).\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.639942-08:00 CRITICAL [(default) magma_1/kvstore-81/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.1/kvstore-81/rev-000000001/seqIndex/sstable.2011.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1).\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.640057-08:00 CRITICAL [(default) magma_1/kvstore-81/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.1/kvstore-81/rev-000000001/seqIndex/sstable.2012.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1).\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.640175-08:00 CRITICAL [(default) magma_1/kvstore-81/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.1/kvstore-81/rev-000000001/seqIndex/sstable.2013.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1).\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.640323-08:00 CRITICAL [(default) magma_1/kvstore-81/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.1/kvstore-81/rev-000000001/seqIndex/sstable.2014.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1).\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.640451-08:00 CRITICAL [(default) magma_1/kvstore-81/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.1/kvstore-81/rev-000000001/seqIndex/sstable.2015.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1).\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.640567-08:00 CRITICAL [(default) magma_1/kvstore-81/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.1/kvstore-81/rev-000000001/seqIndex/sstable.2016.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1).\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.640682-08:00 CRITICAL [(default) magma_1/kvstore-81/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.1/kvstore-81/rev-000000001/seqIndex/sstable.2017.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1).\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.641560-08:00 CRITICAL [(default) magma_1/kvstore-81/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.1/kvstore-81/rev-000000001/seqIndex/sstable.2018.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1).\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.641698-08:00 CRITICAL [(default) magma_1/kvstore-81/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.1/kvstore-81/rev-000000001/seqIndex/sstable.2019.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1).\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.641817-08:00 CRITICAL [(default) magma_1/kvstore-81/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.1/kvstore-81/rev-000000001/seqIndex/sstable.2020.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1).\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.641933-08:00 CRITICAL [(default) magma_1/kvstore-81/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.1/kvstore-81/rev-000000001/seqIndex/sstable.2021.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1).\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.642051-08:00 CRITICAL [(default) magma_1/kvstore-81/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.1/kvstore-81/rev-000000001/seqIndex/sstable.2022.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1).\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.642165-08:00 CRITICAL [(default) magma_1/kvstore-81/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.1/kvstore-81/rev-000000001/seqIndex/sstable.2023.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1).\n', '[ns_server:info,2022-03-06T11:54:10.010-08:00,babysitter_of_ns_1@cb.local:<0.133.0>:ns_port_server:log:226]memcached<0.133.0>: 2022-03-06T11:54:09.801654-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.246: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.807374-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.247: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.812797-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.248: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.819689-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.249: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.827371-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.250: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.833183-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.251: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.839349-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.252: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.845590-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.253: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.851750-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.254: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.857380-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.255: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.863005-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.256: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.868561-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.257: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.874087-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.258: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.879755-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.259: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.884175-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.260: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.889627-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.261: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.895215-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.262: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.900305-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.263: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.905933-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.264: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.911699-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.265: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.916262-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.266: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.921940-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.267: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.928989-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.268: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.936273-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.269: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.943527-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.270: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.950872-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.271: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.955864-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.272: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.961634-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.273: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.963807-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.149: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1/kvstore-89/rev-000000001\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.964893-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.150: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1/kvstore-89/rev-000000001\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.965484-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.151: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1/kvstore-89/rev-000000001\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.966046-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.152: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1/kvstore-89/rev-000000001\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.966769-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.274: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.966890-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.153: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1/kvstore-89/rev-000000001\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.967677-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.154: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.968450-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.155: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.969281-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.156: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.970147-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.157: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.971518-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.158: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.973884-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.275: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.974790-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.159: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.976000-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.160: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.977035-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.161: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.978323-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.162: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.979540-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.276: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.979563-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.163: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.980763-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.164: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.981981-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.165: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.983286-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.166: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.984618-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.167: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.985013-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.277: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.986463-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.168: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.988512-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.169: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.990006-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.170: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.992195-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.171: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.994202-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.172: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.995032-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.278: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.996462-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.173: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:09.998728-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.174: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.000489-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.175: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', '[ns_server:info,2022-03-06T11:54:10.222-08:00,babysitter_of_ns_1@cb.local:<0.133.0>:ns_port_server:log:226]memcached<0.133.0>: 2022-03-06T11:54:10.012089-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.180: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.014206-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.181: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.015192-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.281: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.016320-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.182: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.018522-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.183: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.020701-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.282: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.020877-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.184: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.023598-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.185: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.026776-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.186: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.029372-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.283: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.029912-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.187: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.032966-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.188: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.036203-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.189: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.039509-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.190: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.041463-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.284: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.042781-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.191: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.046146-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.192: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.049709-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.193: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.052898-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.194: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.054137-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.285: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.056257-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.195: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.059833-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.196: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.063065-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.286: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.066319-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.197: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.069299-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.198: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.069360-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.287: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.072553-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.199: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.075848-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.200: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.078542-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.288: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.080261-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.201: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.083551-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.202: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.084378-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.289: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.086674-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.203: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.090145-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.204: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.092194-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.290: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.094974-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.205: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.099548-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.206: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.104609-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.207: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.105449-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.291: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.108172-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.208: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.112514-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.209: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.113326-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.292: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.115989-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.210: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.123276-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.293: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.123360-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.211: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.127024-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.212: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.130040-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.294: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.133867-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.213: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.137875-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.295: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.138838-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.214: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.143721-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.215: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.149298-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.296: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.151047-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.216: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.156823-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.217: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.158827-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.297: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.163029-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.218: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.167837-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.219: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.171686-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.298: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.173942-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.220: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.179037-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.221: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', '[ns_server:info,2022-03-06T11:54:10.431-08:00,babysitter_of_ns_1@cb.local:<0.133.0>:ns_port_server:log:226]memcached<0.133.0>: 2022-03-06T11:54:10.228339-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.230: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.228819-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.305: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.234217-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.231: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.237927-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.306: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.240987-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.232: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.246914-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.307: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.247943-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.233: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.253165-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.234: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.254264-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.308: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.259611-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.235: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.264161-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.309: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.266674-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.236: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.273297-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.310: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.273811-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.237: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.281022-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.238: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.282907-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.311: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.288353-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.239: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.292631-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.312: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.295342-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.240: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.301615-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.313: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.302004-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.241: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.309483-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.242: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.310864-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.314: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.316625-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.243: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.319982-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.315: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.323584-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.244: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.328359-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.316: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.329538-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.245: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.335916-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.317: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.340831-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.246: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.346121-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.318: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.348094-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.247: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.355451-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.248: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.356480-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.319: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.362398-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.249: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.365790-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.320: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.369282-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.250: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.375344-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.321: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.376189-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.251: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.383212-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.252: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.384647-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.322: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.390376-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.253: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.394001-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.323: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.397640-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.254: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.403334-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.324: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.404744-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.255: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.411772-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.256: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.412491-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.325: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.419612-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.257: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.422902-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.326: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.427501-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.258: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', '[ns_server:info,2022-03-06T11:54:10.634-08:00,babysitter_of_ns_1@cb.local:<0.133.0>:ns_port_server:log:226]memcached<0.133.0>: 2022-03-06T11:54:10.433448-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.327: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.435587-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.259: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.441670-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.260: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.441731-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.328: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.449883-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.261: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.452407-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.329: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.457896-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.262: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.462573-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.330: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.466017-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.263: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.473160-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.331: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.474544-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.264: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.483116-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.265: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.483923-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.332: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.491873-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.266: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.494384-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.333: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.499817-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.267: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.504305-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.334: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.507796-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.268: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.514713-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.335: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.516293-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.269: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.523656-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.270: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.524170-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.336: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.532488-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.271: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.535831-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.337: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.541614-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.272: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.547108-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.338: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.550566-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.273: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.558158-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.339: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.559313-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.274: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.568024-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.275: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.568350-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.340: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.575972-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.276: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.578897-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.341: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.584478-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.277: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.589201-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.342: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.593474-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.278: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.601090-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.343: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.603583-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.279: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.613884-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.344: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.614217-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.280: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.624588-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.281: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.626576-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.345: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', '[ns_server:info,2022-03-06T11:54:10.837-08:00,babysitter_of_ns_1@cb.local:<0.133.0>:ns_port_server:log:226]memcached<0.133.0>: 2022-03-06T11:54:10.637201-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.282: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.640353-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.346: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.648009-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.283: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.653703-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.347: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.658780-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.284: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.666721-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.348: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.670161-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.285: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.680116-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.349: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.681105-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.286: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.691788-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.287: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.693264-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.350: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.700957-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.288: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.704738-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.351: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.712252-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.289: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.717657-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.352: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.723653-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.290: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.731311-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.353: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.734491-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.291: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.744639-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.354: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.745983-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.292: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.757693-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.355: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.757838-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.293: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.769325-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.294: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.770859-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.356: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.781161-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.295: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.784540-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.357: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.792678-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.296: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.798428-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.358: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.806697-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.297: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.811266-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.359: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.819368-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.298: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.824310-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.360: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.829644-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.299: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.836327-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.361: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', '[ns_server:info,2022-03-06T11:54:11.042-08:00,babysitter_of_ns_1@cb.local:<0.133.0>:ns_port_server:log:226]memcached<0.133.0>: 2022-03-06T11:54:10.839974-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.300: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.849048-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.362: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.852497-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.301: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.863376-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.363: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.865632-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.302: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.877400-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.364: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.878251-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.303: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.890625-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.304: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.891625-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.365: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.903302-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.305: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.906070-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.366: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.915669-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.306: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.920795-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.367: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.927830-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.307: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.933283-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.368: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.939754-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.308: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.946388-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.369: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.953131-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.309: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.961441-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.370: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.966339-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.310: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.974986-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.371: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.980023-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.311: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.988527-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.372: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:10.993906-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.312: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.002267-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.373: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.007421-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.313: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.015915-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.374: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.021752-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.314: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.035614-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.315: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', '[ns_server:info,2022-03-06T11:54:11.249-08:00,babysitter_of_ns_1@cb.local:<0.133.0>:ns_port_server:log:226]memcached<0.133.0>: 2022-03-06T11:54:11.048955-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.316: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.062119-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.317: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.073012-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.318: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.086160-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.319: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.100024-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.320: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.111327-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.321: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.124712-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.322: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.137855-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.323: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.152209-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.324: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.166213-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.325: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.180386-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.326: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.193948-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.327: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.208174-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.328: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.222177-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.329: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.236268-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.330: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', '[ns_server:info,2022-03-06T11:54:11.450-08:00,babysitter_of_ns_1@cb.local:<0.133.0>:ns_port_server:log:226]memcached<0.133.0>: 2022-03-06T11:54:11.250458-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.331: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.264618-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.332: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.279016-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.333: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.293408-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.334: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.307807-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.335: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.322559-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.336: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.336665-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.337: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.351157-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.338: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.365524-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.339: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.379877-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.340: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.394275-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.341: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.408849-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.342: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.423404-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.343: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.438250-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.344: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', '[ns_server:info,2022-03-06T11:54:11.653-08:00,babysitter_of_ns_1@cb.local:<0.133.0>:ns_port_server:log:226]memcached<0.133.0>: 2022-03-06T11:54:11.453246-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.345: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.468871-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.346: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.483758-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.347: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.499188-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.348: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.514196-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.349: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.529297-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.350: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.544377-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.351: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.559689-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.352: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.573193-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.353: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.588739-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.354: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.603615-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.355: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.617532-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.356: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.631958-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.357: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.645090-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.358: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', '[ns_server:info,2022-03-06T11:54:11.860-08:00,babysitter_of_ns_1@cb.local:<0.133.0>:ns_port_server:log:226]memcached<0.133.0>: 2022-03-06T11:54:11.659904-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.359: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.675644-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.360: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.691463-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.361: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.706848-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.362: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.722618-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.363: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.739185-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.364: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.755362-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.365: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.771553-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.366: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.787896-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.367: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.804571-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.368: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.821365-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.369: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.838031-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.370: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.854273-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.371: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', '[ns_server:info,2022-03-06T11:54:12.070-08:00,babysitter_of_ns_1@cb.local:<0.133.0>:ns_port_server:log:226]memcached<0.133.0>: 2022-03-06T11:54:11.870022-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.372: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.885702-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.373: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.901636-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.374: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.918874-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.375: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.935547-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.376: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.951448-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.377: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.967586-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.378: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:11.983782-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.379: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.000834-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.380: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.018074-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.381: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.035346-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.382: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.052645-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.383: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.070262-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.384: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', '[ns_server:info,2022-03-06T11:54:12.288-08:00,babysitter_of_ns_1@cb.local:<0.133.0>:ns_port_server:log:226]memcached<0.133.0>: 2022-03-06T11:54:12.087863-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.385: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.105471-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.386: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.123097-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.387: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.140769-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.388: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.158797-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.389: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.176793-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.390: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.194804-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.391: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.212794-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.392: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.231744-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.393: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.249982-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.394: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.268488-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.395: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.286841-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.396: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', '[ns_server:info,2022-03-06T11:54:12.505-08:00,babysitter_of_ns_1@cb.local:<0.133.0>:ns_port_server:log:226]memcached<0.133.0>: 2022-03-06T11:54:12.305074-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.397: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.323659-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.398: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.341566-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.399: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.360702-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.400: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.380011-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.401: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.411367-08:00 CRITICAL [(default) magma_0/kvstore-0/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-0/rev-000000001/seqIndex/state.70: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.415663-08:00 CRITICAL [(default) magma_0/kvstore-8/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-8/rev-000000001/seqIndex/state.75: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.420628-08:00 CRITICAL [(default) magma_0/kvstore-16/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-16/rev-000000001/seqIndex/state.97: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.425746-08:00 CRITICAL [(default) magma_0/kvstore-24/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-24/rev-000000001/seqIndex/state.150: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.430471-08:00 CRITICAL [(default) magma_0/kvstore-32/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-32/rev-000000001/seqIndex/state.187: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.436603-08:00 CRITICAL [(default) magma_0/kvstore-40/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-40/rev-000000001/seqIndex/state.233: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.442203-08:00 CRITICAL [(default) magma_0/kvstore-48/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-48/rev-000000001/seqIndex/state.344: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.450056-08:00 CRITICAL [(default) magma_0/kvstore-56/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-56/rev-000000001/seqIndex/state.413: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.459610-08:00 CRITICAL [(default) magma_0/kvstore-64/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-64/rev-000000001/seqIndex/state.455: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.470300-08:00 CRITICAL [(default) magma_0/kvstore-72/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-72/rev-000000001/seqIndex/state.485: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.482374-08:00 CRITICAL [(default) magma_0/kvstore-80/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-80/rev-000000001/seqIndex/state.510: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.494773-08:00 CRITICAL [(default) magma_0/kvstore-88/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-88/rev-000000001/seqIndex/state.116: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.499113-08:00 CRITICAL [(default) magma_0/kvstore-96/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-96/rev-000000001/seqIndex/state.144: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.503347-08:00 CRITICAL [(default) magma_0/kvstore-104/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-104/rev-000000001/seqIndex/state.176: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', '[ns_server:info,2022-03-06T11:54:12.709-08:00,babysitter_of_ns_1@cb.local:<0.133.0>:ns_port_server:log:226]memcached<0.133.0>: 2022-03-06T11:54:12.508943-08:00 CRITICAL [(default) magma_0/kvstore-112/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-112/rev-000000001/seqIndex/state.227: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.515497-08:00 CRITICAL [(default) magma_0/kvstore-120/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-120/rev-000000001/seqIndex/state.277: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.522053-08:00 CRITICAL [(default) magma_0/kvstore-128/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-128/rev-000000001/seqIndex/state.368: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.529679-08:00 CRITICAL [(default) magma_0/kvstore-136/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-136/rev-000000001/seqIndex/state.422: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.538247-08:00 CRITICAL [(default) magma_0/kvstore-144/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-144/rev-000000001/seqIndex/state.448: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.547411-08:00 CRITICAL [(default) magma_0/kvstore-152/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-152/rev-000000001/seqIndex/state.512: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.559973-08:00 CRITICAL [(default) magma_0/kvstore-160/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-160/rev-000000001/seqIndex/state.540: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.572721-08:00 CRITICAL [(default) magma_0/kvstore-168/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-168/rev-000000001/seqIndex/state.560: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.583424-08:00 CRITICAL [(default) magma_0/kvstore-176/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-176/rev-000000001/seqIndex/state.108: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.588460-08:00 CRITICAL [(default) magma_0/kvstore-184/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-184/rev-000000001/seqIndex/state.131: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.592529-08:00 CRITICAL [(default) magma_0/kvstore-192/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-192/rev-000000001/seqIndex/state.178: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.599012-08:00 CRITICAL [(default) magma_0/kvstore-200/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-200/rev-000000001/seqIndex/state.199: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.604799-08:00 CRITICAL [(default) magma_0/kvstore-208/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-208/rev-000000001/seqIndex/state.229: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.611553-08:00 CRITICAL [(default) magma_0/kvstore-216/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-216/rev-000000001/seqIndex/state.304: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.619005-08:00 CRITICAL [(default) magma_0/kvstore-224/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-224/rev-000000001/seqIndex/state.367: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.625947-08:00 CRITICAL [(default) magma_0/kvstore-232/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-232/rev-000000001/seqIndex/state.960: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.628831-08:00 CRITICAL [(default) magma_0/kvstore-240/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-240/rev-000000001/seqIndex/state.1041: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.632140-08:00 CRITICAL [(default) magma_0/kvstore-248/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-248/rev-000000001/seqIndex/state.1048: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.634720-08:00 CRITICAL [(default) magma_0/kvstore-256/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-256/rev-000000001/seqIndex/state.559: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.637381-08:00 CRITICAL [(default) magma_0/kvstore-264/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-264/rev-000000001/seqIndex/state.375: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.649042-08:00 CRITICAL [(default) magma_0/kvstore-272/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-272/rev-000000001/seqIndex/state.33: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.654511-08:00 CRITICAL [(default) magma_0/kvstore-280/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-280/rev-000000001/seqIndex/state.31: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.659178-08:00 CRITICAL [(default) magma_0/kvstore-288/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-288/rev-000000001/seqIndex/state.22: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.663103-08:00 CRITICAL [(default) magma_0/kvstore-296/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-296/rev-000000001/seqIndex/state.26: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.667944-08:00 CRITICAL [(default) magma_0/kvstore-304/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-304/rev-000000001/seqIndex/state.24: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.672168-08:00 CRITICAL [(default) magma_0/kvstore-312/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-312/rev-000000001/seqIndex/state.25: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.676607-08:00 CRITICAL [(default) magma_0/kvstore-320/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-320/rev-000000001/seqIndex/state.28: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.679467-08:00 CRITICAL [(default) magma_0/kvstore-328/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-328/rev-000000001/seqIndex/state.44: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.683819-08:00 CRITICAL [(default) magma_0/kvstore-336/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-336/rev-000000001/seqIndex/state.56: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.687709-08:00 CRITICAL [(default) magma_0/kvstore-512/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-512/rev-000000001/seqIndex/state.20: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.692671-08:00 CRITICAL [(default) magma_0/kvstore-520/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-520/rev-000000001/seqIndex/state.31: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.695994-08:00 CRITICAL [(default) magma_0/kvstore-528/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-528/rev-000000001/seqIndex/state.19: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.699975-08:00 CRITICAL [(default) magma_0/kvstore-536/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-536/rev-000000001/seqIndex/state.24: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.703321-08:00 CRITICAL [(default) magma_0/kvstore-544/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-544/rev-000000001/seqIndex/state.24: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.707405-08:00 CRITICAL [(default) magma_0/kvstore-552/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-552/rev-000000001/seqIndex/state.20: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', '[ns_server:info,2022-03-06T11:54:12.911-08:00,babysitter_of_ns_1@cb.local:<0.133.0>:ns_port_server:log:226]memcached<0.133.0>: 2022-03-06T11:54:12.710939-08:00 CRITICAL [(default) magma_0/kvstore-560/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-560/rev-000000001/seqIndex/state.547: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.713290-08:00 CRITICAL [(default) magma_0/kvstore-568/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-568/rev-000000001/seqIndex/state.564: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.714476-08:00 CRITICAL [(default) magma_0/kvstore-576/rev-000000001/localIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-576/rev-000000001/localIndex/sstable.624.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default).\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.717526-08:00 CRITICAL [(default) magma_0/kvstore-584/rev-000000001/localIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-584/rev-000000001/localIndex/sstable.666.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default).\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.719546-08:00 CRITICAL [(default) magma_0/kvstore-592/rev-000000001/localIndex]Error occured during memtable flush (IOError: /data/default/magma.0/kvstore-592/rev-000000001/localIndex/sstable.716.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default).\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.724004-08:00 CRITICAL [(default) magma_0/kvstore-768/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-768/rev-000000001/seqIndex/state.26: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.728213-08:00 CRITICAL [(default) magma_0/kvstore-776/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-776/rev-000000001/seqIndex/state.27: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.731911-08:00 CRITICAL [(default) magma_0/kvstore-784/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-784/rev-000000001/seqIndex/state.32: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.736485-08:00 CRITICAL [(default) magma_0/kvstore-792/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-792/rev-000000001/seqIndex/state.32: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.740285-08:00 CRITICAL [(default) magma_0/kvstore-800/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-800/rev-000000001/seqIndex/state.26: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.744266-08:00 CRITICAL [(default) magma_0/kvstore-808/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-808/rev-000000001/seqIndex/state.31: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.747985-08:00 CRITICAL [(default) magma_0/kvstore-816/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-816/rev-000000001/seqIndex/state.27: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.751416-08:00 CRITICAL [(default) magma_0/kvstore-824/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-824/rev-000000001/seqIndex/state.43: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.755002-08:00 CRITICAL [(default) magma_0/kvstore-832/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-832/rev-000000001/seqIndex/state.57: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.758520-08:00 CRITICAL [(default) magma_0/kvstore-840/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-840/rev-000000001/seqIndex/state.53: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.762692-08:00 CRITICAL [(default) magma_0/kvstore-848/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.0/kvstore-848/rev-000000001/seqIndex/state.88: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.768038-08:00 CRITICAL [(default) magma_1/kvstore-1/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-1/rev-000000001/seqIndex/state.77: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.771954-08:00 CRITICAL [(default) magma_1/kvstore-9/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-9/rev-000000001/seqIndex/state.101: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.776116-08:00 CRITICAL [(default) magma_1/kvstore-17/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-17/rev-000000001/seqIndex/state.97: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.780757-08:00 CRITICAL [(default) magma_1/kvstore-25/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-25/rev-000000001/seqIndex/state.148: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.784883-08:00 CRITICAL [(default) magma_1/kvstore-33/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-33/rev-000000001/seqIndex/state.194: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.789833-08:00 CRITICAL [(default) magma_1/kvstore-41/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-41/rev-000000001/seqIndex/state.755: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.791579-08:00 CRITICAL [(default) magma_1/kvstore-49/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-49/rev-000000001/seqIndex/state.871: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.793971-08:00 CRITICAL [(default) magma_1/kvstore-57/rev-000000001/localIndex]Error occured during memtable flush (IOError: /data/default/magma.1/kvstore-57/rev-000000001/localIndex/sstable.1400.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1).\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.797376-08:00 CRITICAL [(default) magma_1/kvstore-65/rev-000000001/localIndex]Error occured during memtable flush (IOError: /data/default/magma.1/kvstore-65/rev-000000001/localIndex/sstable.1785.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1).\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.799201-08:00 CRITICAL [(default) magma_1/kvstore-73/rev-000000001/localIndex]Error occured during memtable flush (IOError: /data/default/magma.1/kvstore-73/rev-000000001/localIndex/sstable.1895.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1).\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.801500-08:00 CRITICAL [(default) magma_1/kvstore-81/rev-000000001/seqIndex]Error occured during memtable flush (IOError: /data/default/magma.1/kvstore-81/rev-000000001/seqIndex/sstable.2038.data: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1).\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.806129-08:00 CRITICAL [(default) magma_1/kvstore-89/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-89/rev-000000001/seqIndex/state.402: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.816827-08:00 CRITICAL [(default) magma_1/kvstore-97/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-97/rev-000000001/seqIndex/state.145: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.821605-08:00 CRITICAL [(default) magma_1/kvstore-105/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-105/rev-000000001/seqIndex/state.193: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.825616-08:00 CRITICAL [(default) magma_1/kvstore-113/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-113/rev-000000001/seqIndex/state.212: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.831183-08:00 CRITICAL [(default) magma_1/kvstore-121/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-121/rev-000000001/seqIndex/state.258: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.838509-08:00 CRITICAL [(default) magma_1/kvstore-129/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-129/rev-000000001/seqIndex/state.357: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.845406-08:00 CRITICAL [(default) magma_1/kvstore-137/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-137/rev-000000001/seqIndex/state.375: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.854625-08:00 CRITICAL [(default) magma_1/kvstore-145/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-145/rev-000000001/seqIndex/state.418: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.863577-08:00 CRITICAL [(default) magma_1/kvstore-153/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-153/rev-000000001/seqIndex/state.511: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.875455-08:00 CRITICAL [(default) magma_1/kvstore-161/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-161/rev-000000001/seqIndex/state.536: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.886286-08:00 CRITICAL [(default) magma_1/kvstore-169/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-169/rev-000000001/seqIndex/state.555: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.895529-08:00 CRITICAL [(default) magma_1/kvstore-177/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-177/rev-000000001/seqIndex/state.110: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.899497-08:00 CRITICAL [(default) magma_1/kvstore-185/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-185/rev-000000001/seqIndex/state.132: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.904144-08:00 CRITICAL [(default) magma_1/kvstore-193/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-193/rev-000000001/seqIndex/state.172: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.908928-08:00 CRITICAL [(default) magma_1/kvstore-201/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-201/rev-000000001/seqIndex/state.207: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', '[ns_server:info,2022-03-06T11:54:13.114-08:00,babysitter_of_ns_1@cb.local:<0.133.0>:ns_port_server:log:226]memcached<0.133.0>: 2022-03-06T11:54:12.914526-08:00 CRITICAL [(default) magma_1/kvstore-209/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-209/rev-000000001/seqIndex/state.236: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.920118-08:00 CRITICAL [(default) magma_1/kvstore-217/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-217/rev-000000001/seqIndex/state.309: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.927262-08:00 CRITICAL [(default) magma_1/kvstore-225/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-225/rev-000000001/seqIndex/state.365: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.952062-08:00 CRITICAL [(default) magma_1/kvstore-233/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-233/rev-000000001/seqIndex/state.426: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.962865-08:00 CRITICAL [(default) magma_1/kvstore-241/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-241/rev-000000001/seqIndex/state.488: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:12.972871-08:00 CRITICAL [(default) magma_1/kvstore-249/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-249/rev-000000001/seqIndex/state.501: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:13.012813-08:00 CRITICAL [(default) magma_1/kvstore-313/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-313/rev-000000001/seqIndex/state.23: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:13.016835-08:00 CRITICAL [(default) magma_1/kvstore-321/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-321/rev-000000001/seqIndex/state.23: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:13.019981-08:00 CRITICAL [(default) magma_1/kvstore-329/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-329/rev-000000001/seqIndex/state.578: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:13.023211-08:00 CRITICAL [(default) magma_1/kvstore-337/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-337/rev-000000001/seqIndex/state.160: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:13.056469-08:00 CRITICAL [(default) magma_1/kvstore-561/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-561/rev-000000001/seqIndex/state.20: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:13.059711-08:00 CRITICAL [(default) magma_1/kvstore-569/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-569/rev-000000001/seqIndex/state.33: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:13.063053-08:00 CRITICAL [(default) magma_1/kvstore-577/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-577/rev-000000001/seqIndex/state.51: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:13.066329-08:00 CRITICAL [(default) magma_1/kvstore-585/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-585/rev-000000001/seqIndex/state.85: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:13.069748-08:00 CRITICAL [(default) magma_1/kvstore-593/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-593/rev-000000001/seqIndex/state.158: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:13.086846-08:00 CRITICAL [(default) magma_1/kvstore-801/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-801/rev-000000001/seqIndex/state.27: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1/kvstore-801/rev-000000001\n', 'memcached<0.133.0>: 2022-03-06T11:54:13.089551-08:00 CRITICAL [(default) magma_1/kvstore-809/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-809/rev-000000001/seqIndex/state.26: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:13.092873-08:00 CRITICAL [(default) magma_1/kvstore-817/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-817/rev-000000001/seqIndex/state.27: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:13.095836-08:00 CRITICAL [(default) magma_1/kvstore-825/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-825/rev-000000001/seqIndex/state.43: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:13.099037-08:00 CRITICAL [(default) magma_1/kvstore-833/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-833/rev-000000001/seqIndex/state.55: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:13.102764-08:00 CRITICAL [(default) magma_1/kvstore-841/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-841/rev-000000001/seqIndex/state.69: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n', 'memcached<0.133.0>: 2022-03-06T11:54:13.106255-08:00 CRITICAL [(default) magma_1/kvstore-849/rev-000000001/seqIndex]Failed to write tree state /data/default/magma.1/kvstore-849/rev-000000001/seqIndex/state.279: Unable to open file err=No such file or directory. Closest non-empty parent directory: /data/default/magma.1\n'] 2022-03-06 13:45:36,453 | test | WARNING | MainThread | [rest_client:get_nodes:1896] 172.23.100.35 - Node not part of cluster inactiveAdded 2022-03-06 13:45:36,453 | test | INFO | MainThread | [cluster_ready_functions:trigger_cb_collect_on_cluster:1130] Running cbcollect on node ns_1@172.23.100.35,ns_1@172.23.105.164,ns_1@172.23.105.206,ns_1@172.23.106.177 2022-03-06 13:45:46,484 | test | INFO | MainThread | [cluster_ready_functions:trigger_cb_collect_on_cluster:1133] ns_1@172.23.100.35,ns_1@172.23.105.164,ns_1@172.23.105.206,ns_1@172.23.106.177 - cbcollect status: True 2022-03-06 13:45:46,486 | test | INFO | MainThread | [cluster_ready_functions:wait_for_cb_collect_to_complete:1138] Polling active_tasks to check cbcollect status 2022-03-06 13:49:36,921 | test | INFO | MainThread | [cluster_ready_functions:copy_cb_collect_logs:1165] ns_1@172.23.100.35: Copying cbcollect ZIP file to Client 2022-03-06 13:49:37,990 | test | INFO | MainThread | [cluster_ready_functions:copy_cb_collect_logs:1165] ns_1@172.23.105.164: Copying cbcollect ZIP file to Client 2022-03-06 13:49:39,276 | test | INFO | MainThread | [cluster_ready_functions:copy_cb_collect_logs:1165] ns_1@172.23.105.206: Copying cbcollect ZIP file to Client 2022-03-06 13:49:40,365 | test | INFO | MainThread | [cluster_ready_functions:copy_cb_collect_logs:1165] ns_1@172.23.106.177: Copying cbcollect ZIP file to Client 2022-03-06 13:49:41,296 | test | CRITICAL | MainThread | [basetestcase:tearDownEverything:590] Skipping get_trace !! ERROR During the test, Remote Connections: 147, Disconnections: 147 SDK Connections: 30, Disconnections: 30 ====================================================================== ERROR: test_data_load_collections_with_hard_failover_recovery (bucket_collections.collections_rebalance.CollectionsRebalance) ---------------------------------------------------------------------- Traceback (most recent call last): File "pytests/bucket_collections/collections_rebalance.py", line 92, in tearDown super(CollectionsRebalance, self).tearDown() File "pytests/bucket_collections/collections_base.py", line 101, in tearDown super(CollectionBase, self).tearDown() File "pytests/basetestcase.py", line 1109, in tearDown super(ClusterSetup, self).tearDown() File "pytests/basetestcase.py", line 552, in tearDown self.assertFalse(result, msg="Cb_log file validation failed") AssertionError: Cb_log file validation failed ====================================================================== FAIL: test_data_load_collections_with_hard_failover_recovery (bucket_collections.collections_rebalance.CollectionsRebalance) ---------------------------------------------------------------------- Traceback (most recent call last): File "pytests/bucket_collections/collections_rebalance.py", line 1119, in test_data_load_collections_with_hard_failover_recovery self.load_collections_with_rebalance(rebalance_operation="hard_failover_recovery") File "pytests/bucket_collections/collections_rebalance.py", line 1080, in load_collections_with_rebalance self.wait_for_rebalance_to_complete(rebalance) File "pytests/bucket_collections/collections_rebalance.py", line 869, in wait_for_rebalance_to_complete self.assertTrue(task.result, "Rebalance Failed") AssertionError: Rebalance Failed summary so far suite bucket_collections.collections_rebalance.CollectionsRebalance , pass 2 , fail 2 failures so far... bucket_collections.collections_rebalance.CollectionsRebalance.test_data_load_collections_with_graceful_failover_recovery bucket_collections.collections_rebalance.CollectionsRebalance.test_data_load_collections_with_hard_failover_recovery testrunner logs, diags and results are available under /data/workspace/temp_rebalance_magma/logs/testrunner-22-Mar-06_03-39-55/test_4 ---------------------------------------------------------------------- Ran 1 test in 7341.799s FAILED (failures=1, errors=1) Logs will be stored at /data/workspace/temp_rebalance_magma/logs/testrunner-22-Mar-06_03-39-55/test_5 guides/gradlew --refresh-dependencies testrunner -P jython=/opt/jython/bin/jython -P 'args=-i /tmp/win10-bucket-ops-temp_rebalance_magma.ini rerun=False,get-cbcollect-info=True -t bucket_collections.collections_rebalance.CollectionsRebalance.test_data_load_collections_with_graceful_failover_rebalance_out,nodes_init=4,nodes_failover=1,bucket_spec=magma_dgm.bloom_filter.4_node_1_replica_magma_1024,doc_size=30,randomize_value=True,data_load_stage=during,skip_validations=False,data_load_spec=volume_test_load_with_CRUD_on_collections,change_magma_quota=True,GROUP=failover' Test Input params: {'doc_size': '30', 'data_load_stage': 'during', 'cluster_name': 'win10-bucket-ops-temp_rebalance_magma', 'nodes_failover': '1', 'ini': '/tmp/win10-bucket-ops-temp_rebalance_magma.ini', 'get-cbcollect-info': 'True', 'conf_file': 'conf/magma/dgm_collections_bloom_filter.conf', 'change_magma_quota': 'True', 'skip_validations': 'False', 'spec': 'dgm_collections_bloom_filter', 'rerun': 'False', 'num_nodes': 7, 'logs_folder': '/data/workspace/temp_rebalance_magma/logs/testrunner-22-Mar-06_03-39-55/test_5', 'nodes_init': '4', 'GROUP': 'failover', 'bucket_spec': 'magma_dgm.bloom_filter.4_node_1_replica_magma_1024', 'case_number': 5, 'randomize_value': 'True', 'data_load_spec': 'volume_test_load_with_CRUD_on_collections'} test_data_load_collections_with_graceful_failover_rebalance_out (bucket_collections.collections_rebalance.CollectionsRebalance) ... 2022-03-06 13:49:41,721 | test | INFO | MainThread | [basetestcase:log_setup_status:647] ========= CollectionsRebalance setup started for test #5 test_data_load_collections_with_graceful_failover_rebalance_out ========= 2022-03-06 13:49:41,844 | test | INFO | MainThread | [basetestcase:log_setup_status:647] ========= BaseTestCase setup started for test #5 test_data_load_collections_with_graceful_failover_rebalance_out ========= 2022-03-06 13:50:22,311 | test | INFO | MainThread | [rest_client:monitorRebalance:1610] Rebalance done. Taken 11.0540001392 seconds to complete 2022-03-06 13:50:22,319 | test | INFO | MainThread | [common_lib:sleep:23] Sleep 5 seconds. Reason: Wait after rebalance complete 2022-03-06 13:50:27,359 | test | ERROR | MainThread | [rest_client:_http_request:834] GET http://172.23.100.35:8091/nodes/self body: headers: {'Accept': '*/*', 'Connection': 'close', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==', 'Content-Type': 'application/json'} error: 404 reason: unknown "Node is unknown to this cluster." auth: Administrator:password http://172.23.100.35:8091/nodes/self with status 0: Node is unknown to this cluster. 2022-03-06 13:50:27,362 | test | ERROR | MainThread | [rest_client:__init__:312] Error Node is unknown to this cluster. was gotten, 5 seconds sleep before retry 2022-03-06 13:50:32,378 | test | ERROR | MainThread | [rest_client:_http_request:834] GET http://172.23.100.35:8091/nodes/self body: headers: {'Accept': '*/*', 'Connection': 'close', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==', 'Content-Type': 'application/json'} error: 404 reason: unknown "Node is unknown to this cluster." auth: Administrator:password http://172.23.100.35:8091/nodes/self with status 0: Node is unknown to this cluster. 2022-03-06 13:50:32,380 | test | ERROR | MainThread | [rest_client:__init__:312] Error Node is unknown to this cluster. was gotten, 5 seconds sleep before retry 2022-03-06 13:50:37,394 | test | ERROR | MainThread | [rest_client:_http_request:834] GET http://172.23.100.35:8091/nodes/self body: headers: {'Accept': '*/*', 'Connection': 'close', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==', 'Content-Type': 'application/json'} error: 404 reason: unknown "Node is unknown to this cluster." auth: Administrator:password http://172.23.100.35:8091/nodes/self with status 0: Node is unknown to this cluster. 2022-03-06 13:50:37,395 | test | ERROR | MainThread | [rest_client:__init__:312] Error Node is unknown to this cluster. was gotten, 5 seconds sleep before retry 2022-03-06 13:50:42,404 | test | ERROR | MainThread | [rest_client:__init__:317] Node 172.23.100.35:8091 is in a broken state! 2022-03-06 13:50:42,404 | test | ERROR | MainThread | [cluster_ready_functions:cleanup_cluster:232] Can't create rest connection after rebalance out for ejected nodes, will retry after 10 seconds according to MB-8430: Unable to reach the host @ 172.23.100.35 2022-03-06 13:50:42,411 | test | INFO | MainThread | [common_lib:sleep:23] Sleep 10 seconds. Reason: MB-8430 2022-03-06 13:50:52,420 | test | ERROR | MainThread | [rest_client:_http_request:834] GET http://172.23.100.35:8091/nodes/self body: headers: {'Accept': '*/*', 'Connection': 'close', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==', 'Content-Type': 'application/json'} error: 404 reason: unknown "Node is unknown to this cluster." auth: Administrator:password http://172.23.100.35:8091/nodes/self with status 0: Node is unknown to this cluster. 2022-03-06 13:50:52,421 | test | ERROR | MainThread | [rest_client:__init__:312] Error Node is unknown to this cluster. was gotten, 5 seconds sleep before retry 2022-03-06 13:50:57,436 | test | ERROR | MainThread | [rest_client:_http_request:834] GET http://172.23.100.35:8091/nodes/self body: headers: {'Accept': '*/*', 'Connection': 'close', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==', 'Content-Type': 'application/json'} error: 404 reason: unknown "Node is unknown to this cluster." auth: Administrator:password http://172.23.100.35:8091/nodes/self with status 0: Node is unknown to this cluster. 2022-03-06 13:50:57,437 | test | ERROR | MainThread | [rest_client:__init__:312] Error Node is unknown to this cluster. was gotten, 5 seconds sleep before retry 2022-03-06 13:51:02,453 | test | ERROR | MainThread | [rest_client:_http_request:834] GET http://172.23.100.35:8091/nodes/self body: headers: {'Accept': '*/*', 'Connection': 'close', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==', 'Content-Type': 'application/json'} error: 404 reason: unknown "Node is unknown to this cluster." auth: Administrator:password http://172.23.100.35:8091/nodes/self with status 0: Node is unknown to this cluster. 2022-03-06 13:51:02,456 | test | ERROR | MainThread | [rest_client:__init__:312] Error Node is unknown to this cluster. was gotten, 5 seconds sleep before retry 2022-03-06 13:51:07,463 | test | ERROR | MainThread | [rest_client:__init__:317] Node 172.23.100.35:8091 is in a broken state! Traceback (most recent call last): File "pytests/basetestcase.py", line 363, in setUp self.cluster_util.cluster_cleanup(cluster, File "pytests/basetestcase.py", line 363, in setUp self.cluster_util.cluster_cleanup(cluster, File "couchbase_utils/cluster_utils/cluster_ready_functions.py", line 169, in cluster_cleanup self.cleanup_cluster(cluster, master=cluster.master) File "couchbase_utils/cluster_utils/cluster_ready_functions.py", line 237, in cleanup_cluster rest = RestConnection(removed) File "lib/membase/api/rest_client.py", line 319, in __init__ raise ServerUnavailableException(self.ip) ServerUnavailableException: Unable to reach the host @ 172.23.100.35 ERROR ====================================================================== ERROR: test_data_load_collections_with_graceful_failover_rebalance_out (bucket_collections.collections_rebalance.CollectionsRebalance) ---------------------------------------------------------------------- Traceback (most recent call last): File "pytests/bucket_collections/collections_rebalance.py", line 22, in setUp super(CollectionsRebalance, self).setUp() File "pytests/bucket_collections/collections_base.py", line 19, in setUp super(CollectionBase, self).setUp() File "pytests/basetestcase.py", line 1068, in setUp super(ClusterSetup, self).setUp() File "pytests/basetestcase.py", line 463, in setUp self.fail(e) AssertionError: Unable to reach the host @ 172.23.100.35 ---------------------------------------------------------------------- During the test, Remote Connections: 8, Disconnections: 8 Ran 1 test in 86.121s FAILED (errors=1) SDK Connections: 0, Disconnections: 0 summary so far suite bucket_collections.collections_rebalance.CollectionsRebalance , pass 2 , fail 3 failures so far... bucket_collections.collections_rebalance.CollectionsRebalance.test_data_load_collections_with_graceful_failover_recovery bucket_collections.collections_rebalance.CollectionsRebalance.test_data_load_collections_with_hard_failover_recovery bucket_collections.collections_rebalance.CollectionsRebalance.test_data_load_collections_with_graceful_failover_rebalance_out testrunner logs, diags and results are available under /data/workspace/temp_rebalance_magma/logs/testrunner-22-Mar-06_03-39-55/test_5 Logs will be stored at /data/workspace/temp_rebalance_magma/logs/testrunner-22-Mar-06_03-39-55/test_6 guides/gradlew --refresh-dependencies testrunner -P jython=/opt/jython/bin/jython -P 'args=-i /tmp/win10-bucket-ops-temp_rebalance_magma.ini rerun=False,get-cbcollect-info=True -t bucket_collections.collections_rebalance.CollectionsRebalance.test_data_load_collections_with_hard_failover_rebalance_out,nodes_init=4,nodes_failover=1,bucket_spec=magma_dgm.bloom_filter.4_node_1_replica_magma_1024,doc_size=30,randomize_value=True,data_load_stage=during,skip_validations=False,data_load_spec=volume_test_load_with_CRUD_on_collections,change_magma_quota=True,GROUP=failover' Test Input params: {'doc_size': '30', 'data_load_stage': 'during', 'cluster_name': 'win10-bucket-ops-temp_rebalance_magma', 'nodes_failover': '1', 'ini': '/tmp/win10-bucket-ops-temp_rebalance_magma.ini', 'get-cbcollect-info': 'True', 'conf_file': 'conf/magma/dgm_collections_bloom_filter.conf', 'change_magma_quota': 'True', 'skip_validations': 'False', 'spec': 'dgm_collections_bloom_filter', 'rerun': 'False', 'num_nodes': 7, 'logs_folder': '/data/workspace/temp_rebalance_magma/logs/testrunner-22-Mar-06_03-39-55/test_6', 'nodes_init': '4', 'GROUP': 'failover', 'bucket_spec': 'magma_dgm.bloom_filter.4_node_1_replica_magma_1024', 'case_number': 6, 'randomize_value': 'True', 'data_load_spec': 'volume_test_load_with_CRUD_on_collections'} test_data_load_collections_with_hard_failover_rebalance_out (bucket_collections.collections_rebalance.CollectionsRebalance) ... 2022-03-06 13:51:07,874 | test | INFO | MainThread | [basetestcase:log_setup_status:647] ========= CollectionsRebalance setup started for test #6 test_data_load_collections_with_hard_failover_rebalance_out ========= 2022-03-06 13:51:07,960 | test | INFO | MainThread | [basetestcase:log_setup_status:647] ========= BaseTestCase setup started for test #6 test_data_load_collections_with_hard_failover_rebalance_out ========= 2022-03-06 13:51:08,033 | infra | ERROR | MainThread | [Rest_Connection:_http_request:286] GET http://172.23.100.35:8091/nodes/self body: headers: {'Accept': '*/*', 'Connection': 'close', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==', 'Content-Type': 'application/json'} error: 404 reason: unknown "Node is unknown to this cluster." auth: Administrator:password http://172.23.100.35:8091/nodes/self with status 0: Node is unknown to this cluster. 2022-03-06 13:51:08,036 | infra | ERROR | MainThread | [Rest_Connection:__init__:148] Error Node is unknown to this cluster., 5 seconds sleep before retry 2022-03-06 13:51:13,056 | infra | ERROR | MainThread | [Rest_Connection:_http_request:286] GET http://172.23.100.35:8091/nodes/self body: headers: {'Accept': '*/*', 'Connection': 'close', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==', 'Content-Type': 'application/json'} error: 404 reason: unknown "Node is unknown to this cluster." auth: Administrator:password http://172.23.100.35:8091/nodes/self with status 0: Node is unknown to this cluster. 2022-03-06 13:51:13,058 | infra | ERROR | MainThread | [Rest_Connection:__init__:148] Error Node is unknown to this cluster., 5 seconds sleep before retry 2022-03-06 13:51:18,075 | infra | ERROR | MainThread | [Rest_Connection:_http_request:286] GET http://172.23.100.35:8091/nodes/self body: headers: {'Accept': '*/*', 'Connection': 'close', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==', 'Content-Type': 'application/json'} error: 404 reason: unknown "Node is unknown to this cluster." auth: Administrator:password http://172.23.100.35:8091/nodes/self with status 0: Node is unknown to this cluster. 2022-03-06 13:51:18,076 | infra | ERROR | MainThread | [Rest_Connection:__init__:148] Error Node is unknown to this cluster., 5 seconds sleep before retry 2022-03-06 13:51:23,086 | infra | ERROR | MainThread | [Rest_Connection:__init__:152] Node 172.23.100.35:8091 is in a broken state! Traceback (most recent call last): File "pytests/basetestcase.py", line 363, in setUp self.cluster_util.cluster_cleanup(cluster, File "pytests/basetestcase.py", line 363, in setUp self.cluster_util.cluster_cleanup(cluster, File "couchbase_utils/cluster_utils/cluster_ready_functions.py", line 168, in cluster_cleanup bucket_util.delete_all_buckets(cluster) File "couchbase_utils/bucket_utils/bucket_ready_functions.py", line 1675, in delete_all_buckets buckets = self.get_all_buckets(cluster, cluster_node=server) File "couchbase_utils/bucket_utils/bucket_ready_functions.py", line 3909, in get_all_buckets rest = BucketHelper(cluster_node) File "lib/BucketLib/BucketOperations_Rest.py", line 22, in __init__ super(BucketHelper, self).__init__(server) File "connections/Rest_Connection.py", line 154, in __init__ raise ServerUnavailableException(self.ip) ServerUnavailableException: Unable to reach the host @ 172.23.100.35 During the test, Remote Connections: 8, Disconnections: 8 SDK Connections: 0, Disconnections: 0 ERROR ====================================================================== ERROR: test_data_load_collections_with_hard_failover_rebalance_out (bucket_collections.collections_rebalance.CollectionsRebalance) ---------------------------------------------------------------------- Traceback (most recent call last): File "pytests/bucket_collections/collections_rebalance.py", line 22, in setUp super(CollectionsRebalance, self).setUp() File "pytests/bucket_collections/collections_base.py", line 19, in setUp super(CollectionBase, self).setUp() File "pytests/basetestcase.py", line 1068, in setUp super(ClusterSetup, self).setUp() File "pytests/basetestcase.py", line 463, in setUp self.fail(e) AssertionError: Unable to reach the host @ 172.23.100.35 ---------------------------------------------------------------------- Ran 1 test in 15.610s FAILED (errors=1) summary so far suite bucket_collections.collections_rebalance.CollectionsRebalance , pass 2 , fail 4 failures so far... bucket_collections.collections_rebalance.CollectionsRebalance.test_data_load_collections_with_graceful_failover_recovery bucket_collections.collections_rebalance.CollectionsRebalance.test_data_load_collections_with_hard_failover_recovery bucket_collections.collections_rebalance.CollectionsRebalance.test_data_load_collections_with_graceful_failover_rebalance_out bucket_collections.collections_rebalance.CollectionsRebalance.test_data_load_collections_with_hard_failover_rebalance_out testrunner logs, diags and results are available under /data/workspace/temp_rebalance_magma/logs/testrunner-22-Mar-06_03-39-55/test_6 Logs will be stored at /data/workspace/temp_rebalance_magma/logs/testrunner-22-Mar-06_03-39-55/test_7 guides/gradlew --refresh-dependencies testrunner -P jython=/opt/jython/bin/jython -P 'args=-i /tmp/win10-bucket-ops-temp_rebalance_magma.ini rerun=False,get-cbcollect-info=True -t bucket_collections.collections_rebalance.CollectionsRebalance.test_data_load_collections_with_rebalance_in,nodes_init=4,nodes_in=2,bucket_spec=magma_dgm.bloom_filter.4_node_1_replica_magma_1024,doc_size=30,randomize_value=True,data_load_stage=during,skip_validations=False,data_load_spec=volume_test_load_with_CRUD_on_collections,change_magma_quota=True,GROUP=rebalance' Test Input params: {'doc_size': '30', 'data_load_stage': 'during', 'cluster_name': 'win10-bucket-ops-temp_rebalance_magma', 'ini': '/tmp/win10-bucket-ops-temp_rebalance_magma.ini', 'get-cbcollect-info': 'True', 'conf_file': 'conf/magma/dgm_collections_bloom_filter.conf', 'change_magma_quota': 'True', 'nodes_in': '2', 'skip_validations': 'False', 'spec': 'dgm_collections_bloom_filter', 'rerun': 'False', 'num_nodes': 7, 'logs_folder': '/data/workspace/temp_rebalance_magma/logs/testrunner-22-Mar-06_03-39-55/test_7', 'nodes_init': '4', 'GROUP': 'rebalance', 'bucket_spec': 'magma_dgm.bloom_filter.4_node_1_replica_magma_1024', 'case_number': 7, 'randomize_value': 'True', 'data_load_spec': 'volume_test_load_with_CRUD_on_collections'} test_data_load_collections_with_rebalance_in (bucket_collections.collections_rebalance.CollectionsRebalance) ... 2022-03-06 13:51:23,463 | test | INFO | MainThread | [basetestcase:log_setup_status:647] ========= CollectionsRebalance setup started for test #7 test_data_load_collections_with_rebalance_in ========= 2022-03-06 13:51:23,555 | test | INFO | MainThread | [basetestcase:log_setup_status:647] ========= BaseTestCase setup started for test #7 test_data_load_collections_with_rebalance_in ========= 2022-03-06 13:51:23,625 | infra | ERROR | MainThread | [Rest_Connection:_http_request:286] GET http://172.23.100.35:8091/nodes/self body: headers: {'Accept': '*/*', 'Connection': 'close', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==', 'Content-Type': 'application/json'} error: 404 reason: unknown "Node is unknown to this cluster." auth: Administrator:password http://172.23.100.35:8091/nodes/self with status 0: Node is unknown to this cluster. 2022-03-06 13:51:23,627 | infra | ERROR | MainThread | [Rest_Connection:__init__:148] Error Node is unknown to this cluster., 5 seconds sleep before retry 2022-03-06 13:51:28,641 | infra | ERROR | MainThread | [Rest_Connection:_http_request:286] GET http://172.23.100.35:8091/nodes/self body: headers: {'Accept': '*/*', 'Connection': 'close', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==', 'Content-Type': 'application/json'} error: 404 reason: unknown "Node is unknown to this cluster." auth: Administrator:password http://172.23.100.35:8091/nodes/self with status 0: Node is unknown to this cluster. 2022-03-06 13:51:28,644 | infra | ERROR | MainThread | [Rest_Connection:__init__:148] Error Node is unknown to this cluster., 5 seconds sleep before retry 2022-03-06 13:51:33,658 | infra | ERROR | MainThread | [Rest_Connection:_http_request:286] GET http://172.23.100.35:8091/nodes/self body: headers: {'Accept': '*/*', 'Connection': 'close', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==', 'Content-Type': 'application/json'} error: 404 reason: unknown "Node is unknown to this cluster." auth: Administrator:password http://172.23.100.35:8091/nodes/self with status 0: Node is unknown to this cluster. 2022-03-06 13:51:33,661 | infra | ERROR | MainThread | [Rest_Connection:__init__:148] Error Node is unknown to this cluster., 5 seconds sleep before retry 2022-03-06 13:51:38,667 | infra | ERROR | MainThread | [Rest_Connection:__init__:152] Node 172.23.100.35:8091 is in a broken state! Traceback (most recent call last): File "pytests/basetestcase.py", line 363, in setUp self.cluster_util.cluster_cleanup(cluster, File "pytests/basetestcase.py", line 363, in setUp self.cluster_util.cluster_cleanup(cluster, File "couchbase_utils/cluster_utils/cluster_ready_functions.py", line 168, in cluster_cleanup bucket_util.delete_all_buckets(cluster) File "couchbase_utils/bucket_utils/bucket_ready_functions.py", line 1675, in delete_all_buckets buckets = self.get_all_buckets(cluster, cluster_node=server) File "couchbase_utils/bucket_utils/bucket_ready_functions.py", line 3909, in get_all_buckets rest = BucketHelper(cluster_node) File "lib/BucketLib/BucketOperations_Rest.py", line 22, in __init__ super(BucketHelper, self).__init__(server) File "connections/Rest_Connection.py", line 154, in __init__ raise ServerUnavailableException(self.ip) ServerUnavailableException: Unable to reach the host @ 172.23.100.35 ERROR ====================================================================== ERROR: test_data_load_collections_with_rebalance_in (bucket_collections.collections_rebalance.CollectionsRebalance) ---------------------------------------------------------------------- Traceback (most recent call last): File "pytests/bucket_collections/collections_rebalance.py", line 22, in setUp super(CollectionsRebalance, self).setUp() File "pytests/bucket_collections/collections_base.py", line 19, in setUp super(CollectionBase, self).setUp() File "pytests/basetestcase.py", line 1068, in setUp super(ClusterSetup, self).setUp() File "pytests/basetestcase.py", line 463, in setUp self.fail(e) AssertionError: Unable to reach the host @ 172.23.100.35 ---------------------------------------------------------------------- Ran 1 test in 15.568s FAILED (errors=1) During the test, Remote Connections: 8, Disconnections: 8 SDK Connections: 0, Disconnections: 0 summary so far suite bucket_collections.collections_rebalance.CollectionsRebalance , pass 2 , fail 5 failures so far... bucket_collections.collections_rebalance.CollectionsRebalance.test_data_load_collections_with_graceful_failover_recovery bucket_collections.collections_rebalance.CollectionsRebalance.test_data_load_collections_with_hard_failover_recovery bucket_collections.collections_rebalance.CollectionsRebalance.test_data_load_collections_with_graceful_failover_rebalance_out bucket_collections.collections_rebalance.CollectionsRebalance.test_data_load_collections_with_hard_failover_rebalance_out bucket_collections.collections_rebalance.CollectionsRebalance.test_data_load_collections_with_rebalance_in testrunner logs, diags and results are available under /data/workspace/temp_rebalance_magma/logs/testrunner-22-Mar-06_03-39-55/test_7 Logs will be stored at /data/workspace/temp_rebalance_magma/logs/testrunner-22-Mar-06_03-39-55/test_8 guides/gradlew --refresh-dependencies testrunner -P jython=/opt/jython/bin/jython -P 'args=-i /tmp/win10-bucket-ops-temp_rebalance_magma.ini rerun=False,get-cbcollect-info=True -t bucket_collections.collections_rebalance.CollectionsRebalance.test_data_load_collections_with_rebalance_out,nodes_init=4,nodes_out=1,bucket_spec=magma_dgm.bloom_filter.4_node_1_replica_magma_1024,doc_size=30,randomize_value=True,data_load_stage=during,skip_validations=False,data_load_spec=volume_test_load_with_CRUD_on_collections,change_magma_quota=True,GROUP=rebalance' Test Input params: {'doc_size': '30', 'data_load_stage': 'during', 'cluster_name': 'win10-bucket-ops-temp_rebalance_magma', 'ini': '/tmp/win10-bucket-ops-temp_rebalance_magma.ini', 'get-cbcollect-info': 'True', 'conf_file': 'conf/magma/dgm_collections_bloom_filter.conf', 'change_magma_quota': 'True', 'skip_validations': 'False', 'spec': 'dgm_collections_bloom_filter', 'rerun': 'False', 'num_nodes': 7, 'logs_folder': '/data/workspace/temp_rebalance_magma/logs/testrunner-22-Mar-06_03-39-55/test_8', 'nodes_init': '4', 'GROUP': 'rebalance', 'nodes_out': '1', 'bucket_spec': 'magma_dgm.bloom_filter.4_node_1_replica_magma_1024', 'case_number': 8, 'randomize_value': 'True', 'data_load_spec': 'volume_test_load_with_CRUD_on_collections'} test_data_load_collections_with_rebalance_out (bucket_collections.collections_rebalance.CollectionsRebalance) ... 2022-03-06 13:51:39,009 | test | INFO | MainThread | [basetestcase:log_setup_status:647] ========= CollectionsRebalance setup started for test #8 test_data_load_collections_with_rebalance_out ========= 2022-03-06 13:51:39,091 | test | INFO | MainThread | [basetestcase:log_setup_status:647] ========= BaseTestCase setup started for test #8 test_data_load_collections_with_rebalance_out ========= 2022-03-06 13:51:39,157 | infra | ERROR | MainThread | [Rest_Connection:_http_request:286] GET http://172.23.100.35:8091/nodes/self body: headers: {'Accept': '*/*', 'Connection': 'close', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==', 'Content-Type': 'application/json'} error: 404 reason: unknown "Node is unknown to this cluster." auth: Administrator:password http://172.23.100.35:8091/nodes/self with status 0: Node is unknown to this cluster. 2022-03-06 13:51:39,160 | infra | ERROR | MainThread | [Rest_Connection:__init__:148] Error Node is unknown to this cluster., 5 seconds sleep before retry 2022-03-06 13:51:44,174 | infra | ERROR | MainThread | [Rest_Connection:_http_request:286] GET http://172.23.100.35:8091/nodes/self body: headers: {'Accept': '*/*', 'Connection': 'close', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==', 'Content-Type': 'application/json'} error: 404 reason: unknown "Node is unknown to this cluster." auth: Administrator:password http://172.23.100.35:8091/nodes/self with status 0: Node is unknown to this cluster. 2022-03-06 13:51:44,176 | infra | ERROR | MainThread | [Rest_Connection:__init__:148] Error Node is unknown to this cluster., 5 seconds sleep before retry 2022-03-06 13:51:49,190 | infra | ERROR | MainThread | [Rest_Connection:_http_request:286] GET http://172.23.100.35:8091/nodes/self body: headers: {'Accept': '*/*', 'Connection': 'close', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==', 'Content-Type': 'application/json'} error: 404 reason: unknown "Node is unknown to this cluster." auth: Administrator:password http://172.23.100.35:8091/nodes/self with status 0: Node is unknown to this cluster. 2022-03-06 13:51:49,193 | infra | ERROR | MainThread | [Rest_Connection:__init__:148] Error Node is unknown to this cluster., 5 seconds sleep before retry 2022-03-06 13:51:54,200 | infra | ERROR | MainThread | [Rest_Connection:__init__:152] Node 172.23.100.35:8091 is in a broken state! Traceback (most recent call last): File "pytests/basetestcase.py", line 363, in setUp self.cluster_util.cluster_cleanup(cluster, File "pytests/basetestcase.py", line 363, in setUp self.cluster_util.cluster_cleanup(cluster, File "couchbase_utils/cluster_utils/cluster_ready_functions.py", line 168, in cluster_cleanup bucket_util.delete_all_buckets(cluster) File "couchbase_utils/bucket_utils/bucket_ready_functions.py", line 1675, in delete_all_buckets buckets = self.get_all_buckets(cluster, cluster_node=server) File "couchbase_utils/bucket_utils/bucket_ready_functions.py", line 3909, in get_all_buckets rest = BucketHelper(cluster_node) File "lib/BucketLib/BucketOperations_Rest.py", line 22, in __init__ super(BucketHelper, self).__init__(server) File "connections/Rest_Connection.py", line 154, in __init__ raise ServerUnavailableException(self.ip) ServerUnavailableException: Unable to reach the host @ 172.23.100.35 ERROR ====================================================================== ERROR: test_data_load_collections_with_rebalance_out (bucket_collections.collections_rebalance.CollectionsRebalance) ---------------------------------------------------------------------- Traceback (most recent call last): File "pytests/bucket_collections/collections_rebalance.py", line 22, in setUp super(CollectionsRebalance, self).setUp() File "pytests/bucket_collections/collections_base.py", line 19, in setUp super(CollectionBase, self).setUp() File "pytests/basetestcase.py", line 1068, in setUp super(ClusterSetup, self).setUp() File "pytests/basetestcase.py", line 463, in setUp self.fail(e) AssertionError: Unable to reach the host @ 172.23.100.35 ---------------------------------------------------------------------- During the test, Remote Connections: 8, Disconnections: 8 SDK Connections: 0, Disconnections: 0 Ran 1 test in 15.518s FAILED (errors=1) summary so far suite bucket_collections.collections_rebalance.CollectionsRebalance , pass 2 , fail 6 failures so far... bucket_collections.collections_rebalance.CollectionsRebalance.test_data_load_collections_with_graceful_failover_recovery bucket_collections.collections_rebalance.CollectionsRebalance.test_data_load_collections_with_hard_failover_recovery bucket_collections.collections_rebalance.CollectionsRebalance.test_data_load_collections_with_graceful_failover_rebalance_out bucket_collections.collections_rebalance.CollectionsRebalance.test_data_load_collections_with_hard_failover_rebalance_out bucket_collections.collections_rebalance.CollectionsRebalance.test_data_load_collections_with_rebalance_in bucket_collections.collections_rebalance.CollectionsRebalance.test_data_load_collections_with_rebalance_out testrunner logs, diags and results are available under /data/workspace/temp_rebalance_magma/logs/testrunner-22-Mar-06_03-39-55/test_8 Logs will be stored at /data/workspace/temp_rebalance_magma/logs/testrunner-22-Mar-06_03-39-55/test_9 guides/gradlew --refresh-dependencies testrunner -P jython=/opt/jython/bin/jython -P 'args=-i /tmp/win10-bucket-ops-temp_rebalance_magma.ini rerun=False,get-cbcollect-info=True -t bucket_collections.collections_rebalance.CollectionsRebalance.test_data_load_collections_with_swap_rebalance,nodes_init=4,nodes_swap=1,bucket_spec=magma_dgm.bloom_filter.4_node_1_replica_magma_1024,doc_size=30,randomize_value=True,data_load_stage=during,skip_validations=False,data_load_spec=volume_test_load_with_CRUD_on_collections,change_magma_quota=True,GROUP=rebalance' Test Input params: {'doc_size': '30', 'data_load_stage': 'during', 'cluster_name': 'win10-bucket-ops-temp_rebalance_magma', 'ini': '/tmp/win10-bucket-ops-temp_rebalance_magma.ini', 'get-cbcollect-info': 'True', 'conf_file': 'conf/magma/dgm_collections_bloom_filter.conf', 'change_magma_quota': 'True', 'skip_validations': 'False', 'spec': 'dgm_collections_bloom_filter', 'rerun': 'False', 'num_nodes': 7, 'logs_folder': '/data/workspace/temp_rebalance_magma/logs/testrunner-22-Mar-06_03-39-55/test_9', 'nodes_init': '4', 'GROUP': 'rebalance', 'nodes_swap': '1', 'bucket_spec': 'magma_dgm.bloom_filter.4_node_1_replica_magma_1024', 'case_number': 9, 'randomize_value': 'True', 'data_load_spec': 'volume_test_load_with_CRUD_on_collections'} test_data_load_collections_with_swap_rebalance (bucket_collections.collections_rebalance.CollectionsRebalance) ... 2022-03-06 13:51:54,565 | test | INFO | MainThread | [basetestcase:log_setup_status:647] ========= CollectionsRebalance setup started for test #9 test_data_load_collections_with_swap_rebalance ========= 2022-03-06 13:51:54,650 | test | INFO | MainThread | [basetestcase:log_setup_status:647] ========= BaseTestCase setup started for test #9 test_data_load_collections_with_swap_rebalance ========= 2022-03-06 13:51:54,717 | infra | ERROR | MainThread | [Rest_Connection:_http_request:286] GET http://172.23.100.35:8091/nodes/self body: headers: {'Accept': '*/*', 'Connection': 'close', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==', 'Content-Type': 'application/json'} error: 404 reason: unknown "Node is unknown to this cluster." auth: Administrator:password http://172.23.100.35:8091/nodes/self with status 0: Node is unknown to this cluster. 2022-03-06 13:51:54,717 | infra | ERROR | MainThread | [Rest_Connection:__init__:148] Error Node is unknown to this cluster., 5 seconds sleep before retry 2022-03-06 13:51:59,737 | infra | ERROR | MainThread | [Rest_Connection:_http_request:286] GET http://172.23.100.35:8091/nodes/self body: headers: {'Accept': '*/*', 'Connection': 'close', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==', 'Content-Type': 'application/json'} error: 404 reason: unknown "Node is unknown to this cluster." auth: Administrator:password http://172.23.100.35:8091/nodes/self with status 0: Node is unknown to this cluster. 2022-03-06 13:51:59,740 | infra | ERROR | MainThread | [Rest_Connection:__init__:148] Error Node is unknown to this cluster., 5 seconds sleep before retry 2022-03-06 13:52:04,756 | infra | ERROR | MainThread | [Rest_Connection:_http_request:286] GET http://172.23.100.35:8091/nodes/self body: headers: {'Accept': '*/*', 'Connection': 'close', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==', 'Content-Type': 'application/json'} error: 404 reason: unknown "Node is unknown to this cluster." auth: Administrator:password http://172.23.100.35:8091/nodes/self with status 0: Node is unknown to this cluster. 2022-03-06 13:52:04,756 | infra | ERROR | MainThread | [Rest_Connection:__init__:148] Error Node is unknown to this cluster., 5 seconds sleep before retry 2022-03-06 13:52:09,765 | infra | ERROR | MainThread | [Rest_Connection:__init__:152] Node 172.23.100.35:8091 is in a broken state! Traceback (most recent call last): File "pytests/basetestcase.py", line 363, in setUp self.cluster_util.cluster_cleanup(cluster, File "pytests/basetestcase.py", line 363, in setUp self.cluster_util.cluster_cleanup(cluster, File "couchbase_utils/cluster_utils/cluster_ready_functions.py", line 168, in cluster_cleanup bucket_util.delete_all_buckets(cluster) File "couchbase_utils/bucket_utils/bucket_ready_functions.py", line 1675, in delete_all_buckets buckets = self.get_all_buckets(cluster, cluster_node=server) File "couchbase_utils/bucket_utils/bucket_ready_functions.py", line 3909, in get_all_buckets rest = BucketHelper(cluster_node) File "lib/BucketLib/BucketOperations_Rest.py", line 22, in __init__ super(BucketHelper, self).__init__(server) File "connections/Rest_Connection.py", line 154, in __init__ raise ServerUnavailableException(self.ip) ServerUnavailableException: Unable to reach the host @ 172.23.100.35 ERROR During the test, Remote Connections: 8, Disconnections: 8 SDK Connections: 0, Disconnections: 0 ====================================================================== ERROR: test_data_load_collections_with_swap_rebalance (bucket_collections.collections_rebalance.CollectionsRebalance) ---------------------------------------------------------------------- Traceback (most recent call last): File "pytests/bucket_collections/collections_rebalance.py", line 22, in setUp super(CollectionsRebalance, self).setUp() File "pytests/bucket_collections/collections_base.py", line 19, in setUp super(CollectionBase, self).setUp() File "pytests/basetestcase.py", line 1068, in setUp super(ClusterSetup, self).setUp() File "pytests/basetestcase.py", line 463, in setUp self.fail(e) AssertionError: Unable to reach the host @ 172.23.100.35 ---------------------------------------------------------------------- Ran 1 test in 15.552s FAILED (errors=1) summary so far suite bucket_collections.collections_rebalance.CollectionsRebalance , pass 2 , fail 7 failures so far... bucket_collections.collections_rebalance.CollectionsRebalance.test_data_load_collections_with_graceful_failover_recovery bucket_collections.collections_rebalance.CollectionsRebalance.test_data_load_collections_with_hard_failover_recovery bucket_collections.collections_rebalance.CollectionsRebalance.test_data_load_collections_with_graceful_failover_rebalance_out bucket_collections.collections_rebalance.CollectionsRebalance.test_data_load_collections_with_hard_failover_rebalance_out bucket_collections.collections_rebalance.CollectionsRebalance.test_data_load_collections_with_rebalance_in bucket_collections.collections_rebalance.CollectionsRebalance.test_data_load_collections_with_rebalance_out bucket_collections.collections_rebalance.CollectionsRebalance.test_data_load_collections_with_swap_rebalance testrunner logs, diags and results are available under /data/workspace/temp_rebalance_magma/logs/testrunner-22-Mar-06_03-39-55/test_9 Logs will be stored at /data/workspace/temp_rebalance_magma/logs/testrunner-22-Mar-06_03-39-55/test_10 guides/gradlew --refresh-dependencies testrunner -P jython=/opt/jython/bin/jython -P 'args=-i /tmp/win10-bucket-ops-temp_rebalance_magma.ini rerun=False,get-cbcollect-info=True -t bucket_collections.collections_rebalance.CollectionsRebalance.test_data_load_collections_with_rebalance_in_out,nodes_init=4,nodes_in=1,nodes_out=2,bucket_spec=magma_dgm.bloom_filter.4_node_1_replica_magma_1024,doc_size=30,randomize_value=True,data_load_stage=during,skip_validations=False,data_load_spec=volume_test_load_with_CRUD_on_collections,change_magma_quota=True,GROUP=rebalance' Test Input params: {'doc_size': '30', 'data_load_stage': 'during', 'cluster_name': 'win10-bucket-ops-temp_rebalance_magma', 'ini': '/tmp/win10-bucket-ops-temp_rebalance_magma.ini', 'get-cbcollect-info': 'True', 'conf_file': 'conf/magma/dgm_collections_bloom_filter.conf', 'change_magma_quota': 'True', 'nodes_in': '1', 'skip_validations': 'False', 'spec': 'dgm_collections_bloom_filter', 'rerun': 'False', 'num_nodes': 7, 'logs_folder': '/data/workspace/temp_rebalance_magma/logs/testrunner-22-Mar-06_03-39-55/test_10', 'nodes_init': '4', 'GROUP': 'rebalance', 'nodes_out': '2', 'bucket_spec': 'magma_dgm.bloom_filter.4_node_1_replica_magma_1024', 'case_number': 10, 'randomize_value': 'True', 'data_load_spec': 'volume_test_load_with_CRUD_on_collections'} test_data_load_collections_with_rebalance_in_out (bucket_collections.collections_rebalance.CollectionsRebalance) ... 2022-03-06 13:52:10,170 | test | INFO | MainThread | [basetestcase:log_setup_status:647] ========= CollectionsRebalance setup started for test #10 test_data_load_collections_with_rebalance_in_out ========= 2022-03-06 13:52:10,263 | test | INFO | MainThread | [basetestcase:log_setup_status:647] ========= BaseTestCase setup started for test #10 test_data_load_collections_with_rebalance_in_out ========= 2022-03-06 13:52:10,331 | infra | ERROR | MainThread | [Rest_Connection:_http_request:286] GET http://172.23.100.35:8091/nodes/self body: headers: {'Accept': '*/*', 'Connection': 'close', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==', 'Content-Type': 'application/json'} error: 404 reason: unknown "Node is unknown to this cluster." auth: Administrator:password http://172.23.100.35:8091/nodes/self with status 0: Node is unknown to this cluster. 2022-03-06 13:52:10,332 | infra | ERROR | MainThread | [Rest_Connection:__init__:148] Error Node is unknown to this cluster., 5 seconds sleep before retry 2022-03-06 13:52:15,348 | infra | ERROR | MainThread | [Rest_Connection:_http_request:286] GET http://172.23.100.35:8091/nodes/self body: headers: {'Accept': '*/*', 'Connection': 'close', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==', 'Content-Type': 'application/json'} error: 404 reason: unknown "Node is unknown to this cluster." auth: Administrator:password http://172.23.100.35:8091/nodes/self with status 0: Node is unknown to this cluster. 2022-03-06 13:52:15,351 | infra | ERROR | MainThread | [Rest_Connection:__init__:148] Error Node is unknown to this cluster., 5 seconds sleep before retry 2022-03-06 13:52:20,368 | infra | ERROR | MainThread | [Rest_Connection:_http_request:286] GET http://172.23.100.35:8091/nodes/self body: headers: {'Accept': '*/*', 'Connection': 'close', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==', 'Content-Type': 'application/json'} error: 404 reason: unknown "Node is unknown to this cluster." auth: Administrator:password http://172.23.100.35:8091/nodes/self with status 0: Node is unknown to this cluster. 2022-03-06 13:52:20,369 | infra | ERROR | MainThread | [Rest_Connection:__init__:148] Error Node is unknown to this cluster., 5 seconds sleep before retry Traceback (most recent call last): File "pytests/basetestcase.py", line 363, in setUp 2022-03-06 13:52:25,378 | infra | ERROR | MainThread | [Rest_Connection:__init__:152] Node 172.23.100.35:8091 is in a broken state! self.cluster_util.cluster_cleanup(cluster, File "pytests/basetestcase.py", line 363, in setUp self.cluster_util.cluster_cleanup(cluster, File "couchbase_utils/cluster_utils/cluster_ready_functions.py", line 168, in cluster_cleanup bucket_util.delete_all_buckets(cluster) File "couchbase_utils/bucket_utils/bucket_ready_functions.py", line 1675, in delete_all_buckets buckets = self.get_all_buckets(cluster, cluster_node=server) File "couchbase_utils/bucket_utils/bucket_ready_functions.py", line 3909, in get_all_buckets rest = BucketHelper(cluster_node) File "lib/BucketLib/BucketOperations_Rest.py", line 22, in __init__ super(BucketHelper, self).__init__(server) File "connections/Rest_Connection.py", line 154, in __init__ raise ServerUnavailableException(self.ip) ServerUnavailableException: Unable to reach the host @ 172.23.100.35 ERROR ====================================================================== ERROR: test_data_load_collections_with_rebalance_in_out (bucket_collections.collections_rebalance.CollectionsRebalance) ---------------------------------------------------------------------- Traceback (most recent call last): File "pytests/bucket_collections/collections_rebalance.py", line 22, in setUp super(CollectionsRebalance, self).setUp() File "pytests/bucket_collections/collections_base.py", line 19, in setUp super(CollectionBase, self).setUp() File "pytests/basetestcase.py", line 1068, in setUp super(ClusterSetup, self).setUp() File "pytests/basetestcase.py", line 463, in setUp self.fail(e) AssertionError: Unable to reach the host @ 172.23.100.35 ---------------------------------------------------------------------- Ran 1 test in 15.599s FAILED (errors=1) During the test, Remote Connections: 8, Disconnections: 8 SDK Connections: 0, Disconnections: 0 summary so far suite bucket_collections.collections_rebalance.CollectionsRebalance , pass 2 , fail 8 failures so far... bucket_collections.collections_rebalance.CollectionsRebalance.test_data_load_collections_with_graceful_failover_recovery bucket_collections.collections_rebalance.CollectionsRebalance.test_data_load_collections_with_hard_failover_recovery bucket_collections.collections_rebalance.CollectionsRebalance.test_data_load_collections_with_graceful_failover_rebalance_out bucket_collections.collections_rebalance.CollectionsRebalance.test_data_load_collections_with_hard_failover_rebalance_out bucket_collections.collections_rebalance.CollectionsRebalance.test_data_load_collections_with_rebalance_in bucket_collections.collections_rebalance.CollectionsRebalance.test_data_load_collections_with_rebalance_out bucket_collections.collections_rebalance.CollectionsRebalance.test_data_load_collections_with_swap_rebalance bucket_collections.collections_rebalance.CollectionsRebalance.test_data_load_collections_with_rebalance_in_out testrunner logs, diags and results are available under /data/workspace/temp_rebalance_magma/logs/testrunner-22-Mar-06_03-39-55/test_10 Logs will be stored at /data/workspace/temp_rebalance_magma/logs/testrunner-22-Mar-06_03-39-55/test_11 guides/gradlew --refresh-dependencies testrunner -P jython=/opt/jython/bin/jython -P 'args=-i /tmp/win10-bucket-ops-temp_rebalance_magma.ini rerun=False,get-cbcollect-info=True -t bucket_collections.collections_rebalance.CollectionsRebalance.test_data_load_collections_with_rebalance_in_out,nodes_init=4,nodes_in=2,nodes_out=1,bucket_spec=magma_dgm.bloom_filter.4_node_1_replica_magma_1024,doc_size=30,randomize_value=True,data_load_stage=during,skip_validations=False,data_load_spec=volume_test_load_with_CRUD_on_collections,change_magma_quota=True,GROUP=rebalance' Test Input params: {'doc_size': '30', 'data_load_stage': 'during', 'cluster_name': 'win10-bucket-ops-temp_rebalance_magma', 'ini': '/tmp/win10-bucket-ops-temp_rebalance_magma.ini', 'get-cbcollect-info': 'True', 'conf_file': 'conf/magma/dgm_collections_bloom_filter.conf', 'change_magma_quota': 'True', 'nodes_in': '2', 'skip_validations': 'False', 'spec': 'dgm_collections_bloom_filter', 'rerun': 'False', 'num_nodes': 7, 'logs_folder': '/data/workspace/temp_rebalance_magma/logs/testrunner-22-Mar-06_03-39-55/test_11', 'nodes_init': '4', 'GROUP': 'rebalance', 'nodes_out': '1', 'bucket_spec': 'magma_dgm.bloom_filter.4_node_1_replica_magma_1024', 'case_number': 11, 'randomize_value': 'True', 'data_load_spec': 'volume_test_load_with_CRUD_on_collections'} test_data_load_collections_with_rebalance_in_out (bucket_collections.collections_rebalance.CollectionsRebalance) ... 2022-03-06 13:52:25,737 | test | INFO | MainThread | [basetestcase:log_setup_status:647] ========= CollectionsRebalance setup started for test #11 test_data_load_collections_with_rebalance_in_out ========= 2022-03-06 13:52:25,834 | test | INFO | MainThread | [basetestcase:log_setup_status:647] ========= BaseTestCase setup started for test #11 test_data_load_collections_with_rebalance_in_out ========= 2022-03-06 13:52:25,903 | infra | ERROR | MainThread | [Rest_Connection:_http_request:286] GET http://172.23.100.35:8091/nodes/self body: headers: {'Accept': '*/*', 'Connection': 'close', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==', 'Content-Type': 'application/json'} error: 404 reason: unknown "Node is unknown to this cluster." auth: Administrator:password http://172.23.100.35:8091/nodes/self with status 0: Node is unknown to this cluster. 2022-03-06 13:52:25,904 | infra | ERROR | MainThread | [Rest_Connection:__init__:148] Error Node is unknown to this cluster., 5 seconds sleep before retry 2022-03-06 13:52:30,920 | infra | ERROR | MainThread | [Rest_Connection:_http_request:286] GET http://172.23.100.35:8091/nodes/self body: headers: {'Accept': '*/*', 'Connection': 'close', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==', 'Content-Type': 'application/json'} error: 404 reason: unknown "Node is unknown to this cluster." auth: Administrator:password http://172.23.100.35:8091/nodes/self with status 0: Node is unknown to this cluster. 2022-03-06 13:52:30,921 | infra | ERROR | MainThread | [Rest_Connection:__init__:148] Error Node is unknown to this cluster., 5 seconds sleep before retry 2022-03-06 13:52:35,940 | infra | ERROR | MainThread | [Rest_Connection:_http_request:286] GET http://172.23.100.35:8091/nodes/self body: headers: {'Accept': '*/*', 'Connection': 'close', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==', 'Content-Type': 'application/json'} error: 404 reason: unknown "Node is unknown to this cluster." auth: Administrator:password http://172.23.100.35:8091/nodes/self with status 0: Node is unknown to this cluster. 2022-03-06 13:52:35,940 | infra | ERROR | MainThread | [Rest_Connection:__init__:148] Error Node is unknown to this cluster., 5 seconds sleep before retry Traceback (most recent call last): File "pytests/basetestcase.py", line 363, in setUp 2022-03-06 13:52:40,948 | infra | ERROR | MainThread | [Rest_Connection:__init__:152] Node 172.23.100.35:8091 is in a broken state! self.cluster_util.cluster_cleanup(cluster, File "pytests/basetestcase.py", line 363, in setUp self.cluster_util.cluster_cleanup(cluster, File "couchbase_utils/cluster_utils/cluster_ready_functions.py", line 168, in cluster_cleanup bucket_util.delete_all_buckets(cluster) File "couchbase_utils/bucket_utils/bucket_ready_functions.py", line 1675, in delete_all_buckets buckets = self.get_all_buckets(cluster, cluster_node=server) File "couchbase_utils/bucket_utils/bucket_ready_functions.py", line 3909, in get_all_buckets rest = BucketHelper(cluster_node) File "lib/BucketLib/BucketOperations_Rest.py", line 22, in __init__ super(BucketHelper, self).__init__(server) File "connections/Rest_Connection.py", line 154, in __init__ raise ServerUnavailableException(self.ip) ServerUnavailableException: Unable to reach the host @ 172.23.100.35 ERROR ====================================================================== ERROR: test_data_load_collections_with_rebalance_in_out (bucket_collections.collections_rebalance.CollectionsRebalance) ---------------------------------------------------------------------- Traceback (most recent call last): File "pytests/bucket_collections/collections_rebalance.py", line 22, in setUp super(CollectionsRebalance, self).setUp() File "pytests/bucket_collections/collections_base.py", line 19, in setUp super(CollectionBase, self).setUp() File "pytests/basetestcase.py", line 1068, in setUp super(ClusterSetup, self).setUp() File "pytests/basetestcase.py", line 463, in setUp self.fail(e) AssertionError: Unable to reach the host @ 172.23.100.35 ---------------------------------------------------------------------- Ran 1 test in 15.554s FAILED (errors=1) During the test, Remote Connections: 8, Disconnections: 8 SDK Connections: 0, Disconnections: 0 summary so far suite bucket_collections.collections_rebalance.CollectionsRebalance , pass 2 , fail 9 failures so far... bucket_collections.collections_rebalance.CollectionsRebalance.test_data_load_collections_with_graceful_failover_recovery bucket_collections.collections_rebalance.CollectionsRebalance.test_data_load_collections_with_hard_failover_recovery bucket_collections.collections_rebalance.CollectionsRebalance.test_data_load_collections_with_graceful_failover_rebalance_out bucket_collections.collections_rebalance.CollectionsRebalance.test_data_load_collections_with_hard_failover_rebalance_out bucket_collections.collections_rebalance.CollectionsRebalance.test_data_load_collections_with_rebalance_in bucket_collections.collections_rebalance.CollectionsRebalance.test_data_load_collections_with_rebalance_out bucket_collections.collections_rebalance.CollectionsRebalance.test_data_load_collections_with_swap_rebalance bucket_collections.collections_rebalance.CollectionsRebalance.test_data_load_collections_with_rebalance_in_out bucket_collections.collections_rebalance.CollectionsRebalance.test_data_load_collections_with_rebalance_in_out testrunner logs, diags and results are available under /data/workspace/temp_rebalance_magma/logs/testrunner-22-Mar-06_03-39-55/test_11 Logs will be stored at /data/workspace/temp_rebalance_magma/logs/testrunner-22-Mar-06_03-39-55/test_12 guides/gradlew --refresh-dependencies testrunner -P jython=/opt/jython/bin/jython -P 'args=-i /tmp/win10-bucket-ops-temp_rebalance_magma.ini rerun=False,get-cbcollect-info=True -t bucket_collections.collections_rebalance.CollectionsRebalance.test_data_load_collections_with_rebalance_in,nodes_init=4,nodes_in=0,update_replica=True,updated_num_replicas=2,bucket_spec=magma_dgm.bloom_filter.4_node_1_replica_magma_1024,doc_size=30,randomize_value=True,data_load_stage=during,skip_validations=False,data_load_spec=volume_test_load_with_CRUD_on_collections,change_magma_quota=True,GROUP=rebalance' Test Input params: {'doc_size': '30', 'data_load_stage': 'during', 'cluster_name': 'win10-bucket-ops-temp_rebalance_magma', 'ini': '/tmp/win10-bucket-ops-temp_rebalance_magma.ini', 'get-cbcollect-info': 'True', 'conf_file': 'conf/magma/dgm_collections_bloom_filter.conf', 'change_magma_quota': 'True', 'nodes_in': '0', 'update_replica': 'True', 'skip_validations': 'False', 'spec': 'dgm_collections_bloom_filter', 'rerun': 'False', 'num_nodes': 7, 'logs_folder': '/data/workspace/temp_rebalance_magma/logs/testrunner-22-Mar-06_03-39-55/test_12', 'nodes_init': '4', 'updated_num_replicas': '2', 'GROUP': 'rebalance', 'bucket_spec': 'magma_dgm.bloom_filter.4_node_1_replica_magma_1024', 'case_number': 12, 'randomize_value': 'True', 'data_load_spec': 'volume_test_load_with_CRUD_on_collections'} test_data_load_collections_with_rebalance_in (bucket_collections.collections_rebalance.CollectionsRebalance) ... 2022-03-06 13:52:41,305 | test | INFO | MainThread | [basetestcase:log_setup_status:647] ========= CollectionsRebalance setup started for test #12 test_data_load_collections_with_rebalance_in ========= 2022-03-06 13:52:41,385 | test | INFO | MainThread | [basetestcase:log_setup_status:647] ========= BaseTestCase setup started for test #12 test_data_load_collections_with_rebalance_in ========= 2022-03-06 13:52:41,456 | infra | ERROR | MainThread | [Rest_Connection:_http_request:286] GET http://172.23.100.35:8091/nodes/self body: headers: {'Accept': '*/*', 'Connection': 'close', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==', 'Content-Type': 'application/json'} error: 404 reason: unknown "Node is unknown to this cluster." auth: Administrator:password http://172.23.100.35:8091/nodes/self with status 0: Node is unknown to this cluster. 2022-03-06 13:52:41,457 | infra | ERROR | MainThread | [Rest_Connection:__init__:148] Error Node is unknown to this cluster., 5 seconds sleep before retry 2022-03-06 13:52:46,473 | infra | ERROR | MainThread | [Rest_Connection:_http_request:286] GET http://172.23.100.35:8091/nodes/self body: headers: {'Accept': '*/*', 'Connection': 'close', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==', 'Content-Type': 'application/json'} error: 404 reason: unknown "Node is unknown to this cluster." auth: Administrator:password http://172.23.100.35:8091/nodes/self with status 0: Node is unknown to this cluster. 2022-03-06 13:52:46,476 | infra | ERROR | MainThread | [Rest_Connection:__init__:148] Error Node is unknown to this cluster., 5 seconds sleep before retry 2022-03-06 13:52:51,493 | infra | ERROR | MainThread | [Rest_Connection:_http_request:286] GET http://172.23.100.35:8091/nodes/self body: headers: {'Accept': '*/*', 'Connection': 'close', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==', 'Content-Type': 'application/json'} error: 404 reason: unknown "Node is unknown to this cluster." auth: Administrator:password http://172.23.100.35:8091/nodes/self with status 0: Node is unknown to this cluster. 2022-03-06 13:52:51,493 | infra | ERROR | MainThread | [Rest_Connection:__init__:148] Error Node is unknown to this cluster., 5 seconds sleep before retry 2022-03-06 13:52:56,502 | infra | ERROR | MainThread | [Rest_Connection:__init__:152] Node 172.23.100.35:8091 is in a broken state! Traceback (most recent call last): File "pytests/basetestcase.py", line 363, in setUp self.cluster_util.cluster_cleanup(cluster, File "pytests/basetestcase.py", line 363, in setUp self.cluster_util.cluster_cleanup(cluster, File "couchbase_utils/cluster_utils/cluster_ready_functions.py", line 168, in cluster_cleanup bucket_util.delete_all_buckets(cluster) File "couchbase_utils/bucket_utils/bucket_ready_functions.py", line 1675, in delete_all_buckets buckets = self.get_all_buckets(cluster, cluster_node=server) File "couchbase_utils/bucket_utils/bucket_ready_functions.py", line 3909, in get_all_buckets rest = BucketHelper(cluster_node) File "lib/BucketLib/BucketOperations_Rest.py", line 22, in __init__ super(BucketHelper, self).__init__(server) File "connections/Rest_Connection.py", line 154, in __init__ raise ServerUnavailableException(self.ip) ServerUnavailableException: Unable to reach the host @ 172.23.100.35 ERROR ====================================================================== ERROR: test_data_load_collections_with_rebalance_in (bucket_collections.collections_rebalance.CollectionsRebalance) ---------------------------------------------------------------------- Traceback (most recent call last): File "pytests/bucket_collections/collections_rebalance.py", line 22, in setUp super(CollectionsRebalance, self).setUp() File "pytests/bucket_collections/collections_base.py", line 19, in setUp super(CollectionBase, self).setUp() File "pytests/basetestcase.py", line 1068, in setUp super(ClusterSetup, self).setUp() File "pytests/basetestcase.py", line 463, in setUp self.fail(e) AssertionError: Unable to reach the host @ 172.23.100.35 ---------------------------------------------------------------------- Ran 1 test in 15.536s FAILED (errors=1) During the test, Remote Connections: 8, Disconnections: 8 SDK Connections: 0, Disconnections: 0 summary so far suite bucket_collections.collections_rebalance.CollectionsRebalance , pass 2 , fail 10 failures so far... bucket_collections.collections_rebalance.CollectionsRebalance.test_data_load_collections_with_graceful_failover_recovery bucket_collections.collections_rebalance.CollectionsRebalance.test_data_load_collections_with_hard_failover_recovery bucket_collections.collections_rebalance.CollectionsRebalance.test_data_load_collections_with_graceful_failover_rebalance_out bucket_collections.collections_rebalance.CollectionsRebalance.test_data_load_collections_with_hard_failover_rebalance_out bucket_collections.collections_rebalance.CollectionsRebalance.test_data_load_collections_with_rebalance_in bucket_collections.collections_rebalance.CollectionsRebalance.test_data_load_collections_with_rebalance_out bucket_collections.collections_rebalance.CollectionsRebalance.test_data_load_collections_with_swap_rebalance bucket_collections.collections_rebalance.CollectionsRebalance.test_data_load_collections_with_rebalance_in_out bucket_collections.collections_rebalance.CollectionsRebalance.test_data_load_collections_with_rebalance_in_out bucket_collections.collections_rebalance.CollectionsRebalance.test_data_load_collections_with_rebalance_in testrunner logs, diags and results are available under /data/workspace/temp_rebalance_magma/logs/testrunner-22-Mar-06_03-39-55/test_12 Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0. Use '--warning-mode all' to show the individual deprecation warnings. See https://docs.gradle.org/6.2.2/userguide/command_line_interface.html#sec:command_line_warnings BUILD SUCCESSFUL in 10h 13m 18s 2 actionable tasks: 2 executed [temp_rebalance_magma] $ /bin/sh -xe /tmp/jenkins3010793252134494934.sh Archiving artifacts Recording test results Build step 'Publish JUnit test result report' changed build result to UNSTABLE [description-setter] Description set: 1.1.8 -P client_version=3.1.6' Finished: UNSTABLE