[2023-07-22 01:57:14,174] - [eventing_base] [140279370630912] - INFO - ============== EventingBaseTest setup has started ============== [2023-07-22 01:57:14,175] - [eventing_base] [140279370630912] - INFO - Starting Test: test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously [2023-07-22 01:57:14,179] - [remote_util] [140279370630912] - INFO - SSH Connecting to 172.23.138.138 with username:Administrator, attempt#1 of 5 [2023-07-22 01:57:14,262] - [remote_util] [140279370630912] - INFO - SSH Connected to 172.23.138.138 as Administrator [2023-07-22 01:57:14,753] - [remote_util] [140279370630912] - INFO - File /cygdrive/c/Windows/win.ini was found [2023-07-22 01:57:14,753] - [remote_util] [140279370630912] - INFO - This is windows server! [2023-07-22 01:57:15,083] - [remote_util] [140279370630912] - INFO - File /cygdrive/c/tmp/windows_info.txt was found [2023-07-22 01:57:15,174] - [remote_util] [140279370630912] - INFO - get windows information [2023-07-22 01:57:15,259] - [remote_util] [140279370630912] - INFO - running command.raw on 172.23.138.138: echo "ipconfig" > /tmp/cmd.bat; chmod u=rwx /tmp/cmd.bat; /tmp/cmd.bat [2023-07-22 01:57:15,394] - [remote_util] [140279370630912] - INFO - command executed successfully with Administrator [2023-07-22 01:57:15,402] - [rest_client] [140279370630912] - INFO - -->is_ns_server_running? [2023-07-22 01:57:15,414] - [rest_client] [140279370630912] - INFO - Node version in cluster 7.2.1-5878-enterprise [2023-07-22 01:57:15,415] - [basetestcase] [140279370630912] - INFO - ============== basetestcase setup was started for test #2 test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously============== [2023-07-22 01:57:15,416] - [basetestcase] [140279370630912] - INFO - Building docker image with java sdk client [2023-07-22 01:57:26,561] - [basetestcase] [140279370630912] - INFO - initializing cluster [2023-07-22 01:57:26,568] - [remote_util] [140279370630912] - INFO - SSH Connecting to 172.23.138.138 with username:Administrator, attempt#1 of 5 [2023-07-22 01:57:26,663] - [remote_util] [140279370630912] - INFO - SSH Connected to 172.23.138.138 as Administrator [2023-07-22 01:57:27,044] - [remote_util] [140279370630912] - INFO - File /cygdrive/c/Windows/win.ini was found [2023-07-22 01:57:27,044] - [remote_util] [140279370630912] - INFO - This is windows server! [2023-07-22 01:57:27,154] - [remote_util] [140279370630912] - INFO - File /cygdrive/c/tmp/windows_info.txt was found [2023-07-22 01:57:27,356] - [remote_util] [140279370630912] - INFO - get windows information [2023-07-22 01:57:27,445] - [remote_util] [140279370630912] - INFO - running command.raw on 172.23.138.138: echo "ipconfig" > /tmp/cmd.bat; chmod u=rwx /tmp/cmd.bat; /tmp/cmd.bat [2023-07-22 01:57:27,581] - [remote_util] [140279370630912] - INFO - command executed successfully with Administrator [2023-07-22 01:57:27,596] - [remote_util] [140279370630912] - INFO - SSH Connecting to 172.23.138.138 with username:Administrator, attempt#1 of 5 [2023-07-22 01:57:27,701] - [remote_util] [140279370630912] - INFO - SSH Connected to 172.23.138.138 as Administrator [2023-07-22 01:57:28,129] - [remote_util] [140279370630912] - INFO - File /cygdrive/c/Windows/win.ini was found [2023-07-22 01:57:28,129] - [remote_util] [140279370630912] - INFO - This is windows server! [2023-07-22 01:57:28,239] - [remote_util] [140279370630912] - INFO - File /cygdrive/c/tmp/windows_info.txt was found [2023-07-22 01:57:28,441] - [remote_util] [140279370630912] - INFO - get windows information [2023-07-22 01:57:28,526] - [remote_util] [140279370630912] - INFO - running command.raw on 172.23.138.138: echo "ipconfig" > /tmp/cmd.bat; chmod u=rwx /tmp/cmd.bat; /tmp/cmd.bat [2023-07-22 01:57:28,663] - [remote_util] [140279370630912] - INFO - command executed successfully with Administrator [2023-07-22 01:57:28,739] - [remote_util] [140279370630912] - INFO - running command.raw on 172.23.138.138: cat '/cygdrive/c/Program Files/Couchbase/Server/VERSION.txt' [2023-07-22 01:57:28,802] - [remote_util] [140279370630912] - INFO - command executed successfully with Administrator [2023-07-22 01:57:28,803] - [remote_util] [140279370630912] - INFO - 172.23.138.138 **** The version file /cygdrive/c/Program\ Files/Couchbase/Server/ VERSION.txt exists [2023-07-22 01:57:28,804] - [remote_util] [140279370630912] - INFO - Checking for process memcached on windows [2023-07-22 01:57:29,013] - [remote_util] [140279370630912] - INFO - process is running on 172.23.138.138: ['memcached.exe', '4524', 'Services', '0', '161,620', 'K'] [2023-07-22 01:57:29,013] - [remote_util] [140279370630912] - INFO - Checking for process erl on windows [2023-07-22 01:57:29,225] - [remote_util] [140279370630912] - INFO - process is running on 172.23.138.138: ['erlsrv.exe', '3728', 'Services', '0', '3,724', 'K'] [2023-07-22 01:57:29,225] - [remote_util] [140279370630912] - INFO - running command.raw on 172.23.138.138: net stop couchbaseserver [2023-07-22 01:57:31,827] - [remote_util] [140279370630912] - INFO - command executed successfully with Administrator [2023-07-22 01:57:31,827] - [remote_util] [140279370630912] - INFO - The CouchbaseServer service is stopping. [2023-07-22 01:57:31,828] - [remote_util] [140279370630912] - INFO - The CouchbaseServer service was stopped successfully. [2023-07-22 01:57:31,828] - [remote_util] [140279370630912] - INFO - [2023-07-22 01:57:31,828] - [remote_util] [140279370630912] - INFO - 172.23.138.138:sleep for 10 secs. Wait to stop service completely ... [2023-07-22 01:57:41,839] - [remote_util] [140279370630912] - INFO - running command.raw on 172.23.138.138: sc query couchbaseserver [2023-07-22 01:57:41,904] - [remote_util] [140279370630912] - INFO - command executed successfully with Administrator [2023-07-22 01:57:41,905] - [remote_util] [140279370630912] - INFO - Checking for process erl.exe on windows [2023-07-22 01:57:42,102] - [basetestcase] [140279370630912] - INFO - Couchbase stopped [2023-07-22 01:57:42,103] - [remote_util] [140279370630912] - INFO - running command.raw on 172.23.138.138: rm -rf /cygdrive/c/Program\ Files/Couchbase/Server/var/lib/couchbase/data/* [2023-07-22 01:57:42,164] - [remote_util] [140279370630912] - INFO - command executed successfully with Administrator [2023-07-22 01:57:42,165] - [remote_util] [140279370630912] - INFO - running command.raw on 172.23.138.138: rm -rf /cygdrive/c/Program\ Files/Couchbase/Server/var/lib/couchbase/config/* [2023-07-22 01:57:42,245] - [remote_util] [140279370630912] - INFO - command executed successfully with Administrator [2023-07-22 01:57:42,249] - [remote_util] [140279370630912] - INFO - SSH Connecting to 172.23.138.138 with username:Administrator, attempt#1 of 5 [2023-07-22 01:57:42,337] - [remote_util] [140279370630912] - INFO - SSH Connected to 172.23.138.138 as Administrator [2023-07-22 01:57:42,716] - [remote_util] [140279370630912] - INFO - File /cygdrive/c/Windows/win.ini was found [2023-07-22 01:57:42,716] - [remote_util] [140279370630912] - INFO - This is windows server! [2023-07-22 01:57:42,826] - [remote_util] [140279370630912] - INFO - File /cygdrive/c/tmp/windows_info.txt was found [2023-07-22 01:57:42,916] - [remote_util] [140279370630912] - INFO - get windows information [2023-07-22 01:57:42,999] - [remote_util] [140279370630912] - INFO - running command.raw on 172.23.138.138: echo "ipconfig" > /tmp/cmd.bat; chmod u=rwx /tmp/cmd.bat; /tmp/cmd.bat [2023-07-22 01:57:43,133] - [remote_util] [140279370630912] - INFO - command executed successfully with Administrator [2023-07-22 01:57:43,213] - [remote_util] [140279370630912] - INFO - running command.raw on 172.23.138.138: cat '/cygdrive/c/Program Files/Couchbase/Server/VERSION.txt' [2023-07-22 01:57:43,277] - [remote_util] [140279370630912] - INFO - command executed successfully with Administrator [2023-07-22 01:57:43,279] - [remote_util] [140279370630912] - INFO - 172.23.138.138 **** The version file /cygdrive/c/Program\ Files/Couchbase/Server/ VERSION.txt exists [2023-07-22 01:57:43,279] - [remote_util] [140279370630912] - INFO - Checking for process memcached on windows [2023-07-22 01:57:43,462] - [remote_util] [140279370630912] - INFO - Checking for process erl on windows [2023-07-22 01:57:43,648] - [remote_util] [140279370630912] - INFO - Checking for process erl.exe on windows [2023-07-22 01:57:43,836] - [remote_util] [140279370630912] - INFO - Starting couchbase server [2023-07-22 01:57:43,836] - [remote_util] [140279370630912] - INFO - running command.raw on 172.23.138.138: net start couchbaseserver [2023-07-22 01:57:45,937] - [remote_util] [140279370630912] - INFO - command executed successfully with Administrator [2023-07-22 01:57:45,938] - [remote_util] [140279370630912] - INFO - The CouchbaseServer service is starting. [2023-07-22 01:57:45,938] - [remote_util] [140279370630912] - INFO - The CouchbaseServer service was started successfully. [2023-07-22 01:57:45,939] - [remote_util] [140279370630912] - INFO - [2023-07-22 01:57:45,939] - [remote_util] [140279370630912] - INFO - 172.23.138.138:sleep for 5 secs. Waiting for 5 secs to start...on 172.23.138.138 ... [2023-07-22 01:57:50,942] - [remote_util] [140279370630912] - INFO - Checking for process erl.exe on windows [2023-07-22 01:57:51,115] - [remote_util] [140279370630912] - INFO - process is running on 172.23.138.138: ['erl.exe', '4400', 'Services', '0', '49,964', 'K'] [2023-07-22 01:57:51,116] - [basetestcase] [140279370630912] - INFO - Couchbase started [2023-07-22 01:57:51,121] - [remote_util] [140279370630912] - INFO - SSH Connecting to 172.23.138.141 with username:Administrator, attempt#1 of 5 [2023-07-22 01:57:51,212] - [remote_util] [140279370630912] - INFO - SSH Connected to 172.23.138.141 as Administrator [2023-07-22 01:57:51,636] - [remote_util] [140279370630912] - INFO - File /cygdrive/c/Windows/win.ini was found [2023-07-22 01:57:51,636] - [remote_util] [140279370630912] - INFO - This is windows server! [2023-07-22 01:57:51,778] - [remote_util] [140279370630912] - INFO - File /cygdrive/c/tmp/windows_info.txt was found [2023-07-22 01:57:51,872] - [remote_util] [140279370630912] - INFO - get windows information [2023-07-22 01:57:51,973] - [remote_util] [140279370630912] - INFO - running command.raw on 172.23.138.141: echo "ipconfig" > /tmp/cmd.bat; chmod u=rwx /tmp/cmd.bat; /tmp/cmd.bat [2023-07-22 01:57:52,114] - [remote_util] [140279370630912] - INFO - command executed successfully with Administrator [2023-07-22 01:57:52,129] - [remote_util] [140279370630912] - INFO - SSH Connecting to 172.23.138.141 with username:Administrator, attempt#1 of 5 [2023-07-22 01:57:52,217] - [remote_util] [140279370630912] - INFO - SSH Connected to 172.23.138.141 as Administrator [2023-07-22 01:57:52,641] - [remote_util] [140279370630912] - INFO - File /cygdrive/c/Windows/win.ini was found [2023-07-22 01:57:52,641] - [remote_util] [140279370630912] - INFO - This is windows server! [2023-07-22 01:57:52,768] - [remote_util] [140279370630912] - INFO - File /cygdrive/c/tmp/windows_info.txt was found [2023-07-22 01:57:52,861] - [remote_util] [140279370630912] - INFO - get windows information [2023-07-22 01:57:52,976] - [remote_util] [140279370630912] - INFO - running command.raw on 172.23.138.141: echo "ipconfig" > /tmp/cmd.bat; chmod u=rwx /tmp/cmd.bat; /tmp/cmd.bat [2023-07-22 01:57:53,114] - [remote_util] [140279370630912] - INFO - command executed successfully with Administrator [2023-07-22 01:57:53,191] - [remote_util] [140279370630912] - INFO - running command.raw on 172.23.138.141: cat '/cygdrive/c/Program Files/Couchbase/Server/VERSION.txt' [2023-07-22 01:57:53,256] - [remote_util] [140279370630912] - INFO - command executed successfully with Administrator [2023-07-22 01:57:53,257] - [remote_util] [140279370630912] - INFO - 172.23.138.141 **** The version file /cygdrive/c/Program\ Files/Couchbase/Server/ VERSION.txt exists [2023-07-22 01:57:53,257] - [remote_util] [140279370630912] - INFO - Checking for process memcached on windows [2023-07-22 01:57:53,507] - [remote_util] [140279370630912] - INFO - process is running on 172.23.138.141: ['memcached.exe', '5136', 'Services', '0', '67,716', 'K'] [2023-07-22 01:57:53,507] - [remote_util] [140279370630912] - INFO - Checking for process erl on windows [2023-07-22 01:57:53,746] - [remote_util] [140279370630912] - INFO - process is running on 172.23.138.141: ['erlsrv.exe', '7012', 'Services', '0', '3,720', 'K'] [2023-07-22 01:57:53,747] - [remote_util] [140279370630912] - INFO - running command.raw on 172.23.138.141: net stop couchbaseserver [2023-07-22 01:57:56,358] - [remote_util] [140279370630912] - INFO - command executed successfully with Administrator [2023-07-22 01:57:56,359] - [remote_util] [140279370630912] - INFO - The CouchbaseServer service is stopping. [2023-07-22 01:57:56,359] - [remote_util] [140279370630912] - INFO - The CouchbaseServer service was stopped successfully. [2023-07-22 01:57:56,359] - [remote_util] [140279370630912] - INFO - [2023-07-22 01:57:56,360] - [remote_util] [140279370630912] - INFO - 172.23.138.141:sleep for 10 secs. Wait to stop service completely ... [2023-07-22 01:58:06,370] - [remote_util] [140279370630912] - INFO - running command.raw on 172.23.138.141: sc query couchbaseserver [2023-07-22 01:58:06,441] - [remote_util] [140279370630912] - INFO - command executed successfully with Administrator [2023-07-22 01:58:06,442] - [remote_util] [140279370630912] - INFO - Checking for process erl.exe on windows [2023-07-22 01:58:06,661] - [basetestcase] [140279370630912] - INFO - Couchbase stopped [2023-07-22 01:58:06,662] - [remote_util] [140279370630912] - INFO - running command.raw on 172.23.138.141: rm -rf /cygdrive/c/Program\ Files/Couchbase/Server/var/lib/couchbase/data/* [2023-07-22 01:58:06,726] - [remote_util] [140279370630912] - INFO - command executed successfully with Administrator [2023-07-22 01:58:06,726] - [remote_util] [140279370630912] - INFO - running command.raw on 172.23.138.141: rm -rf /cygdrive/c/Program\ Files/Couchbase/Server/var/lib/couchbase/config/* [2023-07-22 01:58:06,809] - [remote_util] [140279370630912] - INFO - command executed successfully with Administrator [2023-07-22 01:58:06,812] - [remote_util] [140279370630912] - INFO - SSH Connecting to 172.23.138.141 with username:Administrator, attempt#1 of 5 [2023-07-22 01:58:06,899] - [remote_util] [140279370630912] - INFO - SSH Connected to 172.23.138.141 as Administrator [2023-07-22 01:58:07,431] - [remote_util] [140279370630912] - INFO - File /cygdrive/c/Windows/win.ini was found [2023-07-22 01:58:07,431] - [remote_util] [140279370630912] - INFO - This is windows server! [2023-07-22 01:58:07,571] - [remote_util] [140279370630912] - INFO - File /cygdrive/c/tmp/windows_info.txt was found [2023-07-22 01:58:07,665] - [remote_util] [140279370630912] - INFO - get windows information [2023-07-22 01:58:07,753] - [remote_util] [140279370630912] - INFO - running command.raw on 172.23.138.141: echo "ipconfig" > /tmp/cmd.bat; chmod u=rwx /tmp/cmd.bat; /tmp/cmd.bat [2023-07-22 01:58:07,890] - [remote_util] [140279370630912] - INFO - command executed successfully with Administrator [2023-07-22 01:58:07,993] - [remote_util] [140279370630912] - INFO - running command.raw on 172.23.138.141: cat '/cygdrive/c/Program Files/Couchbase/Server/VERSION.txt' [2023-07-22 01:58:08,073] - [remote_util] [140279370630912] - INFO - command executed successfully with Administrator [2023-07-22 01:58:08,074] - [remote_util] [140279370630912] - INFO - 172.23.138.141 **** The version file /cygdrive/c/Program\ Files/Couchbase/Server/ VERSION.txt exists [2023-07-22 01:58:08,074] - [remote_util] [140279370630912] - INFO - Checking for process memcached on windows [2023-07-22 01:58:08,418] - [remote_util] [140279370630912] - INFO - Checking for process erl on windows [2023-07-22 01:58:08,630] - [remote_util] [140279370630912] - INFO - Checking for process erl.exe on windows [2023-07-22 01:58:08,848] - [remote_util] [140279370630912] - INFO - Starting couchbase server [2023-07-22 01:58:08,849] - [remote_util] [140279370630912] - INFO - running command.raw on 172.23.138.141: net start couchbaseserver [2023-07-22 01:58:10,968] - [remote_util] [140279370630912] - INFO - command executed successfully with Administrator [2023-07-22 01:58:10,968] - [remote_util] [140279370630912] - INFO - The CouchbaseServer service is starting. [2023-07-22 01:58:10,969] - [remote_util] [140279370630912] - INFO - The CouchbaseServer service was started successfully. [2023-07-22 01:58:10,969] - [remote_util] [140279370630912] - INFO - [2023-07-22 01:58:10,970] - [remote_util] [140279370630912] - INFO - 172.23.138.141:sleep for 5 secs. Waiting for 5 secs to start...on 172.23.138.141 ... [2023-07-22 01:58:15,975] - [remote_util] [140279370630912] - INFO - Checking for process erl.exe on windows [2023-07-22 01:58:16,189] - [remote_util] [140279370630912] - INFO - process is running on 172.23.138.141: ['erl.exe', '2952', 'Services', '0', '49,644', 'K'] [2023-07-22 01:58:16,189] - [basetestcase] [140279370630912] - INFO - Couchbase started [2023-07-22 01:58:16,192] - [remote_util] [140279370630912] - INFO - SSH Connecting to 172.23.138.137 with username:Administrator, attempt#1 of 5 [2023-07-22 01:58:16,294] - [remote_util] [140279370630912] - INFO - SSH Connected to 172.23.138.137 as Administrator [2023-07-22 01:58:16,844] - [remote_util] [140279370630912] - INFO - File /cygdrive/c/Windows/win.ini was found [2023-07-22 01:58:16,844] - [remote_util] [140279370630912] - INFO - This is windows server! [2023-07-22 01:58:16,985] - [remote_util] [140279370630912] - INFO - File /cygdrive/c/tmp/windows_info.txt was found [2023-07-22 01:58:17,079] - [remote_util] [140279370630912] - INFO - get windows information [2023-07-22 01:58:17,198] - [remote_util] [140279370630912] - INFO - running command.raw on 172.23.138.137: echo "ipconfig" > /tmp/cmd.bat; chmod u=rwx /tmp/cmd.bat; /tmp/cmd.bat [2023-07-22 01:58:17,341] - [remote_util] [140279370630912] - INFO - command executed successfully with Administrator [2023-07-22 01:58:17,356] - [remote_util] [140279370630912] - INFO - SSH Connecting to 172.23.138.137 with username:Administrator, attempt#1 of 5 [2023-07-22 01:58:17,552] - [remote_util] [140279370630912] - INFO - SSH Connected to 172.23.138.137 as Administrator [2023-07-22 01:58:17,964] - [remote_util] [140279370630912] - INFO - File /cygdrive/c/Windows/win.ini was found [2023-07-22 01:58:17,964] - [remote_util] [140279370630912] - INFO - This is windows server! [2023-07-22 01:58:18,074] - [remote_util] [140279370630912] - INFO - File /cygdrive/c/tmp/windows_info.txt was found [2023-07-22 01:58:18,165] - [remote_util] [140279370630912] - INFO - get windows information [2023-07-22 01:58:18,289] - [remote_util] [140279370630912] - INFO - running command.raw on 172.23.138.137: echo "ipconfig" > /tmp/cmd.bat; chmod u=rwx /tmp/cmd.bat; /tmp/cmd.bat [2023-07-22 01:58:18,438] - [remote_util] [140279370630912] - INFO - command executed successfully with Administrator [2023-07-22 01:58:18,542] - [remote_util] [140279370630912] - INFO - running command.raw on 172.23.138.137: cat '/cygdrive/c/Program Files/Couchbase/Server/VERSION.txt' [2023-07-22 01:58:18,834] - [remote_util] [140279370630912] - INFO - command executed successfully with Administrator [2023-07-22 01:58:18,837] - [remote_util] [140279370630912] - INFO - 172.23.138.137 **** The version file /cygdrive/c/Program\ Files/Couchbase/Server/ VERSION.txt exists [2023-07-22 01:58:18,837] - [remote_util] [140279370630912] - INFO - Checking for process memcached on windows [2023-07-22 01:58:19,090] - [remote_util] [140279370630912] - INFO - process is running on 172.23.138.137: ['memcached.exe', '6464', 'Services', '0', '67,696', 'K'] [2023-07-22 01:58:19,091] - [remote_util] [140279370630912] - INFO - Checking for process erl on windows [2023-07-22 01:58:19,341] - [remote_util] [140279370630912] - INFO - process is running on 172.23.138.137: ['erlsrv.exe', '4516', 'Services', '0', '3,708', 'K'] [2023-07-22 01:58:19,341] - [remote_util] [140279370630912] - INFO - running command.raw on 172.23.138.137: net stop couchbaseserver [2023-07-22 01:58:21,944] - [remote_util] [140279370630912] - INFO - command executed successfully with Administrator [2023-07-22 01:58:21,945] - [remote_util] [140279370630912] - INFO - The CouchbaseServer service is stopping. [2023-07-22 01:58:21,945] - [remote_util] [140279370630912] - INFO - The CouchbaseServer service was stopped successfully. [2023-07-22 01:58:21,946] - [remote_util] [140279370630912] - INFO - [2023-07-22 01:58:21,946] - [remote_util] [140279370630912] - INFO - 172.23.138.137:sleep for 10 secs. Wait to stop service completely ... [2023-07-22 01:58:31,956] - [remote_util] [140279370630912] - INFO - running command.raw on 172.23.138.137: sc query couchbaseserver [2023-07-22 01:58:32,027] - [remote_util] [140279370630912] - INFO - command executed successfully with Administrator [2023-07-22 01:58:32,028] - [remote_util] [140279370630912] - INFO - Checking for process erl.exe on windows [2023-07-22 01:58:32,244] - [basetestcase] [140279370630912] - INFO - Couchbase stopped [2023-07-22 01:58:32,245] - [remote_util] [140279370630912] - INFO - running command.raw on 172.23.138.137: rm -rf /cygdrive/c/Program\ Files/Couchbase/Server/var/lib/couchbase/data/* [2023-07-22 01:58:32,309] - [remote_util] [140279370630912] - INFO - command executed successfully with Administrator [2023-07-22 01:58:32,309] - [remote_util] [140279370630912] - INFO - running command.raw on 172.23.138.137: rm -rf /cygdrive/c/Program\ Files/Couchbase/Server/var/lib/couchbase/config/* [2023-07-22 01:58:32,394] - [remote_util] [140279370630912] - INFO - command executed successfully with Administrator [2023-07-22 01:58:32,398] - [remote_util] [140279370630912] - INFO - SSH Connecting to 172.23.138.137 with username:Administrator, attempt#1 of 5 [2023-07-22 01:58:32,485] - [remote_util] [140279370630912] - INFO - SSH Connected to 172.23.138.137 as Administrator [2023-07-22 01:58:33,079] - [remote_util] [140279370630912] - INFO - File /cygdrive/c/Windows/win.ini was found [2023-07-22 01:58:33,079] - [remote_util] [140279370630912] - INFO - This is windows server! [2023-07-22 01:58:33,219] - [remote_util] [140279370630912] - INFO - File /cygdrive/c/tmp/windows_info.txt was found [2023-07-22 01:58:33,313] - [remote_util] [140279370630912] - INFO - get windows information [2023-07-22 01:58:33,412] - [remote_util] [140279370630912] - INFO - running command.raw on 172.23.138.137: echo "ipconfig" > /tmp/cmd.bat; chmod u=rwx /tmp/cmd.bat; /tmp/cmd.bat [2023-07-22 01:58:33,550] - [remote_util] [140279370630912] - INFO - command executed successfully with Administrator [2023-07-22 01:58:33,641] - [remote_util] [140279370630912] - INFO - running command.raw on 172.23.138.137: cat '/cygdrive/c/Program Files/Couchbase/Server/VERSION.txt' [2023-07-22 01:58:33,706] - [remote_util] [140279370630912] - INFO - command executed successfully with Administrator [2023-07-22 01:58:33,707] - [remote_util] [140279370630912] - INFO - 172.23.138.137 **** The version file /cygdrive/c/Program\ Files/Couchbase/Server/ VERSION.txt exists [2023-07-22 01:58:33,708] - [remote_util] [140279370630912] - INFO - Checking for process memcached on windows [2023-07-22 01:58:33,911] - [remote_util] [140279370630912] - INFO - Checking for process erl on windows [2023-07-22 01:58:34,122] - [remote_util] [140279370630912] - INFO - Checking for process erl.exe on windows [2023-07-22 01:58:34,320] - [remote_util] [140279370630912] - INFO - Starting couchbase server [2023-07-22 01:58:34,320] - [remote_util] [140279370630912] - INFO - running command.raw on 172.23.138.137: net start couchbaseserver [2023-07-22 01:58:36,444] - [remote_util] [140279370630912] - INFO - command executed successfully with Administrator [2023-07-22 01:58:36,445] - [remote_util] [140279370630912] - INFO - The CouchbaseServer service is starting. [2023-07-22 01:58:36,445] - [remote_util] [140279370630912] - INFO - The CouchbaseServer service was started successfully. [2023-07-22 01:58:36,445] - [remote_util] [140279370630912] - INFO - [2023-07-22 01:58:36,446] - [remote_util] [140279370630912] - INFO - 172.23.138.137:sleep for 5 secs. Waiting for 5 secs to start...on 172.23.138.137 ... [2023-07-22 01:58:41,451] - [remote_util] [140279370630912] - INFO - Checking for process erl.exe on windows [2023-07-22 01:58:41,666] - [remote_util] [140279370630912] - INFO - process is running on 172.23.138.137: ['erl.exe', '2520', 'Services', '0', '50,772', 'K'] [2023-07-22 01:58:41,666] - [basetestcase] [140279370630912] - INFO - Couchbase started [2023-07-22 01:58:41,670] - [remote_util] [140279370630912] - INFO - SSH Connecting to 172.23.138.142 with username:Administrator, attempt#1 of 5 [2023-07-22 01:58:41,767] - [remote_util] [140279370630912] - INFO - SSH Connected to 172.23.138.142 as Administrator [2023-07-22 01:58:42,222] - [remote_util] [140279370630912] - INFO - File /cygdrive/c/Windows/win.ini was found [2023-07-22 01:58:42,223] - [remote_util] [140279370630912] - INFO - This is windows server! [2023-07-22 01:58:42,348] - [remote_util] [140279370630912] - INFO - File /cygdrive/c/tmp/windows_info.txt was found [2023-07-22 01:58:42,443] - [remote_util] [140279370630912] - INFO - get windows information [2023-07-22 01:58:42,563] - [remote_util] [140279370630912] - INFO - running command.raw on 172.23.138.142: echo "ipconfig" > /tmp/cmd.bat; chmod u=rwx /tmp/cmd.bat; /tmp/cmd.bat [2023-07-22 01:58:42,709] - [remote_util] [140279370630912] - INFO - command executed successfully with Administrator [2023-07-22 01:58:42,723] - [remote_util] [140279370630912] - INFO - SSH Connecting to 172.23.138.142 with username:Administrator, attempt#1 of 5 [2023-07-22 01:58:42,819] - [remote_util] [140279370630912] - INFO - SSH Connected to 172.23.138.142 as Administrator [2023-07-22 01:58:43,275] - [remote_util] [140279370630912] - INFO - File /cygdrive/c/Windows/win.ini was found [2023-07-22 01:58:43,275] - [remote_util] [140279370630912] - INFO - This is windows server! [2023-07-22 01:58:43,416] - [remote_util] [140279370630912] - INFO - File /cygdrive/c/tmp/windows_info.txt was found [2023-07-22 01:58:43,510] - [remote_util] [140279370630912] - INFO - get windows information [2023-07-22 01:58:43,633] - [remote_util] [140279370630912] - INFO - running command.raw on 172.23.138.142: echo "ipconfig" > /tmp/cmd.bat; chmod u=rwx /tmp/cmd.bat; /tmp/cmd.bat [2023-07-22 01:58:43,774] - [remote_util] [140279370630912] - INFO - command executed successfully with Administrator [2023-07-22 01:58:43,856] - [remote_util] [140279370630912] - INFO - running command.raw on 172.23.138.142: cat '/cygdrive/c/Program Files/Couchbase/Server/VERSION.txt' [2023-07-22 01:58:43,928] - [remote_util] [140279370630912] - INFO - command executed successfully with Administrator [2023-07-22 01:58:43,930] - [remote_util] [140279370630912] - INFO - 172.23.138.142 **** The version file /cygdrive/c/Program\ Files/Couchbase/Server/ VERSION.txt exists [2023-07-22 01:58:43,930] - [remote_util] [140279370630912] - INFO - Checking for process memcached on windows [2023-07-22 01:58:44,161] - [remote_util] [140279370630912] - INFO - process is running on 172.23.138.142: ['memcached.exe', '2060', 'Services', '0', '63,872', 'K'] [2023-07-22 01:58:44,161] - [remote_util] [140279370630912] - INFO - Checking for process erl on windows [2023-07-22 01:58:44,399] - [remote_util] [140279370630912] - INFO - process is running on 172.23.138.142: ['erlsrv.exe', '5944', 'Services', '0', '3,692', 'K'] [2023-07-22 01:58:44,400] - [remote_util] [140279370630912] - INFO - running command.raw on 172.23.138.142: net stop couchbaseserver [2023-07-22 01:58:49,540] - [remote_util] [140279370630912] - INFO - command executed successfully with Administrator [2023-07-22 01:58:49,541] - [remote_util] [140279370630912] - INFO - The CouchbaseServer service is stopping.. [2023-07-22 01:58:49,541] - [remote_util] [140279370630912] - INFO - The CouchbaseServer service was stopped successfully. [2023-07-22 01:58:49,542] - [remote_util] [140279370630912] - INFO - [2023-07-22 01:58:49,542] - [remote_util] [140279370630912] - INFO - 172.23.138.142:sleep for 10 secs. Wait to stop service completely ... [2023-07-22 01:58:59,549] - [remote_util] [140279370630912] - INFO - running command.raw on 172.23.138.142: sc query couchbaseserver [2023-07-22 01:58:59,628] - [remote_util] [140279370630912] - INFO - command executed successfully with Administrator [2023-07-22 01:58:59,628] - [remote_util] [140279370630912] - INFO - Checking for process erl.exe on windows [2023-07-22 01:58:59,857] - [basetestcase] [140279370630912] - INFO - Couchbase stopped [2023-07-22 01:58:59,858] - [remote_util] [140279370630912] - INFO - running command.raw on 172.23.138.142: rm -rf /cygdrive/c/Program\ Files/Couchbase/Server/var/lib/couchbase/data/* [2023-07-22 01:58:59,935] - [remote_util] [140279370630912] - INFO - command executed successfully with Administrator [2023-07-22 01:58:59,935] - [remote_util] [140279370630912] - INFO - running command.raw on 172.23.138.142: rm -rf /cygdrive/c/Program\ Files/Couchbase/Server/var/lib/couchbase/config/* [2023-07-22 01:59:00,034] - [remote_util] [140279370630912] - INFO - command executed successfully with Administrator [2023-07-22 01:59:00,038] - [remote_util] [140279370630912] - INFO - SSH Connecting to 172.23.138.142 with username:Administrator, attempt#1 of 5 [2023-07-22 01:59:00,128] - [remote_util] [140279370630912] - INFO - SSH Connected to 172.23.138.142 as Administrator [2023-07-22 01:59:00,581] - [remote_util] [140279370630912] - INFO - File /cygdrive/c/Windows/win.ini was found [2023-07-22 01:59:00,582] - [remote_util] [140279370630912] - INFO - This is windows server! [2023-07-22 01:59:00,737] - [remote_util] [140279370630912] - INFO - File /cygdrive/c/tmp/windows_info.txt was found [2023-07-22 01:59:00,831] - [remote_util] [140279370630912] - INFO - get windows information [2023-07-22 01:59:00,955] - [remote_util] [140279370630912] - INFO - running command.raw on 172.23.138.142: echo "ipconfig" > /tmp/cmd.bat; chmod u=rwx /tmp/cmd.bat; /tmp/cmd.bat [2023-07-22 01:59:01,097] - [remote_util] [140279370630912] - INFO - command executed successfully with Administrator [2023-07-22 01:59:01,190] - [remote_util] [140279370630912] - INFO - running command.raw on 172.23.138.142: cat '/cygdrive/c/Program Files/Couchbase/Server/VERSION.txt' [2023-07-22 01:59:01,263] - [remote_util] [140279370630912] - INFO - command executed successfully with Administrator [2023-07-22 01:59:01,265] - [remote_util] [140279370630912] - INFO - 172.23.138.142 **** The version file /cygdrive/c/Program\ Files/Couchbase/Server/ VERSION.txt exists [2023-07-22 01:59:01,266] - [remote_util] [140279370630912] - INFO - Checking for process memcached on windows [2023-07-22 01:59:01,481] - [remote_util] [140279370630912] - INFO - Checking for process erl on windows [2023-07-22 01:59:01,704] - [remote_util] [140279370630912] - INFO - Checking for process erl.exe on windows [2023-07-22 01:59:01,906] - [remote_util] [140279370630912] - INFO - Starting couchbase server [2023-07-22 01:59:01,907] - [remote_util] [140279370630912] - INFO - running command.raw on 172.23.138.142: net start couchbaseserver [2023-07-22 01:59:04,024] - [remote_util] [140279370630912] - INFO - command executed successfully with Administrator [2023-07-22 01:59:04,025] - [remote_util] [140279370630912] - INFO - The CouchbaseServer service is starting. [2023-07-22 01:59:04,025] - [remote_util] [140279370630912] - INFO - The CouchbaseServer service was started successfully. [2023-07-22 01:59:04,026] - [remote_util] [140279370630912] - INFO - [2023-07-22 01:59:04,026] - [remote_util] [140279370630912] - INFO - 172.23.138.142:sleep for 5 secs. Waiting for 5 secs to start...on 172.23.138.142 ... [2023-07-22 01:59:09,032] - [remote_util] [140279370630912] - INFO - Checking for process erl.exe on windows [2023-07-22 01:59:09,238] - [remote_util] [140279370630912] - INFO - process is running on 172.23.138.142: ['erl.exe', '3700', 'Services', '0', '51,220', 'K'] [2023-07-22 01:59:09,238] - [basetestcase] [140279370630912] - INFO - Couchbase started [2023-07-22 01:59:09,242] - [remote_util] [140279370630912] - INFO - SSH Connecting to 172.23.138.143 with username:Administrator, attempt#1 of 5 [2023-07-22 01:59:09,342] - [remote_util] [140279370630912] - INFO - SSH Connected to 172.23.138.143 as Administrator [2023-07-22 01:59:09,815] - [remote_util] [140279370630912] - INFO - File /cygdrive/c/Windows/win.ini was found [2023-07-22 01:59:09,815] - [remote_util] [140279370630912] - INFO - This is windows server! [2023-07-22 01:59:09,957] - [remote_util] [140279370630912] - INFO - File /cygdrive/c/tmp/windows_info.txt was found [2023-07-22 01:59:10,051] - [remote_util] [140279370630912] - INFO - get windows information [2023-07-22 01:59:10,166] - [remote_util] [140279370630912] - INFO - running command.raw on 172.23.138.143: echo "ipconfig" > /tmp/cmd.bat; chmod u=rwx /tmp/cmd.bat; /tmp/cmd.bat [2023-07-22 01:59:10,329] - [remote_util] [140279370630912] - INFO - command executed successfully with Administrator [2023-07-22 01:59:10,342] - [remote_util] [140279370630912] - INFO - SSH Connecting to 172.23.138.143 with username:Administrator, attempt#1 of 5 [2023-07-22 01:59:10,442] - [remote_util] [140279370630912] - INFO - SSH Connected to 172.23.138.143 as Administrator [2023-07-22 01:59:10,898] - [remote_util] [140279370630912] - INFO - File /cygdrive/c/Windows/win.ini was found [2023-07-22 01:59:10,899] - [remote_util] [140279370630912] - INFO - This is windows server! [2023-07-22 01:59:11,040] - [remote_util] [140279370630912] - INFO - File /cygdrive/c/tmp/windows_info.txt was found [2023-07-22 01:59:11,150] - [remote_util] [140279370630912] - INFO - get windows information [2023-07-22 01:59:11,274] - [remote_util] [140279370630912] - INFO - running command.raw on 172.23.138.143: echo "ipconfig" > /tmp/cmd.bat; chmod u=rwx /tmp/cmd.bat; /tmp/cmd.bat [2023-07-22 01:59:11,519] - [remote_util] [140279370630912] - INFO - command executed successfully with Administrator [2023-07-22 01:59:11,621] - [remote_util] [140279370630912] - INFO - running command.raw on 172.23.138.143: cat '/cygdrive/c/Program Files/Couchbase/Server/VERSION.txt' [2023-07-22 01:59:11,693] - [remote_util] [140279370630912] - INFO - command executed successfully with Administrator [2023-07-22 01:59:11,696] - [remote_util] [140279370630912] - INFO - 172.23.138.143 **** The version file /cygdrive/c/Program\ Files/Couchbase/Server/ VERSION.txt exists [2023-07-22 01:59:11,697] - [remote_util] [140279370630912] - INFO - Checking for process memcached on windows [2023-07-22 01:59:11,938] - [remote_util] [140279370630912] - INFO - process is running on 172.23.138.143: ['memcached.exe', '2888', 'Services', '0', '63,756', 'K'] [2023-07-22 01:59:11,939] - [remote_util] [140279370630912] - INFO - Checking for process erl on windows [2023-07-22 01:59:12,186] - [remote_util] [140279370630912] - INFO - process is running on 172.23.138.143: ['erlsrv.exe', '3716', 'Services', '0', '3,732', 'K'] [2023-07-22 01:59:12,187] - [remote_util] [140279370630912] - INFO - running command.raw on 172.23.138.143: net stop couchbaseserver [2023-07-22 01:59:14,802] - [remote_util] [140279370630912] - INFO - command executed successfully with Administrator [2023-07-22 01:59:14,802] - [remote_util] [140279370630912] - INFO - The CouchbaseServer service is stopping. [2023-07-22 01:59:14,803] - [remote_util] [140279370630912] - INFO - The CouchbaseServer service was stopped successfully. [2023-07-22 01:59:14,803] - [remote_util] [140279370630912] - INFO - [2023-07-22 01:59:14,804] - [remote_util] [140279370630912] - INFO - 172.23.138.143:sleep for 10 secs. Wait to stop service completely ... [2023-07-22 01:59:24,814] - [remote_util] [140279370630912] - INFO - running command.raw on 172.23.138.143: sc query couchbaseserver [2023-07-22 01:59:24,893] - [remote_util] [140279370630912] - INFO - command executed successfully with Administrator [2023-07-22 01:59:24,894] - [remote_util] [140279370630912] - INFO - Checking for process erl.exe on windows [2023-07-22 01:59:25,116] - [basetestcase] [140279370630912] - INFO - Couchbase stopped [2023-07-22 01:59:25,117] - [remote_util] [140279370630912] - INFO - running command.raw on 172.23.138.143: rm -rf /cygdrive/c/Program\ Files/Couchbase/Server/var/lib/couchbase/data/* [2023-07-22 01:59:25,189] - [remote_util] [140279370630912] - INFO - command executed successfully with Administrator [2023-07-22 01:59:25,190] - [remote_util] [140279370630912] - INFO - running command.raw on 172.23.138.143: rm -rf /cygdrive/c/Program\ Files/Couchbase/Server/var/lib/couchbase/config/* [2023-07-22 01:59:25,277] - [remote_util] [140279370630912] - INFO - command executed successfully with Administrator [2023-07-22 01:59:25,281] - [remote_util] [140279370630912] - INFO - SSH Connecting to 172.23.138.143 with username:Administrator, attempt#1 of 5 [2023-07-22 01:59:25,374] - [remote_util] [140279370630912] - INFO - SSH Connected to 172.23.138.143 as Administrator [2023-07-22 01:59:25,828] - [remote_util] [140279370630912] - INFO - File /cygdrive/c/Windows/win.ini was found [2023-07-22 01:59:25,828] - [remote_util] [140279370630912] - INFO - This is windows server! [2023-07-22 01:59:25,968] - [remote_util] [140279370630912] - INFO - File /cygdrive/c/tmp/windows_info.txt was found [2023-07-22 01:59:26,062] - [remote_util] [140279370630912] - INFO - get windows information [2023-07-22 01:59:26,185] - [remote_util] [140279370630912] - INFO - running command.raw on 172.23.138.143: echo "ipconfig" > /tmp/cmd.bat; chmod u=rwx /tmp/cmd.bat; /tmp/cmd.bat [2023-07-22 01:59:26,329] - [remote_util] [140279370630912] - INFO - command executed successfully with Administrator [2023-07-22 01:59:26,422] - [remote_util] [140279370630912] - INFO - running command.raw on 172.23.138.143: cat '/cygdrive/c/Program Files/Couchbase/Server/VERSION.txt' [2023-07-22 01:59:26,501] - [remote_util] [140279370630912] - INFO - command executed successfully with Administrator [2023-07-22 01:59:26,502] - [remote_util] [140279370630912] - INFO - 172.23.138.143 **** The version file /cygdrive/c/Program\ Files/Couchbase/Server/ VERSION.txt exists [2023-07-22 01:59:26,503] - [remote_util] [140279370630912] - INFO - Checking for process memcached on windows [2023-07-22 01:59:26,707] - [remote_util] [140279370630912] - INFO - Checking for process erl on windows [2023-07-22 01:59:26,924] - [remote_util] [140279370630912] - INFO - Checking for process erl.exe on windows [2023-07-22 01:59:27,128] - [remote_util] [140279370630912] - INFO - Starting couchbase server [2023-07-22 01:59:27,129] - [remote_util] [140279370630912] - INFO - running command.raw on 172.23.138.143: net start couchbaseserver [2023-07-22 01:59:29,255] - [remote_util] [140279370630912] - INFO - command executed successfully with Administrator [2023-07-22 01:59:29,256] - [remote_util] [140279370630912] - INFO - The CouchbaseServer service is starting. [2023-07-22 01:59:29,256] - [remote_util] [140279370630912] - INFO - The CouchbaseServer service was started successfully. [2023-07-22 01:59:29,256] - [remote_util] [140279370630912] - INFO - [2023-07-22 01:59:29,257] - [remote_util] [140279370630912] - INFO - 172.23.138.143:sleep for 5 secs. Waiting for 5 secs to start...on 172.23.138.143 ... [2023-07-22 01:59:34,262] - [remote_util] [140279370630912] - INFO - Checking for process erl.exe on windows [2023-07-22 01:59:34,470] - [remote_util] [140279370630912] - INFO - process is running on 172.23.138.143: ['erl.exe', '3672', 'Services', '0', '49,368', 'K'] [2023-07-22 01:59:34,470] - [basetestcase] [140279370630912] - INFO - Couchbase started [2023-07-22 01:59:34,474] - [remote_util] [140279370630912] - INFO - SSH Connecting to 172.23.138.139 with username:Administrator, attempt#1 of 5 [2023-07-22 01:59:34,688] - [remote_util] [140279370630912] - INFO - SSH Connected to 172.23.138.139 as Administrator [2023-07-22 01:59:35,144] - [remote_util] [140279370630912] - INFO - File /cygdrive/c/Windows/win.ini was found [2023-07-22 01:59:35,145] - [remote_util] [140279370630912] - INFO - This is windows server! [2023-07-22 01:59:35,285] - [remote_util] [140279370630912] - INFO - File /cygdrive/c/tmp/windows_info.txt was found [2023-07-22 01:59:35,379] - [remote_util] [140279370630912] - INFO - get windows information [2023-07-22 01:59:35,496] - [remote_util] [140279370630912] - INFO - running command.raw on 172.23.138.139: echo "ipconfig" > /tmp/cmd.bat; chmod u=rwx /tmp/cmd.bat; /tmp/cmd.bat [2023-07-22 01:59:35,655] - [remote_util] [140279370630912] - INFO - command executed successfully with Administrator [2023-07-22 01:59:35,671] - [remote_util] [140279370630912] - INFO - SSH Connecting to 172.23.138.139 with username:Administrator, attempt#1 of 5 [2023-07-22 01:59:35,787] - [remote_util] [140279370630912] - INFO - SSH Connected to 172.23.138.139 as Administrator [2023-07-22 01:59:36,259] - [remote_util] [140279370630912] - INFO - File /cygdrive/c/Windows/win.ini was found [2023-07-22 01:59:36,260] - [remote_util] [140279370630912] - INFO - This is windows server! [2023-07-22 01:59:36,400] - [remote_util] [140279370630912] - INFO - File /cygdrive/c/tmp/windows_info.txt was found [2023-07-22 01:59:36,494] - [remote_util] [140279370630912] - INFO - get windows information [2023-07-22 01:59:36,619] - [remote_util] [140279370630912] - INFO - running command.raw on 172.23.138.139: echo "ipconfig" > /tmp/cmd.bat; chmod u=rwx /tmp/cmd.bat; /tmp/cmd.bat [2023-07-22 01:59:36,757] - [remote_util] [140279370630912] - INFO - command executed successfully with Administrator [2023-07-22 01:59:36,855] - [remote_util] [140279370630912] - INFO - running command.raw on 172.23.138.139: cat '/cygdrive/c/Program Files/Couchbase/Server/VERSION.txt' [2023-07-22 01:59:36,926] - [remote_util] [140279370630912] - INFO - command executed successfully with Administrator [2023-07-22 01:59:36,928] - [remote_util] [140279370630912] - INFO - 172.23.138.139 **** The version file /cygdrive/c/Program\ Files/Couchbase/Server/ VERSION.txt exists [2023-07-22 01:59:36,929] - [remote_util] [140279370630912] - INFO - Checking for process memcached on windows [2023-07-22 01:59:37,179] - [remote_util] [140279370630912] - INFO - process is running on 172.23.138.139: ['memcached.exe', '4116', 'Services', '0', '62,652', 'K'] [2023-07-22 01:59:37,179] - [remote_util] [140279370630912] - INFO - Checking for process erl on windows [2023-07-22 01:59:37,437] - [remote_util] [140279370630912] - INFO - process is running on 172.23.138.139: ['erlsrv.exe', '1688', 'Services', '0', '3,700', 'K'] [2023-07-22 01:59:37,437] - [remote_util] [140279370630912] - INFO - running command.raw on 172.23.138.139: net stop couchbaseserver [2023-07-22 01:59:40,054] - [remote_util] [140279370630912] - INFO - command executed successfully with Administrator [2023-07-22 01:59:40,055] - [remote_util] [140279370630912] - INFO - The CouchbaseServer service is stopping. [2023-07-22 01:59:40,055] - [remote_util] [140279370630912] - INFO - The CouchbaseServer service was stopped successfully. [2023-07-22 01:59:40,055] - [remote_util] [140279370630912] - INFO - [2023-07-22 01:59:40,056] - [remote_util] [140279370630912] - INFO - 172.23.138.139:sleep for 10 secs. Wait to stop service completely ... [2023-07-22 01:59:50,057] - [remote_util] [140279370630912] - INFO - running command.raw on 172.23.138.139: sc query couchbaseserver [2023-07-22 01:59:50,135] - [remote_util] [140279370630912] - INFO - command executed successfully with Administrator [2023-07-22 01:59:50,136] - [remote_util] [140279370630912] - INFO - Checking for process erl.exe on windows [2023-07-22 01:59:50,339] - [basetestcase] [140279370630912] - INFO - Couchbase stopped [2023-07-22 01:59:50,340] - [remote_util] [140279370630912] - INFO - running command.raw on 172.23.138.139: rm -rf /cygdrive/c/Program\ Files/Couchbase/Server/var/lib/couchbase/data/* [2023-07-22 01:59:50,414] - [remote_util] [140279370630912] - INFO - command executed successfully with Administrator [2023-07-22 01:59:50,414] - [remote_util] [140279370630912] - INFO - running command.raw on 172.23.138.139: rm -rf /cygdrive/c/Program\ Files/Couchbase/Server/var/lib/couchbase/config/* [2023-07-22 01:59:50,514] - [remote_util] [140279370630912] - INFO - command executed successfully with Administrator [2023-07-22 01:59:50,520] - [remote_util] [140279370630912] - INFO - SSH Connecting to 172.23.138.139 with username:Administrator, attempt#1 of 5 [2023-07-22 01:59:50,735] - [remote_util] [140279370630912] - INFO - SSH Connected to 172.23.138.139 as Administrator [2023-07-22 01:59:51,175] - [remote_util] [140279370630912] - INFO - File /cygdrive/c/Windows/win.ini was found [2023-07-22 01:59:51,175] - [remote_util] [140279370630912] - INFO - This is windows server! [2023-07-22 01:59:51,286] - [remote_util] [140279370630912] - INFO - File /cygdrive/c/tmp/windows_info.txt was found [2023-07-22 01:59:51,376] - [remote_util] [140279370630912] - INFO - get windows information [2023-07-22 01:59:51,472] - [remote_util] [140279370630912] - INFO - running command.raw on 172.23.138.139: echo "ipconfig" > /tmp/cmd.bat; chmod u=rwx /tmp/cmd.bat; /tmp/cmd.bat [2023-07-22 01:59:51,609] - [remote_util] [140279370630912] - INFO - command executed successfully with Administrator [2023-07-22 01:59:51,704] - [remote_util] [140279370630912] - INFO - running command.raw on 172.23.138.139: cat '/cygdrive/c/Program Files/Couchbase/Server/VERSION.txt' [2023-07-22 01:59:51,776] - [remote_util] [140279370630912] - INFO - command executed successfully with Administrator [2023-07-22 01:59:51,777] - [remote_util] [140279370630912] - INFO - 172.23.138.139 **** The version file /cygdrive/c/Program\ Files/Couchbase/Server/ VERSION.txt exists [2023-07-22 01:59:51,778] - [remote_util] [140279370630912] - INFO - Checking for process memcached on windows [2023-07-22 01:59:51,978] - [remote_util] [140279370630912] - INFO - Checking for process erl on windows [2023-07-22 01:59:52,307] - [remote_util] [140279370630912] - INFO - Checking for process erl.exe on windows [2023-07-22 01:59:52,527] - [remote_util] [140279370630912] - INFO - Starting couchbase server [2023-07-22 01:59:52,528] - [remote_util] [140279370630912] - INFO - running command.raw on 172.23.138.139: net start couchbaseserver [2023-07-22 01:59:54,663] - [remote_util] [140279370630912] - INFO - command executed successfully with Administrator [2023-07-22 01:59:54,664] - [remote_util] [140279370630912] - INFO - The CouchbaseServer service is starting. [2023-07-22 01:59:54,664] - [remote_util] [140279370630912] - INFO - The CouchbaseServer service was started successfully. [2023-07-22 01:59:54,665] - [remote_util] [140279370630912] - INFO - [2023-07-22 01:59:54,665] - [remote_util] [140279370630912] - INFO - 172.23.138.139:sleep for 5 secs. Waiting for 5 secs to start...on 172.23.138.139 ... [2023-07-22 01:59:59,671] - [remote_util] [140279370630912] - INFO - Checking for process erl.exe on windows [2023-07-22 01:59:59,873] - [remote_util] [140279370630912] - INFO - process is running on 172.23.138.139: ['erl.exe', '1412', 'Services', '0', '52,472', 'K'] [2023-07-22 01:59:59,873] - [basetestcase] [140279370630912] - INFO - Couchbase started [2023-07-22 01:59:59,877] - [remote_util] [140279370630912] - INFO - SSH Connecting to 172.23.138.140 with username:Administrator, attempt#1 of 5 [2023-07-22 01:59:59,994] - [remote_util] [140279370630912] - INFO - SSH Connected to 172.23.138.140 as Administrator [2023-07-22 02:00:00,482] - [remote_util] [140279370630912] - INFO - File /cygdrive/c/Windows/win.ini was found [2023-07-22 02:00:00,482] - [remote_util] [140279370630912] - INFO - This is windows server! [2023-07-22 02:00:00,638] - [remote_util] [140279370630912] - INFO - File /cygdrive/c/tmp/windows_info.txt was found [2023-07-22 02:00:00,732] - [remote_util] [140279370630912] - INFO - get windows information [2023-07-22 02:00:00,862] - [remote_util] [140279370630912] - INFO - running command.raw on 172.23.138.140: echo "ipconfig" > /tmp/cmd.bat; chmod u=rwx /tmp/cmd.bat; /tmp/cmd.bat [2023-07-22 02:00:01,027] - [remote_util] [140279370630912] - INFO - command executed successfully with Administrator [2023-07-22 02:00:01,042] - [remote_util] [140279370630912] - INFO - SSH Connecting to 172.23.138.140 with username:Administrator, attempt#1 of 5 [2023-07-22 02:00:01,250] - [remote_util] [140279370630912] - INFO - SSH Connected to 172.23.138.140 as Administrator [2023-07-22 02:00:01,720] - [remote_util] [140279370630912] - INFO - File /cygdrive/c/Windows/win.ini was found [2023-07-22 02:00:01,720] - [remote_util] [140279370630912] - INFO - This is windows server! [2023-07-22 02:00:01,877] - [remote_util] [140279370630912] - INFO - File /cygdrive/c/tmp/windows_info.txt was found [2023-07-22 02:00:01,972] - [remote_util] [140279370630912] - INFO - get windows information [2023-07-22 02:00:02,303] - [remote_util] [140279370630912] - INFO - running command.raw on 172.23.138.140: echo "ipconfig" > /tmp/cmd.bat; chmod u=rwx /tmp/cmd.bat; /tmp/cmd.bat [2023-07-22 02:00:02,444] - [remote_util] [140279370630912] - INFO - command executed successfully with Administrator [2023-07-22 02:00:02,537] - [remote_util] [140279370630912] - INFO - running command.raw on 172.23.138.140: cat '/cygdrive/c/Program Files/Couchbase/Server/VERSION.txt' [2023-07-22 02:00:02,609] - [remote_util] [140279370630912] - INFO - command executed successfully with Administrator [2023-07-22 02:00:02,611] - [remote_util] [140279370630912] - INFO - 172.23.138.140 **** The version file /cygdrive/c/Program\ Files/Couchbase/Server/ VERSION.txt exists [2023-07-22 02:00:02,611] - [remote_util] [140279370630912] - INFO - Checking for process memcached on windows [2023-07-22 02:00:02,845] - [remote_util] [140279370630912] - INFO - process is running on 172.23.138.140: ['memcached.exe', '6620', 'Services', '0', '62,860', 'K'] [2023-07-22 02:00:02,845] - [remote_util] [140279370630912] - INFO - Checking for process erl on windows [2023-07-22 02:00:03,097] - [remote_util] [140279370630912] - INFO - process is running on 172.23.138.140: ['erlsrv.exe', '1040', 'Services', '0', '3,708', 'K'] [2023-07-22 02:00:03,098] - [remote_util] [140279370630912] - INFO - running command.raw on 172.23.138.140: net stop couchbaseserver [2023-07-22 02:00:08,234] - [remote_util] [140279370630912] - INFO - command executed successfully with Administrator [2023-07-22 02:00:08,235] - [remote_util] [140279370630912] - INFO - The CouchbaseServer service is stopping.. [2023-07-22 02:00:08,236] - [remote_util] [140279370630912] - INFO - The CouchbaseServer service was stopped successfully. [2023-07-22 02:00:08,236] - [remote_util] [140279370630912] - INFO - [2023-07-22 02:00:08,237] - [remote_util] [140279370630912] - INFO - 172.23.138.140:sleep for 10 secs. Wait to stop service completely ... [2023-07-22 02:00:18,245] - [remote_util] [140279370630912] - INFO - running command.raw on 172.23.138.140: sc query couchbaseserver [2023-07-22 02:00:18,326] - [remote_util] [140279370630912] - INFO - command executed successfully with Administrator [2023-07-22 02:00:18,327] - [remote_util] [140279370630912] - INFO - Checking for process erl.exe on windows [2023-07-22 02:00:18,535] - [basetestcase] [140279370630912] - INFO - Couchbase stopped [2023-07-22 02:00:18,537] - [remote_util] [140279370630912] - INFO - running command.raw on 172.23.138.140: rm -rf /cygdrive/c/Program\ Files/Couchbase/Server/var/lib/couchbase/data/* [2023-07-22 02:00:18,816] - [remote_util] [140279370630912] - INFO - command executed successfully with Administrator [2023-07-22 02:00:18,817] - [remote_util] [140279370630912] - INFO - running command.raw on 172.23.138.140: rm -rf /cygdrive/c/Program\ Files/Couchbase/Server/var/lib/couchbase/config/* [2023-07-22 02:00:18,912] - [remote_util] [140279370630912] - INFO - command executed successfully with Administrator [2023-07-22 02:00:18,916] - [remote_util] [140279370630912] - INFO - SSH Connecting to 172.23.138.140 with username:Administrator, attempt#1 of 5 [2023-07-22 02:00:19,010] - [remote_util] [140279370630912] - INFO - SSH Connected to 172.23.138.140 as Administrator [2023-07-22 02:00:19,436] - [remote_util] [140279370630912] - INFO - File /cygdrive/c/Windows/win.ini was found [2023-07-22 02:00:19,437] - [remote_util] [140279370630912] - INFO - This is windows server! [2023-07-22 02:00:19,573] - [remote_util] [140279370630912] - INFO - File /cygdrive/c/tmp/windows_info.txt was found [2023-07-22 02:00:19,666] - [remote_util] [140279370630912] - INFO - get windows information [2023-07-22 02:00:19,787] - [remote_util] [140279370630912] - INFO - running command.raw on 172.23.138.140: echo "ipconfig" > /tmp/cmd.bat; chmod u=rwx /tmp/cmd.bat; /tmp/cmd.bat [2023-07-22 02:00:19,930] - [remote_util] [140279370630912] - INFO - command executed successfully with Administrator [2023-07-22 02:00:20,026] - [remote_util] [140279370630912] - INFO - running command.raw on 172.23.138.140: cat '/cygdrive/c/Program Files/Couchbase/Server/VERSION.txt' [2023-07-22 02:00:20,096] - [remote_util] [140279370630912] - INFO - command executed successfully with Administrator [2023-07-22 02:00:20,099] - [remote_util] [140279370630912] - INFO - 172.23.138.140 **** The version file /cygdrive/c/Program\ Files/Couchbase/Server/ VERSION.txt exists [2023-07-22 02:00:20,100] - [remote_util] [140279370630912] - INFO - Checking for process memcached on windows [2023-07-22 02:00:20,305] - [remote_util] [140279370630912] - INFO - Checking for process erl on windows [2023-07-22 02:00:20,523] - [remote_util] [140279370630912] - INFO - Checking for process erl.exe on windows [2023-07-22 02:00:20,724] - [remote_util] [140279370630912] - INFO - Starting couchbase server [2023-07-22 02:00:20,724] - [remote_util] [140279370630912] - INFO - running command.raw on 172.23.138.140: net start couchbaseserver [2023-07-22 02:00:22,844] - [remote_util] [140279370630912] - INFO - command executed successfully with Administrator [2023-07-22 02:00:22,845] - [remote_util] [140279370630912] - INFO - The CouchbaseServer service is starting. [2023-07-22 02:00:22,845] - [remote_util] [140279370630912] - INFO - The CouchbaseServer service was started successfully. [2023-07-22 02:00:22,845] - [remote_util] [140279370630912] - INFO - [2023-07-22 02:00:22,846] - [remote_util] [140279370630912] - INFO - 172.23.138.140:sleep for 5 secs. Waiting for 5 secs to start...on 172.23.138.140 ... [2023-07-22 02:00:27,848] - [remote_util] [140279370630912] - INFO - Checking for process erl.exe on windows [2023-07-22 02:00:28,063] - [remote_util] [140279370630912] - INFO - process is running on 172.23.138.140: ['erl.exe', '1580', 'Services', '0', '48,352', 'K'] [2023-07-22 02:00:28,063] - [basetestcase] [140279370630912] - INFO - Couchbase started [2023-07-22 02:00:28,172] - [rest_client] [140279370630912] - ERROR - socket error while connecting to http://172.23.138.140:8091/nodes/self error [Errno 111] Connection refused [2023-07-22 02:00:31,177] - [rest_client] [140279370630912] - ERROR - socket error while connecting to http://172.23.138.140:8091/nodes/self error [Errno 111] Connection refused [2023-07-22 02:00:37,406] - [task] [140278505654016] - INFO - server: ip:172.23.138.138 port:8091 ssh_username:Administrator, nodes/self [2023-07-22 02:00:37,411] - [task] [140278505654016] - INFO - {'uptime': '171', 'memoryTotal': 6433652736, 'memoryFree': 4714139648, 'mcdMemoryReserved': 4908, 'mcdMemoryAllocated': 4908, 'status': 'healthy', 'hostname': '172.23.138.138:8091', 'clusterCompatibility': 458754, 'clusterMembership': 'active', 'recoveryType': 'none', 'version': '7.2.1-5878-enterprise', 'os': 'win64', 'ports': [], 'availableStorage': [], 'storage': [], 'memoryQuota': 3015, 'memcached': 11210, 'id': 'ns_1@172.23.138.138', 'ip': '172.23.138.138', 'internal_ip': '', 'rest_username': '', 'rest_password': '', 'port': '8091', 'services': ['kv'], 'storageTotalRam': 6135, 'curr_items': 0} [2023-07-22 02:00:37,414] - [rest_client] [140278505654016] - ERROR - GET http://172.23.138.138:8091/pools/default body: headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==', 'Accept': '*/*'} error: 404 reason: unknown b'"unknown pool"' auth: Administrator:password [2023-07-22 02:00:37,417] - [rest_client] [140278505654016] - INFO - pools/default params : memoryQuota=3288 [2023-07-22 02:00:37,426] - [rest_client] [140278505654016] - INFO - --> init_node_services(Administrator,password,172.23.138.138,8091,['kv']) [2023-07-22 02:00:37,427] - [rest_client] [140278505654016] - INFO - /node/controller/setupServices params on 172.23.138.138: 8091:hostname=172.23.138.138&user=Administrator&password=password&services=kv [2023-07-22 02:00:38,101] - [rest_client] [140278505654016] - INFO - --> in init_cluster...Administrator,password,8091 [2023-07-22 02:00:38,102] - [rest_client] [140278505654016] - INFO - settings/web params on 172.23.138.138:8091:port=8091&username=Administrator&password=password [2023-07-22 02:00:38,193] - [rest_client] [140278505654016] - INFO - --> status:True [2023-07-22 02:00:38,197] - [remote_util] [140278505654016] - INFO - SSH Connecting to 172.23.138.138 with username:Administrator, attempt#1 of 5 [2023-07-22 02:00:38,302] - [remote_util] [140278505654016] - INFO - SSH Connected to 172.23.138.138 as Administrator [2023-07-22 02:00:38,728] - [remote_util] [140278505654016] - INFO - File /cygdrive/c/Windows/win.ini was found [2023-07-22 02:00:38,728] - [remote_util] [140278505654016] - INFO - This is windows server! [2023-07-22 02:00:38,838] - [remote_util] [140278505654016] - INFO - File /cygdrive/c/tmp/windows_info.txt was found [2023-07-22 02:00:39,039] - [remote_util] [140278505654016] - INFO - get windows information [2023-07-22 02:00:39,129] - [remote_util] [140278505654016] - INFO - running command.raw on 172.23.138.138: echo "ipconfig" > /tmp/cmd.bat; chmod u=rwx /tmp/cmd.bat; /tmp/cmd.bat [2023-07-22 02:00:39,265] - [remote_util] [140278505654016] - INFO - command executed successfully with Administrator [2023-07-22 02:00:39,267] - [remote_util] [140278505654016] - INFO - running command.raw on 172.23.138.138: curl --silent --show-error http://Administrator:password@localhost:8091/diag/eval -X POST -d 'ns_config:set(allow_nonlocal_eval, true).' [2023-07-22 02:00:39,360] - [remote_util] [140278505654016] - INFO - command executed successfully with Administrator [2023-07-22 02:00:39,360] - [remote_util] [140278505654016] - INFO - ['ok'] [2023-07-22 02:00:39,364] - [rest_client] [140278505654016] - INFO - /diag/eval status on 172.23.138.138:8091: True content: [7,2] command: cluster_compat_mode:get_compat_version(). [2023-07-22 02:00:39,368] - [rest_client] [140278505654016] - INFO - /diag/eval status on 172.23.138.138:8091: True content: [7,2] command: cluster_compat_mode:get_compat_version(). [2023-07-22 02:00:39,373] - [rest_client] [140278505654016] - INFO - settings/indexes params : storageMode=plasma [2023-07-22 02:00:39,396] - [task] [140278505654016] - INFO - server: ip:172.23.138.141 port:8091 ssh_username:Administrator, nodes/self [2023-07-22 02:00:39,401] - [task] [140278505654016] - INFO - {'uptime': '148', 'memoryTotal': 6433652736, 'memoryFree': 4778459136, 'mcdMemoryReserved': 4908, 'mcdMemoryAllocated': 4908, 'status': 'healthy', 'hostname': '172.23.138.141:8091', 'clusterCompatibility': 458754, 'clusterMembership': 'active', 'recoveryType': 'none', 'version': '7.2.1-5878-enterprise', 'os': 'win64', 'ports': [], 'availableStorage': [], 'storage': [], 'memoryQuota': 3015, 'memcached': 11210, 'id': 'ns_1@172.23.138.141', 'ip': '172.23.138.141', 'internal_ip': '', 'rest_username': '', 'rest_password': '', 'port': '8091', 'services': ['kv'], 'storageTotalRam': 6135, 'curr_items': 0} [2023-07-22 02:00:39,404] - [rest_client] [140278505654016] - ERROR - GET http://172.23.138.141:8091/pools/default body: headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==', 'Accept': '*/*'} error: 404 reason: unknown b'"unknown pool"' auth: Administrator:password [2023-07-22 02:00:39,405] - [rest_client] [140278505654016] - INFO - pools/default params : memoryQuota=3288 [2023-07-22 02:00:39,414] - [rest_client] [140278505654016] - INFO - --> in init_cluster...Administrator,password,8091 [2023-07-22 02:00:39,415] - [rest_client] [140278505654016] - INFO - settings/web params on 172.23.138.141:8091:port=8091&username=Administrator&password=password [2023-07-22 02:00:39,527] - [rest_client] [140278505654016] - INFO - --> status:True [2023-07-22 02:00:39,530] - [remote_util] [140278505654016] - INFO - SSH Connecting to 172.23.138.141 with username:Administrator, attempt#1 of 5 [2023-07-22 02:00:39,634] - [remote_util] [140278505654016] - INFO - SSH Connected to 172.23.138.141 as Administrator [2023-07-22 02:00:40,152] - [remote_util] [140278505654016] - INFO - File /cygdrive/c/Windows/win.ini was found [2023-07-22 02:00:40,153] - [remote_util] [140278505654016] - INFO - This is windows server! [2023-07-22 02:00:40,290] - [remote_util] [140278505654016] - INFO - File /cygdrive/c/tmp/windows_info.txt was found [2023-07-22 02:00:40,384] - [remote_util] [140278505654016] - INFO - get windows information [2023-07-22 02:00:40,502] - [remote_util] [140278505654016] - INFO - running command.raw on 172.23.138.141: echo "ipconfig" > /tmp/cmd.bat; chmod u=rwx /tmp/cmd.bat; /tmp/cmd.bat [2023-07-22 02:00:40,643] - [remote_util] [140278505654016] - INFO - command executed successfully with Administrator [2023-07-22 02:00:40,645] - [remote_util] [140278505654016] - INFO - running command.raw on 172.23.138.141: curl --silent --show-error http://Administrator:password@localhost:8091/diag/eval -X POST -d 'ns_config:set(allow_nonlocal_eval, true).' [2023-07-22 02:00:40,843] - [remote_util] [140278505654016] - INFO - command executed successfully with Administrator [2023-07-22 02:00:40,843] - [remote_util] [140278505654016] - INFO - ['ok'] [2023-07-22 02:00:40,848] - [rest_client] [140278505654016] - INFO - /diag/eval status on 172.23.138.141:8091: True content: [7,2] command: cluster_compat_mode:get_compat_version(). [2023-07-22 02:00:40,851] - [rest_client] [140278505654016] - INFO - /diag/eval status on 172.23.138.141:8091: True content: [7,2] command: cluster_compat_mode:get_compat_version(). [2023-07-22 02:00:40,856] - [rest_client] [140278505654016] - INFO - settings/indexes params : storageMode=plasma [2023-07-22 02:00:40,878] - [task] [140278505654016] - INFO - server: ip:172.23.138.137 port:8091 ssh_username:Administrator, nodes/self [2023-07-22 02:00:40,882] - [task] [140278505654016] - INFO - {'uptime': '119', 'memoryTotal': 6433652736, 'memoryFree': 4781764608, 'mcdMemoryReserved': 4908, 'mcdMemoryAllocated': 4908, 'status': 'healthy', 'hostname': '172.23.138.137:8091', 'clusterCompatibility': 458754, 'clusterMembership': 'active', 'recoveryType': 'none', 'version': '7.2.1-5878-enterprise', 'os': 'win64', 'ports': [], 'availableStorage': [], 'storage': [], 'memoryQuota': 3015, 'memcached': 11210, 'id': 'ns_1@172.23.138.137', 'ip': '172.23.138.137', 'internal_ip': '', 'rest_username': '', 'rest_password': '', 'port': '8091', 'services': ['kv'], 'storageTotalRam': 6135, 'curr_items': 0} [2023-07-22 02:00:40,885] - [rest_client] [140278505654016] - ERROR - GET http://172.23.138.137:8091/pools/default body: headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==', 'Accept': '*/*'} error: 404 reason: unknown b'"unknown pool"' auth: Administrator:password [2023-07-22 02:00:40,886] - [rest_client] [140278505654016] - INFO - pools/default params : memoryQuota=3288 [2023-07-22 02:00:40,896] - [rest_client] [140278505654016] - INFO - --> in init_cluster...Administrator,password,8091 [2023-07-22 02:00:40,896] - [rest_client] [140278505654016] - INFO - settings/web params on 172.23.138.137:8091:port=8091&username=Administrator&password=password [2023-07-22 02:00:40,998] - [rest_client] [140278505654016] - INFO - --> status:True [2023-07-22 02:00:41,001] - [remote_util] [140278505654016] - INFO - SSH Connecting to 172.23.138.137 with username:Administrator, attempt#1 of 5 [2023-07-22 02:00:41,106] - [remote_util] [140278505654016] - INFO - SSH Connected to 172.23.138.137 as Administrator [2023-07-22 02:00:41,562] - [remote_util] [140278505654016] - INFO - File /cygdrive/c/Windows/win.ini was found [2023-07-22 02:00:41,562] - [remote_util] [140278505654016] - INFO - This is windows server! [2023-07-22 02:00:41,703] - [remote_util] [140278505654016] - INFO - File /cygdrive/c/tmp/windows_info.txt was found [2023-07-22 02:00:41,797] - [remote_util] [140278505654016] - INFO - get windows information [2023-07-22 02:00:41,913] - [remote_util] [140278505654016] - INFO - running command.raw on 172.23.138.137: echo "ipconfig" > /tmp/cmd.bat; chmod u=rwx /tmp/cmd.bat; /tmp/cmd.bat [2023-07-22 02:00:42,056] - [remote_util] [140278505654016] - INFO - command executed successfully with Administrator [2023-07-22 02:00:42,059] - [remote_util] [140278505654016] - INFO - running command.raw on 172.23.138.137: curl --silent --show-error http://Administrator:password@localhost:8091/diag/eval -X POST -d 'ns_config:set(allow_nonlocal_eval, true).' [2023-07-22 02:00:42,149] - [remote_util] [140278505654016] - INFO - command executed successfully with Administrator [2023-07-22 02:00:42,150] - [remote_util] [140278505654016] - INFO - ['ok'] [2023-07-22 02:00:42,154] - [rest_client] [140278505654016] - INFO - /diag/eval status on 172.23.138.137:8091: True content: [7,2] command: cluster_compat_mode:get_compat_version(). [2023-07-22 02:00:42,157] - [rest_client] [140278505654016] - INFO - /diag/eval status on 172.23.138.137:8091: True content: [7,2] command: cluster_compat_mode:get_compat_version(). [2023-07-22 02:00:42,162] - [rest_client] [140278505654016] - INFO - settings/indexes params : storageMode=plasma [2023-07-22 02:00:42,183] - [task] [140278505654016] - INFO - server: ip:172.23.138.142 port:8091 ssh_username:Administrator, nodes/self [2023-07-22 02:00:42,187] - [task] [140278505654016] - INFO - {'uptime': '94', 'memoryTotal': 6433652736, 'memoryFree': 4767391744, 'mcdMemoryReserved': 4908, 'mcdMemoryAllocated': 4908, 'status': 'healthy', 'hostname': '172.23.138.142:8091', 'clusterCompatibility': 458754, 'clusterMembership': 'active', 'recoveryType': 'none', 'version': '7.2.1-5878-enterprise', 'os': 'win64', 'ports': [], 'availableStorage': [], 'storage': [], 'memoryQuota': 3015, 'memcached': 11210, 'id': 'ns_1@172.23.138.142', 'ip': '172.23.138.142', 'internal_ip': '', 'rest_username': '', 'rest_password': '', 'port': '8091', 'services': ['kv'], 'storageTotalRam': 6135, 'curr_items': 0} [2023-07-22 02:00:42,191] - [rest_client] [140278505654016] - ERROR - GET http://172.23.138.142:8091/pools/default body: headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==', 'Accept': '*/*'} error: 404 reason: unknown b'"unknown pool"' auth: Administrator:password [2023-07-22 02:00:42,191] - [rest_client] [140278505654016] - INFO - pools/default params : memoryQuota=3288 [2023-07-22 02:00:42,201] - [rest_client] [140278505654016] - INFO - --> in init_cluster...Administrator,password,8091 [2023-07-22 02:00:42,201] - [rest_client] [140278505654016] - INFO - settings/web params on 172.23.138.142:8091:port=8091&username=Administrator&password=password [2023-07-22 02:00:42,285] - [rest_client] [140278505654016] - INFO - --> status:True [2023-07-22 02:00:42,288] - [remote_util] [140278505654016] - INFO - SSH Connecting to 172.23.138.142 with username:Administrator, attempt#1 of 5 [2023-07-22 02:00:42,394] - [remote_util] [140278505654016] - INFO - SSH Connected to 172.23.138.142 as Administrator [2023-07-22 02:00:43,273] - [remote_util] [140278505654016] - INFO - File /cygdrive/c/Windows/win.ini was found [2023-07-22 02:00:43,274] - [remote_util] [140278505654016] - INFO - This is windows server! [2023-07-22 02:00:43,399] - [remote_util] [140278505654016] - INFO - File /cygdrive/c/tmp/windows_info.txt was found [2023-07-22 02:00:43,493] - [remote_util] [140278505654016] - INFO - get windows information [2023-07-22 02:00:43,600] - [remote_util] [140278505654016] - INFO - running command.raw on 172.23.138.142: echo "ipconfig" > /tmp/cmd.bat; chmod u=rwx /tmp/cmd.bat; /tmp/cmd.bat [2023-07-22 02:00:43,742] - [remote_util] [140278505654016] - INFO - command executed successfully with Administrator [2023-07-22 02:00:43,743] - [remote_util] [140278505654016] - INFO - running command.raw on 172.23.138.142: curl --silent --show-error http://Administrator:password@localhost:8091/diag/eval -X POST -d 'ns_config:set(allow_nonlocal_eval, true).' [2023-07-22 02:00:43,838] - [remote_util] [140278505654016] - INFO - command executed successfully with Administrator [2023-07-22 02:00:43,839] - [remote_util] [140278505654016] - INFO - ['ok'] [2023-07-22 02:00:43,844] - [rest_client] [140278505654016] - INFO - /diag/eval status on 172.23.138.142:8091: True content: [7,2] command: cluster_compat_mode:get_compat_version(). [2023-07-22 02:00:43,847] - [rest_client] [140278505654016] - INFO - /diag/eval status on 172.23.138.142:8091: True content: [7,2] command: cluster_compat_mode:get_compat_version(). [2023-07-22 02:00:43,852] - [rest_client] [140278505654016] - INFO - settings/indexes params : storageMode=plasma [2023-07-22 02:00:43,874] - [task] [140278505654016] - INFO - server: ip:172.23.138.143 port:8091 ssh_username:Administrator, nodes/self [2023-07-22 02:00:43,878] - [task] [140278505654016] - INFO - {'uptime': '70', 'memoryTotal': 6433652736, 'memoryFree': 4788867072, 'mcdMemoryReserved': 4908, 'mcdMemoryAllocated': 4908, 'status': 'healthy', 'hostname': '172.23.138.143:8091', 'clusterCompatibility': 458754, 'clusterMembership': 'active', 'recoveryType': 'none', 'version': '7.2.1-5878-enterprise', 'os': 'win64', 'ports': [], 'availableStorage': [], 'storage': [], 'memoryQuota': 3015, 'memcached': 11210, 'id': 'ns_1@172.23.138.143', 'ip': '172.23.138.143', 'internal_ip': '', 'rest_username': '', 'rest_password': '', 'port': '8091', 'services': ['kv'], 'storageTotalRam': 6135, 'curr_items': 0} [2023-07-22 02:00:43,882] - [rest_client] [140278505654016] - ERROR - GET http://172.23.138.143:8091/pools/default body: headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==', 'Accept': '*/*'} error: 404 reason: unknown b'"unknown pool"' auth: Administrator:password [2023-07-22 02:00:43,883] - [rest_client] [140278505654016] - INFO - pools/default params : memoryQuota=3288 [2023-07-22 02:00:43,892] - [rest_client] [140278505654016] - INFO - --> in init_cluster...Administrator,password,8091 [2023-07-22 02:00:43,892] - [rest_client] [140278505654016] - INFO - settings/web params on 172.23.138.143:8091:port=8091&username=Administrator&password=password [2023-07-22 02:00:43,981] - [rest_client] [140278505654016] - INFO - --> status:True [2023-07-22 02:00:43,984] - [remote_util] [140278505654016] - INFO - SSH Connecting to 172.23.138.143 with username:Administrator, attempt#1 of 5 [2023-07-22 02:00:44,088] - [remote_util] [140278505654016] - INFO - SSH Connected to 172.23.138.143 as Administrator [2023-07-22 02:00:44,558] - [remote_util] [140278505654016] - INFO - File /cygdrive/c/Windows/win.ini was found [2023-07-22 02:00:44,558] - [remote_util] [140278505654016] - INFO - This is windows server! [2023-07-22 02:00:44,715] - [remote_util] [140278505654016] - INFO - File /cygdrive/c/tmp/windows_info.txt was found [2023-07-22 02:00:44,809] - [remote_util] [140278505654016] - INFO - get windows information [2023-07-22 02:00:44,931] - [remote_util] [140278505654016] - INFO - running command.raw on 172.23.138.143: echo "ipconfig" > /tmp/cmd.bat; chmod u=rwx /tmp/cmd.bat; /tmp/cmd.bat [2023-07-22 02:00:45,073] - [remote_util] [140278505654016] - INFO - command executed successfully with Administrator [2023-07-22 02:00:45,075] - [remote_util] [140278505654016] - INFO - running command.raw on 172.23.138.143: curl --silent --show-error http://Administrator:password@localhost:8091/diag/eval -X POST -d 'ns_config:set(allow_nonlocal_eval, true).' [2023-07-22 02:00:45,165] - [remote_util] [140278505654016] - INFO - command executed successfully with Administrator [2023-07-22 02:00:45,166] - [remote_util] [140278505654016] - INFO - ['ok'] [2023-07-22 02:00:45,170] - [rest_client] [140278505654016] - INFO - /diag/eval status on 172.23.138.143:8091: True content: [7,2] command: cluster_compat_mode:get_compat_version(). [2023-07-22 02:00:45,173] - [rest_client] [140278505654016] - INFO - /diag/eval status on 172.23.138.143:8091: True content: [7,2] command: cluster_compat_mode:get_compat_version(). [2023-07-22 02:00:45,179] - [rest_client] [140278505654016] - INFO - settings/indexes params : storageMode=plasma [2023-07-22 02:00:45,201] - [task] [140278505654016] - INFO - server: ip:172.23.138.139 port:8091 ssh_username:Administrator, nodes/self [2023-07-22 02:00:45,205] - [task] [140278505654016] - INFO - {'uptime': '47', 'memoryTotal': 6433652736, 'memoryFree': 4812488704, 'mcdMemoryReserved': 4908, 'mcdMemoryAllocated': 4908, 'status': 'healthy', 'hostname': '172.23.138.139:8091', 'clusterCompatibility': 458754, 'clusterMembership': 'active', 'recoveryType': 'none', 'version': '7.2.1-5878-enterprise', 'os': 'win64', 'ports': [], 'availableStorage': [], 'storage': [], 'memoryQuota': 3015, 'memcached': 11210, 'id': 'ns_1@172.23.138.139', 'ip': '172.23.138.139', 'internal_ip': '', 'rest_username': '', 'rest_password': '', 'port': '8091', 'services': ['kv'], 'storageTotalRam': 6135, 'curr_items': 0} [2023-07-22 02:00:45,208] - [rest_client] [140278505654016] - ERROR - GET http://172.23.138.139:8091/pools/default body: headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==', 'Accept': '*/*'} error: 404 reason: unknown b'"unknown pool"' auth: Administrator:password [2023-07-22 02:00:45,209] - [rest_client] [140278505654016] - INFO - pools/default params : memoryQuota=3288 [2023-07-22 02:00:45,218] - [rest_client] [140278505654016] - INFO - --> in init_cluster...Administrator,password,8091 [2023-07-22 02:00:45,218] - [rest_client] [140278505654016] - INFO - settings/web params on 172.23.138.139:8091:port=8091&username=Administrator&password=password [2023-07-22 02:00:45,321] - [rest_client] [140278505654016] - INFO - --> status:True [2023-07-22 02:00:45,325] - [remote_util] [140278505654016] - INFO - SSH Connecting to 172.23.138.139 with username:Administrator, attempt#1 of 5 [2023-07-22 02:00:45,424] - [remote_util] [140278505654016] - INFO - SSH Connected to 172.23.138.139 as Administrator [2023-07-22 02:00:45,898] - [remote_util] [140278505654016] - INFO - File /cygdrive/c/Windows/win.ini was found [2023-07-22 02:00:45,899] - [remote_util] [140278505654016] - INFO - This is windows server! [2023-07-22 02:00:46,035] - [remote_util] [140278505654016] - INFO - File /cygdrive/c/tmp/windows_info.txt was found [2023-07-22 02:00:46,129] - [remote_util] [140278505654016] - INFO - get windows information [2023-07-22 02:00:46,241] - [remote_util] [140278505654016] - INFO - running command.raw on 172.23.138.139: echo "ipconfig" > /tmp/cmd.bat; chmod u=rwx /tmp/cmd.bat; /tmp/cmd.bat [2023-07-22 02:00:46,385] - [remote_util] [140278505654016] - INFO - command executed successfully with Administrator [2023-07-22 02:00:46,387] - [remote_util] [140278505654016] - INFO - running command.raw on 172.23.138.139: curl --silent --show-error http://Administrator:password@localhost:8091/diag/eval -X POST -d 'ns_config:set(allow_nonlocal_eval, true).' [2023-07-22 02:00:46,489] - [remote_util] [140278505654016] - INFO - command executed successfully with Administrator [2023-07-22 02:00:46,489] - [remote_util] [140278505654016] - INFO - ['ok'] [2023-07-22 02:00:46,495] - [rest_client] [140278505654016] - INFO - /diag/eval status on 172.23.138.139:8091: True content: [7,2] command: cluster_compat_mode:get_compat_version(). [2023-07-22 02:00:46,499] - [rest_client] [140278505654016] - INFO - /diag/eval status on 172.23.138.139:8091: True content: [7,2] command: cluster_compat_mode:get_compat_version(). [2023-07-22 02:00:46,504] - [rest_client] [140278505654016] - INFO - settings/indexes params : storageMode=plasma [2023-07-22 02:00:46,526] - [task] [140278505654016] - INFO - server: ip:172.23.138.140 port:8091 ssh_username:Administrator, nodes/self [2023-07-22 02:00:46,531] - [task] [140278505654016] - INFO - {'uptime': '22', 'memoryTotal': 6433652736, 'memoryFree': 4821880832, 'mcdMemoryReserved': 4908, 'mcdMemoryAllocated': 4908, 'status': 'healthy', 'hostname': '172.23.138.140:8091', 'clusterCompatibility': 458754, 'clusterMembership': 'active', 'recoveryType': 'none', 'version': '7.2.1-5878-enterprise', 'os': 'win64', 'ports': [], 'availableStorage': [], 'storage': [], 'memoryQuota': 3015, 'memcached': 11210, 'id': 'ns_1@172.23.138.140', 'ip': '172.23.138.140', 'internal_ip': '', 'rest_username': '', 'rest_password': '', 'port': '8091', 'services': ['kv'], 'storageTotalRam': 6135, 'curr_items': 0} [2023-07-22 02:00:46,534] - [rest_client] [140278505654016] - ERROR - GET http://172.23.138.140:8091/pools/default body: headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==', 'Accept': '*/*'} error: 404 reason: unknown b'"unknown pool"' auth: Administrator:password [2023-07-22 02:00:46,535] - [rest_client] [140278505654016] - INFO - pools/default params : memoryQuota=3288 [2023-07-22 02:00:46,545] - [rest_client] [140278505654016] - INFO - --> in init_cluster...Administrator,password,8091 [2023-07-22 02:00:46,545] - [rest_client] [140278505654016] - INFO - settings/web params on 172.23.138.140:8091:port=8091&username=Administrator&password=password [2023-07-22 02:00:46,636] - [rest_client] [140278505654016] - INFO - --> status:True [2023-07-22 02:00:46,639] - [remote_util] [140278505654016] - INFO - SSH Connecting to 172.23.138.140 with username:Administrator, attempt#1 of 5 [2023-07-22 02:00:46,748] - [remote_util] [140278505654016] - INFO - SSH Connected to 172.23.138.140 as Administrator [2023-07-22 02:00:47,248] - [remote_util] [140278505654016] - INFO - File /cygdrive/c/Windows/win.ini was found [2023-07-22 02:00:47,248] - [remote_util] [140278505654016] - INFO - This is windows server! [2023-07-22 02:00:47,389] - [remote_util] [140278505654016] - INFO - File /cygdrive/c/tmp/windows_info.txt was found [2023-07-22 02:00:47,499] - [remote_util] [140278505654016] - INFO - get windows information [2023-07-22 02:00:47,625] - [remote_util] [140278505654016] - INFO - running command.raw on 172.23.138.140: echo "ipconfig" > /tmp/cmd.bat; chmod u=rwx /tmp/cmd.bat; /tmp/cmd.bat [2023-07-22 02:00:47,781] - [remote_util] [140278505654016] - INFO - command executed successfully with Administrator [2023-07-22 02:00:47,783] - [remote_util] [140278505654016] - INFO - running command.raw on 172.23.138.140: curl --silent --show-error http://Administrator:password@localhost:8091/diag/eval -X POST -d 'ns_config:set(allow_nonlocal_eval, true).' [2023-07-22 02:00:47,882] - [remote_util] [140278505654016] - INFO - command executed successfully with Administrator [2023-07-22 02:00:47,882] - [remote_util] [140278505654016] - INFO - ['ok'] [2023-07-22 02:00:47,888] - [rest_client] [140278505654016] - INFO - /diag/eval status on 172.23.138.140:8091: True content: [7,2] command: cluster_compat_mode:get_compat_version(). [2023-07-22 02:00:47,892] - [rest_client] [140278505654016] - INFO - /diag/eval status on 172.23.138.140:8091: True content: [7,2] command: cluster_compat_mode:get_compat_version(). [2023-07-22 02:00:47,898] - [rest_client] [140278505654016] - INFO - settings/indexes params : storageMode=plasma [2023-07-22 02:00:47,917] - [basetestcase] [140279370630912] - INFO - **** add built-in 'cbadminbucket' user to node 172.23.138.138 **** [2023-07-22 02:00:47,934] - [rest_client] [140279370630912] - ERROR - DELETE http://172.23.138.138:8091/settings/rbac/users/local/cbadminbucket body: headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==', 'Accept': '*/*'} error: 404 reason: unknown b'"User was not found."' auth: Administrator:password [2023-07-22 02:00:47,935] - [internal_user] [140279370630912] - INFO - Exception while deleting user. Exception is -b'"User was not found."' [2023-07-22 02:00:48,018] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. ... [2023-07-22 02:00:53,023] - [basetestcase] [140279370630912] - INFO - **** add 'admin' role to 'cbadminbucket' user **** [2023-07-22 02:00:53,040] - [basetestcase] [140279370630912] - INFO - done initializing cluster [2023-07-22 02:00:53,923] - [task] [140278505654016] - INFO - adding node 172.23.138.141:8091 to cluster [2023-07-22 02:00:53,932] - [rest_client] [140278505654016] - INFO - adding remote node @172.23.138.141:18091 to this cluster @172.23.138.138:8091 [2023-07-22 02:01:03,950] - [rest_client] [140278505654016] - INFO - rebalance progress took 10.02 seconds [2023-07-22 02:01:03,950] - [rest_client] [140278505654016] - INFO - sleep for 10 seconds after rebalance... [2023-07-22 02:01:18,021] - [task] [140278505654016] - INFO - adding node 172.23.138.137:8091 to cluster [2023-07-22 02:01:18,031] - [rest_client] [140278505654016] - INFO - adding remote node @172.23.138.137:18091 to this cluster @172.23.138.138:8091 [2023-07-22 02:01:28,044] - [rest_client] [140278505654016] - INFO - rebalance progress took 10.01 seconds [2023-07-22 02:01:28,045] - [rest_client] [140278505654016] - INFO - sleep for 10 seconds after rebalance... [2023-07-22 02:01:42,726] - [task] [140278505654016] - INFO - adding node 172.23.138.142:8091 to cluster [2023-07-22 02:01:42,735] - [rest_client] [140278505654016] - INFO - adding remote node @172.23.138.142:18091 to this cluster @172.23.138.138:8091 [2023-07-22 02:01:52,749] - [rest_client] [140278505654016] - INFO - rebalance progress took 10.01 seconds [2023-07-22 02:01:52,749] - [rest_client] [140278505654016] - INFO - sleep for 10 seconds after rebalance... [2023-07-22 02:02:08,593] - [task] [140278505654016] - INFO - adding node 172.23.138.143:8091 to cluster [2023-07-22 02:02:08,605] - [rest_client] [140278505654016] - INFO - adding remote node @172.23.138.143:18091 to this cluster @172.23.138.138:8091 [2023-07-22 02:02:18,620] - [rest_client] [140278505654016] - INFO - rebalance progress took 10.01 seconds [2023-07-22 02:02:18,620] - [rest_client] [140278505654016] - INFO - sleep for 10 seconds after rebalance... [2023-07-22 02:02:34,824] - [rest_client] [140278505654016] - INFO - Node 172.23.138.137 not part of cluster inactiveAdded [2023-07-22 02:02:34,824] - [rest_client] [140278505654016] - INFO - Node 172.23.138.141 not part of cluster inactiveAdded [2023-07-22 02:02:34,825] - [rest_client] [140278505654016] - INFO - Node 172.23.138.142 not part of cluster inactiveAdded [2023-07-22 02:02:34,825] - [rest_client] [140278505654016] - INFO - Node 172.23.138.143 not part of cluster inactiveAdded [2023-07-22 02:02:34,843] - [rest_client] [140278505654016] - INFO - rebalance params : {'knownNodes': 'ns_1@172.23.138.137,ns_1@172.23.138.138,ns_1@172.23.138.141,ns_1@172.23.138.142,ns_1@172.23.138.143', 'ejectedNodes': '', 'user': 'Administrator', 'password': 'password'} [2023-07-22 02:02:34,917] - [rest_client] [140278505654016] - INFO - rebalance operation started [2023-07-22 02:02:44,928] - [task] [140278505654016] - INFO - Rebalance - status: none, progress: 100.00% [2023-07-22 02:02:44,937] - [task] [140278505654016] - INFO - rebalancing was completed with progress: 100% in 10.019586563110352 sec [2023-07-22 02:02:44,940] - [remote_util] [140279370630912] - INFO - SSH Connecting to 172.23.138.138 with username:Administrator, attempt#1 of 5 [2023-07-22 02:02:45,037] - [remote_util] [140279370630912] - INFO - SSH Connected to 172.23.138.138 as Administrator [2023-07-22 02:02:45,462] - [remote_util] [140279370630912] - INFO - File /cygdrive/c/Windows/win.ini was found [2023-07-22 02:02:45,462] - [remote_util] [140279370630912] - INFO - This is windows server! [2023-07-22 02:02:45,603] - [remote_util] [140279370630912] - INFO - File /cygdrive/c/tmp/windows_info.txt was found [2023-07-22 02:02:45,698] - [remote_util] [140279370630912] - INFO - get windows information [2023-07-22 02:02:45,816] - [remote_util] [140279370630912] - INFO - running command.raw on 172.23.138.138: echo "ipconfig" > /tmp/cmd.bat; chmod u=rwx /tmp/cmd.bat; /tmp/cmd.bat [2023-07-22 02:02:45,959] - [remote_util] [140279370630912] - INFO - command executed successfully with Administrator [2023-07-22 02:02:45,961] - [remote_util] [140279370630912] - INFO - running command.raw on 172.23.138.138: curl --silent --show-error http://Administrator:password@localhost:8091/diag/eval -X POST -d 'ns_config:set(allow_nonlocal_eval, true).' [2023-07-22 02:02:46,058] - [remote_util] [140279370630912] - INFO - command executed successfully with Administrator [2023-07-22 02:02:46,059] - [remote_util] [140279370630912] - INFO - ['ok'] [2023-07-22 02:02:46,060] - [basetestcase] [140279370630912] - INFO - Enabled diag/eval for non-local hosts from 172.23.138.138 [2023-07-22 02:02:46,060] - [basetestcase] [140279370630912] - INFO - ============== basetestcase setup was finished for test #2 test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously ============== [2023-07-22 02:02:46,071] - [remote_util] [140279370630912] - INFO - SSH Connecting to 172.23.138.138 with username:Administrator, attempt#1 of 5 [2023-07-22 02:02:46,168] - [remote_util] [140279370630912] - INFO - SSH Connected to 172.23.138.138 as Administrator [2023-07-22 02:02:46,609] - [remote_util] [140279370630912] - INFO - File /cygdrive/c/Windows/win.ini was found [2023-07-22 02:02:46,609] - [remote_util] [140279370630912] - INFO - This is windows server! [2023-07-22 02:02:46,750] - [remote_util] [140279370630912] - INFO - File /cygdrive/c/tmp/windows_info.txt was found [2023-07-22 02:02:46,844] - [remote_util] [140279370630912] - INFO - get windows information [2023-07-22 02:02:46,970] - [remote_util] [140279370630912] - INFO - running command.raw on 172.23.138.138: echo "ipconfig" > /tmp/cmd.bat; chmod u=rwx /tmp/cmd.bat; /tmp/cmd.bat [2023-07-22 02:02:47,111] - [remote_util] [140279370630912] - INFO - command executed successfully with Administrator [2023-07-22 02:02:47,118] - [remote_util] [140279370630912] - INFO - SSH Connecting to 172.23.138.141 with username:Administrator, attempt#1 of 5 [2023-07-22 02:02:47,208] - [remote_util] [140279370630912] - INFO - SSH Connected to 172.23.138.141 as Administrator [2023-07-22 02:02:47,665] - [remote_util] [140279370630912] - INFO - File /cygdrive/c/Windows/win.ini was found [2023-07-22 02:02:47,666] - [remote_util] [140279370630912] - INFO - This is windows server! [2023-07-22 02:02:47,807] - [remote_util] [140279370630912] - INFO - File /cygdrive/c/tmp/windows_info.txt was found [2023-07-22 02:02:47,901] - [remote_util] [140279370630912] - INFO - get windows information [2023-07-22 02:02:48,028] - [remote_util] [140279370630912] - INFO - running command.raw on 172.23.138.141: echo "ipconfig" > /tmp/cmd.bat; chmod u=rwx /tmp/cmd.bat; /tmp/cmd.bat [2023-07-22 02:02:48,172] - [remote_util] [140279370630912] - INFO - command executed successfully with Administrator [2023-07-22 02:02:48,177] - [remote_util] [140279370630912] - INFO - SSH Connecting to 172.23.138.137 with username:Administrator, attempt#1 of 5 [2023-07-22 02:02:48,278] - [remote_util] [140279370630912] - INFO - SSH Connected to 172.23.138.137 as Administrator [2023-07-22 02:02:48,828] - [remote_util] [140279370630912] - INFO - File /cygdrive/c/Windows/win.ini was found [2023-07-22 02:02:48,828] - [remote_util] [140279370630912] - INFO - This is windows server! [2023-07-22 02:02:48,969] - [remote_util] [140279370630912] - INFO - File /cygdrive/c/tmp/windows_info.txt was found [2023-07-22 02:02:49,064] - [remote_util] [140279370630912] - INFO - get windows information [2023-07-22 02:02:49,151] - [remote_util] [140279370630912] - INFO - running command.raw on 172.23.138.137: echo "ipconfig" > /tmp/cmd.bat; chmod u=rwx /tmp/cmd.bat; /tmp/cmd.bat [2023-07-22 02:02:49,290] - [remote_util] [140279370630912] - INFO - command executed successfully with Administrator [2023-07-22 02:02:49,296] - [remote_util] [140279370630912] - INFO - SSH Connecting to 172.23.138.142 with username:Administrator, attempt#1 of 5 [2023-07-22 02:02:49,393] - [remote_util] [140279370630912] - INFO - SSH Connected to 172.23.138.142 as Administrator [2023-07-22 02:02:49,850] - [remote_util] [140279370630912] - INFO - File /cygdrive/c/Windows/win.ini was found [2023-07-22 02:02:49,850] - [remote_util] [140279370630912] - INFO - This is windows server! [2023-07-22 02:02:49,991] - [remote_util] [140279370630912] - INFO - File /cygdrive/c/tmp/windows_info.txt was found [2023-07-22 02:02:50,085] - [remote_util] [140279370630912] - INFO - get windows information [2023-07-22 02:02:50,182] - [remote_util] [140279370630912] - INFO - running command.raw on 172.23.138.142: echo "ipconfig" > /tmp/cmd.bat; chmod u=rwx /tmp/cmd.bat; /tmp/cmd.bat [2023-07-22 02:02:50,319] - [remote_util] [140279370630912] - INFO - command executed successfully with Administrator [2023-07-22 02:02:50,326] - [remote_util] [140279370630912] - INFO - SSH Connecting to 172.23.138.143 with username:Administrator, attempt#1 of 5 [2023-07-22 02:02:50,426] - [remote_util] [140279370630912] - INFO - SSH Connected to 172.23.138.143 as Administrator [2023-07-22 02:02:50,882] - [remote_util] [140279370630912] - INFO - File /cygdrive/c/Windows/win.ini was found [2023-07-22 02:02:50,882] - [remote_util] [140279370630912] - INFO - This is windows server! [2023-07-22 02:02:50,996] - [remote_util] [140279370630912] - INFO - File /cygdrive/c/tmp/windows_info.txt was found [2023-07-22 02:02:51,086] - [remote_util] [140279370630912] - INFO - get windows information [2023-07-22 02:02:51,211] - [remote_util] [140279370630912] - INFO - running command.raw on 172.23.138.143: echo "ipconfig" > /tmp/cmd.bat; chmod u=rwx /tmp/cmd.bat; /tmp/cmd.bat [2023-07-22 02:02:51,349] - [remote_util] [140279370630912] - INFO - command executed successfully with Administrator [2023-07-22 02:02:51,353] - [remote_util] [140279370630912] - INFO - SSH Connecting to 172.23.138.139 with username:Administrator, attempt#1 of 5 [2023-07-22 02:02:51,482] - [remote_util] [140279370630912] - INFO - SSH Connected to 172.23.138.139 as Administrator [2023-07-22 02:02:51,984] - [remote_util] [140279370630912] - INFO - File /cygdrive/c/Windows/win.ini was found [2023-07-22 02:02:51,985] - [remote_util] [140279370630912] - INFO - This is windows server! [2023-07-22 02:02:52,141] - [remote_util] [140279370630912] - INFO - File /cygdrive/c/tmp/windows_info.txt was found [2023-07-22 02:02:52,236] - [remote_util] [140279370630912] - INFO - get windows information [2023-07-22 02:02:52,334] - [remote_util] [140279370630912] - INFO - running command.raw on 172.23.138.139: echo "ipconfig" > /tmp/cmd.bat; chmod u=rwx /tmp/cmd.bat; /tmp/cmd.bat [2023-07-22 02:02:52,502] - [remote_util] [140279370630912] - INFO - command executed successfully with Administrator [2023-07-22 02:02:52,508] - [remote_util] [140279370630912] - INFO - SSH Connecting to 172.23.138.140 with username:Administrator, attempt#1 of 5 [2023-07-22 02:02:52,623] - [remote_util] [140279370630912] - INFO - SSH Connected to 172.23.138.140 as Administrator [2023-07-22 02:02:53,094] - [remote_util] [140279370630912] - INFO - File /cygdrive/c/Windows/win.ini was found [2023-07-22 02:02:53,094] - [remote_util] [140279370630912] - INFO - This is windows server! [2023-07-22 02:02:53,235] - [remote_util] [140279370630912] - INFO - File /cygdrive/c/tmp/windows_info.txt was found [2023-07-22 02:02:53,329] - [remote_util] [140279370630912] - INFO - get windows information [2023-07-22 02:02:53,459] - [remote_util] [140279370630912] - INFO - running command.raw on 172.23.138.140: echo "ipconfig" > /tmp/cmd.bat; chmod u=rwx /tmp/cmd.bat; /tmp/cmd.bat [2023-07-22 02:02:53,622] - [remote_util] [140279370630912] - INFO - command executed successfully with Administrator [2023-07-22 02:02:59,617] - [basetestcase] [140279370630912] - INFO - ------- Cluster statistics ------- [2023-07-22 02:02:59,617] - [basetestcase] [140279370630912] - INFO - 172.23.138.137:8091 => {'services': ['eventing'], 'cpu_utilization': 4.166666666666667, 'mem_free': 4737703936, 'mem_total': 6433652736, 'swap_mem_used': 1952641024, 'swap_mem_total': 7507394560} [2023-07-22 02:02:59,617] - [basetestcase] [140279370630912] - INFO - 172.23.138.138:8091 => {'services': ['kv'], 'cpu_utilization': 9.803468208092486, 'mem_free': 4667781120, 'mem_total': 6433652736, 'swap_mem_used': 2014482432, 'swap_mem_total': 7507394560} [2023-07-22 02:02:59,618] - [basetestcase] [140279370630912] - INFO - 172.23.138.141:8091 => {'services': ['kv'], 'cpu_utilization': 4.193333333333333, 'mem_free': 4733456384, 'mem_total': 6433652736, 'swap_mem_used': 1943818240, 'swap_mem_total': 7507394560} [2023-07-22 02:02:59,618] - [basetestcase] [140279370630912] - INFO - 172.23.138.142:8091 => {'services': ['eventing'], 'cpu_utilization': 6.458333333333333, 'mem_free': 4764250112, 'mem_total': 6433652736, 'swap_mem_used': 1937784832, 'swap_mem_total': 7507394560} [2023-07-22 02:02:59,619] - [basetestcase] [140279370630912] - INFO - 172.23.138.143:8091 => {'services': ['eventing'], 'cpu_utilization': 3.648272528791187, 'mem_free': 4745146368, 'mem_total': 6433652736, 'swap_mem_used': 1943851008, 'swap_mem_total': 7507394560} [2023-07-22 02:02:59,619] - [basetestcase] [140279370630912] - INFO - --- End of cluster statistics --- [2023-07-22 02:02:59,619] - [basetestcase] [140279370630912] - INFO - Time to execute basesetup : 345.4438519477844 [2023-07-22 02:02:59,622] - [remote_util] [140279370630912] - INFO - SSH Connecting to 172.23.138.138 with username:Administrator, attempt#1 of 5 [2023-07-22 02:02:59,719] - [remote_util] [140279370630912] - INFO - SSH Connected to 172.23.138.138 as Administrator [2023-07-22 02:03:00,160] - [remote_util] [140279370630912] - INFO - File /cygdrive/c/Windows/win.ini was found [2023-07-22 02:03:00,160] - [remote_util] [140279370630912] - INFO - This is windows server! [2023-07-22 02:03:00,301] - [remote_util] [140279370630912] - INFO - File /cygdrive/c/tmp/windows_info.txt was found [2023-07-22 02:03:00,396] - [remote_util] [140279370630912] - INFO - get windows information [2023-07-22 02:03:00,524] - [remote_util] [140279370630912] - INFO - running command.raw on 172.23.138.138: echo "ipconfig" > /tmp/cmd.bat; chmod u=rwx /tmp/cmd.bat; /tmp/cmd.bat [2023-07-22 02:03:00,666] - [remote_util] [140279370630912] - INFO - command executed successfully with Administrator [2023-07-22 02:03:07,783] - [basetestcase] [140279370630912] - INFO - cannot find service node n1ql in cluster [2023-07-22 02:03:07,787] - [remote_util] [140279370630912] - INFO - SSH Connecting to 172.23.138.138 with username:Administrator, attempt#1 of 5 [2023-07-22 02:03:07,910] - [remote_util] [140279370630912] - INFO - SSH Connected to 172.23.138.138 as Administrator [2023-07-22 02:03:08,381] - [remote_util] [140279370630912] - INFO - File /cygdrive/c/Windows/win.ini was found [2023-07-22 02:03:08,381] - [remote_util] [140279370630912] - INFO - This is windows server! [2023-07-22 02:03:08,494] - [remote_util] [140279370630912] - INFO - File /cygdrive/c/tmp/windows_info.txt was found [2023-07-22 02:03:08,585] - [remote_util] [140279370630912] - INFO - get windows information [2023-07-22 02:03:08,674] - [remote_util] [140279370630912] - INFO - running command.raw on 172.23.138.138: echo "ipconfig" > /tmp/cmd.bat; chmod u=rwx /tmp/cmd.bat; /tmp/cmd.bat [2023-07-22 02:03:08,816] - [remote_util] [140279370630912] - INFO - command executed successfully with Administrator [2023-07-22 02:03:08,832] - [basetestcase] [140279370630912] - INFO - list of eventing nodes in cluster: [ip:172.23.138.137 port:8091 ssh_username:Administrator, ip:172.23.138.142 port:8091 ssh_username:Administrator, ip:172.23.138.143 port:8091 ssh_username:Administrator] [2023-07-22 02:03:08,837] - [rest_client] [140279370630912] - INFO - settings/indexes params : storageMode=plasma [2023-07-22 02:03:08,843] - [rest_client] [140279370630912] - INFO - pools/default params : eventingMemoryQuota=512 [2023-07-22 02:03:08,865] - [basetestcase] [140279370630912] - INFO - Creating bucket src_bucket [2023-07-22 02:03:08,874] - [basetestcase] [140279370630912] - INFO - Creating bucket metadata [2023-07-22 02:03:08,883] - [basetestcase] [140279370630912] - INFO - Creating bucket dst_bucket [2023-07-22 02:03:09,687] - [rest_client] [140278505654016] - INFO - http://172.23.138.138:8091/pools/default/buckets with param: name=src_bucket&ramQuotaMB=1096&replicaNumber=1&bucketType=membase&replicaIndex=1&threadsNumber=3&flushEnabled=1&evictionPolicy=valueOnly&compressionMode=passive&storageBackend=couchstore [2023-07-22 02:03:09,721] - [rest_client] [140278505654016] - INFO - 0.03 seconds to create bucket src_bucket [2023-07-22 02:03:09,744] - [rest_client] [140278505654016] - INFO - http://172.23.138.138:8091/pools/default/buckets with param: name=metadata&ramQuotaMB=1096&replicaNumber=1&bucketType=membase&replicaIndex=1&threadsNumber=3&flushEnabled=1&evictionPolicy=valueOnly&compressionMode=passive&storageBackend=couchstore [2023-07-22 02:03:09,768] - [rest_client] [140278505654016] - INFO - 0.02 seconds to create bucket metadata [2023-07-22 02:03:09,791] - [rest_client] [140278505654016] - INFO - http://172.23.138.138:8091/pools/default/buckets with param: name=dst_bucket&ramQuotaMB=1096&replicaNumber=1&bucketType=membase&replicaIndex=1&threadsNumber=3&flushEnabled=1&evictionPolicy=valueOnly&compressionMode=passive&storageBackend=couchstore [2023-07-22 02:03:09,823] - [rest_client] [140278505654016] - INFO - 0.03 seconds to create bucket dst_bucket [2023-07-22 02:03:09,824] - [bucket_helper] [140278505654016] - INFO - waiting for memcached bucket : src_bucket in 172.23.138.138 to accept set ops [2023-07-22 02:03:11,711] - [data_helper] [140278505654016] - INFO - creating direct client 172.23.138.138:11210 src_bucket [2023-07-22 02:03:11,819] - [data_helper] [140278505654016] - INFO - creating direct client 172.23.138.141:11210 src_bucket [2023-07-22 02:03:11,996] - [data_helper] [140278505654016] - INFO - creating direct client 172.23.138.138:11210 src_bucket [2023-07-22 02:03:12,245] - [data_helper] [140278505654016] - INFO - creating direct client 172.23.138.141:11210 src_bucket [2023-07-22 02:03:13,062] - [task] [140278505654016] - INFO - bucket 'src_bucket' was created with per node RAM quota: 1096 [2023-07-22 02:03:13,063] - [bucket_helper] [140278505654016] - INFO - waiting for memcached bucket : metadata in 172.23.138.138 to accept set ops [2023-07-22 02:03:15,951] - [data_helper] [140278505654016] - INFO - creating direct client 172.23.138.138:11210 metadata [2023-07-22 02:03:17,066] - [data_helper] [140278505654016] - INFO - creating direct client 172.23.138.141:11210 metadata [2023-07-22 02:03:18,502] - [data_helper] [140278505654016] - INFO - creating direct client 172.23.138.138:11210 metadata [2023-07-22 02:03:18,616] - [data_helper] [140278505654016] - INFO - creating direct client 172.23.138.141:11210 metadata [2023-07-22 02:03:19,318] - [task] [140278505654016] - INFO - bucket 'metadata' was created with per node RAM quota: 1096 [2023-07-22 02:03:19,319] - [bucket_helper] [140278505654016] - INFO - waiting for memcached bucket : dst_bucket in 172.23.138.138 to accept set ops [2023-07-22 02:03:20,322] - [data_helper] [140278505654016] - INFO - creating direct client 172.23.138.138:11210 dst_bucket [2023-07-22 02:03:24,467] - [data_helper] [140278505654016] - INFO - creating direct client 172.23.138.141:11210 dst_bucket [2023-07-22 02:03:24,752] - [data_helper] [140278505654016] - INFO - creating direct client 172.23.138.138:11210 dst_bucket [2023-07-22 02:03:24,847] - [data_helper] [140278505654016] - INFO - creating direct client 172.23.138.141:11210 dst_bucket [2023-07-22 02:03:25,758] - [task] [140278505654016] - INFO - bucket 'dst_bucket' was created with per node RAM quota: 1096 [2023-07-22 02:03:25,759] - [eventing_base] [140279370630912] - INFO - ============== EventingBaseTest setup has completed ============== [2023-07-22 02:03:27,401] - [eventing_base] [140279370630912] - INFO - saving function b'{\n "code": 0,\n "info": "Function: src_bucket/_default/Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously stored in temp store"\n}' [2023-07-22 02:03:27,402] - [eventing_base] [140279370630912] - INFO - Deploying the following handler code : Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously with bindings: {'buckets': [{'alias': 'dst_bucket', 'bucket_name': 'dst_bucket', 'access': 'rw'}], 'metadata_bucket': 'metadata', 'source_bucket': 'src_bucket', 'curl': []} and settings: {'checkpoint_interval': 20000, 'cleanup_timers': False, 'dcp_stream_boundary': 'everything', 'deployment_status': False, 'description': 'Sample Description', 'log_level': 'INFO', 'skip_timer_threshold': 86400, 'sock_batch_size': 1, 'tick_duration': 5000, 'timer_processing_tick_interval': 500, 'timer_worker_pool_size': 3, 'worker_count': 3, 'processing_status': False, 'cpp_worker_thread_count': 1, 'execution_timeout': 20, 'data_chan_size': 10000, 'worker_queue_cap': 100000, 'use_memory_manager': True, 'deadline_timeout': 21, 'timer_storage_chan_size': 10000, 'dcp_gen_chan_size': 10000} [2023-07-22 02:03:27,402] - [eventing_base] [140279370630912] - INFO - function OnUpdate(doc, meta) { log("Doc created/updated", meta.id); var result1= couchbase.insert(dst_bucket,meta,doc); log(result1); } function OnDelete(meta, options) { log("Doc deleted/expired", meta.id); var doc={"id":meta.id} var result = couchbase.delete(dst_bucket,doc); log(result); } [2023-07-22 02:03:28,523] - [eventing_base] [140279370630912] - INFO - deploy Application : b'{\n "code": 0,\n "info": "Function: src_bucket/_default/Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously is deploying"\n}' [2023-07-22 02:03:28,524] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:03:33,545] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:03:38,567] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:03:43,588] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:03:48,613] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:03:53,636] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:03:58,655] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:04:03,677] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:04:08,696] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:04:13,718] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:04:18,741] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:04:23,763] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:04:28,786] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:04:33,808] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:04:38,830] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:04:43,851] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:04:48,874] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:04:53,897] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:04:58,920] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:05:03,942] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:05:08,965] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:05:13,984] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:05:19,007] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:05:24,029] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:05:29,053] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:05:34,075] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:05:39,098] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:05:44,120] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:05:49,143] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:05:54,165] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:05:59,187] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:06:04,210] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:06:09,235] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:06:14,258] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:06:19,281] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:06:24,304] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:06:29,327] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:06:34,349] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:06:39,372] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:06:44,395] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:06:49,415] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:06:54,438] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:06:59,462] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:07:04,484] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:07:09,508] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:07:14,531] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:07:19,553] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:07:24,575] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:07:29,598] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:07:34,622] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:07:39,645] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:07:44,668] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:07:49,690] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:07:54,711] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:07:59,734] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:08:04,754] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:08:09,777] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:08:14,803] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:08:19,825] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:08:24,848] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:08:29,871] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:08:34,893] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:08:39,915] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:08:44,938] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:08:49,959] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:08:54,984] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:09:00,007] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:09:05,029] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:09:10,052] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:09:15,074] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:09:20,094] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:09:25,112] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:09:30,134] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:09:35,155] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:09:40,178] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:09:45,202] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:09:50,224] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:09:55,246] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:10:00,270] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:10:05,293] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:10:10,313] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:10:15,330] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:10:20,349] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:10:25,366] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:10:30,389] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:10:35,408] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:10:40,429] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:10:45,448] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:10:50,471] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:10:55,493] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:11:00,515] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:11:05,539] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:11:10,560] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:11:15,582] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:11:20,604] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:11:25,622] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:11:30,644] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:11:35,668] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:11:40,692] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:11:45,716] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:11:50,746] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:11:55,765] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:12:00,788] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:12:05,810] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:12:10,839] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:12:15,856] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:12:20,876] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:12:25,895] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:12:30,917] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:12:35,935] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:12:40,959] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:12:45,980] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:12:51,006] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:12:56,026] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:13:01,050] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:13:06,074] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:13:11,104] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:13:16,125] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:13:21,147] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:13:26,167] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:13:31,198] - [basetestcase] [140279370630912] - INFO - sleep for 5 secs. Waiting for Function_841642593_test_kv_eventing_failover_and_kv_eventing_rebalance_simultaneously to deployed... ... [2023-07-22 02:13:36,221] - [eventing_base] [140279370630912] - INFO - ============== EventingBaseTest tearDown has started ============== [2023-07-22 02:13:36,249] - [basetestcase] [140279370630912] - INFO - list of eventing nodes in cluster: [ip:172.23.138.137 port:8091 ssh_username:Administrator, ip:172.23.138.142 port:8091 ssh_username:Administrator, ip:172.23.138.143 port:8091 ssh_username:Administrator] [2023-07-22 02:13:36,252] - [remote_util] [140279370630912] - INFO - SSH Connecting to 172.23.138.137 with username:Administrator, attempt#1 of 5 [2023-07-22 02:13:36,340] - [remote_util] [140279370630912] - INFO - SSH Connected to 172.23.138.137 as Administrator [2023-07-22 02:13:36,781] - [remote_util] [140279370630912] - INFO - File /cygdrive/c/Windows/win.ini was found [2023-07-22 02:13:36,781] - [remote_util] [140279370630912] - INFO - This is windows server! [2023-07-22 02:13:36,922] - [remote_util] [140279370630912] - INFO - File /cygdrive/c/tmp/windows_info.txt was found [2023-07-22 02:13:37,016] - [remote_util] [140279370630912] - INFO - get windows information [2023-07-22 02:13:37,104] - [remote_util] [140279370630912] - INFO - running command.raw on 172.23.138.137: echo "ipconfig" > /tmp/cmd.bat; chmod u=rwx /tmp/cmd.bat; /tmp/cmd.bat [2023-07-22 02:13:37,251] - [remote_util] [140279370630912] - INFO - command executed successfully with Administrator [2023-07-22 02:13:37,263] - [rest_client] [140279370630912] - INFO - /diag/eval status on 172.23.138.137:8091: True content: "c:/Program Files/Couchbase/Server/var/lib/couchbase/logs" command: filename:absname(element(2, application:get_env(ns_server,error_logger_mf_dir))). [2023-07-22 02:13:37,263] - [remote_util] [140279370630912] - INFO - running command.raw on 172.23.138.137: zgrep "panic" "c:/Program Files/Couchbase/Server/var/lib/couchbase/logs"/eventing.log* | wc -l [2023-07-22 02:13:37,461] - [remote_util] [140279370630912] - INFO - command executed successfully with Administrator [2023-07-22 02:13:37,462] - [remote_util] [140279370630912] - INFO - running command.raw on 172.23.138.137: ls c://CrashDumps| wc -l [2023-07-22 02:13:37,572] - [remote_util] [140279370630912] - INFO - command executed successfully with Administrator [2023-07-22 02:13:37,575] - [remote_util] [140279370630912] - INFO - SSH Connecting to 172.23.138.142 with username:Administrator, attempt#1 of 5 [2023-07-22 02:13:37,688] - [remote_util] [140279370630912] - INFO - SSH Connected to 172.23.138.142 as Administrator [2023-07-22 02:13:38,317] - [remote_util] [140279370630912] - INFO - File /cygdrive/c/Windows/win.ini was found [2023-07-22 02:13:38,318] - [remote_util] [140279370630912] - INFO - This is windows server! [2023-07-22 02:13:38,443] - [remote_util] [140279370630912] - INFO - File /cygdrive/c/tmp/windows_info.txt was found [2023-07-22 02:13:38,537] - [remote_util] [140279370630912] - INFO - get windows information [2023-07-22 02:13:38,647] - [remote_util] [140279370630912] - INFO - running command.raw on 172.23.138.142: echo "ipconfig" > /tmp/cmd.bat; chmod u=rwx /tmp/cmd.bat; /tmp/cmd.bat [2023-07-22 02:13:38,797] - [remote_util] [140279370630912] - INFO - command executed successfully with Administrator [2023-07-22 02:13:38,808] - [rest_client] [140279370630912] - INFO - /diag/eval status on 172.23.138.142:8091: True content: "c:/Program Files/Couchbase/Server/var/lib/couchbase/logs" command: filename:absname(element(2, application:get_env(ns_server,error_logger_mf_dir))). [2023-07-22 02:13:38,808] - [remote_util] [140279370630912] - INFO - running command.raw on 172.23.138.142: zgrep "panic" "c:/Program Files/Couchbase/Server/var/lib/couchbase/logs"/eventing.log* | wc -l [2023-07-22 02:13:38,984] - [remote_util] [140279370630912] - INFO - command executed successfully with Administrator [2023-07-22 02:13:38,984] - [remote_util] [140279370630912] - INFO - running command.raw on 172.23.138.142: ls c://CrashDumps| wc -l [2023-07-22 02:13:39,093] - [remote_util] [140279370630912] - INFO - command executed successfully with Administrator [2023-07-22 02:13:39,097] - [remote_util] [140279370630912] - INFO - SSH Connecting to 172.23.138.143 with username:Administrator, attempt#1 of 5 [2023-07-22 02:13:39,201] - [remote_util] [140279370630912] - INFO - SSH Connected to 172.23.138.143 as Administrator [2023-07-22 02:13:39,738] - [remote_util] [140279370630912] - INFO - File /cygdrive/c/Windows/win.ini was found [2023-07-22 02:13:39,739] - [remote_util] [140279370630912] - INFO - This is windows server! [2023-07-22 02:13:39,876] - [remote_util] [140279370630912] - INFO - File /cygdrive/c/tmp/windows_info.txt was found [2023-07-22 02:13:39,971] - [remote_util] [140279370630912] - INFO - get windows information [2023-07-22 02:13:40,096] - [remote_util] [140279370630912] - INFO - running command.raw on 172.23.138.143: echo "ipconfig" > /tmp/cmd.bat; chmod u=rwx /tmp/cmd.bat; /tmp/cmd.bat [2023-07-22 02:13:40,238] - [remote_util] [140279370630912] - INFO - command executed successfully with Administrator [2023-07-22 02:13:40,251] - [rest_client] [140279370630912] - INFO - /diag/eval status on 172.23.138.143:8091: True content: "c:/Program Files/Couchbase/Server/var/lib/couchbase/logs" command: filename:absname(element(2, application:get_env(ns_server,error_logger_mf_dir))). [2023-07-22 02:13:40,251] - [remote_util] [140279370630912] - INFO - running command.raw on 172.23.138.143: zgrep "panic" "c:/Program Files/Couchbase/Server/var/lib/couchbase/logs"/eventing.log* | wc -l [2023-07-22 02:13:40,412] - [remote_util] [140279370630912] - INFO - command executed successfully with Administrator [2023-07-22 02:13:40,413] - [remote_util] [140279370630912] - INFO - running command.raw on 172.23.138.143: ls c://CrashDumps| wc -l [2023-07-22 02:13:40,515] - [remote_util] [140279370630912] - INFO - command executed successfully with Administrator [2023-07-22 02:13:40,709] - [rest_client] [140279370630912] - INFO - Node versions in cluster ['7.2.1-5878-enterprise', '7.2.1-5878-enterprise', '7.2.1-5878-enterprise', '7.2.1-5878-enterprise', '7.2.1-5878-enterprise'] [2023-07-22 02:13:40,718] - [basetestcase] [140279370630912] - INFO - list of kv nodes in cluster: [ip:172.23.138.138 port:8091 ssh_username:Administrator, ip:172.23.138.141 port:8091 ssh_username:Administrator] [2023-07-22 02:13:40,721] - [remote_util] [140279370630912] - INFO - SSH Connecting to 172.23.138.138 with username:Administrator, attempt#1 of 5 [2023-07-22 02:13:40,811] - [remote_util] [140279370630912] - INFO - SSH Connected to 172.23.138.138 as Administrator [2023-07-22 02:13:41,267] - [remote_util] [140279370630912] - INFO - File /cygdrive/c/Windows/win.ini was found [2023-07-22 02:13:41,267] - [remote_util] [140279370630912] - INFO - This is windows server! [2023-07-22 02:13:41,408] - [remote_util] [140279370630912] - INFO - File /cygdrive/c/tmp/windows_info.txt was found [2023-07-22 02:13:41,502] - [remote_util] [140279370630912] - INFO - get windows information [2023-07-22 02:13:41,606] - [remote_util] [140279370630912] - INFO - running command.raw on 172.23.138.138: echo "ipconfig" > /tmp/cmd.bat; chmod u=rwx /tmp/cmd.bat; /tmp/cmd.bat [2023-07-22 02:13:41,752] - [remote_util] [140279370630912] - INFO - command executed successfully with Administrator [2023-07-22 02:13:42,022] - [remote_util] [140279370630912] - INFO - SSH Connecting to 172.23.138.141 with username:Administrator, attempt#1 of 5 [2023-07-22 02:13:42,126] - [remote_util] [140279370630912] - INFO - SSH Connected to 172.23.138.141 as Administrator [2023-07-22 02:13:42,648] - [remote_util] [140279370630912] - INFO - File /cygdrive/c/Windows/win.ini was found [2023-07-22 02:13:42,648] - [remote_util] [140279370630912] - INFO - This is windows server! [2023-07-22 02:13:42,786] - [remote_util] [140279370630912] - INFO - File /cygdrive/c/tmp/windows_info.txt was found [2023-07-22 02:13:43,085] - [remote_util] [140279370630912] - INFO - get windows information [2023-07-22 02:13:43,164] - [remote_util] [140279370630912] - INFO - running command.raw on 172.23.138.141: echo "ipconfig" > /tmp/cmd.bat; chmod u=rwx /tmp/cmd.bat; /tmp/cmd.bat [2023-07-22 02:13:43,296] - [remote_util] [140279370630912] - INFO - command executed successfully with Administrator [2023-07-22 02:13:43,576] - [eventing_base] [140279370630912] - INFO - number of documents in metadata keyspace 220 [2023-07-22 02:13:43,585] - [remote_util] [140278179559168] - INFO - SSH Connecting to 172.23.138.141 with username:Administrator, attempt#1 of 5 [2023-07-22 02:13:43,588] - [remote_util] [140279370630912] - INFO - SSH Connecting to 172.23.138.138 with username:Administrator, attempt#1 of 5 [2023-07-22 02:13:43,596] - [remote_util] [140278162773760] - INFO - SSH Connecting to 172.23.138.137 with username:Administrator, attempt#1 of 5 [2023-07-22 02:13:43,598] - [remote_util] [140277784241920] - INFO - SSH Connecting to 172.23.138.143 with username:Administrator, attempt#1 of 5 [2023-07-22 02:13:43,603] - [remote_util] [140277775849216] - INFO - SSH Connecting to 172.23.138.139 with username:Administrator, attempt#1 of 5 [2023-07-22 02:13:43,608] - [remote_util] [140278149129984] - INFO - SSH Connecting to 172.23.138.142 with username:Administrator, attempt#1 of 5 [2023-07-22 02:13:43,615] - [remote_util] [140277759063808] - INFO - SSH Connecting to 172.23.138.140 with username:Administrator, attempt#1 of 5 [2023-07-22 02:13:43,704] - [remote_util] [140279370630912] - INFO - SSH Connected to 172.23.138.138 as Administrator [2023-07-22 02:13:43,709] - [remote_util] [140277784241920] - INFO - SSH Connected to 172.23.138.143 as Administrator [2023-07-22 02:13:43,712] - [remote_util] [140278162773760] - INFO - SSH Connected to 172.23.138.137 as Administrator [2023-07-22 02:13:43,717] - [remote_util] [140277775849216] - INFO - SSH Connected to 172.23.138.139 as Administrator [2023-07-22 02:13:43,732] - [remote_util] [140277759063808] - INFO - SSH Connected to 172.23.138.140 as Administrator [2023-07-22 02:13:43,734] - [remote_util] [140278149129984] - INFO - SSH Connected to 172.23.138.142 as Administrator [2023-07-22 02:13:43,792] - [remote_util] [140278179559168] - INFO - SSH Connected to 172.23.138.141 as Administrator [2023-07-22 02:13:44,143] - [remote_util] [140279370630912] - INFO - File /cygdrive/c/Windows/win.ini was found [2023-07-22 02:13:44,143] - [remote_util] [140279370630912] - INFO - This is windows server! [2023-07-22 02:13:44,173] - [remote_util] [140278162773760] - INFO - File /cygdrive/c/Windows/win.ini was found [2023-07-22 02:13:44,175] - [remote_util] [140278162773760] - INFO - This is windows server! [2023-07-22 02:13:44,180] - [remote_util] [140277784241920] - INFO - File /cygdrive/c/Windows/win.ini was found [2023-07-22 02:13:44,181] - [remote_util] [140277784241920] - INFO - This is windows server! [2023-07-22 02:13:44,195] - [remote_util] [140278149129984] - INFO - File /cygdrive/c/Windows/win.ini was found [2023-07-22 02:13:44,195] - [remote_util] [140278149129984] - INFO - This is windows server! [2023-07-22 02:13:44,205] - [remote_util] [140277775849216] - INFO - File /cygdrive/c/Windows/win.ini was found [2023-07-22 02:13:44,206] - [remote_util] [140277775849216] - INFO - This is windows server! [2023-07-22 02:13:44,207] - [remote_util] [140277759063808] - INFO - File /cygdrive/c/Windows/win.ini was found [2023-07-22 02:13:44,208] - [remote_util] [140277759063808] - INFO - This is windows server! [2023-07-22 02:13:44,211] - [remote_util] [140278179559168] - INFO - File /cygdrive/c/Windows/win.ini was found [2023-07-22 02:13:44,212] - [remote_util] [140278179559168] - INFO - This is windows server! [2023-07-22 02:13:44,257] - [remote_util] [140279370630912] - INFO - File /cygdrive/c/tmp/windows_info.txt was found [2023-07-22 02:13:44,295] - [remote_util] [140278162773760] - INFO - File /cygdrive/c/tmp/windows_info.txt was found [2023-07-22 02:13:44,321] - [remote_util] [140277784241920] - INFO - File /cygdrive/c/tmp/windows_info.txt was found [2023-07-22 02:13:44,334] - [remote_util] [140278149129984] - INFO - File /cygdrive/c/tmp/windows_info.txt was found [2023-07-22 02:13:44,340] - [remote_util] [140277775849216] - INFO - File /cygdrive/c/tmp/windows_info.txt was found [2023-07-22 02:13:44,344] - [remote_util] [140278179559168] - INFO - File /cygdrive/c/tmp/windows_info.txt was found [2023-07-22 02:13:44,346] - [remote_util] [140277759063808] - INFO - File /cygdrive/c/tmp/windows_info.txt was found [2023-07-22 02:13:44,348] - [remote_util] [140279370630912] - INFO - get windows information [2023-07-22 02:13:44,388] - [remote_util] [140278162773760] - INFO - get windows information [2023-07-22 02:13:44,416] - [remote_util] [140277784241920] - INFO - get windows information [2023-07-22 02:13:44,437] - [remote_util] [140278179559168] - INFO - get windows information [2023-07-22 02:13:44,441] - [remote_util] [140278149129984] - INFO - get windows information [2023-07-22 02:13:44,442] - [remote_util] [140277775849216] - INFO - get windows information [2023-07-22 02:13:44,451] - [remote_util] [140279370630912] - INFO - running command.raw on 172.23.138.138: echo "ipconfig" > /tmp/cmd.bat; chmod u=rwx /tmp/cmd.bat; /tmp/cmd.bat [2023-07-22 02:13:44,456] - [remote_util] [140277759063808] - INFO - get windows information [2023-07-22 02:13:44,509] - [remote_util] [140278162773760] - INFO - running command.raw on 172.23.138.137: echo "ipconfig" > /tmp/cmd.bat; chmod u=rwx /tmp/cmd.bat; /tmp/cmd.bat [2023-07-22 02:13:44,532] - [remote_util] [140278179559168] - INFO - running command.raw on 172.23.138.141: echo "ipconfig" > /tmp/cmd.bat; chmod u=rwx /tmp/cmd.bat; /tmp/cmd.bat [2023-07-22 02:13:44,541] - [remote_util] [140277784241920] - INFO - running command.raw on 172.23.138.143: echo "ipconfig" > /tmp/cmd.bat; chmod u=rwx /tmp/cmd.bat; /tmp/cmd.bat [2023-07-22 02:13:44,573] - [remote_util] [140277775849216] - INFO - running command.raw on 172.23.138.139: echo "ipconfig" > /tmp/cmd.bat; chmod u=rwx /tmp/cmd.bat; /tmp/cmd.bat [2023-07-22 02:13:44,592] - [remote_util] [140277759063808] - INFO - running command.raw on 172.23.138.140: echo "ipconfig" > /tmp/cmd.bat; chmod u=rwx /tmp/cmd.bat; /tmp/cmd.bat [2023-07-22 02:13:44,595] - [remote_util] [140279370630912] - INFO - command executed successfully with Administrator [2023-07-22 02:13:44,598] - [remote_util] [140279370630912] - INFO - running command.raw on 172.23.138.138: /cygdrive/c/Program\ Files/Couchbase/Server/bin/cbcollect_info.exe 172.23.138.138-20230722-0213-diag.zip [2023-07-22 02:13:44,646] - [remote_util] [140278162773760] - INFO - command executed successfully with Administrator [2023-07-22 02:13:44,647] - [remote_util] [140278162773760] - INFO - running command.raw on 172.23.138.137: /cygdrive/c/Program\ Files/Couchbase/Server/bin/cbcollect_info.exe 172.23.138.137-20230722-0213-diag.zip [2023-07-22 02:13:44,664] - [remote_util] [140278179559168] - INFO - command executed successfully with Administrator [2023-07-22 02:13:44,665] - [remote_util] [140278179559168] - INFO - running command.raw on 172.23.138.141: /cygdrive/c/Program\ Files/Couchbase/Server/bin/cbcollect_info.exe 172.23.138.141-20230722-0213-diag.zip [2023-07-22 02:13:44,677] - [remote_util] [140278149129984] - INFO - running command.raw on 172.23.138.142: echo "ipconfig" > /tmp/cmd.bat; chmod u=rwx /tmp/cmd.bat; /tmp/cmd.bat [2023-07-22 02:13:44,683] - [remote_util] [140277784241920] - INFO - command executed successfully with Administrator [2023-07-22 02:13:44,684] - [remote_util] [140277784241920] - INFO - running command.raw on 172.23.138.143: /cygdrive/c/Program\ Files/Couchbase/Server/bin/cbcollect_info.exe 172.23.138.143-20230722-0213-diag.zip [2023-07-22 02:13:44,734] - [remote_util] [140277775849216] - INFO - command executed successfully with Administrator [2023-07-22 02:13:44,735] - [remote_util] [140277775849216] - INFO - running command.raw on 172.23.138.139: /cygdrive/c/Program\ Files/Couchbase/Server/bin/cbcollect_info.exe 172.23.138.139-20230722-0213-diag.zip [2023-07-22 02:13:44,819] - [remote_util] [140278149129984] - INFO - command executed successfully with Administrator [2023-07-22 02:13:44,820] - [remote_util] [140278149129984] - INFO - running command.raw on 172.23.138.142: /cygdrive/c/Program\ Files/Couchbase/Server/bin/cbcollect_info.exe 172.23.138.142-20230722-0213-diag.zip [2023-07-22 02:13:44,852] - [remote_util] [140277759063808] - INFO - command executed successfully with Administrator [2023-07-22 02:13:44,855] - [remote_util] [140277759063808] - INFO - running command.raw on 172.23.138.140: /cygdrive/c/Program\ Files/Couchbase/Server/bin/cbcollect_info.exe 172.23.138.140-20230722-0213-diag.zip [2023-07-22 02:15:58,634] - [remote_util] [140277775849216] - INFO - command executed successfully with Administrator [2023-07-22 02:15:59,384] - [remote_util] [140277775849216] - INFO - running command.raw on 172.23.138.139: rm -f /home/Administrator/172.23.138.139-20230722-0213-diag.zip [2023-07-22 02:15:59,466] - [remote_util] [140277775849216] - INFO - command executed successfully with Administrator [2023-07-22 02:16:00,079] - [remote_util] [140277759063808] - INFO - command executed successfully with Administrator [2023-07-22 02:16:00,719] - [remote_util] [140277759063808] - INFO - running command.raw on 172.23.138.140: rm -f /home/Administrator/172.23.138.140-20230722-0213-diag.zip [2023-07-22 02:16:00,893] - [remote_util] [140277759063808] - INFO - command executed successfully with Administrator [2023-07-22 02:16:12,261] - [remote_util] [140278162773760] - INFO - command executed successfully with Administrator [2023-07-22 02:16:12,935] - [remote_util] [140278162773760] - INFO - running command.raw on 172.23.138.137: rm -f /home/Administrator/172.23.138.137-20230722-0213-diag.zip [2023-07-22 02:16:13,002] - [remote_util] [140278162773760] - INFO - command executed successfully with Administrator [2023-07-22 02:16:14,072] - [remote_util] [140278149129984] - INFO - command executed successfully with Administrator [2023-07-22 02:16:14,545] - [remote_util] [140277784241920] - INFO - command executed successfully with Administrator [2023-07-22 02:16:14,731] - [remote_util] [140278149129984] - INFO - running command.raw on 172.23.138.142: rm -f /home/Administrator/172.23.138.142-20230722-0213-diag.zip [2023-07-22 02:16:14,805] - [remote_util] [140278149129984] - INFO - command executed successfully with Administrator [2023-07-22 02:16:15,215] - [remote_util] [140277784241920] - INFO - running command.raw on 172.23.138.143: rm -f /home/Administrator/172.23.138.143-20230722-0213-diag.zip [2023-07-22 02:16:15,287] - [remote_util] [140277784241920] - INFO - command executed successfully with Administrator [2023-07-22 02:17:38,010] - [remote_util] [140278179559168] - INFO - command executed successfully with Administrator [2023-07-22 02:17:38,854] - [remote_util] [140278179559168] - INFO - running command.raw on 172.23.138.141: rm -f /home/Administrator/172.23.138.141-20230722-0213-diag.zip [2023-07-22 02:17:38,921] - [remote_util] [140278179559168] - INFO - command executed successfully with Administrator [2023-07-22 02:17:44,316] - [remote_util] [140279370630912] - INFO - command executed successfully with Administrator [2023-07-22 02:17:45,254] - [remote_util] [140279370630912] - INFO - running command.raw on 172.23.138.138: rm -f /home/Administrator/172.23.138.138-20230722-0213-diag.zip [2023-07-22 02:17:45,323] - [remote_util] [140279370630912] - INFO - command executed successfully with Administrator