2022-02-15T20:22:36.197-08:00 INFO ClusterTestBase [main] configured sslWorkDir to /home/couchbase/jenkins/workspace/cbas-cbcluster-docker-test/analytics/cbas/cbas-test/cbas-docker-test/target/sslWork
|
2022-02-15T20:22:36.241-08:00 INFO DockerTestBase [main] resolved dockerHostAddr using 'host.docker.internal': 172.17.0.1
|
2022-02-15T20:22:36.241-08:00 INFO DockerTestBase [main] Docker image id=build-docker.couchbase.com/couchbase/server-toy:analytics-jenkins-enterprise-7.1.0-1708340308 dockerHostAddr=172.17.0.1 dockerContainerId=9b20e912998c0e16489be9c1d8cfba83eefc2ed32e87578904a401097eefe1ea
|
2022-02-15T20:22:36.275-08:00 WARN DockerClientBuilder [main] 'dockerHttpClient' should be set.Falling back to Jersey, will be an error in future releases.
|
2022-02-15T20:22:37.206-08:00 WARN DockerClientBuilder [main] 'dockerHttpClient' should be set.Falling back to Jersey, will be an error in future releases.
|
2022-02-15T20:22:37.363-08:00 INFO DockerTestBase [main] Executing the command: docker pull build-docker.couchbase.com/couchbase/server-toy:analytics-jenkins-enterprise-7.1.0-1708340308
|
2022-02-15T20:22:37.430-08:00 INFO DockerTestBase [main+] >> analytics-jenkins-enterprise-7.1.0-1708340308: Pulling from couchbase/server-toy
|
2022-02-15T20:22:37.431-08:00 INFO DockerTestBase [main+] >> Digest: sha256:041fb42be9d199cb7ae59b7d009c49dce0062d302f6e89a811cb2e1e52ff8ce6
|
2022-02-15T20:22:37.432-08:00 INFO DockerTestBase [main+] >> Status: Image is up to date for build-docker.couchbase.com/couchbase/server-toy:analytics-jenkins-enterprise-7.1.0-1708340308
|
2022-02-15T20:22:37.432-08:00 INFO DockerTestBase [main+] >> build-docker.couchbase.com/couchbase/server-toy:analytics-jenkins-enterprise-7.1.0-1708340308
|
2022-02-15T20:22:37.442-08:00 WARN TestMethodTracer [main] ### testQuorumFailover START
|
2022-02-15T20:22:37.444-08:00 WARN DockerClientBuilder [main] 'dockerHttpClient' should be set.Falling back to Jersey, will be an error in future releases.
|
2022-02-15T20:22:37.481-08:00 INFO DockerTestBase [main] Creating docker network (name = cbas-docker-test-0001-067a117a-a50f-4e8b-9b70-ef31ffd28bec)
|
2022-02-15T20:22:37.563-08:00 INFO DockerTestBase [main] Created docker network: { name = cbas-docker-test-0001-067a117a-a50f-4e8b-9b70-ef31ffd28bec, id = 36c3e9364ce683091dff2988340437a2f0e6267642602b053f36cdc5bd4006cc }
|
2022-02-15T20:22:37.567-08:00 INFO UpgradeTestBase [pool-0] Creating n1 node using 'this build' docker image...
|
2022-02-15T20:22:37.567-08:00 INFO UpgradeTestBase [pool-2] Creating n3 node using 'this build' docker image...
|
2022-02-15T20:22:37.573-08:00 INFO UpgradeTestBase [pool-4] Creating n4 node using 'this build' docker image...
|
2022-02-15T20:22:37.576-08:00 INFO UpgradeTestBase [pool-2] binding /home/couchbase/jenkins/workspace/cbas-cbcluster-docker-test/analytics/cbas/cbas-test/cbas-docker-test/target/binds/0002n3 to /opt/couchbase/var
|
2022-02-15T20:22:37.573-08:00 INFO UpgradeTestBase [pool-1] Creating n2 node using 'this build' docker image...
|
2022-02-15T20:22:37.576-08:00 INFO UpgradeTestBase [pool-0] binding /home/couchbase/jenkins/workspace/cbas-cbcluster-docker-test/analytics/cbas/cbas-test/cbas-docker-test/target/binds/0001n1 to /opt/couchbase/var
|
2022-02-15T20:22:37.577-08:00 INFO UpgradeTestBase [pool-1] binding /home/couchbase/jenkins/workspace/cbas-cbcluster-docker-test/analytics/cbas/cbas-test/cbas-docker-test/target/binds/0003n2 to /opt/couchbase/var
|
2022-02-15T20:22:37.574-08:00 INFO UpgradeTestBase [pool-3] Creating n5 node using 'this build' docker image...
|
2022-02-15T20:22:37.578-08:00 INFO UpgradeTestBase [pool-3] binding /home/couchbase/jenkins/workspace/cbas-cbcluster-docker-test/analytics/cbas/cbas-test/cbas-docker-test/target/binds/0004n5 to /opt/couchbase/var
|
2022-02-15T20:22:37.582-08:00 INFO DockerTestBase [pool-0] Creating docker container (test dir = /home/couchbase/jenkins/workspace/cbas-cbcluster-docker-test/analytics/cbas/cbas-test/cbas-docker-test/target/com.couchbase.analytics.test.docker.lifecycle.QuorumFailoverITD/testQuorumFailover/0001_n1_thisbuild, network = { name = cbas-docker-test-0001-067a117a-a50f-4e8b-9b70-ef31ffd28bec, id = 36c3e9364ce683091dff2988340437a2f0e6267642602b053f36cdc5bd4006cc })
|
2022-02-15T20:22:37.582-08:00 INFO DockerTestBase [pool-1] Creating docker container (test dir = /home/couchbase/jenkins/workspace/cbas-cbcluster-docker-test/analytics/cbas/cbas-test/cbas-docker-test/target/com.couchbase.analytics.test.docker.lifecycle.QuorumFailoverITD/testQuorumFailover/0002_n2_thisbuild, network = { name = cbas-docker-test-0001-067a117a-a50f-4e8b-9b70-ef31ffd28bec, id = 36c3e9364ce683091dff2988340437a2f0e6267642602b053f36cdc5bd4006cc })
|
2022-02-15T20:22:37.583-08:00 INFO UpgradeTestBase [pool-4] binding /home/couchbase/jenkins/workspace/cbas-cbcluster-docker-test/analytics/cbas/cbas-test/cbas-docker-test/target/binds/0005n4 to /opt/couchbase/var
|
2022-02-15T20:22:37.583-08:00 INFO DockerTestBase [pool-2] Creating docker container (test dir = /home/couchbase/jenkins/workspace/cbas-cbcluster-docker-test/analytics/cbas/cbas-test/cbas-docker-test/target/com.couchbase.analytics.test.docker.lifecycle.QuorumFailoverITD/testQuorumFailover/0003_n3_thisbuild, network = { name = cbas-docker-test-0001-067a117a-a50f-4e8b-9b70-ef31ffd28bec, id = 36c3e9364ce683091dff2988340437a2f0e6267642602b053f36cdc5bd4006cc })
|
2022-02-15T20:22:37.583-08:00 INFO DockerTestBase [pool-4] Creating docker container (test dir = /home/couchbase/jenkins/workspace/cbas-cbcluster-docker-test/analytics/cbas/cbas-test/cbas-docker-test/target/com.couchbase.analytics.test.docker.lifecycle.QuorumFailoverITD/testQuorumFailover/0004_n4_thisbuild, network = { name = cbas-docker-test-0001-067a117a-a50f-4e8b-9b70-ef31ffd28bec, id = 36c3e9364ce683091dff2988340437a2f0e6267642602b053f36cdc5bd4006cc })
|
2022-02-15T20:22:37.588-08:00 INFO DockerTestBase [pool-3] Creating docker container (test dir = /home/couchbase/jenkins/workspace/cbas-cbcluster-docker-test/analytics/cbas/cbas-test/cbas-docker-test/target/com.couchbase.analytics.test.docker.lifecycle.QuorumFailoverITD/testQuorumFailover/0005_n5_thisbuild, network = { name = cbas-docker-test-0001-067a117a-a50f-4e8b-9b70-ef31ffd28bec, id = 36c3e9364ce683091dff2988340437a2f0e6267642602b053f36cdc5bd4006cc })
|
2022-02-15T20:22:37.618-08:00 INFO DockerTestBase [pool-0] binding exposed port n1:8091/tcp -> 172.17.0.1:34673/tcp
|
2022-02-15T20:22:37.618-08:00 INFO DockerTestBase [pool-3] binding exposed port n5:8091/tcp -> 172.17.0.1:34231/tcp
|
2022-02-15T20:22:37.618-08:00 INFO DockerTestBase [pool-1] binding exposed port n2:8091/tcp -> 172.17.0.1:42871/tcp
|
2022-02-15T20:22:37.618-08:00 INFO DockerTestBase [pool-2] binding exposed port n3:8091/tcp -> 172.17.0.1:46321/tcp
|
2022-02-15T20:22:37.618-08:00 INFO DockerTestBase [pool-3] binding exposed port n5:8095/tcp -> 172.17.0.1:45329/tcp
|
2022-02-15T20:22:37.619-08:00 INFO DockerTestBase [pool-2] binding exposed port n3:8095/tcp -> 172.17.0.1:45301/tcp
|
2022-02-15T20:22:37.618-08:00 INFO DockerTestBase [pool-0] binding exposed port n1:8095/tcp -> 172.17.0.1:34901/tcp
|
2022-02-15T20:22:37.619-08:00 INFO DockerTestBase [pool-3] binding exposed port n5:11210/tcp -> 172.17.0.1:33605/tcp
|
2022-02-15T20:22:37.619-08:00 INFO DockerTestBase [pool-2] binding exposed port n3:11210/tcp -> 172.17.0.1:36447/tcp
|
2022-02-15T20:22:37.619-08:00 INFO DockerTestBase [pool-0] binding exposed port n1:11210/tcp -> 172.17.0.1:34151/tcp
|
2022-02-15T20:22:37.621-08:00 INFO DockerTestBase [pool-4] binding exposed port n4:8091/tcp -> 172.17.0.1:40585/tcp
|
2022-02-15T20:22:37.622-08:00 INFO DockerTestBase [pool-4] binding exposed port n4:8095/tcp -> 172.17.0.1:39937/tcp
|
2022-02-15T20:22:37.622-08:00 INFO DockerTestBase [pool-1] binding exposed port n2:8095/tcp -> 172.17.0.1:41609/tcp
|
2022-02-15T20:22:37.623-08:00 INFO DockerTestBase [pool-1] binding exposed port n2:11210/tcp -> 172.17.0.1:36501/tcp
|
2022-02-15T20:22:37.623-08:00 INFO DockerTestBase [pool-4] binding exposed port n4:11210/tcp -> 172.17.0.1:39369/tcp
|
2022-02-15T20:22:37.766-08:00 INFO DockerTestBase [pool-2] Starting docker container: 5a8ae6a3d52f7
|
2022-02-15T20:22:37.766-08:00 INFO DockerTestBase [pool-0] Starting docker container: 52a524a2312e4
|
2022-02-15T20:22:37.770-08:00 INFO DockerTestBase [pool-1] Starting docker container: d0d327f7e810e
|
2022-02-15T20:22:37.770-08:00 INFO DockerTestBase [pool-4] Starting docker container: debfd4cd1d830
|
2022-02-15T20:22:37.770-08:00 INFO DockerTestBase [pool-3] Starting docker container: 6dd8ae5125a57
|
2022-02-15T20:22:39.703-08:00 INFO DockerTestBase [pool-0] Started docker container: 52a524a2312e4
|
2022-02-15T20:22:39.836-08:00 INFO DockerTestBase [pool-0] Container information for 52a524a2312e4: host: n1.couchbase.host, image: analytics-jenkins-enterprise-7.1.0-1708340308, ip: 192.168.176.2, 172.17.0.1:34901->192.168.176.2:8095, 172.17.0.1:34151->192.168.176.2:11210, 172.17.0.1:34673->192.168.176.2:8091
|
2022-02-15T20:22:40.197-08:00 INFO DockerTestBase [pool-2] Started docker container: 5a8ae6a3d52f7
|
2022-02-15T20:22:40.206-08:00 INFO DockerTestBase [pool-2] Container information for 5a8ae6a3d52f7: host: n3.couchbase.host, image: analytics-jenkins-enterprise-7.1.0-1708340308, ip: 192.168.176.4, 172.17.0.1:45301->192.168.176.4:8095, 172.17.0.1:36447->192.168.176.4:11210, 172.17.0.1:46321->192.168.176.4:8091
|
2022-02-15T20:22:40.798-08:00 INFO DockerTestBase [pool-3] Started docker container: 6dd8ae5125a57
|
2022-02-15T20:22:40.803-08:00 INFO DockerTestBase [pool-3] Container information for 6dd8ae5125a57: host: n5.couchbase.host, image: analytics-jenkins-enterprise-7.1.0-1708340308, ip: 192.168.176.3, 172.17.0.1:45329->192.168.176.3:8095, 172.17.0.1:33605->192.168.176.3:11210, 172.17.0.1:34231->192.168.176.3:8091
|
2022-02-15T20:22:41.328-08:00 INFO DockerTestBase [pool-1] Started docker container: d0d327f7e810e
|
2022-02-15T20:22:41.341-08:00 INFO DockerTestBase [pool-1] Container information for d0d327f7e810e: host: n2.couchbase.host, image: analytics-jenkins-enterprise-7.1.0-1708340308, ip: 192.168.176.5, 172.17.0.1:41609->192.168.176.5:8095, 172.17.0.1:36501->192.168.176.5:11210, 172.17.0.1:42871->192.168.176.5:8091
|
2022-02-15T20:22:42.444-08:00 INFO DockerTestBase [pool-4] Started docker container: debfd4cd1d830
|
2022-02-15T20:22:42.451-08:00 INFO DockerTestBase [pool-4] Container information for debfd4cd1d830: host: n4.couchbase.host, image: analytics-jenkins-enterprise-7.1.0-1708340308, ip: 192.168.176.6, 172.17.0.1:39937->192.168.176.6:8095, 172.17.0.1:39369->192.168.176.6:11210, 172.17.0.1:40585->192.168.176.6:8091
|
2022-02-15T20:22:51.858-08:00 INFO DockerTestBase [pool-0] Running 'node-init' on node n1 (192.168.176.2)
|
2022-02-15T20:22:51.859-08:00 INFO DockerTestBase [pool-0] Executing the command: docker exec n1.couchbase.host /opt/couchbase/bin/couchbase-cli node-init -c 192.168.176.2:8091 -u couchbase -p couchbase --node-init-hostname n1.couchbase.host
|
2022-02-15T20:22:52.366-08:00 INFO DockerTestBase [pool-2] Running 'node-init' on node n3 (192.168.176.4)
|
2022-02-15T20:22:52.366-08:00 INFO DockerTestBase [pool-2] Executing the command: docker exec n3.couchbase.host /opt/couchbase/bin/couchbase-cli node-init -c 192.168.176.4:8091 -u couchbase -p couchbase --node-init-hostname n3.couchbase.host
|
2022-02-15T20:22:52.997-08:00 INFO DockerTestBase [pool-3] Running 'node-init' on node n5 (192.168.176.3)
|
2022-02-15T20:22:52.998-08:00 INFO DockerTestBase [pool-3] Executing the command: docker exec n5.couchbase.host /opt/couchbase/bin/couchbase-cli node-init -c 192.168.176.3:8091 -u couchbase -p couchbase --node-init-hostname n5.couchbase.host
|
2022-02-15T20:22:53.844-08:00 INFO DockerTestBase [pool-1] Running 'node-init' on node n2 (192.168.176.5)
|
2022-02-15T20:22:53.844-08:00 INFO DockerTestBase [pool-1] Executing the command: docker exec n2.couchbase.host /opt/couchbase/bin/couchbase-cli node-init -c 192.168.176.5:8091 -u couchbase -p couchbase --node-init-hostname n2.couchbase.host
|
2022-02-15T20:22:54.148-08:00 INFO DockerTestBase [pool-0+] >> SUCCESS: Node initialized
|
2022-02-15T20:22:54.258-08:00 INFO DockerTestBase [pool-0] Running 'cluster-init' on node n1 (192.168.176.2) with services: [DATA]
|
2022-02-15T20:22:54.258-08:00 INFO DockerTestBase [pool-0] Executing the command: docker exec n1.couchbase.host /opt/couchbase/bin/couchbase-cli cluster-init -c 127.0.0.1:8091 --cluster-username couchbase --cluster-password couchbase --cluster-ramsize 512 --cluster-analytics-ramsize 1024 --services data
|
2022-02-15T20:22:54.756-08:00 INFO DockerTestBase [pool-2+] >> SUCCESS: Node initialized
|
2022-02-15T20:22:55.103-08:00 INFO DockerTestBase [pool-0+] >> SUCCESS: Cluster initialized
|
2022-02-15T20:22:55.629-08:00 INFO DockerTestBase [pool-4] Running 'node-init' on node n4 (192.168.176.6)
|
2022-02-15T20:22:55.630-08:00 INFO DockerTestBase [pool-4] Executing the command: docker exec n4.couchbase.host /opt/couchbase/bin/couchbase-cli node-init -c 192.168.176.6:8091 -u couchbase -p couchbase --node-init-hostname n4.couchbase.host
|
2022-02-15T20:23:02.244-08:00 INFO DockerTestBase [pool-3+] >> SUCCESS: Node initialized
|
2022-02-15T20:23:02.807-08:00 INFO DockerTestBase [pool-1+] >> SUCCESS: Node initialized
|
2022-02-15T20:23:04.246-08:00 INFO DockerTestBase [pool-4+] >> SUCCESS: Node initialized
|
2022-02-15T20:23:04.292-08:00 INFO DockerTestBase [pool-1] Running 'server-add' for node n3 (192.168.176.4) with services: [DATA]
|
2022-02-15T20:23:04.293-08:00 INFO DockerTestBase [pool-3] Running 'server-add' for node n5 (192.168.176.3) with services: [DATA]
|
2022-02-15T20:23:04.292-08:00 INFO DockerTestBase [pool-9] Running 'server-add' for node n4 (192.168.176.6) with services: [DATA, ANALYTICS]
|
2022-02-15T20:23:04.293-08:00 INFO DockerTestBase [pool-3] Executing the command: docker exec n1.couchbase.host /opt/couchbase/bin/couchbase-cli server-add --no-ssl-verify -c https://192.168.176.2:18091 -u couchbase -p couchbase --server-add https://n5.couchbase.host:18091 --server-add-username couchbase --server-add-password couchbase --services data
|
2022-02-15T20:23:04.293-08:00 INFO DockerTestBase [pool-9] Executing the command: docker exec n1.couchbase.host /opt/couchbase/bin/couchbase-cli server-add --no-ssl-verify -c https://192.168.176.2:18091 -u couchbase -p couchbase --server-add https://n4.couchbase.host:18091 --server-add-username couchbase --server-add-password couchbase --services data,analytics
|
2022-02-15T20:23:04.292-08:00 INFO DockerTestBase [pool-4] Running 'server-add' for node n2 (192.168.176.5) with services: [DATA]
|
2022-02-15T20:23:04.294-08:00 INFO DockerTestBase [pool-4] Executing the command: docker exec n1.couchbase.host /opt/couchbase/bin/couchbase-cli server-add --no-ssl-verify -c https://192.168.176.2:18091 -u couchbase -p couchbase --server-add https://n2.couchbase.host:18091 --server-add-username couchbase --server-add-password couchbase --services data
|
2022-02-15T20:23:04.293-08:00 INFO DockerTestBase [pool-1] Executing the command: docker exec n1.couchbase.host /opt/couchbase/bin/couchbase-cli server-add --no-ssl-verify -c https://192.168.176.2:18091 -u couchbase -p couchbase --server-add https://n3.couchbase.host:18091 --server-add-username couchbase --server-add-password couchbase --services data
|
2022-02-15T20:23:08.382-08:00 INFO DockerTestBase [pool-1+] >> SUCCESS: Server added
|
2022-02-15T20:23:11.777-08:00 INFO DockerTestBase [pool-3+] >> SUCCESS: Server added
|
2022-02-15T20:23:14.853-08:00 INFO DockerTestBase [pool-9+] >> SUCCESS: Server added
|
2022-02-15T20:23:17.979-08:00 INFO DockerTestBase [pool-4+] >> SUCCESS: Server added
|
2022-02-15T20:23:18.028-08:00 INFO UpgradeTestBase [main] Running 'rebalance'
|
2022-02-15T20:23:18.029-08:00 INFO DockerTestBase [main] Executing the command: docker exec n1.couchbase.host /opt/couchbase/bin/couchbase-cli rebalance -c 192.168.176.2:8091 -u couchbase -p couchbase
|
2022-02-15T20:23:19.566-08:00 INFO DockerTestBase [main+] >> Unable to display progress bar on this os
|
2022-02-15T20:23:19.566-08:00 INFO DockerTestBase [main+] >> SUCCESS: Rebalance complete
|
2022-02-15T20:23:19.626-08:00 INFO DockerTestBase [main] Waiting for 'rebalance completion' condition for up to 120 seconds...
|
2022-02-15T20:23:19.640-08:00 INFO DockerTestBase [pool-4] rebalance tasks complete
|
2022-02-15T20:23:19.641-08:00 INFO DockerTestBase [main] 'rebalance completion' condition satisfied
|
2022-02-15T20:23:19.641-08:00 INFO UpgradeTestBase [main] ... rebalance succeeded!
|
2022-02-15T20:23:19.652-08:00 INFO DockerTestBase [pool-4] Waiting for NCs == 1 & Cluster state in {ACTIVE} for up to 120s...
|
2022-02-15T20:23:19.698-08:00 INFO KvStoreHttpHelper [pool-9] Initializing KvStoreHttpHelper with nodes [/172.17.0.1:34673]
|
2022-02-15T20:23:19.702-08:00 INFO KvStoreHttpHelper [pool-9] installing sample(s): [travel-sample] on 172.17.0.1:34673
|
2022-02-15T20:23:20.809-08:00 ERRO asterix [pool-3-thread-1] Failure executing POST http://172.17.0.1:34901/analytics/service HTTP/1.1
|
org.apache.http.NoHttpResponseException: 172.17.0.1:34901 failed to respond
|
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:141) ~[httpclient-4.5.13.jar:4.5.13]
|
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) ~[httpclient-4.5.13.jar:4.5.13]
|
at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259) ~[httpcore-4.4.15.jar:4.4.15]
|
at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:163) ~[httpcore-4.4.15.jar:4.4.15]
|
at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:157) ~[httpclient-4.5.13.jar:4.5.13]
|
at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273) ~[httpcore-4.4.15.jar:4.4.15]
|
at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125) ~[httpcore-4.4.15.jar:4.4.15]
|
at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) ~[httpclient-4.5.13.jar:4.5.13]
|
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) ~[httpclient-4.5.13.jar:4.5.13]
|
at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) ~[httpclient-4.5.13.jar:4.5.13]
|
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) ~[httpclient-4.5.13.jar:4.5.13]
|
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) ~[httpclient-4.5.13.jar:4.5.13]
|
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) ~[httpclient-4.5.13.jar:4.5.13]
|
at org.apache.asterix.test.common.TestExecutor.lambda$executeHttpRequest$1(TestExecutor.java:680) ~[asterix-app-0.9.8-SNAPSHOT-tests.jar:0.9.8-SNAPSHOT]
|
at java.util.concurrent.FutureTask.run(FutureTask.java:264) [?:?]
|
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
|
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
|
at java.lang.Thread.run(Thread.java:834) [?:?]
|
Task
Major