Details
-
Bug
-
Resolution: Not a Bug
-
Major
-
Columnar 1.0.0
-
1.0.0-2237
-
Untriaged
-
0
-
Unknown
-
Analytics Sprint 47
Description
There was no functional impact. But there are exceptions seen with this message a few times -
Seen on 002
org.apache.asterix.common.exceptions.RuntimeDataException: ASX0023: 600s passed before getting back the responses from NCs
|
at com.couchbase.analytics.adapter.CouchbaseConnectorFactory.waitForShadowStateFutures(CouchbaseConnectorFactory.java:550) ~[columnar-connector.jar:1.0.0-2237] |
at com.couchbase.analytics.adapter.CouchbaseConnectorFactory.calculateStartingPoint(CouchbaseConnectorFactory.java:487) ~[columnar-connector.jar:1.0.0-2237] |
at com.couchbase.analytics.adapter.CouchbaseConnectorFactory.lambda$startCalculateStartingPointTask$5(CouchbaseConnectorFactory.java:472) ~[columnar-connector.jar:1.0.0-2237] |
at com.couchbase.analytics.util.BlockingUnwindFuture.lambda$submit$1(BlockingUnwindFuture.java:50) ~[columnar-common.jar:1.0.0-2237] |
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) [?:?] |
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) [?:?] |
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) [?:?] |
at java.base/java.lang.Thread.run(Thread.java:840) [?:?] |
2024-07-25T06:44:21.607+00:00 WARN CBAS.lang.ConnectLinkStatement [Rebalancer (d3420e1d3c7b9560b0ab0d078b75deb8)] Failed to connect bucket { "link" : "linkKepNyACL", "bucket" : "default1", "uuid" : "eb59ce96235781884156270fe08e897f", "running" : true } |
org.apache.asterix.common.exceptions.RuntimeDataException: ASX0023: 600s passed before getting back the responses from NCs
|
at com.couchbase.analytics.adapter.CouchbaseConnectorFactory.waitForShadowStateFutures(CouchbaseConnectorFactory.java:550) ~[columnar-connector.jar:1.0.0-2237] |
at com.couchbase.analytics.adapter.CouchbaseConnectorFactory.calculateStartingPoint(CouchbaseConnectorFactory.java:487) ~[columnar-connector.jar:1.0.0-2237] |
at com.couchbase.analytics.adapter.CouchbaseConnectorFactory.lambda$startCalculateStartingPointTask$5(CouchbaseConnectorFactory.java:472) ~[columnar-connector.jar:1.0.0-2237] |
at com.couchbase.analytics.util.BlockingUnwindFuture.lambda$submit$1(BlockingUnwindFuture.java:50) ~[columnar-common.jar:1.0.0-2237] |
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) [?:?] |
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) [?:?] |
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) [?:?] |
at java.base/java.lang.Thread.run(Thread.java:840) [?:?] |
2024-07-25T06:44:21.608+00:00 WARN CBAS.active.RecoveryTask [Rebalancer (d3420e1d3c7b9560b0ab0d078b75deb8)] Attempt to resume linkKepNyACL/default1 Failed |
org.apache.asterix.common.exceptions.RuntimeDataException: ASX0023: 600s passed before getting back the responses from NCs
|
at com.couchbase.analytics.adapter.CouchbaseConnectorFactory.waitForShadowStateFutures(CouchbaseConnectorFactory.java:550) ~[columnar-connector.jar:1.0.0-2237] |
at com.couchbase.analytics.adapter.CouchbaseConnectorFactory.calculateStartingPoint(CouchbaseConnectorFactory.java:487) ~[columnar-connector.jar:1.0.0-2237] |
at com.couchbase.analytics.adapter.CouchbaseConnectorFactory.lambda$startCalculateStartingPointTask$5(CouchbaseConnectorFactory.java:472) ~[columnar-connector.jar:1.0.0-2237] |
at com.couchbase.analytics.util.BlockingUnwindFuture.lambda$submit$1(BlockingUnwindFuture.java:50) ~[columnar-common.jar:1.0.0-2237] |
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) [?:?] |
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) [?:?] |
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) [?:?] |
at java.base/java.lang.Thread.run(Thread.java:840) [?:?] |
This appears to be during the rebalance stage ( 4 nodes to 8 nodes).
Rebalance report
"rebalanceId":"99283526e4e397d2df15a05a68a44dcf" |
"startTime":"2024-07-25T06:22:25.595Z", |
"completedTime":"2024-07-25T06:54:22.271Z" |
Not sure if these errors are expected.
cbcollect ->