Details
-
Bug
-
Resolution: Not a Bug
-
Critical
-
6.5.0
-
Untriaged
-
-
Unknown
-
CX Sprint 155
Description
Build : 6.5.0-3506
Test : -test tests/analytics/test_analytics_rebalance.yml -scope tests/analytics/scope_analytics_rebalance.yml
Scale : 4
Iteration : All
In the system test, every time the step to disconnect link Local is executed, analytics service crashes after the statement completes execution.
Here is an excerpt from the analytics_error log from one of such crashes on 172.23.96.148:
2019-06-14T11:57:17.334-07:00 FATA CBAS.runtime.DcpUpdateCallback [org.apache.hyracks.api.rewriter.runtime.SuperActivity:JID:3.50729:TAID:TID:ANID:ODID:2:0:7:0:0] Restarting process to ensure data integrity
|
org.apache.hyracks.api.exceptions.HyracksDataException: java.lang.InterruptedException
|
at org.apache.hyracks.api.exceptions.HyracksDataException.create(HyracksDataException.java:51) ~[hyracks-api.jar:6.5.0-3506]
|
at org.apache.hyracks.control.nc.io.IoRequest.queue(IoRequest.java:105) ~[hyracks-control-nc.jar:6.5.0-3506]
|
at org.apache.hyracks.control.nc.io.IoRequest.read(IoRequest.java:76) ~[hyracks-control-nc.jar:6.5.0-3506]
|
at org.apache.hyracks.control.nc.io.IOManager.asyncRead(IOManager.java:318) ~[hyracks-control-nc.jar:6.5.0-3506]
|
at org.apache.hyracks.control.nc.io.IOManager.syncRead(IOManager.java:249) ~[hyracks-control-nc.jar:6.5.0-3506]
|
at org.apache.hyracks.storage.common.buffercache.AbstractBufferedFileIOManager.readToBuffer(AbstractBufferedFileIOManager.java:250) ~[hyracks-storage-common.jar:6.5.0-3506]
|
at org.apache.hyracks.storage.common.file.BufferedFileHandle.read(BufferedFileHandle.java:75) ~[hyracks-storage-common.jar:6.5.0-3506]
|
at org.apache.hyracks.storage.common.buffercache.BufferCache.read(BufferCache.java:549) ~[hyracks-storage-common.jar:6.5.0-3506]
|
at org.apache.hyracks.storage.common.buffercache.BufferCache.tryRead(BufferCache.java:522) ~[hyracks-storage-common.jar:6.5.0-3506]
|
at org.apache.hyracks.storage.common.buffercache.BufferCache.pin(BufferCache.java:192) ~[hyracks-storage-common.jar:6.5.0-3506]
|
at org.apache.hyracks.storage.am.btree.impls.DiskBTree.searchDown(DiskBTree.java:170) ~[hyracks-storage-am-btree.jar:6.5.0-3506]
|
at org.apache.hyracks.storage.am.btree.impls.DiskBTree.search(DiskBTree.java:95) ~[hyracks-storage-am-btree.jar:6.5.0-3506]
|
at org.apache.hyracks.storage.am.btree.impls.DiskBTree.access$000(DiskBTree.java:44) ~[hyracks-storage-am-btree.jar:6.5.0-3506]
|
at org.apache.hyracks.storage.am.btree.impls.DiskBTree$DiskBTreeAccessor.search(DiskBTree.java:243) ~[hyracks-storage-am-btree.jar:6.5.0-3506]
|
at org.apache.hyracks.storage.am.lsm.btree.impls.LSMBTreePointSearchCursor.doHasNext(LSMBTreePointSearchCursor.java:82) ~[hyracks-storage-am-lsm-btree.jar:6.5.0-3506]
|
at org.apache.hyracks.storage.common.EnforcedIndexCursor.hasNext(EnforcedIndexCursor.java:69) ~[hyracks-storage-common.jar:6.5.0-3506]
|
at org.apache.hyracks.storage.am.lsm.btree.impls.LSMBTreeSearchCursor.doHasNext(LSMBTreeSearchCursor.java:60) ~[hyracks-storage-am-lsm-btree.jar:6.5.0-3506]
|
at org.apache.hyracks.storage.common.EnforcedIndexCursor.hasNext(EnforcedIndexCursor.java:69) ~[hyracks-storage-common.jar:6.5.0-3506]
|
at org.apache.asterix.runtime.operators.LSMPrimaryUpsertOperatorNodePushable$1.process(LSMPrimaryUpsertOperatorNodePushable.java:159) [asterix-runtime.jar:6.5.0-3506]
|
at org.apache.hyracks.storage.am.lsm.common.impls.LSMHarness.processFrame(LSMHarness.java:854) ~[hyracks-storage-am-lsm-common.jar:6.5.0-3506]
|
at org.apache.hyracks.storage.am.lsm.common.impls.LSMHarness.batchOperate(LSMHarness.java:705) [hyracks-storage-am-lsm-common.jar:6.5.0-3506]
|
at org.apache.hyracks.storage.am.lsm.common.impls.LSMTreeIndexAccessor.batchOperate(LSMTreeIndexAccessor.java:214) [hyracks-storage-am-lsm-common.jar:6.5.0-3506]
|
at org.apache.asterix.runtime.operators.LSMPrimaryUpsertOperatorNodePushable.nextFrame(LSMPrimaryUpsertOperatorNodePushable.java:323) [asterix-runtime.jar:6.5.0-3506]
|
at org.apache.asterix.external.feed.dataflow.SyncFeedRuntimeInputHandler.nextFrame(SyncFeedRuntimeInputHandler.java:46) [asterix-external-data.jar:6.5.0-3506]
|
at org.apache.asterix.external.operators.FeedMetaStoreNodePushable.nextFrame(FeedMetaStoreNodePushable.java:151) [asterix-external-data.jar:6.5.0-3506]
|
at org.apache.hyracks.control.nc.Task.pushFrames(Task.java:396) [hyracks-control-nc.jar:6.5.0-3506]
|
at org.apache.hyracks.control.nc.Task.run(Task.java:330) [hyracks-control-nc.jar:6.5.0-3506]
|
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
|
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
|
at java.lang.Thread.run(Thread.java:834) [?:?]
|
Caused by: java.lang.InterruptedException
|
at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(AbstractQueuedSynchronizer.java:1261) ~[?:?]
|
at java.util.concurrent.locks.ReentrantLock.lockInterruptibly(ReentrantLock.java:317) ~[?:?]
|
at java.util.concurrent.ArrayBlockingQueue.put(ArrayBlockingQueue.java:364) ~[?:?]
|
at org.apache.hyracks.control.nc.io.IoRequest.queue(IoRequest.java:103) ~[hyracks-control-nc.jar:6.5.0-3506]
|
... 28 more
|
2019-06-14T11:57:17.318-07:00 FATA CBAS.util.ExitUtil [org.apache.hyracks.api.rewriter.runtime.SuperActivity:JID:3.50729:TAID:TID:ANID:ODID:2:0:5:0:0] JVM halting with status 103; thread dump at halt:
|
