Details
-
Bug
-
Resolution: Fixed
-
Critical
-
Columnar 1.0.0
-
Columnar Edition 1.0.0 build 2172
-
Untriaged
-
0
-
Unknown
-
Analytics Sprint 44, Analytics Sprint 45, Analytics Sprint 46
Description
It's a 4-node cluster (16 VCPUs + 64 GB).
Seen on node 002-
2024-06-25T07:00:53.660+00:00 INFO CBAS.adapter.TopicRecordReader [Executor-11080:366d39b76e6b79e287b27e683fc2a51d] Data Polling started... |
2024-06-25T07:00:53.660+00:00 INFO CBAS.adapter.TopicRecordReader [Executor-11075:366d39b76e6b79e287b27e683fc2a51d] Data Polling started... |
2024-06-25T07:00:53.660+00:00 INFO CBAS.adapter.TopicRecordReader [Executor-11081:366d39b76e6b79e287b27e683fc2a51d] Data Polling started... |
2024-06-25T07:00:53.660+00:00 INFO CBAS.adapter.TopicRecordReader [Executor-11073:366d39b76e6b79e287b27e683fc2a51d] Data Polling started... |
2024-06-25T07:00:53.660+00:00 INFO CBAS.adapter.TopicRecordReader [Executor-11078:366d39b76e6b79e287b27e683fc2a51d] Data Polling started... |
2024-06-25T07:00:53.661+00:00 INFO CBAS.adapter.TopicRecordReader [Executor-11079:366d39b76e6b79e287b27e683fc2a51d] Data Polling started... |
2024-06-25T07:00:53.661+00:00 INFO CBAS.adapter.TopicRecordReader [Executor-11074:366d39b76e6b79e287b27e683fc2a51d] Data Polling started... |
2024-06-25T07:00:53.661+00:00 INFO CBAS.adapter.TopicRecordReader [Executor-11076:366d39b76e6b79e287b27e683fc2a51d] Data Polling started... |
2024-06-25T07:00:55.198+00:00 INFO CBAS.bootstrap.ClusterMonitor [linkyPpdoUMQ cluster monitor] interrupted during pool stream; will retry |
2024-06-25T07:00:55.198+00:00 INFO CBAS.bootstrap.ClusterMonitor [linkEhWaXEIm cluster monitor] interrupted during pool stream; will retry |
2024-06-25T07:00:55.198+00:00 INFO CBAS.bootstrap.ClusterMonitor [linkLGikBFPE cluster monitor] interrupted during pool stream; will retry |
2024-06-25T07:00:57.306+00:00 INFO CBAS.context.DatasetLifecycleManager [Checkpoint Thread (178)] Async flushing Dataset (365), Partition (2) |
2024-06-25T07:00:57.306+00:00 INFO CBAS.context.DatasetLifecycleManager [Checkpoint Thread (178)] Async flushing Dataset (365), Partition (3) |
2024-06-25T07:00:57.306+00:00 INFO CBAS.context.DatasetLifecycleManager [Checkpoint Thread (178)] Async flushing Dataset (365), Partition (4) |
2024-06-25T07:00:57.306+00:00 INFO CBAS.context.DatasetLifecycleManager [Checkpoint Thread (178)] Async flushing Dataset (365), Partition (5) |
2024-06-25T07:00:57.306+00:00 INFO CBAS.context.DatasetLifecycleManager [Checkpoint Thread (178)] Async flushing Dataset (365), Partition (6) |
2024-06-25T07:00:57.306+00:00 INFO CBAS.context.DatasetLifecycleManager [Checkpoint Thread (178)] Async flushing Dataset (365), Partition (7) |
2024-06-25T07:00:57.306+00:00 INFO CBAS.context.DatasetLifecycleManager [Checkpoint Thread (178)] Async flushing Dataset (365), Partition (10) |
2024-06-25T07:00:57.306+00:00 INFO CBAS.context.DatasetLifecycleManager [Checkpoint Thread (178)] Async flushing Dataset (365), Partition (11) |
2024-06-25T07:00:57.306+00:00 INFO CBAS.context.DatasetLifecycleManager [Checkpoint Thread (178)] Async flushing Dataset (365), Partition (12) |
2024-06-25T07:00:57.306+00:00 INFO CBAS.context.DatasetLifecycleManager [Checkpoint Thread (178)] Async flushing Dataset (365), Partition (13) |
2024-06-25T07:00:57.306+00:00 INFO CBAS.context.DatasetLifecycleManager [Checkpoint Thread (178)] Async flushing Dataset (365), Partition (14) |
2024-06-25T07:00:57.306+00:00 INFO CBAS.context.DatasetLifecycleManager [Checkpoint Thread (178)] Async flushing Dataset (365), Partition (15) |
2024-06-25T07:00:57.306+00:00 INFO CBAS.context.DatasetLifecycleManager [Checkpoint Thread (178)] Async flushing Dataset (365), Partition (18) |
2024-06-25T07:00:57.306+00:00 INFO CBAS.context.DatasetLifecycleManager [Checkpoint Thread (178)] Async flushing Dataset (365), Partition (19) |
2024-06-25T07:00:57.306+00:00 INFO CBAS.context.DatasetLifecycleManager [Checkpoint Thread (178)] Async flushing Dataset (365), Partition (20) |
2024-06-25T07:00:57.306+00:00 INFO CBAS.context.DatasetLifecycleManager [Checkpoint Thread (178)] Async flushing Dataset (365), Partition (21) |
2024-06-25T07:00:57.306+00:00 INFO CBAS.context.DatasetLifecycleManager [Checkpoint Thread (178)] Async flushing Dataset (365), Partition (22) |
2024-06-25T07:00:57.306+00:00 INFO CBAS.context.DatasetLifecycleManager [Checkpoint Thread (178)] Async flushing Dataset (365), Partition (23) |
2024-06-25T07:00:57.306+00:00 INFO CBAS.context.DatasetLifecycleManager [Checkpoint Thread (178)] Async flushing Dataset (365), Partition (26) |
2024-06-25T07:00:57.306+00:00 INFO CBAS.context.DatasetLifecycleManager [Checkpoint Thread (178)] Async flushing Dataset (365), Partition (27) |
2024-06-25T07:00:57.306+00:00 INFO CBAS.context.DatasetLifecycleManager [Checkpoint Thread (178)] Async flushing Dataset (365), Partition (28) |
2024-06-25T07:00:57.306+00:00 INFO CBAS.context.DatasetLifecycleManager [Checkpoint Thread (178)] Async flushing Dataset (365), Partition (29) |
2024-06-25T07:00:57.306+00:00 INFO CBAS.context.DatasetLifecycleManager [Checkpoint Thread (178)] Async flushing Dataset (365), Partition (30) |
2024-06-25T07:00:57.306+00:00 INFO CBAS.context.DatasetLifecycleManager [Checkpoint Thread (178)] Async flushing Dataset (365), Partition (31) |
2024-06-25T07:00:57.306+00:00 INFO CBAS.context.DatasetLifecycleManager [Checkpoint Thread (178)] Async flushing Dataset (365), Partition (34) |
2024-06-25T07:00:57.306+00:00 INFO CBAS.context.DatasetLifecycleManager [Checkpoint Thread (178)] Async flushing Dataset (365), Partition (35) |
2024-06-25T07:00:57.306+00:00 INFO CBAS.context.DatasetLifecycleManager [Checkpoint Thread (178)] Async flushing Dataset (365), Partition (36) |
2024-06-25T07:00:57.306+00:00 INFO CBAS.context.DatasetLifecycleManager [Checkpoint Thread (178)] Async flushing Dataset (365), Partition (37) |
2024-06-25T07:00:57.306+00:00 INFO CBAS.context.DatasetLifecycleManager [Checkpoint Thread (178)] Async flushing Dataset (365), Partition (38) |
2024-06-25T07:00:57.306+00:00 INFO CBAS.context.DatasetLifecycleManager [Checkpoint Thread (178)] Async flushing Dataset (365), Partition (39) |
2024-06-25T07:00:57.306+00:00 INFO CBAS.context.DatasetLifecycleManager [Checkpoint Thread (178)] Async flushing Dataset (365), Partition (42) |
2024-06-25T07:00:57.306+00:00 INFO CBAS.context.DatasetLifecycleManager [Checkpoint Thread (178)] Async flushing Dataset (365), Partition (43) |
2024-06-25T07:00:57.306+00:00 INFO CBAS.context.DatasetLifecycleManager [Checkpoint Thread (178)] Async flushing Dataset (368), Partition (2) |
2024-06-25T07:00:57.306+00:00 INFO CBAS.context.DatasetLifecycleManager [Checkpoint Thread (178)] Async flushing Dataset (368), Partition (3) |
2024-06-25T07:00:57.307+00:00 INFO CBAS.context.DatasetLifecycleManager [Checkpoint Thread (178)] Async flushing Dataset (368), Partition (4) |
2024-06-25T07:00:57.307+00:00 INFO CBAS.context.DatasetLifecycleManager [Checkpoint Thread (178)] Async flushing Dataset (368), Partition (5) |
2024-06-25T07:00:57.307+00:00 INFO CBAS.context.DatasetLifecycleManager [Checkpoint Thread (178)] Async flushing Dataset (368), Partition (6) |
2024-06-25T07:00:57.307+00:00 INFO CBAS.context.DatasetLifecycleManager [Checkpoint Thread (178)] Async flushing Dataset (368), Partition (7) |
2024-06-25T07:00:57.307+00:00 INFO CBAS.context.DatasetLifecycleManager [Checkpoint Thread (178)] Async flushing Dataset (368), Partition (10) |
2024-06-25T07:00:57.307+00:00 INFO CBAS.context.DatasetLifecycleManager [Checkpoint Thread (178)] Async flushing Dataset (368), Partition (11) |
2024-06-25T07:00:57.307+00:00 INFO CBAS.context.DatasetLifecycleManager [Checkpoint Thread (178)] Async flushing Dataset (368), Partition (12) |
2024-06-25T07:00:57.307+00:00 INFO CBAS.context.DatasetLifecycleManager [Checkpoint Thread (178)] Async flushing Dataset (368), Partition (13) |
2024-06-25T07:00:57.307+00:00 INFO CBAS.context.DatasetLifecycleManager [Checkpoint Thread (178)] Async flushing Dataset (368), Partition (14) |
2024-06-25T07:00:57.307+00:00 INFO CBAS.context.DatasetLifecycleManager [Checkpoint Thread (178)] Async flushing Dataset (368), Partition (15) |
2024-06-25T07:00:57.307+00:00 INFO CBAS.context.DatasetLifecycleManager [Checkpoint Thread (178)] Async flushing Dataset (368), Partition (18) |
2024-06-25T07:00:57.307+00:00 INFO CBAS.context.DatasetLifecycleManager [Checkpoint Thread (178)] Async flushing Dataset (368), Partition (19) |
2024-06-25T07:00:57.307+00:00 INFO CBAS.context.DatasetLifecycleManager [Checkpoint Thread (178)] Async flushing Dataset (368), Partition (20) |
2024-06-25T07:00:57.307+00:00 INFO CBAS.context.DatasetLifecycleManager [Checkpoint Thread (178)] Async flushing Dataset (368), Partition (21) |
2024-06-25T07:00:57.307+00:00 INFO CBAS.context.DatasetLifecycleManager [Checkpoint Thread (178)] Async flushing Dataset (368), Partition (22) |
2024-06-25T07:00:57.307+00:00 INFO CBAS.context.DatasetLifecycleManager [Checkpoint Thread (178)] Async flushing Dataset (368), Partition (23) |
2024-06-25T07:00:57.307+00:00 INFO CBAS.context.DatasetLifecycleManager [Checkpoint Thread (178)] Async flushing Dataset (368), Partition (26) |
2024-06-25T07:00:57.307+00:00 INFO CBAS.context.DatasetLifecycleManager [Checkpoint Thread (178)] Async flushing Dataset (368), Partition (27) |
2024-06-25T07:00:57.307+00:00 INFO CBAS.context.DatasetLifecycleManager [Checkpoint Thread (178)] Async flushing Dataset (368), Partition (28) |
2024-06-25T07:00:57.307+00:00 INFO CBAS.context.DatasetLifecycleManager [Checkpoint Thread (178)] Async flushing Dataset (368), Partition (29) |
2024-06-25T07:00:57.307+00:00 INFO CBAS.context.DatasetLifecycleManager [Checkpoint Thread (178)] Async flushing Dataset (368), Partition (30) |
2024-06-25T07:00:57.307+00:00 INFO CBAS.context.DatasetLifecycleManager [Checkpoint Thread (178)] Async flushing Dataset (368), Partition (31) |
2024-06-25T07:00:57.307+00:00 INFO CBAS.context.DatasetLifecycleManager [Checkpoint Thread (178)] Async flushing Dataset (368), Partition (34) |
2024-06-25T07:00:57.307+00:00 INFO CBAS.context.DatasetLifecycleManager [Checkpoint Thread (178)] Async flushing Dataset (368), Partition (35) |
2024-06-25T07:00:57.307+00:00 INFO CBAS.context.DatasetLifecycleManager [Checkpoint Thread (178)] Async flushing Dataset (368), Partition (36) |
2024-06-25T07:00:57.307+00:00 INFO CBAS.context.DatasetLifecycleManager [Checkpoint Thread (178)] Async flushing Dataset (368), Partition (37) |
2024-06-25T07:00:57.307+00:00 INFO CBAS.context.DatasetLifecycleManager [Checkpoint Thread (178)] Async flushing Dataset (368), Partition (38) |
2024-06-25T07:00:57.307+00:00 INFO CBAS.context.DatasetLifecycleManager [Checkpoint Thread (178)] Async flushing Dataset (368), Partition (39) |
2024-06-25T07:00:57.307+00:00 INFO CBAS.context.DatasetLifecycleManager [Checkpoint Thread (178)] Async flushing Dataset (368), Partition (42) |
2024-06-25T07:00:57.307+00:00 INFO CBAS.context.DatasetLifecycleManager [Checkpoint Thread (178)] Async flushing Dataset (368), Partition (43) |
2024-06-25T07:00:57.308+00:00 INFO CBAS.context.DatasetLifecycleManager [Checkpoint Thread (178)] Async flushing Dataset (373), Partition (2) |
2024-06-25T07:00:57.308+00:00 INFO CBAS.context.DatasetLifecycleManager [Checkpoint Thread (178)] Async flushing Dataset (373), Partition (3) |
2024-06-25T07:00:57.308+00:00 INFO CBAS.context.DatasetLifecycleManager [Checkpoint Thread (178)] Async flushing Dataset (373), Partition (4) |
2024-06-25T07:00:57.308+00:00 INFO CBAS.context.DatasetLifecycleManager [Checkpoint Thread (178)] Async flushing Dataset (373), Partition (5) |
2024-06-25T07:00:57.308+00:00 INFO CBAS.context.DatasetLifecycleManager [Checkpoint Thread (178)] Async flushing Dataset (373), Partition (6) |
2024-06-25T07:00:57.308+00:00 INFO CBAS.context.DatasetLifecycleManager [Checkpoint Thread (178)] Async flushing Dataset (373), Partition (7) |
2024-06-25T07:00:57.308+00:00 INFO CBAS.context.DatasetLifecycleManager [Checkpoint Thread (178)] Async flushing Dataset (373), Partition (10) |
2024-06-25T07:00:57.308+00:00 INFO CBAS.context.DatasetLifecycleManager [Checkpoint Thread (178)] Async flushing Dataset (373), Partition (11) |
2024-06-25T07:00:57.308+00:00 INFO CBAS.context.DatasetLifecycleManager [Checkpoint Thread (178)] Async flushing Dataset (373), Partition (12) |
2024-06-25T07:00:57.308+00:00 INFO CBAS.context.DatasetLifecycleManager [Checkpoint Thread (178)] Async flushing Dataset (373), Partition (13) |
2024-06-25T07:00:57.308+00:00 INFO CBAS.context.DatasetLifecycleManager [Checkpoint Thread (178)] Async flushing Dataset (373), Partition (14) |
2024-06-25T07:00:57.308+00:00 INFO CBAS.context.DatasetLifecycleManager [Checkpoint Thread (178)] Async flushing Dataset (373), Partition (15) |
2024-06-25T07:00:57.308+00:00 INFO CBAS.context.DatasetLifecycleManager [Checkpoint Thread (178)] Async flushing Dataset (373), Partition (18) |
2024-06-25T07:00:57.308+00:00 INFO CBAS.context.DatasetLifecycleManager [Checkpoint Thread (178)] Async flushing Dataset (373), Partition (19) |
2024-06-25T07:00:57.308+00:00 INFO CBAS.context.DatasetLifecycleManager [Checkpoint Thread (178)] Async flushing Dataset (373), Partition (20) |
2024-06-25T07:00:57.308+00:00 INFO CBAS.context.DatasetLifecycleManager [Checkpoint Thread (178)] Async flushing Dataset (373), Partition (21) |
2024-06-25T07:00:57.308+00:00 INFO CBAS.context.DatasetLifecycleManager [Checkpoint Thread (178)] Async flushing Dataset (373), Partition (22) |
2024-06-25T07:00:57.308+00:00 INFO CBAS.context.DatasetLifecycleManager [Checkpoint Thread (178)] Async flushing Dataset (373), Partition (23) |
2024-06-25T07:00:57.308+00:00 INFO CBAS.context.DatasetLifecycleManager [Checkpoint Thread (178)] Async flushing Dataset (373), Partition (26) |
2024-06-25T07:00:57.308+00:00 INFO CBAS.context.DatasetLifecycleManager [Checkpoint Thread (178)] Async flushing Dataset (373), Partition (27) |
2024-06-25T07:00:57.308+00:00 INFO CBAS.context.DatasetLifecycleManager [Checkpoint Thread (178)] Async flushing Dataset (373), Partition (28) |
2024-06-25T07:00:57.308+00:00 INFO CBAS.context.DatasetLifecycleManager [Checkpoint Thread (178)] Async flushing Dataset (373), Partition (29) |
2024-06-25T07:00:57.308+00:00 INFO CBAS.context.DatasetLifecycleManager [Checkpoint Thread (178)] Async flushing Dataset (373), Partition (30) |
2024-06-25T07:00:57.308+00:00 INFO CBAS.context.DatasetLifecycleManager [Checkpoint Thread (178)] Async flushing Dataset (373), Partition (31) |
2024-06-25T07:00:57.308+00:00 INFO CBAS.context.DatasetLifecycleManager [Checkpoint Thread (178)] Async flushing Dataset (373), Partition (34) |
2024-06-25T07:00:57.308+00:00 INFO CBAS.context.DatasetLifecycleManager [Checkpoint Thread (178)] Async flushing Dataset (373), Partition (35) |
2024-06-25T07:00:57.308+00:00 INFO CBAS.context.DatasetLifecycleManager [Checkpoint Thread (178)] Async flushing Dataset (373), Partition (36) |
2024-06-25T07:00:57.308+00:00 INFO CBAS.context.DatasetLifecycleManager [Checkpoint Thread (178)] Async flushing Dataset (373), Partition (37) |
2024-06-25T07:00:57.309+00:00 INFO CBAS.context.DatasetLifecycleManager [Checkpoint Thread (178)] Async flushing Dataset (373), Partition (38) |
2024-06-25T07:00:57.309+00:00 INFO CBAS.context.DatasetLifecycleManager [Checkpoint Thread (178)] Async flushing Dataset (373), Partition (39) |
2024-06-25T07:00:57.309+00:00 INFO CBAS.context.DatasetLifecycleManager [Checkpoint Thread (178)] Async flushing Dataset (373), Partition (42) |
2024-06-25T07:00:57.309+00:00 INFO CBAS.context.DatasetLifecycleManager [Checkpoint Thread (178)] Async flushing Dataset (373), Partition (43) |
2024-06-25T07:00:57.826+00:00 INFO CBAS.bootstrap.ClusterMonitor [linkYvZjVsVp cluster monitor] interrupted during pool stream; will retry |
Terminating due to java.lang.OutOfMemoryError: Java heap space
|
2024-06-25T07:01:06.852+00:00 WARN CBAS.cbas analytics driver has exited w/ exit status 3 |
cbcollect ->
https://cb-engineering.s3.amazonaws.com/SysTestColumnarJun24/collectinfo-2024-06-25T070705-ns_1%40svc-da-node-001.nxp9u2dolhjxwjtf.sandbox.nonprod-project-avengers.com.zip
https://cb-engineering.s3.amazonaws.com/SysTestColumnarJun24/collectinfo-2024-06-25T070705-ns_1%40svc-da-node-002.nxp9u2dolhjxwjtf.sandbox.nonprod-project-avengers.com.zip
https://cb-engineering.s3.amazonaws.com/SysTestColumnarJun24/collectinfo-2024-06-25T070705-ns_1%40svc-da-node-003.nxp9u2dolhjxwjtf.sandbox.nonprod-project-avengers.com.zip
https://cb-engineering.s3.amazonaws.com/SysTestColumnarJun24/collectinfo-2024-06-25T070705-ns_1%40svc-da-node-004.nxp9u2dolhjxwjtf.sandbox.nonprod-project-avengers.com.zip
Attachments
Issue Links
- relates to
-
MB-62475 Limit the number of links
- Resolved