Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-45042

[System Test][Analytics] Exception observed in longevity - FATA CBAS.util.ExitUtil [pool-2-thread-1] JVM halting with status 19; thread dump at halt:

    XMLWordPrintable

Details

    • Untriaged
    • 1
    • Unknown
    • CX Sprint 241

    Description

      7.0.0-4707

      Test:
      -test tests/integration/cheshirecat/test_cheshirecat_kv_gsi_coll_xdcr_backup_sgw_fts_itemct_txns_eventing_cbas.yml -scope tests/integration/cheshirecat/scope_cheshirecat_with_backup.yml
      Scale 2
      Iteration 1

      From .106.136:

      2021-03-17T09:39:04.288-07:00 FATA CBAS.util.ExitUtil [pool-2-thread-1] JVM halting with status 19; thread dump at halt: 
      "main" [tid=1 state=WAITING lock=java.util.concurrent.Semaphore$NonfairSync@27c460a8]
      	at java.base@11.0.10/jdk.internal.misc.Unsafe.park(Native Method)
      	at java.base@11.0.10/java.util.concurrent.locks.LockSupport.park(Unknown Source)
      	at java.base@11.0.10/java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(Unknown Source)
      	at java.base@11.0.10/java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(Unknown Source)
      	at java.base@11.0.10/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(Unknown Source)
      	at java.base@11.0.10/java.util.concurrent.Semaphore.acquire(Unknown Source)
      	at app//com.couchbase.analytics.control.AnalyticsDriver.main(AnalyticsDriver.java:112)
       
      "Reference Handler" [tid=2 state=RUNNABLE]
      	at java.base@11.0.10/java.lang.ref.Reference.waitForReferencePendingList(Native Method)
      	at java.base@11.0.10/java.lang.ref.Reference.processPendingReferences(Unknown Source)
      	at java.base@11.0.10/java.lang.ref.Reference$ReferenceHandler.run(Unknown Source)
       
      "Finalizer" [tid=3 state=WAITING lock=java.lang.ref.ReferenceQueue$Lock@aca3c85]
      	at java.base@11.0.10/java.lang.Object.wait(Native Method)
      	at java.base@11.0.10/java.lang.ref.ReferenceQueue.remove(Unknown Source)
      	at java.base@11.0.10/java.lang.ref.ReferenceQueue.remove(Unknown Source)
      	at java.base@11.0.10/java.lang.ref.Finalizer$FinalizerThread.run(Unknown Source)
       
      "Signal Dispatcher" [tid=4 state=RUNNABLE]
       
      "Common-Cleaner" [tid=11 state=TIMED_WAITING lock=java.lang.ref.ReferenceQueue$Lock@45eab322]
      	at java.base@11.0.10/java.lang.Object.wait(Native Method)
      	at java.base@11.0.10/java.lang.ref.ReferenceQueue.remove(Unknown Source)
      	at java.base@11.0.10/jdk.internal.ref.CleanerImpl.run(Unknown Source)
      	at java.base@11.0.10/java.lang.Thread.run(Unknown Source)
      	at java.base@11.0.10/jdk.internal.misc.InnocuousThread.run(Unknown Source)
       
      "AsyncAppender-AsyncDcpDebugLog" [tid=12 state=WAITING lock=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@f017dd0]
      	at java.base@11.0.10/jdk.internal.misc.Unsafe.park(Native Method)
      	at java.base@11.0.10/java.util.concurrent.locks.LockSupport.park(Unknown Source)
      	at java.base@11.0.10/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(Unknown Source)
      	at app//com.conversantmedia.util.concurrent.AbstractCondition.await(AbstractCondition.java:57)
      	at app//com.conversantmedia.util.concurrent.DisruptorBlockingQueue.take(DisruptorBlockingQueue.java:240)
      	at app//org.apache.logging.log4j.core.appender.AsyncAppender$AsyncThread.run(AsyncAppender.java:398)
       
      "AsyncAppender-AsyncAccessLog" [tid=13 state=WAITING lock=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@2424cb9d]
      	at java.base@11.0.10/jdk.internal.misc.Unsafe.park(Native Method)
      	at java.base@11.0.10/java.util.concurrent.locks.LockSupport.park(Unknown Source)
      	at java.base@11.0.10/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(Unknown Source)
      	at app//com.conversantmedia.util.concurrent.AbstractCondition.await(AbstractCondition.java:57)
      	at app//com.conversantmedia.util.concurrent.DisruptorBlockingQueue.take(DisruptorBlockingQueue.java:240)
      	at app//org.apache.logging.log4j.core.appender.AsyncAppender$AsyncThread.run(AsyncAppender.java:398)
       
      "AsyncAppender-AsyncDebugLog" [tid=14 state=WAITING lock=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@7f51f588]
      	at java.base@11.0.10/jdk.internal.misc.Unsafe.park(Native Method)
      	at java.base@11.0.10/java.util.concurrent.locks.LockSupport.park(Unknown Source)
      	at java.base@11.0.10/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(Unknown Source)
      	at app//com.conversantmedia.util.concurrent.AbstractCondition.await(AbstractCondition.java:57)
      	at app//com.conversantmedia.util.concurrent.DisruptorBlockingQueue.take(DisruptorBlockingQueue.java:240)
      	at app//org.apache.logging.log4j.core.appender.AsyncAppender$AsyncThread.run(AsyncAppender.java:398)
       
      "AsyncAppender-AsyncDcpDebugLog" [tid=16 state=WAITING lock=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@6fb22ae3]
      	at java.base@11.0.10/jdk.internal.misc.Unsafe.park(Native Method)
      	at java.base@11.0.10/java.util.concurrent.locks.LockSupport.park(Unknown Source)
      	at java.base@11.0.10/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(Unknown Source)
      	at app//com.conversantmedia.util.concurrent.AbstractCondition.await(AbstractCondition.java:57)
      	at app//com.conversantmedia.util.concurrent.DisruptorBlockingQueue.take(DisruptorBlockingQueue.java:240)
      	at app//org.apache.logging.log4j.core.appender.AsyncAppender$AsyncThread.run(AsyncAppender.java:398)
       
      "AsyncAppender-AsyncAccessLog" [tid=17 state=WAITING lock=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@6bd92538]
      	at java.base@11.0.10/jdk.internal.misc.Unsafe.park(Native Method)
      	at java.base@11.0.10/java.util.concurrent.locks.LockSupport.park(Unknown Source)
      	at java.base@11.0.10/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(Unknown Source)
      	at app//com.conversantmedia.util.concurrent.AbstractCondition.await(AbstractCondition.java:57)
      	at app//com.conversantmedia.util.concurrent.DisruptorBlockingQueue.take(DisruptorBlockingQueue.java:240)
      	at app//org.apache.logging.log4j.core.appender.AsyncAppender$AsyncThread.run(AsyncAppender.java:398)
       
      "AsyncAppender-AsyncDebugLog" [tid=18 state=WAITING lock=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@77771830]
      	at java.base@11.0.10/jdk.internal.misc.Unsafe.park(Native Method)
      	at java.base@11.0.10/java.util.concurrent.locks.LockSupport.park(Unknown Source)
      	at java.base@11.0.10/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(Unknown Source)
      	at app//com.conversantmedia.util.concurrent.AbstractCondition.await(AbstractCondition.java:57)
      	at app//com.conversantmedia.util.concurrent.DisruptorBlockingQueue.take(DisruptorBlockingQueue.java:240)
      	at app//org.apache.logging.log4j.core.appender.AsyncAppender$AsyncThread.run(AsyncAppender.java:398)
       
      "ShutdownWatchdog" [tid=20 state=WAITING lock=java.util.concurrent.Semaphore$NonfairSync@19ae36f4]
      	at java.base@11.0.10/jdk.internal.misc.Unsafe.park(Native Method)
      	at java.base@11.0.10/java.util.concurrent.locks.LockSupport.park(Unknown Source)
      	at java.base@11.0.10/java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(Unknown Source)
      	at java.base@11.0.10/java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireShared(Unknown Source)
      	at java.base@11.0.10/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireShared(Unknown Source)
      	at java.base@11.0.10/java.util.concurrent.Semaphore.acquireUninterruptibly(Unknown Source)
      	at app//org.apache.hyracks.util.ExitUtil$ShutdownWatchdog.run(ExitUtil.java:153)
       
      "IoRequestHandler-0" [tid=21 state=WAITING lock=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@6735f210]
      	at java.base@11.0.10/jdk.internal.misc.Unsafe.park(Native Method)
      	at java.base@11.0.10/java.util.concurrent.locks.LockSupport.park(Unknown Source)
      	at java.base@11.0.10/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(Unknown Source)
      	at java.base@11.0.10/java.util.concurrent.ArrayBlockingQueue.take(Unknown Source)
      	at app//org.apache.hyracks.control.nc.io.IoRequestHandler.run(IoRequestHandler.java:45)
      	at java.base@11.0.10/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
      	at java.base@11.0.10/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
      	at java.base@11.0.10/java.lang.Thread.run(Unknown Source)
       
      	Locked synchronizers:
      	- java.util.concurrent.ThreadPoolExecutor$Worker@3aa41da1
       
      "IoRequestHandler-1" [tid=22 state=WAITING lock=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@6735f210]
      	at java.base@11.0.10/jdk.internal.misc.Unsafe.park(Native Method)
      	at java.base@11.0.10/java.util.concurrent.locks.LockSupport.park(Unknown Source)
      	at java.base@11.0.10/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(Unknown Source)
      	at java.base@11.0.10/java.util.concurrent.ArrayBlockingQueue.take(Unknown Source)
      	at app//org.apache.hyracks.control.nc.io.IoRequestHandler.run(IoRequestHandler.java:45)
      	at java.base@11.0.10/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
      	at java.base@11.0.10/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
      	at java.base@11.0.10/java.lang.Thread.run(Unknown Source)
       
      	Locked synchronizers:
      	- java.util.concurrent.ThreadPoolExecutor$Worker@74fab04a
       
      "IoRequestHandler-2" [tid=23 state=WAITING lock=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@6735f210]
      	at java.base@11.0.10/jdk.internal.misc.Unsafe.park(Native Method)
      	at java.base@11.0.10/java.util.concurrent.locks.LockSupport.park(Unknown Source)
      	at java.base@11.0.10/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(Unknown Source)
      	at java.base@11.0.10/java.util.concurrent.ArrayBlockingQueue.take(Unknown Source)
      	at app//org.apache.hyracks.control.nc.io.IoRequestHandler.run(IoRequestHandler.java:45)
      	at java.base@11.0.10/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
      	at java.base@11.0.10/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
      	at java.base@11.0.10/java.lang.Thread.run(Unknown Source)
       
      	Locked synchronizers:
      	- java.util.concurrent.ThreadPoolExecutor$Worker@3c83468e
       
      "IoRequestHandler-3" [tid=24 state=WAITING lock=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@6735f210]
      	at java.base@11.0.10/jdk.internal.misc.Unsafe.park(Native Method)
      	at java.base@11.0.10/java.util.concurrent.locks.LockSupport.park(Unknown Source)
      	at java.base@11.0.10/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(Unknown Source)
      	at java.base@11.0.10/java.util.concurrent.ArrayBlockingQueue.take(Unknown Source)
      	at app//org.apache.hyracks.control.nc.io.IoRequestHandler.run(IoRequestHandler.java:45)
      	at java.base@11.0.10/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
      	at java.base@11.0.10/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
      	at java.base@11.0.10/java.lang.Thread.run(Unknown Source)
      

      Logs:
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1616000927/collectinfo-2021-03-17T170848-ns_1%40172.23.106.134.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1616000927/collectinfo-2021-03-17T170848-ns_1%40172.23.106.136.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1616000927/collectinfo-2021-03-17T170848-ns_1%40172.23.106.137.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1616000927/collectinfo-2021-03-17T170848-ns_1%40172.23.120.58.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1616000927/collectinfo-2021-03-17T170848-ns_1%40172.23.120.73.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1616000927/collectinfo-2021-03-17T170848-ns_1%40172.23.120.74.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1616000927/collectinfo-2021-03-17T170848-ns_1%40172.23.120.75.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1616000927/collectinfo-2021-03-17T170848-ns_1%40172.23.120.77.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1616000927/collectinfo-2021-03-17T170848-ns_1%40172.23.120.81.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1616000927/collectinfo-2021-03-17T170848-ns_1%40172.23.120.86.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1616000927/collectinfo-2021-03-17T170848-ns_1%40172.23.121.77.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1616000927/collectinfo-2021-03-17T170848-ns_1%40172.23.123.24.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1616000927/collectinfo-2021-03-17T170848-ns_1%40172.23.123.25.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1616000927/collectinfo-2021-03-17T170848-ns_1%40172.23.123.26.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1616000927/collectinfo-2021-03-17T170848-ns_1%40172.23.123.31.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1616000927/collectinfo-2021-03-17T170848-ns_1%40172.23.123.32.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1616000927/collectinfo-2021-03-17T170848-ns_1%40172.23.123.33.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1616000927/collectinfo-2021-03-17T170848-ns_1%40172.23.96.122.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1616000927/collectinfo-2021-03-17T170848-ns_1%40172.23.96.14.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1616000927/collectinfo-2021-03-17T170848-ns_1%40172.23.96.243.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1616000927/collectinfo-2021-03-17T170848-ns_1%40172.23.96.254.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1616000927/collectinfo-2021-03-17T170848-ns_1%40172.23.96.48.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1616000927/collectinfo-2021-03-17T170848-ns_1%40172.23.97.110.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1616000927/collectinfo-2021-03-17T170848-ns_1%40172.23.97.112.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1616000927/collectinfo-2021-03-17T170848-ns_1%40172.23.97.148.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1616000927/collectinfo-2021-03-17T170848-ns_1%40172.23.97.149.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1616000927/collectinfo-2021-03-17T170848-ns_1%40172.23.97.150.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1616000927/collectinfo-2021-03-17T170848-ns_1%40172.23.97.151.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1616000927/collectinfo-2021-03-17T170848-ns_1%40172.23.97.241.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1616000927/collectinfo-2021-03-17T170848-ns_1%40172.23.97.74.zip

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              arunkumar Arunkumar Senthilnathan (Inactive)
              arunkumar Arunkumar Senthilnathan (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty