Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-61894

[System Test] Fatal messages seen - appears to be a deadlock

    XMLWordPrintable

Details

    • 0
    • Yes
    • Analytics Sprint 42

    Description

      It's a lengthy trace so I'm just pasting the first few lines -

      2024-05-14T17:02:24.722+00:00 INFO CBAS.bootstrap.AnalyticsNCApplication [main] Starting Couchbase Analytics driver, version: {"build.date":"Mon Apr 29 12:39:42 PDT 2024","build.number":"2045","build.version":"7.6.100"}
      2024-05-14T17:02:47.289+00:00 INFO CBAS.bootstrap.AnalyticsNCApplication [main] Starting Couchbase Analytics driver, version: {"build.date":"Mon Apr 29 12:39:42 PDT 2024","build.number":"2045","build.version":"7.6.100"}
      2024-05-14T19:02:49.493+00:00 FATA CBAS.bootstrap.DeadlockWatchdog [Timer-0] Detected 3 threads ([116, 3270, 3314]); thread dump: 
      "main" [tid=1 state=WAITING lock=java.util.concurrent.Semaphore$NonfairSync@44ca0f04]
      	at java.base@17.0.10/jdk.internal.misc.Unsafe.park(Native Method)
      	at java.base@17.0.10/java.util.concurrent.locks.LockSupport.park(LockSupport.java:211)
      	at java.base@17.0.10/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:715)
      	at java.base@17.0.10/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1047)
      	at java.base@17.0.10/java.util.concurrent.Semaphore.acquire(Semaphore.java:318)
      	at app//com.couchbase.analytics.control.AnalyticsDriver.main(AnalyticsDriver.java:109)
      	at app//com.couchbase.columnar.ColumnarDriver.main(ColumnarDriver.java:10)
       
      "Reference Handler" [tid=2 state=RUNNABLE]
      	at java.base@17.0.10/java.lang.ref.Reference.waitForReferencePendingList(Native Method)
      	at java.base@17.0.10/java.lang.ref.Reference.processPendingReferences(Reference.java:253)
      	at java.base@17.0.10/java.lang.ref.Reference$ReferenceHandler.run(Reference.java:215)
       
      "Finalizer" [tid=3 state=WAITING lock=java.lang.ref.ReferenceQueue$Lock@53d87b2d]
      	at java.base@17.0.10/java.lang.Object.wait(Native Method)
      	at java.base@17.0.10/java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:155)
      	at java.base@17.0.10/java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:176)
      	at java.base@17.0.10/java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:172)
       
      "Signal Dispatcher" [tid=4 state=RUNNABLE]
      

      Seen on nodes da-node-001 and da-node-004

      https://cb-engineering.s3.amazonaws.com/SysTestColumnarMay14/collectinfo-2024-05-15T050246-ns_1%40svc-da-node-001.roww7re5gsxne0fr.sandbox.nonprod-project-avengers.com.zip
      https://cb-engineering.s3.amazonaws.com/SysTestColumnarMay14/collectinfo-2024-05-15T050246-ns_1%40svc-da-node-002.roww7re5gsxne0fr.sandbox.nonprod-project-avengers.com.zip
      https://cb-engineering.s3.amazonaws.com/SysTestColumnarMay14/collectinfo-2024-05-15T050246-ns_1%40svc-da-node-003.roww7re5gsxne0fr.sandbox.nonprod-project-avengers.com.zip
      https://cb-engineering.s3.amazonaws.com/SysTestColumnarMay14/collectinfo-2024-05-15T050246-ns_1%40svc-da-node-004.roww7re5gsxne0fr.sandbox.nonprod-project-avengers.com.zip

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              pavan.pb Pavan PB
              pavan.pb Pavan PB
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty