DEV Community

Cover image for How to Debug Deadlocks Using Thread Dumps and Scripts (Bash/Python)
Sanjay Ghosh
Sanjay Ghosh

Posted on

How to Debug Deadlocks Using Thread Dumps and Scripts (Bash/Python)

Have you ever had an application completely freeze in production without throwing a single error? You likely encountered a Deadlock. While Java’s JVM is powerful, circular dependencies between threads can bring your entire system to a halt.

In this post, we’ll analyze a real-world deadlock scenario and I’ll share two scripts (Bash and Python) to help you automate the detection of these "silent killers."

🏗️ The Scenario

The scenario is like this with 2 Threads, Thread1 and Thread2.
Thread 1 acquired a lock on a Resource1 (Thread2 is trying to lock it) and waiting to acquire a lock on Resource2 (Already locked by Thread2).Thread 2 acquired a lock on the Resource2 (Thread 1 is trying to lock it) and waiting to acquire a lock on Resource1 (Already locked by Thread1).

Example of java code which will cause dead-lock

public class MyDeadLockExample1 {    

    public static Object someResource1 = new Object();

    public static Object someResource2 = new Object();

    public static void main (String[] args) {     

        System.out.println("Test of DeadLock -------------------");
        TestThread1 t1  = new TestThread1();
        TestThread2 t2  = new TestThread2();     
        t1.start();
        t2.start();
    }

    /*Now create a class which will Acquire lock on someResource1 and also wait to Acquire lock on someResource 2*/

    private static class TestThread1 extends Thread {    

        public void run() {
            /*Aquiring lock on someResource1*/

            synchronized (someResource1) {           
                      System.out.println("Thread 1 has locked someResource1");            
                      /*Now sleep for a minute to wait and then try to acquire lock on someResource 2*/           
                       try {
                           Thread.sleep(60)           
                       }           
                       catch (InterruptedException ie)           
                       {           
                           ie.printStackTrace();         
                       }

                       /*Now going to acquire lock on someResource2*/           
                       synchronized(someResource2) {           
                           System.out.println("Thread 1 has locked someResource 2");           
                       }

         }

      }

    } /*End class TestThread1*/



    /*Now create a class which will Acquire lock on someResource2 and also wait to Acquire lock on someResource1*/

    private static class TestThread2 extends Thread {   

        public void run() {

            /*Aquiring lock on someResource1*/

            synchronized (someResource2) {           
                       System.out.println("Thread 2 has locked someResource2");            
                       /*Now sleep for a minute to wait and then try to acquire lock on someResource 1*/          
                       try {           
                           Thread.sleep(60);           
                       }           
                       catch (InterruptedException ie)           
                       {           
                          ie.printStackTrace();           
                       }             
                       /*Now going to acquire lock on someResource1*/           
                       synchronized(someResource1) {           
                           System.out.println("Thread 1 has locked someResource 2");           
                       }
            }

        }

    } /*End class TestThread1*/



} /*End of class MyDeadLockExample1*/
Enter fullscreen mode Exit fullscreen mode

Run the above code:
javac MyDeadLockExample1.java
java MyDeadLockExample1
It will hang forever because of deadlock

🛡️ How to know that there is a Dead Lock

🔍 Step 1: Generate the Thread Dump
Either do
kill -3 //From Linux
or
jcmd Thread.print

This will create the thread dump and you can save in a file (In example td.txt)

Thread dump for this below (td.txt):

     1  25047:
     2  2026-02-07 18:37:20
     3  Full thread dump OpenJDK 64-Bit Server VM (17.0.18+8-Ubuntu-124.04.1 mixed mode, sharing):
     4
     5  Threads class SMR info:
     6  _java_thread_list=0x00007df42c001ed0, length=14, elements={
     7  0x00007df4c017c250, 0x00007df4c017d640, 0x00007df4c0185610, 0x00007df4c01869d0,
     8  0x00007df4c0187df0, 0x00007df4c01897b0, 0x00007df4c018acf0, 0x00007df4c0194160,
     9  0x00007df4c019b8d0, 0x00007df4c019ef20, 0x00007df4c01a17f0, 0x00007df4c01a28b0,
    10  0x00007df4c00182e0, 0x00007df42c000f40
    11  }
    12
    13  "Reference Handler" #2 daemon prio=10 os_prio=0 cpu=0.26ms elapsed=213.94s tid=0x00007df4c017c250 nid=0x61df waiting on condition  [0x00007df4a13a9000]
    14     java.lang.Thread.State: RUNNABLE
    15          at java.lang.ref.Reference.waitForReferencePendingList(java.base@17.0.18/Native Method)
    16          at java.lang.ref.Reference.processPendingReferences(java.base@17.0.18/Reference.java:253)
    17          at java.lang.ref.Reference$ReferenceHandler.run(java.base@17.0.18/Reference.java:215)
    18
    19  "Finalizer" #3 daemon prio=8 os_prio=0 cpu=0.33ms elapsed=213.94s tid=0x00007df4c017d640 nid=0x61e0 in Object.wait()  [0x00007df4a12a9000]
    20     java.lang.Thread.State: WAITING (on object monitor)
    21          at java.lang.Object.wait(java.base@17.0.18/Native Method)
    22          - waiting on <0x0000000717602f40> (a java.lang.ref.ReferenceQueue$Lock)
    23          at java.lang.ref.ReferenceQueue.remove(java.base@17.0.18/ReferenceQueue.java:155)
    24          - locked <0x0000000717602f40> (a java.lang.ref.ReferenceQueue$Lock)
    25          at java.lang.ref.ReferenceQueue.remove(java.base@17.0.18/ReferenceQueue.java:176)
    26          at java.lang.ref.Finalizer$FinalizerThread.run(java.base@17.0.18/Finalizer.java:172)
    27
    28  "Signal Dispatcher" #4 daemon prio=9 os_prio=0 cpu=0.74ms elapsed=213.94s tid=0x00007df4c0185610 nid=0x61e1 waiting on condition  [0x0000000000000000]
    29     java.lang.Thread.State: RUNNABLE
    30
    31  "Service Thread" #5 daemon prio=9 os_prio=0 cpu=0.18ms elapsed=213.94s tid=0x00007df4c01869d0 nid=0x61e2 runnable  [0x0000000000000000]
    32     java.lang.Thread.State: RUNNABLE
    33
    34  "Monitor Deflation Thread" #6 daemon prio=9 os_prio=0 cpu=61.88ms elapsed=213.94s tid=0x00007df4c0187df0 nid=0x61e3 runnable  [0x0000000000000000]
    35     java.lang.Thread.State: RUNNABLE
    36
    37  "C2 CompilerThread0" #7 daemon prio=9 os_prio=0 cpu=5.41ms elapsed=213.94s tid=0x00007df4c01897b0 nid=0x61e4 waiting on condition  [0x0000000000000000]
    38     java.lang.Thread.State: RUNNABLE
    39     No compile task
    40
    41  "C1 CompilerThread0" #10 daemon prio=9 os_prio=0 cpu=4.74ms elapsed=213.94s tid=0x00007df4c018acf0 nid=0x61e5 waiting on condition  [0x0000000000000000]
    42     java.lang.Thread.State: RUNNABLE
    43     No compile task
    44
    45  "Sweeper thread" #11 daemon prio=9 os_prio=0 cpu=0.27ms elapsed=213.94s tid=0x00007df4c0194160 nid=0x61e6 runnable  [0x0000000000000000]
    46     java.lang.Thread.State: RUNNABLE
    47
    48  "Notification Thread" #12 daemon prio=9 os_prio=0 cpu=0.31ms elapsed=213.94s tid=0x00007df4c019b8d0 nid=0x61e7 runnable  [0x0000000000000000]
    49     java.lang.Thread.State: RUNNABLE
    50
    51  "Common-Cleaner" #13 daemon prio=8 os_prio=0 cpu=0.68ms elapsed=213.92s tid=0x00007df4c019ef20 nid=0x61e9 in Object.wait()  [0x00007df4a094f000]
    52     java.lang.Thread.State: TIMED_WAITING (on object monitor)
    53          at java.lang.Object.wait(java.base@17.0.18/Native Method)
    54          - waiting on <0x0000000717618120> (a java.lang.ref.ReferenceQueue$Lock)
    55          at java.lang.ref.ReferenceQueue.remove(java.base@17.0.18/ReferenceQueue.java:155)
    56          - locked <0x0000000717618120> (a java.lang.ref.ReferenceQueue$Lock)
    57          at jdk.internal.ref.CleanerImpl.run(java.base@17.0.18/CleanerImpl.java:140)
    58          at java.lang.Thread.run(java.base@17.0.18/Thread.java:840)
    59          at jdk.internal.misc.InnocuousThread.run(java.base@17.0.18/InnocuousThread.java:162)
    60
    61  "Thread-0" #14 prio=5 os_prio=0 cpu=14.72ms elapsed=213.89s tid=0x00007df4c01a17f0 nid=0x61ea waiting for monitor entry  [0x00007df4a084f000]
    62     java.lang.Thread.State: BLOCKED (on object monitor)
    63          at MyDeadLockExample1$TestThread1.run(MyDeadLockExample1.java:35)
    64          - waiting to lock <0x0000000717619480> (a java.lang.Object)
    65          - locked <0x0000000717619470> (a java.lang.Object)
    66
    67  "Thread-1" #15 prio=5 os_prio=0 cpu=15.43ms elapsed=213.89s tid=0x00007df4c01a28b0 nid=0x61eb waiting for monitor entry  [0x00007df4a074f000]
    68     java.lang.Thread.State: BLOCKED (on object monitor)
    69          at MyDeadLockExample1$TestThread2.run(MyDeadLockExample1.java:60)
    70          - waiting to lock <0x0000000717619470> (a java.lang.Object)
    71          - locked <0x0000000717619480> (a java.lang.Object)
    72
    73  "DestroyJavaVM" #16 prio=5 os_prio=0 cpu=32.70ms elapsed=213.89s tid=0x00007df4c00182e0 nid=0x61d8 waiting on condition  [0x0000000000000000]
    74     java.lang.Thread.State: RUNNABLE
    75
    76  "Attach Listener" #17 daemon prio=9 os_prio=0 cpu=1.51ms elapsed=12.63s tid=0x00007df42c000f40 nid=0x620e waiting on condition  [0x0000000000000000]
    77     java.lang.Thread.State: RUNNABLE
    78
    79  "VM Periodic Task Thread" os_prio=0 cpu=327.22ms elapsed=213.94s tid=0x00007df4c019d220 nid=0x61e8 waiting on condition
    80
    81  "VM Thread" os_prio=0 cpu=17.21ms elapsed=213.94s tid=0x00007df4c0178270 nid=0x61de runnable
    82
    83  "G1 Service" os_prio=0 cpu=81.37ms elapsed=213.96s tid=0x00007df4c01498e0 nid=0x61dd runnable
    84
    85  "G1 Refine#0" os_prio=0 cpu=0.38ms elapsed=213.96s tid=0x00007df4c01489d0 nid=0x61dc runnable
    86
    87  "G1 Conc#0" os_prio=0 cpu=0.23ms elapsed=213.96s tid=0x00007df4c0095020 nid=0x61db runnable
    88
    89  "G1 Main Marker" os_prio=0 cpu=0.23ms elapsed=213.96s tid=0x00007df4c00940a0 nid=0x61da runnable
    90
    91  "GC Thread#0" os_prio=0 cpu=0.36ms elapsed=213.96s tid=0x00007df4c0083280 nid=0x61d9 runnable
    92
    93  JNI global refs: 6, weak refs: 0
    94
    95
    96  Found one Java-level deadlock:
    97  =============================
    98  "Thread-0":
    99    waiting to lock monitor 0x00007df410001000 (object 0x0000000717619480, a java.lang.Object),
   100    which is held by "Thread-1"
   101
   102  "Thread-1":
   103    waiting to lock monitor 0x00007df404000d80 (object 0x0000000717619470, a java.lang.Object),
   104    which is held by "Thread-0"
   105
   106  Java stack information for the threads listed above:
   107  ===================================================
   108  "Thread-0":
   109          at MyDeadLockExample1$TestThread1.run(MyDeadLockExample1.java:35)
   110          - waiting to lock <0x0000000717619480> (a java.lang.Object)
   111          - locked <0x0000000717619470> (a java.lang.Object)
   112  "Thread-1":
   113          at MyDeadLockExample1$TestThread2.run(MyDeadLockExample1.java:60)
   114          - waiting to lock <0x0000000717619470> (a java.lang.Object)
   115          - locked <0x0000000717619480> (a java.lang.Object)
   116
   117  Found 1 deadlock.
   118
Enter fullscreen mode Exit fullscreen mode

📊 Step 2: Analysis of the Dump (td.txt)
Look for the BLOCKED state in your logs:
"Thread-0" #14 ... java.lang.Thread.State: BLOCKED (on object monitor)

  • waiting to lock <0x0000000717619480>
  • locked <0x0000000717619470>

When we look at a thread dump, the smoking gun is usually the BLOCKED state. Let's look at the specific conflict in our example:

Key Findings:

Thread-0 (Lines 61-66) has locked ID ...9470 but is waiting for ...9480.

Thread-1 (Lines 67-71) has locked ID ...9480 but is waiting for ...9470.

Thread-0: Holding Resource 1, Waiting for Resource 2
Looking at lines 61-66, we see Thread-0 in action:

Status: java.lang.Thread.State: BLOCKED (on object monitor)

Locked: 0x0000000717619470 (Source 1)

Waiting to lock: 0x0000000717619480 (Source 2)

Thread-1: Holding Resource 2, Waiting for Resource 1
In lines 67-71, the "Circular Wait" is completed:

Status: BLOCKED

Locked: 0x0000000717619480 (Source 2)

Waiting to lock: 0x0000000717619470 (Source 1)

The Conclusion: Neither thread can proceed because each is holding the key that the other one needs.

🛠️ Automation Scripts
Instead of manually digging through thousands of lines in a production thread dump, you can use these automation scripts to flag deadlocks instantly.

🔗 [GitHub Repository Link Here]
All resources for this example are available on GitHub: GitHub location

Included Files:
MyDeadLockExample1.java: The Java source that reproduces this exact deadlock.

td.txt: The raw thread dump file used in this analysis.

parse.py: A Python script for deep parsing. (Usage: python3 parse.py td.txt)

parse.sh: A lightweight Bash script for quick CLI checks. (Usage: ./parse.sh td.txt)

Pro-Tip
These scripts aren't just for this example—you can run them against any thread dump file to detect monitor-based deadlocks in your own applications.

Top comments (0)