Java Trouble Shooting 4 – [ Thread Dump ]

When a Java application becomes slow, unresponsive, or completely stuck, a thread dump is often the fastest way to understand what’s really happening inside the JVM.

1 – Capture Thread Dump

Generally, when the server hangs, crashes, or has low performance, it is necessary to grab the server’s thread stack (Thread Dump) for subsequent analysis.

It is necessary to do thread dumps several times at an interval of 10-20s.

If each dump points to the same problem, we can determine the root cause of problem.

To capture thread dump, we can do below :

/**
 * Operating system command to get ThreadDump
*/

// get pid
ps -ef | grep -i java

// get thread dump
kill -3 <pid>

/**
 * JVM tools to get ThreadDump
*/

// get pid
jps -l

// get thread dump
jstack -l <pid> | tee -a jstack.log

2 – Structure of Thread Dump

Below is a sample of thread dump :

2026-01-03 08:42:37
Full thread dump Java HotSpot(TM) 64-Bit Server VM (21.0.3+7-LTS-152 mixed mode, sharing):

Threads class SMR info:
_java_thread_list=0x0000750ee4194a50, length=10, elements={
0x0000750ee40285f0, 0x0000750ee4168ff0, 0x0000750ee416a690, 0x0000750ee416c0a0,
0x0000750ee416d6e0, 0x0000750ee416ec80, 0x0000750ee41707c0, 0x0000750ee4171e80,
0x0000750ee41900f0, 0x0000750ee4193910
}

"main" #1 [9074] prio=5 os_prio=0 cpu=158,35ms elapsed=29,56s tid=0x0000750ee40285f0 nid=9074 waiting on condition  [0x0000750eec5fe000]
   java.lang.Thread.State: TIMED_WAITING (sleeping)
        at java.lang.Thread.sleep0(java.base@21.0.3/Native Method)
        at java.lang.Thread.sleep(java.base@21.0.3/Thread.java:558)
        at java.util.concurrent.TimeUnit.sleep(java.base@21.0.3/TimeUnit.java:446)
        at RandomTest.sleep(RandomTest.java:27)
        at RandomTest.process(RandomTest.java:14)
        at RandomTest.main(RandomTest.java:9)

"Reference Handler" #9 [9082] daemon prio=10 os_prio=0 cpu=0,63ms elapsed=29,42s tid=0x0000750ee4168ff0 nid=9082 waiting on condition  [0x0000750ee8aa5000]
   java.lang.Thread.State: RUNNABLE
        at java.lang.ref.Reference.waitForReferencePendingList(java.base@21.0.3/Native Method)
        at java.lang.ref.Reference.processPendingReferences(java.base@21.0.3/Reference.java:246)
        at java.lang.ref.Reference$ReferenceHandler.run(java.base@21.0.3/Reference.java:208)

"Finalizer" #10 [9083] daemon prio=8 os_prio=0 cpu=0,40ms elapsed=29,42s tid=0x0000750ee416a690 nid=9083 in Object.wait()  [0x0000750ee89a5000]
   java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait0(java.base@21.0.3/Native Method)
        - waiting on <0x0000000716401670> (a java.lang.ref.NativeReferenceQueue$Lock)
        at java.lang.Object.wait(java.base@21.0.3/Object.java:366)
        at java.lang.Object.wait(java.base@21.0.3/Object.java:339)
        at java.lang.ref.NativeReferenceQueue.await(java.base@21.0.3/NativeReferenceQueue.java:48)
        at java.lang.ref.ReferenceQueue.remove0(java.base@21.0.3/ReferenceQueue.java:158)
        at java.lang.ref.NativeReferenceQueue.remove(java.base@21.0.3/NativeReferenceQueue.java:89)
        - locked <0x0000000716401670> (a java.lang.ref.NativeReferenceQueue$Lock)
        at java.lang.ref.Finalizer$FinalizerThread.run(java.base@21.0.3/Finalizer.java:173)

"Signal Dispatcher" #11 [9084] daemon prio=9 os_prio=0 cpu=0,45ms elapsed=29,42s tid=0x0000750ee416c0a0 nid=9084 waiting on condition  [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"Service Thread" #12 [9085] daemon prio=9 os_prio=0 cpu=0,57ms elapsed=29,42s tid=0x0000750ee416d6e0 nid=9085 runnable  [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"Monitor Deflation Thread" #13 [9086] daemon prio=9 os_prio=0 cpu=8,45ms elapsed=29,42s tid=0x0000750ee416ec80 nid=9086 runnable  [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"C2 CompilerThread0" #14 [9087] daemon prio=9 os_prio=0 cpu=16,20ms elapsed=29,42s tid=0x0000750ee41707c0 nid=9087 waiting on condition  [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE
   No compile task

"C1 CompilerThread0" #16 [9088] daemon prio=9 os_prio=0 cpu=13,23ms elapsed=29,42s tid=0x0000750ee4171e80 nid=9088 waiting on condition  [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE
   No compile task

"Notification Thread" #17 [9089] daemon prio=9 os_prio=0 cpu=0,38ms elapsed=29,41s tid=0x0000750ee41900f0 nid=9089 runnable  [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"Common-Cleaner" #18 [9090] daemon prio=8 os_prio=0 cpu=0,72ms elapsed=29,40s tid=0x0000750ee4193910 nid=9090 waiting on condition  [0x0000750ee81d5000]
   java.lang.Thread.State: TIMED_WAITING (parking)
        at jdk.internal.misc.Unsafe.park(java.base@21.0.3/Native Method)
        - parking to wait for  <0x0000000716410a68> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.parkNanos(java.base@21.0.3/LockSupport.java:269)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(java.base@21.0.3/AbstractQueuedSynchronizer.java:1847)
        at java.lang.ref.ReferenceQueue.await(java.base@21.0.3/ReferenceQueue.java:71)
        at java.lang.ref.ReferenceQueue.remove0(java.base@21.0.3/ReferenceQueue.java:143)
        at java.lang.ref.ReferenceQueue.remove(java.base@21.0.3/ReferenceQueue.java:218)
        at jdk.internal.ref.CleanerImpl.run(java.base@21.0.3/CleanerImpl.java:140)
        at java.lang.Thread.runWith(java.base@21.0.3/Thread.java:1596)
        at java.lang.Thread.run(java.base@21.0.3/Thread.java:1583)
        at jdk.internal.misc.InnocuousThread.run(java.base@21.0.3/InnocuousThread.java:186)

"VM Thread" os_prio=0 cpu=3,96ms elapsed=29,44s tid=0x0000750ee415af50 nid=9081 runnable  

"GC Thread#0" os_prio=0 cpu=0,27ms elapsed=29,53s tid=0x0000750ee4090d20 nid=9075 runnable  

"G1 Main Marker" os_prio=0 cpu=0,21ms elapsed=29,53s tid=0x0000750ee40a1b90 nid=9076 runnable  

"G1 Conc#0" os_prio=0 cpu=0,18ms elapsed=29,53s tid=0x0000750ee40a2a60 nid=9077 runnable  

"G1 Refine#0" os_prio=0 cpu=0,20ms elapsed=29,53s tid=0x0000750ee4125b40 nid=9078 runnable  

"G1 Service" os_prio=0 cpu=2,49ms elapsed=29,53s tid=0x0000750ee4126af0 nid=9079 runnable  

"VM Periodic Task Thread" os_prio=0 cpu=46,15ms elapsed=29,45s tid=0x0000750ee4140990 nid=9080 waiting on condition  

JNI global refs: 5, weak refs: 0

Heap
 garbage-first heap   total 258048K, used 2283K [0x0000000706c00000, 0x0000000800000000)
  region size 2048K, 1 young (2048K), 0 survivors (0K)
 Metaspace       used 97K, committed 320K, reserved 1114112K
  class space    used 7K, committed 128K, reserved 1048576K

The first and second lines display the timestamp and information about the JVM :

2026-01-03 08:42:37
Full thread dump Java HotSpot(TM) 64-Bit Server VM (21.0.3+7-LTS-152 mixed mode, sharing):

Next section shows the Safe Memory Reclamation (SMR), which enumerates the addresses of all non-JVM internal threads (e.g. non-VM and non-Garbage Collection (GC)) :

Threads class SMR info:
_java_thread_list=0x0000750ee4194a50, length=10, elements={
0x0000750ee40285f0, 0x0000750ee4168ff0, 0x0000750ee416a690, 0x0000750ee416c0a0,
0x0000750ee416d6e0, 0x0000750ee416ec80, 0x0000750ee41707c0, 0x0000750ee4171e80,
0x0000750ee41900f0, 0x0000750ee4193910
}

If we examine these addresses, we see that they correspond to the tid value of each of the numbered threads in the dump :

jstack 9072 | grep 0x0000750ee40285f0
0x0000750ee40285f0, 0x0000750ee4168ff0, 0x0000750ee416a690, 0x0000750ee416c0a0,
"main" #1 [9074] prio=5 os_prio=0 cpu=380,45ms elapsed=772,92s tid=0x0000750ee40285f0 nid=9074 waiting on condition  [0x0000750eec5fe000]

jstack 9072 | grep 0x0000750ee4168ff0
0x0000750ee40285f0, 0x0000750ee4168ff0, 0x0000750ee416a690, 0x0000750ee416c0a0,
"Reference Handler" #9 [9082] daemon prio=10 os_prio=0 cpu=0,63ms elapsed=782,93s tid=0x0000750ee4168ff0 nid=9082 waiting on condition  [0x0000750ee8aa5000]

jstack 9072 | grep 0x0000750ee416a690
0x0000750ee40285f0, 0x0000750ee4168ff0, 0x0000750ee416a690, 0x0000750ee416c0a0,
"Finalizer" #10 [9083] daemon prio=8 os_prio=0 cpu=0,40ms elapsed=810,92s tid=0x0000750ee416a690 nid=9083 in Object.wait()  [0x0000750ee89a5000]

jstack 9072 | grep 0x0000750ee416c0a0
0x0000750ee40285f0, 0x0000750ee4168ff0, 0x0000750ee416a690, 0x0000750ee416c0a0,
"Signal Dispatcher" #11 [9084] daemon prio=9 os_prio=0 cpu=1,39ms elapsed=830,81s tid=0x0000750ee416c0a0 nid=9084 waiting on condition  [0x0000000000000000]

Next section is the list of threads.

Each thread contains the following information :

Thread Summary

The first line of each thread represents the thread summary.

"main" #1 [9074] prio=5 os_prio=0 cpu=380,45ms elapsed=772,92s tid=0x0000750ee40285f0 nid=9074 waiting on condition  [0x0000750eec5fe000]

"Reference Handler" #9 [9082] daemon prio=10 os_prio=0 cpu=0,63ms elapsed=782,93s tid=0x0000750ee4168ff0 nid=9082 waiting on condition  [0x0000750ee8aa5000]

"Finalizer" #10 [9083] daemon prio=8 os_prio=0 cpu=0,40ms elapsed=810,92s tid=0x0000750ee416a690 nid=9083 in Object.wait()  [0x0000750ee89a5000]

"Signal Dispatcher" #11 [9084] daemon prio=9 os_prio=0 cpu=1,39ms elapsed=830,81s tid=0x0000750ee416c0a0 nid=9084 waiting on condition  [0x0000000000000000]

It contains the following items :

SECTIONEXAMPLEDESCRIPTION
NamemainHuman-readable name of the thread.

This name can be set by calling the setName method on a Thread object and be obtained by calling getName on the object.
ID#1A unique ID associated with each Thread object.

This number is generated, starting at 1, for all threads in the system.

Each time a Thread object is created, the sequence number is incremented and then assigned to the newly created Thread.

This ID is read-only and can be obtained by calling getId on a Thread object.
Daemon StatusdaemonA tag to tell if the thread is a daemon thread.

If the thread is a daemon, this tag will be present, else no tag will be present.
Priorityprio=5The numeric priority of the Java thread.

The priority of a Thread object can be set using the setPriority method and obtained using the getPriority method.
OS Thread Priorityos_prio=0The OS thread priority.

This priority can differ from the Java thread priority and corresponds to the OS thread on which the Java thread is dispatched.
Addresstid=0x00007ff084024000The address of the Java thread.

It is the unique ID given by the JVM.
OS Thread IDnid=0xe45The unique ID of the OS thread to which the Java Thread is mapped.

It is useful to extract correlation with CPU or memory processing.
Statuswaiting on conditionHuman-readable current status of the thread.
Last Known Java Stack Pointer[0x00007ff08a36c000]The last known Stack Pointer (SP) for the stack associated with the thread.

For simple thread dumps, this information may not be useful, but for more complex diagnostics, this SP value can be used to trace lock acquisition through a program.
Thread State

The second line represents the current state of the thread.

java.lang.Thread.State: TIMED_WAITING (sleeping)

java.lang.Thread.State: RUNNABLE

java.lang.Thread.State: WAITING (on object monitor)

The possible states for a thread can be found in the Thread.State enumeration :

NEW

Each thread has a corresponding Thread object in the heap memory.

When the Thread object is just created in the heap memory, the thread is in the NEW state before the t.start() method is called.

In this state, a thread is no different from an ordinary java object in the heap memory.

public class ThreadStateTest {
    public static void main(String[] args) throws InterruptedException {
        Thread t = new Thread();
        System.out.println(t.getState());
    }
}
/*
Output:
    NEW
 */

RUNNABLE

This state indicates that the thread has all operating conditions, is ready for operating system scheduling in the run queue, or is running.

The thread in this state is relatively normal, but if the thread stays in this state for a long time, it is abnormal, which means that the thread has been running for a long time (there is a performance problem), or the thread has not been able to execute (there is thread starvation problem).

public class ThreadStateTest2 {
    public static void main(String[] args) throws InterruptedException {
        Thread t = new Thread();
        t.start();
        System.out.println(t.getState());
    }
}
/*
Output:
    RUNNABLE
 */

BLOCKED

A thread is in the BLOCKED state when it’s currently not eligible to run.

It enters this state when it is waiting for a monitor lock and is trying to access a section of code that is locked by some other thread.

public class ThreadStateTest3 {
    public static void main(String[] args) throws InterruptedException {
        Runnable r = () -> {
            synchronized (ThreadStateTest3.class) {
                while (true) {

                }
            }
        };

        Thread t1 = new Thread(r);
        t1.start();
        Thread.sleep(1000);
        Thread t2 = new Thread(r);
        t2.start();
        Thread.sleep(1000);
        System.out.println(t2.getState());
        System.exit(0);
    }
}
/*
Output:
    BLOCKED
 */

WAITING

A thread is in WAITING state when it’s waiting for some other thread to perform a particular action.

Any thread can enter this state by calling any one of the following three methods :

  • object.wait()
  • thread.join()
  • LockSupport.park()
import java.util.concurrent.locks.LockSupport;

public class ThreadStateTest4 {
    public static void main(String[] args) throws InterruptedException {
        Runnable r = () -> {
            LockSupport.park();
        };

        Thread t = new Thread(r);
        t.start();
        Thread.sleep(1000);
        System.out.println(t.getState());
        System.exit(0);
    }
}
/*
Output:
    WAITING
 */

TIMED_WAITING

A thread is in TIMED_WAITING state when it’s waiting for another thread to perform a particular action within a stipulated amount of time.

Any thread can enter this state by calling any one of the following methods :

  • Thread.sleep(timeout)
  • Thread.join(timeout)
  • Object.wait(timeout)
  • LockSupport.parkUntil(timeout)
  • LockSupport.parkNanos(timeout)
import java.util.concurrent.locks.LockSupport;

public class ThreadStateTest5 {
    public static void main(String[] args) throws InterruptedException {
        Runnable r = () -> {
            LockSupport.parkNanos(1_000_000_000);
        };

        Thread t = new Thread(r);
        t.start();
        Thread.sleep(100);
        System.out.println(t.getState());
        System.exit(0);
    }
}
/*
 * Output:
 *   TIMED_WAITING
 */

TERMINATED

After the thread is executed, the run method returns normally after executing the run method, or it ends with a runtime exception, the thread will stay in this state.

import java.util.concurrent.locks.LockSupport;

public class ThreadStateTest6 {
    public static void main(String[] args) throws InterruptedException {
        Thread t = new Thread();
        t.start();
        Thread.sleep(1000);
        System.out.println(t.getState());
        System.exit(0);
    }
}
/*
 * Output:
 *   TERMINATED
 */
Thread Stack Trace

The next section contains the stack trace for the thread at the time of the dump.

The stack information should be interpreted in the reverse direction :

        at java.lang.Thread.sleep0(java.base@21.0.3/Native Method)
        at java.lang.Thread.sleep(java.base@21.0.3/Thread.java:558)
        at java.util.concurrent.TimeUnit.sleep(java.base@21.0.3/TimeUnit.java:446)
        at RandomTest.sleep(RandomTest.java:27)
        at RandomTest.process(RandomTest.java:14)
        at RandomTest.main(RandomTest.java:9)
JVM Threads

The next section of the thread dump contains the threads which are usually composed of GC threads and other threads used by the JVM to run and maintain a Java application in the background :

"VM Thread" os_prio=0 cpu=3,96ms elapsed=29,44s tid=0x0000750ee415af50 nid=9081 runnable  

"GC Thread#0" os_prio=0 cpu=0,27ms elapsed=29,53s tid=0x0000750ee4090d20 nid=9075 runnable  

"G1 Main Marker" os_prio=0 cpu=0,21ms elapsed=29,53s tid=0x0000750ee40a1b90 nid=9076 runnable  

"G1 Conc#0" os_prio=0 cpu=0,18ms elapsed=29,53s tid=0x0000750ee40a2a60 nid=9077 runnable  

"G1 Refine#0" os_prio=0 cpu=0,20ms elapsed=29,53s tid=0x0000750ee4125b40 nid=9078 runnable  

"G1 Service" os_prio=0 cpu=2,49ms elapsed=29,53s tid=0x0000750ee4126af0 nid=9079 runnable  

"VM Periodic Task Thread" os_prio=0 cpu=46,15ms elapsed=29,45s tid=0x0000750ee4140990 nid=9080 waiting on condition

Some of important threads generated during the running of the JVM are listed below :

NAMEDESCRIPTION
Attach ListenerThe Attach Listener thread is responsible for receiving an external command, executing the command and returning the result to the sender.

Usually we will use some commands to ask the JVM to give us some feedback information, such as: java -version, jmap, jstack, etc.

If the thread is not initialized when the JVM is started, it will be started when the user executes the JVM command for the first time.
Signal DispatcherThe responsibility of the Attach Listener thread is to receive external JVM commands.

When the command is successfully received, it will be handed over to the signal dispatcher thread for distribution to various modules to process the command and return the processing result.

The signal dispatcher thread also initializes when it receives an external JVM command for the first time.
CompilerThreadxxxUsed to call JIT, compile and unload classes in real time.

Usually, the JVM will start multiple threads to process this part of the work, and the number after the thread name will also accumulate.
DestroyJavaVMAfter main method is executed, the DestroyJavaVM thread is awaken, and it is in a waiting state, waiting for other threads (Java thread and Native thread) to exit and notify it to unload the JVM.

When each thread exits, it will determine whether it is currently the last non-deamon thread in the entire JVM, and if it is, it will notify the DestroyJavaVM thread to uninstall the JVM.
Finalizer ThreadThis thread is also created after the main thread, it is mainly used to call the finalize() method of the object before garbage collection.
Low Memory DetectorThis thread is responsible for detecting the available memory, and if the available memory is found to be low, new memory space is allocated.
Reference HandlerThis thread is also created after the main thread, it is mainly used to call the finalize() method of the object before garbage collection.
VM ThreadIt is the key thread in the JVM.

It will generate or trigger all other threads and be used by other threads to do some VM operations.
Java Native Interface

Finally, the dump displays the Java Native Interface (JNI) references.

JNI global refs: 5, weak refs: 0

Those references are prone to memory leaks, as they are not automatically garbage collected, and the programmer must explicitly free them.

If you are not writing any JNI code yourself, it is possible that the library you are using has a memory leak.

3 – Recommendations

  • Take a number of thread dumps at close intervals (e.g 10 seconds, 30 seconds …)
  • Comparing the RUNNABLE thread of multiple dump files :
  • If the execution method has a big change, it means it is normal.
  • If it is the same method among those dumps, there are some problems and you found the stuck method.
  • Find the thread id that uses the most CPU by using top -H -p pid, then in thread dump, find the corresponding thread stack information according to the thread id.
  • When the processing performance is abnormally low, it has to focus on BLOCKED threads and to see if it is stucked in i/o, database, etc, and try to locate the cause of the bottleneck.