Handling and Resolving Performance Issues in Java Multithreaded Applications II: Utilizing Java Flight Recorder (JFR) and JDK Mission Control (JMC)

In a previous article[1], I discussed how to identify Java performance issues using thread dumps. As a continuation, this article will explain how Java Flight Recorder (JFR) can be employed to pinpoint Java thread-level slowness that impacts production-level application performance.

Java Flight Recorder (JFR) is a Java Virtual Machine (JVM) feature that enables continuous monitoring and profiling of a running Java application. JFR provides in-depth information about the JVM, application code, and underlying operating system, allowing developers to detect and diagnose performance issues, memory leaks, and other application problems.

JFR records data in real-time and stores it in a binary file format, which can be analyzed offline using various JVM-provided tools, such as JDK Mission Control (JMC). The collected data includes information on CPU usage, memory usage, garbage collection, thread behavior, and more.

Available as a commercial feature in the Oracle JDK and compatible with OpenJDK builds, JFR is widely employed in production environments to troubleshoot performance issues and optimize Java applications.

To capture JFR dumps from your JVM while observing a performance issue, use the following commands, which are compatible with Java 8 and 11. Note that these commands may not work with the latest versions, so ensure you use the correct commands for your Java version, which can be found with a simple Google search.

First, enable Java commercial features with the following command:

jcmd <process_id> VM.unlock_commercial_features

Next, start capturing the JFR dump, which will be saved in the specified location. Be sure to collect JFR dumps throughout the performance issue period, capturing more information and facilitating the identification of problematic threads.

jcmd <process_id> JFR.start name=ESB_profiling duration=1h filename=/home/temp/ESB_profiling.jfr

To stop recording, use this command:

jcmd <process_id> JFR.stop name=ESB_profiling

If you experience intermittent slowness lasting around 10 minutes, capture JFR dumps throughout the issue period. If the issue is constant, capture dumps for approximately 30 minutes and analyze.

After obtaining the JFR dump, use the JDK Mission Control (JMC) tool to analyze it. JMC is freely available from Oracle[2], as well as for OpenJDK[3]. Once the JFR is opened with JMC, focus on thread locking (lock instances) to address Java thread-level performance issues. By clicking on lock instances, you can view the location where threads are stuck, the number of samples that got stuck, the time taken to pass that point, and the threads and thread stack traces. Please refer to the attached screenshot below.

This information will help you identify problematic threads and code sections causing Java application slowness.

If you wish to focus on I/O-level performance issues, JFR provides information on file I/O and network I/O. To concentrate on garbage collections and GC pauses, JFR offers comprehensive information and statistics related to GC, which can be used to address production system performance issues.

This article has provided an overview of capturing JFR dumps and using JMC to analyze JFR dumps to identify Java thread-level performance issues. In a future blog post, I will explain how to use JProfiler to analyze JFR dumps. Until then, goodbye.

[1] https://medium.com/@prabhashdilhanakmeemana/handling-and-resolving-performance-issues-in-java-multithreaded-applications-i-using-thread-dumps-baccab34d2c5

[2] https://www.oracle.com/java/technologies/javase/products-jmc8-downloads.html

[3] https://github.com/openjdk/jmc