Building Data Intensive Systems With Java

Java developers often experience memory-related issues. Due to the complexity of Java memory management, it can be hard to diagnose and pinpoint issues in an application. At TransUnion, we have defined the process of tuning Java applications, which includes memory management. While many books and blogs highlight strategies to fine-tune Java applications, they’re often not comprehensive across tools, under the hood memory details and root cause. Our approach aims to provide that holistic view across both theory and practical implementation.

In Java memory is segregated into multiple parts. Each part is designed to hold data when the JVM performs specific tasks. These parts are called heap memory and non-heap memory. Java heap memory is used by Java runtime to allocate memory to objects. Non-heap memory is used for the execution of a thread. Heap memory is divided into two parts called the young generation and old generation.

Young generation area

Whenever the JVM creates an object, it tries to reserve the space in the young generation area. The young generation area is divided into three parts: Eden, S0, and S1 space. Whenever the JVM creates new objects, it tries to reserve space in the Eden space. If the Eden space is already occupied and does not have any further free space, the JVM will move objects in the Eden space to the S0 or S1 space. This process is called minor garbage collection. During the minor garbage collection process, objects that are no longer used are removed to free memory. In addition, objects that are still in use are shifted to survivor space (i.e., the S0 and S1 area).

Old generation area

Often the JVM cannot reclaim the objects from the young generation area because those objects are still being used by an application. To continue utilizing the young generation area, the JVM moves the objects from the young generation area to the old generation area. Objects in the old generation area are long-lived and have survived multiple rounds of minor garbage collection. Once the old generation area is full, the JVM starts a process called major garbage collection to free up memory.

Why do different parts of Java memory matter to application performance?

When JVM performs garbage collection, all threads are stopped. This means that the application is paused until the garbage collection process is complete. The duration application is paused is called a “Stop the World” event. The frequency of these Stop the World events is a major factor in application performance. More of these events make applications less performant.

The JVM has many options available to configure different parts of heap memory. And the JVM provides multiple algorithms for garbage collection. Choosing an appropriate size for each area, along with the right garbage collection algorithm, can significantly improve and application performance.

First, let’s review the available garbage collection algorithms:

Serial garbage collector: The serial collector uses a single thread to perform all garbage collection work. This makes it relatively efficient because it avoids communication overhead between threads; however, it can’t take advantage of multiprocessor hardware.

Parallel garbage collector: Parallel garbage collection uses multiple threads to perform the garbage collection process. Given the appropriate infrastructure, this can speed up garbage collection. Parallel garbage collection is also called the Throughput collector.

Parallel old garbage collector: By default, Parallel garbage collection uses multiple threads for both young and old generation area. This can be disabled by using Parallel old garbage collection, which will cause the old generation area to use a single thread for the garbage collection process.

Concurrent Mark Sweep collector: The Concurrent Mark Sweep (CMS) implementation uses multiple garbage collector threads for the garbage collection. It’s designed for applications that prefer shorter garbage collection pauses. In order to accelerate the development of other collectors, this collector is being depreciated.

Garbage-First (G1) garbage collector: The G1 collector is a parallel, concurrent, and incrementally compacting low-pause garbage collector. Garbage First Collector doesn’t work like other collectors. Instead of splitting the heap into three spaces (Eden, survivor, and old) like most other GCs, it splits the heap into many (often several hundred) very small regions.

Shenandoah: Shenandoah is the low pause time garbage collector that reduces GC pause times by performing more garbage collection work concurrently with the running Java program.

Z Garbage collection (ZGC): ZGC is scalable, with low latency. It is a completely new GC, written from scratch, with the goal of a few milliseconds pause, handling all heap sizes.

Which Garbage collector is best?

Over the years, many algorithms had been developed and fine-tuned to support various types of applications. Primarily, each garbage collection algorithm is judged based on throughput and latency.

But what do these terms mean? Throughput is the CPU time used by the garbage collector. Latency refers to latency of pauses.

The parallel garbage collector can produce better throughput and latency for small heap size requirements. If heap size is big, latency can be poor, as it will take more time to perform garbage collection (which can be fine for a batch application).

On the other hand, if the application needs the flexibility to control throughput versus latency, the G1 collector provides that flexibility. Configurations can be applied based on the application’s desired goals of throughput and pause time. G1 collector can start showing poor performance if the heap is very large. That’s why new collectors started to show up such as Shenandoah and G1 collectors.

The approach Shenandoah collector takes is to perform garbage collection as work along with the application thread (i.e., shorter pause time). Also, this means pause time does not need to be dependent on heap size. It may sound like Shenandoah is the best garbage collector out there, but Z garbage collector also tries to achieve low latencies, regardless of the heap size. Systems that require a very large heap in accordance with performance may also benefit from the Z garbage collector.

Impact of Java version and vendor on modern garbage collector:

While modern collectors provided improved performance, another challenge an architect needs to deal with is the Java version and vendor. After Oracle announced changes to the Java licensing term, the enterprise either had to shift to an open-source alternative or buy commercial licenses. The default garbage collector, along with the available customization features, can vary from vendor to vendor. For example, the Shenandoah collector is not bundled by Oracle. It’s not included in Oracle OpenJDK builds or in propriety license builds. Oracle does include the ZGC collector, but features and availability can also vary based on the Java version. From Oracle, ZGC is available as an experimental feature starting with Java 11. Java 13 onward, Oracle provides support for the larger heap. The Shenandoah collector is being shipped in Azul, RedHat, AdoptOpenJDK and Corretto. But keep in mind, features and availability still might vary by Java version. Another aspect, in addition to the Java version, is the underlying operating system. It’s worth reviewing the respective vendor’s documentation for these details. Pay special attention to which feature is listed as “experimental” in the respective Java version documentation.

Products/tool that give us insight into a specific application:

JDK includes some tools like jConsole, jstat, jmap, etc. These tools may not be available in certain vendor’s builds. JDK logs can also provide JVM insights, most commonly JVM flag is -XX:+PrintGCDetails. This flag can produce various outputs based on the GC collector used. Logs also can be written to log files by using -Xloggc.

While the log can provide details, it often becomes a tedious task to fetch the required details. A GUI tool may help develop efficiency. The most common visual open-source GUI tool used by developers is VisualVM. VisualVM provides a nice visual representation of the Java memory. It also shows the GC cycles and various options for profiling.

Post-Java licensing changes, Oracle open sourced another tool called Java Mission Control. Java Mission Control produces more fine-tuned options. On a similar path, Azul created the tool Zulu Mission Control. These tools can offer developers, architects and support teams insight into an application in a more granular manner. Along with profiling options, these tools provide a clear view on GC cycles and pause time.

Here are a few example screenshots of various configurations for a sample application.

Memory view of application:

Notice GC pause time, memory allocation, and overall how memory is structured.

Memory object view:

CPU profile & method invocation:

Mission control:

JVM internals in Mission control:

GC Details in Mission control:

— Akshat Sharma is an architect on the Application Development team at TransUnion working across multiple technologies and frameworks. He is an advocate of Microservices, DevOps, Cloud, Data science and Middleware Platforms. He enjoys sharing knowledge.

Technologists at TransUnion have the opportunity to be involved in creating solutions, driven by our collective curiosity. Our environment is a tech sandbox, combining experimentation and new ways of working. Learn more about Technology @TransUnion and see how you can take your skills to the next level.