Managing Java Heap size in Kubernetes

Gonçalo Valente

Published in

Marionete

6 min readJul 8, 2021

A guide on how to handle Java (jdk8) heap size for containerised microservices in Kubernetes.

Problem statement

I have come across the topic for this article while managing Java microservices in a Kubernetes environment for a project where, due to restricted access to external software, we were limited to an older jdk version.

When you specify a pod, you can and should (assuming you are not experimenting on your personal workbench) specify how much of each resource a container needs. I am assuming the reader of this article will already have the understanding of allocating pod resources, but if that is not the case reference [3] specifies the differences between requests and limits and even presents implementation details.

It is possible to control the amount of memory your Java application uses with the Java command-line parameters -Xmx , -Xms and -Xss (thoroughly debated in reference [4]). When working in a non-development scenario, we want to avoid manual overriding Java options. The solution and conclusions discussed below provide a simplification of CI/CD implementation (also discussed below).

Getting back to what motivated this article, we found when using a jdk version below 8u191 to containerise Java microservices, the JVM does not pick up the quotas attributed to that container.

Consequently, before going any further, a disclaimer is required: if you are using a jdk version 8u191 or above, this article will probably not be the most relevant to you, but I always find this sort of non-perfect world scenarios quite useful. Additionally, please stick around for the recommendations section even if that is not your case.

Over the following subsections, I will present a couple of scenarios that illustrate the described topic, as well as a “how to replicate” piece.

Setup and how to replicate

If you want to verify the scenarios described below in your machine, you’ll need the following:

Basic understanding of Docker and Java. Understanding of how to upgrade/downgrade Java versions.
minikube installed (or access to any Kubernetes cluster);
This git repository, which has the Kubernetes manifest files and a cheat sheet bash script (it will automatically replicate all the results presented below). The structure of this repository is illustrated below. You’ll be able to find the cheat sheet bash script right at the root directory and the Kubernetes manifest files under k8s/manifests .

k8s-java-heap
├── LICENSE
├── README.md
├── cheatsheet.sh
├── k8s
│   └── manifests
│       ├── scenario-1.yml
│       ├── scenario-2.yml
│       └── scenario-3.yml
└── screenshots
    ├── scenario-0.jpg
    ├── scenario-1.jpg
    ├── scenario-2.jpg
    ├── scenario-3.jpg
    └── scenario-4.jpg

You’ll need a way to evaluate the maximum Java heap size, which can be done using the following command:

java -XX:+PrintFlagsFinal -version | grep -Ei 'MaxHeapSize|MaxRAMFraction|version'

As a baseline, running the above command in an unrestricted Docker container running in my local machine (similar to Scenario 1 below) yields4198498304 Bytes, which is 4Gigabytes, 25% of the memory allocated to my local Docker engine(32GB).

25% is the default RAM fraction for the Java maximum heap space (1/MaxRAMFraction from the screenshot above). This fraction can be changed, which will be further discussed in the recommendations section below.

Scenario #1: Running a `jdk` version below `8u191` inside an unrestricted pod

In the first scenario, we are running jdk 8u181 and not imposing any limits.

As we may have been expecting, the result for the maximum Java heap space is 4198498304 Bytes (~4GB), which is 25% of the total memory allocated to my Docker engine (which is running a Kubernetes cluster using minikube where this pod is deployed), as we are not imposing any quotas.

Scenario #2: Running a `jdk` version below `8u191` on a pod with a `1GB` memory limit

When deploying to enterprise Kubernetes clusters, good practice dictates we should attribute quotas to our pods/deployments/... (sometimes this is even strongly enforced). To do that, we use a resources block in our Kubernetes manifest file:

resources:
        limits:
          memory: "1Gi"
        requests:
          memory: "600Mi"

In this second scenario (screenshot above), we are also running jdk 8u181 and, unfortunately, the JVM does not pick up the quotas attributed to the pod and maintains the same ~4GBmaximum heap size, which is what motivated this article.

Scenario #3: Running `jdk` version `8u191` on a pod with a `1GB` memory limit

This scenario shows that this is not an issue if you are running jdk version 8u191 or above (again, please stick around for the recommendations section).

The next screenshot features a similar setup (resources block in the manifest file for a pod) but this time running jdk version 8u191.

We can see that the JVM detects the limits correctly, attributing a Java maximum heap size of 25% of the memory available to the pod.

Scenario #4: Running a `jdk` version below `8u191` inside an unrestricted pod with `JVM` command-line option

If we go back to jdk 8u181 and have absolutely no option of upgrading, I found that the most reliable way to actually limit the heap size is to compute it and hardcode it into the Java command to run your application (could be your container entry point, your Kubernetes container command and arguments or something along those lines). In this exposé, I simulate that in the kubectl command I use to obtain the values, but in a Dockerfile it can be something like this:

...
CMD ["/usr/bin/java", "-Xmx256M", "-jar", "/app.jar"]

We can once again conclude that it worked, the JVM is using the specified maximum heap size. However, this is a solution to avoid if possible (if you can upgrade your jdk), as it means manual math and hardcoding, two very dangerous words in the DevOps world!

Alternative scenario — experimental parameter

If we take a look at reference [1], there is also mention of an alternative solution: from JDK 8u131+, there’s an experimental VM option that allows the JVM ergonomics to read the memory values from CGgroups. I am not going to further detail this solution here but it may be worth taking a look at.

Conclusions/Recommendations

The first couple of scenarios illustrate my observations motivation for this article, while the remaining expose possible isolated solutions. After living through this issue, we were able to update to a later version of jdk8. Even so, my recommendation is to go for a hybrid solution:

In a containerised microservice context, using only 25% of the total container available memory for Java maximum heap size is a bit overkill, as the container will not have any other background activities apart from running the Java process and keeping itself alive (specific requirements may contradict this statement).
There’s another Java option that we can use to select the maximum RAM fraction for the Java maximum heap size, the-XX:MaxRAMFraction Java option.
The StackOverflow post in reference [2] quotes a RAM fraction of 50% is safe(ish) to start experimenting. We have been using this without any issues thus far.
You may ask (the StackOverflow post author actually did) why not allocating 100% of the container available memory for the Java heap space. As is pictured in the cover illustration for this article, the JVM is never the only process running on a container. Moreover, the JVM memory usage is not 100% heap space.
Changing the RAM fraction can be achieved also in the container entry point or in the Kubernetes command and arguments, something like:

...
CMD ["/usr/bin/java", "-XX:MaxRAMFraction=2", "-jar", "/app.jar"]

One key advantage of the solution discussed here is the simplification of CI/CD implementation in the terms that we won’t need to adjust the Java options or the requests/limit of Kubernetes resources.