Bridging the memory gap between your device and the cloud
In the past, as developers, we often worked on local development machines with limited memory but deployed our applications to powerful remote servers.
Today, the landscape has changed. These days laptops often come with huge amounts of memory and lots of cpu cores yet our applications find their home in the cloud on servers that can handle various tasks but are deliberately configured with more modest capabilities to save on costs.
The difference in memory resources can sometimes make code execution seem smooth during local development but reveal significant challenges in production.
At Ixor, we experienced this firsthand when transitioning from the maven-docker-plugin to the maven-jib-plugin. This migration was required to make sure that we could support multi-platform builds in order to make sure that developers with new Apple M1/M2 chips could still run the images.
While local development remained trouble-free, we were surprised to see OutOfMemory errors in production. The culprit was the missing -XX:MaxRAMPercentage parameter during service startup. In the process of migrating to the maven-jib-plugin, we noticed that it no longer used the JAVA_TOOLS parameter we had in our Dockerfile to configure the memory.
As a result, we unintentionally forfeited about 25% of the available memory causing the OutOfMemory. The amount of memory we had in production was already highly optimised, and losing out on 25% meant the service could no longer function properly.
On a related note, we opt for MaxRAMPercentage rather than specifying a fixed RAM value. This approach proves advantageous in scenarios involving vertical scaling of the container, as it eliminates the need to adjust the RAM value. Typically, we set it to 75% of the available memory, ensuring ample resources remain unallocated for various (Java) processes such as garbage collection, native memory, metaspace, and beyond.
In this blog post I would like to talk about how to set up your local environment, both using a standard JVM process as well as using Docker, to more closely mirror the resources you would have available on your cloud environment of choice as well as using VisualVM to look into the statistics of your application.
Let’s start of by creating a simple Spring boot app that we can use to deliberately consume memory. I have created a simple application that can be found here: https://github.com/ixorbv/spring-boot-docker-memory-demo
The application has an endpoint that can be triggered to generate a specific amount of memory (in megabytes):
curl localhost:8080/memory/take/{megaBytes}After creating the memory, the application will retain it to make sure the garbage collector does not clean it up.
We also provide an endpoint to release the memory by providing the id that was returned during the take/memory call.
curl localhost:8080/memory/release/{id}When launching a JVM, you can determine the size of the heap, including both its initial and maximum sizes, by executing the following command on a Unix/Linux system:
java -XX:+PrintFlagsFinal -version | grep HeapSizeOn my macbook with 32GB of memory, I get the following output:
size_t ErgoHeapSizeLimit = 0 {product} {default}
size_t HeapSizePerGCThread = 43620760 {product} {default}
size_t InitialHeapSize = 536870912 {product} {ergonomic}
size_t LargePageHeapSizeThreshold = 134217728 {product} {default}
size_t MaxHeapSize = 8589934592 {product} {ergonomic}
size_t MinHeapSize = 8388608 {product} {ergonomic}
uintx NonNMethodCodeHeapSize = 7602480 {pd product} {ergonomic}
uintx NonProfiledCodeHeapSize = 122027880 {pd product} {ergonomic}
uintx ProfiledCodeHeapSize = 122027880 {pd product} {ergonomic}
size_t SoftMaxHeapSize = 8589934592 {manageable} {ergonomic}As we can see the, the default MaxHeapSizeis equal to 8.589.934.592 bytes (8GB). or 1/4th of the available physical memory of the machine.
For the purposes of this demonstration, let’s reduce the heap size to expedite the appearance of an OutOfMemory error. This can be achieved by using the -Xmx argument, allowing you to specify the MaxHeapSize yourself.
So when we type
java -Xmx1024m -XX:+PrintFlagsFinal -version | grep HeapSizewe now see that our heap size is limited to 1.073.741.824 bytes, or roughly 1 GB
size_t ErgoHeapSizeLimit = 0 {product} {default}
size_t HeapSizePerGCThread = 43620760 {product} {default}
size_t InitialHeapSize = 1073741824 {product} {ergonomic}
size_t LargePageHeapSizeThreshold = 134217728 {product} {default}
size_t MaxHeapSize = 1073741824 {product} {command line}
size_t MinHeapSize = 8388608 {product} {ergonomic}
uintx NonNMethodCodeHeapSize = 7602992 {pd product} {ergonomic}
uintx NonProfiledCodeHeapSize = 122027624 {pd product} {ergonomic}
uintx ProfiledCodeHeapSize = 122027624 {pd product} {ergonomic}
size_t SoftMaxHeapSize = 1073741824 {manageable} {ergonomic}So lets launch our sample Spring Boot application with the same parameter. this can be done by launching the JAR directly
java -Xmx1024m -jar spring-boot-docker-memory-demo-0.0.1-SNAPSHOT.jarOr by using the mvn spring-boot plugin
mvn spring-boot:run -Dspring-boot.run.jvmArguments="-Xmx1024m"In this example, the application will be locally launched with a maximum heap size of 1024MB. Now, when we invoke /memory/take/512endpoint twice, each time reserving 512MB of memory, you'll observe the occurrence of an OutOfMemory error.
java.lang.OutOfMemoryError: Java heap space
at com.example.springbootdockermemorydemo.LoadGeneratorController.generateLoad(LoadGeneratorController.java:19) ~[classes!/:0.0.1-SNAPSHOT]
at java.base/java.lang.invoke.LambdaForm$DMH/0x0000000801400400.invokeVirtual(LambdaForm$DMH) ~[na:na]
at java.base/java.lang.invoke.LambdaForm$MH/0x0000000801400c00.invoke(LambdaForm$MH) ~[na:na]
at java.base/java.lang.invoke.Invokers$Holder.invokeExact_MT(Invokers$Holder) ~[na:na]
at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invokeImpl(DirectMethodHandleAccessor.java:155) ~[na:na]
at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104) ~[na:na]
at java.base/java.lang.reflect.Method.invoke(Method.java:578) ~[na:na]
at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205) ~[spring-web-6.0.12.jar!/:6.0.12]
at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:150) ~[spring-web-6.0.12.jar!/:6.0.12]
at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:118) ~[spring-webmvc-6.0.12.jar!/:6.0.12]
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:884) ~[spring-webmvc-6.0.12.jar!/:6.0.12]
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:797) ~[spring-webmvc-6.0.12.jar!/:6.0.12]
at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87) ~[spring-webmvc-6.0.12.jar!/:6.0.12]
at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1081) ~[spring-webmvc-6.0.12.jar!/:6.0.12]
at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:974) ~[spring-webmvc-6.0.12.jar!/:6.0.12]
at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1011) ~[spring-webmvc-6.0.12.jar!/:6.0.12]
at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:903) ~[spring-webmvc-6.0.12.jar!/:6.0.12]
at jakarta.servlet.http.HttpServlet.service(HttpServlet.java:564) ~[tomcat-embed-core-10.1.13.jar!/:na]
at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:885) ~[spring-webmvc-6.0.12.jar!/:6.0.12]
at jakarta.servlet.http.HttpServlet.service(HttpServlet.java:658) ~[tomcat-embed-core-10.1.13.jar!/:na]
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:205) ~[tomcat-embed-core-10.1.13.jar!/:na]
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:149) ~[tomcat-embed-core-10.1.13.jar!/:na]
at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:51) ~[tomcat-embed-websocket-10.1.13.jar!/:na]
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:174) ~[tomcat-embed-core-10.1.13.jar!/:na]
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:149) ~[tomcat-embed-core-10.1.13.jar!/:na]
at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100) ~[spring-web-6.0.12.jar!/:6.0.12]
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116) ~[spring-web-6.0.12.jar!/:6.0.12]
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:174) ~[tomcat-embed-core-10.1.13.jar!/:na]
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:149) ~[tomcat-embed-core-10.1.13.jar!/:na]
at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:93) ~[spring-web-6.0.12.jar!/:6.0.12]
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116) ~[spring-web-6.0.12.jar!/:6.0.12]
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:174) ~[tomcat-embed-core-10.1.13.jar!/:na]Docker:
To be able to run the application in Docker, following commands must be executed in the root folder op the project:
mvn compile jib:dockerBuildBy default, Docker will allocate some resources of the device it is running on. You can check the currently allocated resources in Docker Desktop by navigating to Settings -> Resources:
Or when using the CLI:
docker info | grep -e 'Total Memory' -e 'CPU'In my setup, Docker has access to 8GB of memory and 8 CPUs. When I run a single container, it can use up all 8GB of memory and all the CPUs.
If I want to see what the default heap size would be inside my docker container I can execute the following command
docker run --rm -ti amazoncorretto:19 java -XX:+PrintFlagsFinal -version | grep HeapSizethat will generate the following output
size_t ErgoHeapSizeLimit = 0 {product} {default}
size_t HeapSizePerGCThread = 43620760 {product} {default}
size_t InitialHeapSize = 130023424 {product} {ergonomic}
size_t LargePageHeapSizeThreshold = 134217728 {product} {default}
size_t MaxHeapSize = 2057306112 {product} {ergonomic}
size_t MinHeapSize = 8388608 {product} {ergonomic}
uintx NonNMethodCodeHeapSize = 7602992 {pd product} {ergonomic}
uintx NonProfiledCodeHeapSize = 122027624 {pd product} {ergonomic}
uintx ProfiledCodeHeapSize = 122027624 {pd product} {ergonomic}
size_t SoftMaxHeapSize = 2057306112 {manageable} {ergonomic}Notice how we have 2.057.306.112 bytes available, or roughly 2 GB, 1/4 th of the total amount of memory docker can use.
To control how much of these resources your container can use, you can specify additional settings in the docker-compose file under the deploy section.
version: '3.8'
services:
spring-boot-memory-demo:
container_name: memory-demo
image: ixor/memory-demo:latest
ports:
- "8080:8080"
deploy:
resources:
limits:
memory: 2048M
reservations:
memory: 2048MWith this configuration, the container will have up the 2048MB of memory available during runtime. Once again it is important to check the size of the heap — this time inside of your Docker container:
docker exec -it memory-demo /bin/sh
java -XX:+PrintFlagsFinal -version | grep HeapSize size_t ErgoHeapSizeLimit = 0 {product} {default}
size_t HeapSizePerGCThread = 43620760 {product} {default}
size_t InitialHeapSize = 33554432 {product} {ergonomic}
size_t LargePageHeapSizeThreshold = 134217728 {product} {default}
size_t MaxHeapSize = 536870912 {product} {ergonomic}
size_t MinHeapSize = 8388608 {product} {ergonomic}
uintx NonNMethodCodeHeapSize = 5839372 {pd product} {ergonomic}
uintx NonProfiledCodeHeapSize = 122909434 {pd product} {ergonomic}
uintx ProfiledCodeHeapSize = 122909434 {pd product} {ergonomic}
size_t SoftMaxHeapSize = 536870912 {manageable} {ergonomic}From this command we can see that the MaxHeapSize for this configuration is 536.870.912 bytes(512MB), 25% of the available memory we have reserved for the container.
Now that’s a lot of numbers. Let’s summarize them
In a containerized environment, you have to deal with
- The physical memory of your host running the docker process (32 GB)
- The memory configured on your docker runtime (8GB = your docker configuration))
- The memory assigned to your container (2GB = the value in your docker compose.yml)
- The memory (heap size) assigned to your JVM (512MB = 1/4th of the memory assigned to your container)
If we would increase the container memory to 4096MB, we would see that the default heap size of the JVM would increase to 1024MB.
We can explicitely set the size op the maximum heap by passing an environment variable in our docker-compose file:
version: '3.8'
services:
spring-boot-memory-demo:
container_name: memory-demo
image: ixor/memory-demo:latest
environment:
- JAVA_TOOL_OPTIONS=-Xmx1024m
ports:
- "8080:8080"
deploy:
resources:
limits:
memory: 2048M
reservations:
memory: 2048MIf we now once again execute the /memory/take endpoint with 512 as parameter twice, we see the OutOfMemory error.
Using VisualVM to monitor the memory of your application:
VisualVM is a Java performance tool designed for tracking and analysing memory usage in Java applications. It provides a user-friendly interface to help developers monitor and optimise memory-related issues in their Java applications.
You can download VisualVM here.
Local JVM:
After launching VisualVM, delete the Docker container and launch the application:
java -Xmx1000m -jar spring-boot-docker-memory-demo-0.0.1-SNAPSHOT.jarInside of VisualVM you should be able to locate the application:
If we now generate some load on the application, we can see the Heap space increasing, I am adding 100mb of memory per call this time:
We could also Heap Dump our Heap to see what is taking up the memory and as expected, it are the bytes we are allocating in our demo application:
Docker:
In order to connect to the application running in Docker I altered the docker-compose file to add support for JMX functionality aswel as exposing the port on which to connect in VisualVM:
version: '3.8'
services:
spring-boot-memory-demo:
container_name: memory-demo
image: ixor/memory-demo:latest
environment:
- JAVA_TOOL_OPTIONS=-Xmx1024m -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.port=9010 -Dcom.sun.management.jmxremote.rmi.port=9010 -Dcom.sun.management.jmxremote.host=0.0.0.0 -Djava.rmi.server.hostname=0.0.0.0
ports:
- "8080:8080"
- "9010:9010"
deploy:
resources:
limits:
memory: 2048M
reservations:
memory: 2048MWe can now add the connection in VisualVM:
And we can once again see the Heap increasing when executing the generate-load calls:
Conclusion
In conclusion, the memory resource divide between local development and the cloud is a critical consideration for modern software developers. Ensuring that your code performs optimally in production is not just about efficient coding but also aligning your development environment with the resources available in the cloud. As we've explored, adjusting JVM heap sizes, optimising Docker containers, and leveraging tools like VisualVM can help bridge this gap and minimize unexpected memory-related issues.
