Pod Startup Time Improvements
In a Kubernetes environment, deploying and managing Java backend applications within pods can introduce challenges related to pod startup time. The time it takes for pods to transition from initialization to full operational status can have significant implications for application performance and user experience. This problem becomes particularly pronounced during deployment, restarts, and scaling activities.
When a Java backend application’s pod is initially started or restarted due to updates or scaling, several issues related to pod startup time can arise:
Increased Response Times: During pod startup, the Java application often goes through a warmup phase, involving actions like database connections, caching, and code optimization. This process can cause delays in responding to incoming requests, leading to increased response times. Users might experience slow or even failed responses, impacting the application’s reliability and user satisfaction.
Scaling Challenges: When scaling the application by adding more pods to handle increased traffic, the collective warmup phase across multiple pods can intensify the impact on response times.
“Imagine pods are like athletes preparing to run a race. Before the race starts, they need to put on their running shoes, stretch their muscles and get their bodies ready for peak performance.”
It’s clear that we need pods to be warmed up before taking actual traffic. It is equally important to have this warmup done quickly in a high scale agile infrastructure.
Before moving to optimization techniques we need to understand the process of JVM warmup:
In the Java Virtual Machine (JVM), the process of warmup involves optimizing the execution of Java bytecode through Just-In-Time (JIT) compilation. This process enhances the performance of Java applications by converting frequently executed bytecode into optimized native machine code.
The JVM uses two main JIT compilers: the C1 (Client) compiler and the C2 (Server) compiler. C1 compiler provides faster but less extensive optimizations suitable for quicker application startup, while the C2 compiler focuses on thorough and aggressive optimizations to ensure sustained high performance over an application’s lifetime.
To achieve this we have applied following techniques(Azul Prime JDK 17):
Warmup Script:
A warmup script makes calls across APIs to initialize classes, load necessary data and establish connections to become fully operational and ready to handle incoming requests or tasks efficiently.
A good warmup script should:
- Cover all the APIs with variety of requests across each API.
- Make parallel and bulk requests.
JIT Optimizations:
Tuning JVM parameters in accordance with Azul Prime guides to optimize the methods during C1/C2 compilations
Further reading on details of these properties:
https://docs.azul.com/prime/analyzing-tuning-warmup
https://docs.azul.com/prime/Command-Line-Options
RN Profiling:
The ReadyNow Profile log records optimization decisions made on methods in the run that it was recorded in. When used as input in subsequent runs, it allows the JVM to perform those same optimizations as soon as possible upon startup, even before the JVM starts experiencing actual load, and without having to wait for the load to happen while running non-optimized code.
It is recommended to collect ReadyNow Profile data that can be used to train JVM optimizations. This requires the following steps:
- Perform a run with a representative load (e.g. in a test or production environment) lasting for a minimum of 30 minutes (or more). Extract/save the ReadyNow Profile Log (eg. rnprofileIn.log) for the next step.
- Copy the ReadyNow Profile output log extracted in the previous step (eg. rnprofileIn.log) into a new image to be used in subsequent runs of the application. Place the file at the same location as it was extracted from.
Results
We were able to see ~81% reduction in warmup time. It has reduced from ~1000s(16m) to ~180s(3m).
Warmup time before optimizations:
Warmup time after optimizations:
