Understanding Java JIT Compiler
Few years ago I wrote a blog evaluating solutions for a performance problem I had faced some days before. Checking the blog again I realized that I failed to mention that some performance improvement while the test was running, happened because the Java JIT (just in time) compiler had compiled the code and hence it was running much faster after several iterations.
But, what is this JIT compiler (we will use JIT for short) and why it is important? I’d bet many novice Java developers are not aware of it and how dramatically performance changes as the JIT goes doing its job. Over the years I have had many discussions about the topic I will discuss in this page and I have even seen executives made very expensive and unnecessary decisions for not understanding this topic well enough.
It is well known that code written in a compiled programing language executes much faster than an code written in an interpreted programing language. Also, for most programing languages it is easy enough to know, whether they are either compiled or interpreted. For example Phyton and most shell scripting languages are interpreted while C++ and Go Lang are compiled. Then, what about Java? The answer is that Java is both, interpreted and compiled, and this surely requires some explanation.
Running Java code from the command line takes 2 steps. First you use javac command to “compile” the source code and then you use the java command to execute the compiled code. One may ask, if the code is compiled to machine language then why we need the java command to execute it? The answer is simple: becuse the code is not compiled to machine language. What? another one may ask? Didn’t you just say that we compile the Java code before executing it? Yes, Java code is compiled, but not to machine language. It is compiled to a JVM specific format called byte code that the JVM interprets when the java command is issued.
From the paragraph above we get that the Java code is compiled to byte code and then interpreted by the JVM. Up to this point, performance wise we are talking about an interpreted language, and now is when the JIT enters the game. As the JVM executes the byte code, it keeps track of how many times each method is being called and, when a predefined threshold for a method is reached, bingo, that method is compiled on the fly (just in time) to machine language. From that moment on the machine language compiled method is used, which results in a much faster execution process. The process described is used not just with our proprietary code, but with the standard Java libraries as well. Eventually, after some time, almost all the code is compiled to machine language.
Test and Results
To visualize the performance gains after the JIT compiles the code let’s run the following example that executes 100 batches of 50 calculations of the fibonacci number of 100 (f(100) for short) and after each batch prints average execution time for f(100).
package rr.jittest;
/**
* Class used to measure the effect of JIT on a simple calculation.
* It calculates the fibonacci of some number many times,
* calculating and printing the average time per execution. The
* process is repeated many times to understand the long term
* effect.
*
* @author Rodrigo Ramirez
*
*/
public class Fibonacci {
/**
* Calculates the Fibonacci of n :
* @param n
* @return f(n)
*/
public static int f(int n) {
if (n <= 2) {
return 1;
}
int n1 = 1;
int n2 = 1;
int result = 0;
for(int i = 3; i <= n; i++ ) {
result = n1 + n2;
n2 = n1;
n1 = result;
}
return result;
}
/**
* Prints the average time it takes to calculate f(n)
* @param args
*/
public static void trackExecutionSpeed() {
System.out.println(
"Average time to execute f(100) in nanoseconds");
for(int i = 0; i < 100; i++) {
long startTime = System.nanoTime();
for(int j = 0; j < 50; j++) {
f(100);
}
long totalTime = System.nanoTime() - startTime;
System.out.println(String.format("%f",
totalTime / 50.0));
}
}
public static void main(String[] args) {
trackExecutionSpeed();
}
}The initial and final lines in the output may look as follows:
Average time to execute f(100) in nanoseconds
3144.700000
3528.540000
11019.200000
625.360000
765.600000
568.380000
…
120.180000
Process finished with exit code 0Using excel we created a line chart with the above output. The results follow:
In the chart above you can clearly see that average execution times start high and as time passes, go down several times until they stabilize. In our example the machine compiled code executes about 30 times faster than the byte code. It is also evident that some times there are big spikes in the average execution time before it goes down even more than the previous batches. This the penalty of JIT compilation while it happens.
Checking the results in more detail we see:
- Two batches of 50 iterations each one taking an average of some more than 3000 nanoseconds each.
- One big spike, presumably due to JIT compilation, while the third batch was executing.
- About eleven batches with execution averages around 600 nanoseconds.
- One or two more compilation spikes.
- Batches of 120 nanoseconds average execution time for the rest of the test.
Effect on load and stress testing
Clearly any load or stress test will produce very different results before and after the JIT compiles most or all of the methods used in the test. For code that is intended to be used in production over extended periods of time, it makes sense to discard the initial iterations and use the results after the execution time stabilizes for a constant load, a sign that the code is fully compiled to machine language.
Server warming up
Depending on your specific scenario it may be unacceptable to start with a higher execution time, if even for a few iterations before execution times stabilize after full compilation. If that is your case, it makes sense to “warm it up” by using test data forcing the JIT compilation before the server is used in production.
Scripts vs Server Code
It should be clear now that before JIT compiles your code, you experience performance typically associated with interpreted code, not machine language compiled code. This is the case with code that executes once and exists, for example code that executes from a cron job task, uploads a few files to a server and exits.
JIT compiled benefits are normally seen on code that starts executing and stays running for long periods of time like happens on web, application or database servers, as well as code in embedded devices that may start when you power the device and keep running until the device is disconnected.
Environment used for execution
For the test in this post the following conditions were used:
- MacBook Pro 17'’ late 2010.
- JDK 1.8
- -Xms10m -Xmx10m settings, to minimize the effect of garbage collection on the test.
Where to go from here
The purpose of this post is to help the reader understand what the JIT compiler is and how it affects execution. There is far more to understand if this is your area of interest. I suggest to look for posts explaining how to tune your JIT compiler; note that this information may be specific to your JVM distribution, particularly on embedded platforms. You can find information explaining:
- Details about the order and frequency required before code is compiled.
- Startup switches to tune the JIT compiler for server vs script like scenarios.
- Controlling how setters and getters are optimized.
- Controlling whether all code should be compiled or just a few classes; useful in embedded platforms with limited memory.
Closing comments
In this post I discussed how Java JIT compiler works and created a test to help the reader visualize its benefits and penalties, particularly the fact that JIT compiled code runs about 30 times faster than not compiled code. I discussed how to manage JIT compilation effects on some typical situations like load testing and server warm up as well as some scenarios that may not benefit from JIT compilation.
