Java JIT Compiler
This article will discuss the JIT (Just In Time) compiler of Java virtual machine. If you are a computer science geek, you may have a good understanding of the compiler. In some cases, if you prefer to refresh your knowledge about compilers before reading this article, first read this article, which is written by me. There are two main sections in this article. In the first section, I discuss how a program is executed directly in the OS and executed in a virtual machine. In this case Java virtual machine (JVM). So if you have a good understanding of the first topic, you can directly go to the next section about the JIT compiler.
How a program runs on the computer
Before moving into the JIT compiler, I will discuss how the computer program works on our computer. There are two types of how we can run a program on our computer.
- Run the program directly on our OS
- Run the program on a virtual machine that is run on our OS
Execute a program directly on OS
First, let’s talk about executing a program directly on our OS. So imagine you have written a C code that produces prime numbers. Then you compile that code using the C compiler on your computer. This compiler will generate an executable program or a binary. You can directly run your executable on your OS. If you are running your C compiler on a Linux machine, it will generate Linux compatible executable file. If you are running your C compiler on a Windows machine, it will generate a Windows-compatible executable file. We call this a native code. Because this executable contains instructions that can run on your OS, representing the same behavior of the C code you write in your favorite IDE. So those instructions will be different according to the target OS of your compiler.
Since these instructions can run directly on your OS, it is much faster to execute the program than the second way of executing a program, executing the program on a virtual machine. Why is that?
Execute a program inside a virtual machine
First, let's understand what a virtual machine is. According to this analogy, the virtual machine is also a program that allows running a program (a byte code) on this machine instead of runs the program directly on OS. When we run our code on our OS directly, we call this native code, and when we run on a virtual machine, we call it byte code. JVM, the Java virtual machine, and BEAM, the Erlang virtual machine are good examples of virtual machines. So how this process happens. Let’s assume we write a Java code that will produce prime numbers. Then we can compile that Java code using the javac compiler. In the previous example, when we compile the C code using the C compiler, it generates native code that can execute directly on our OS. But when we use the javac compiler, it will generate byte code that can be run on the Java virtual machine.
This JVM contains the JIT compiler, which is the topic of today's story. Before move into the JIT compiler, let’s try to understand how the byte code (our compiled java code) is executed inside the JVM. For this, let’s see the architecture of the JVM.
In the Java virtual machine (JVM), there are 3 main components.
- Class-Loader
- Runtime Memory
- Execution Engine
First, the class-loader will load all the byte codes to the RAM. In that phase, JVM loads a few types of class files. Those are bootstrap classes, extension classes, and application classes. In this article, I will not talk about these in deep (But for sure, in another article about JVM). Then the byte-code inside those classes will be executed using the execution engine. The runtime memory will manage runtime data. Now the execution engine is the section that we are going to focus on.
The execution engine has a few sub-components. But a few of them are important for us in this article (bolded ones).
- Interpreter
- JIT Compiler
- Profiler
- Garbage Collector
- Java Native Interface
- Native Interface Library
You know how a traditional interpreter works. The same thing happens in the interpreter inside the JVM as well. It read the byte code line by line and translates into machine code, and executes it. So we can see the Java code has been executed without the JIT compiler. Then why the heck we need this JIT compiler? This is why.
Remember the C code that produces prime numbers. When we execute that code, first we compile the whole code into the machine code. Then OS can directly run that machine-code. There is no any time-cost for translations again. But when we run the Java code, the interpreter translates the byte code line by line and executes. It consumes a huge amount of time in contrast to the C code. So the JIT compiler helps to get rid of this time-consuming. The JIT compiler translates re-occurring codes into machine code. When that code phrase has occurred again, the interpreter will not translate that code phrase into machine code, again and again; instead, the machine code that is translated by the JIT compiler will be executed.
So this is how the JIT compiler works in the JVM, and in the next section, let’s see how the JIT compiler does this task and the architecture of the JIT compiler that is designed to do this task in a very efficient manner.
The JIT compiler
Now we know what the JIT compiler does. JIT compiler compiles re-occurring bytecode code blocks into machine code so that the interpreter can directly use that machine code. So how the JIT compiler identify these re-occurring code blocks? To figure out this, let’s analyze the architecture of the execution engine in deep. Here I will only consider the interpreter, JIT compiler, and profiler.
We already know the interpreter compiles the byte code line by line and executes. Profiler is the component that is responsible for identifying re-occurring byte code blocks. Refer to the code below.
Assume this is the byte code executed by the interpreter (Unfortunately, I don’t know to write bytecode yet ☹️). When we call the executed method of JITCompiler class, it continuously calling myMethod(). So profiler can identify that this method gets called multiple times. Profiler maintaining a counter which counts the number of calls to a particular method. When it passes some threshold value predefined in JVM (You can find this value by using the CompileThreashold flag), the JIT compiler compiles that particular method into native code so that the interpreter can use that native code next time. JIT compiler does not stop its job after compiling the code into machine code. It goes even further. If that particular code block passes the next threshold value, the JIT compiler optimizes the machine code again. This happens in 4 stages. For that, there are two compilers in the JIT compiler. The C1 compiler and the C2 compiler. We also call the client compiler for the C1 compiler and the server compiler for the C2 compiler. It is not about traditional client-server architecture but about the time a Java application runs.
C1 compiler is responsible for levels 1, 2, and 3 compilations and optimizations. C2 compiler is responsible for level 4 compilation and optimization.
Still, this is not the end. JVM tries to optimize the time even more. There is a memory area called code cache. It is like the cache memory in our computer system. It has limited memory, and we can tune code cache with JVM flags. When some code block is called even more, then the JIT compiler moves that method into the code cache so that the interpreter can access that particular code block quickly. But the JIT compiler does not add all the codes into this code cache since it makes a tradeoff. JIT compiler is intelligent to manage this as it can gain maximum efficiency.
We can see these compilation steps with this JVM flag.
-XX:+PrintCompilation
Refer to the following code.
This is the output when we run the above code with the JVM flag. Since I pass 50000 as the argument to the code, it runs until it generates 50000 prime numbers.
In the 3rd column, you can see the stages of the code block's compilation process. Some are in level 0, which are not compiled by the JIT compiler. That means code blocks that are not re-occurred. Some code blocks are in levels 1,2, and 3, which are compiled by the C1 compiler, and some are in level 4, which are compiled by the C2 compiler. If you see the above image carefully, you can see in the 2nd column, and there is the percentage(%) symbol in front of some code blocks. That means those code blocks are in the code cache. Since I have passed 50000 as the argument to the above program, it runs until it generates 5000 prime numbers. So the isPrime() method gets called many times. Thus, the JIT compiler has moved it to the code cache so that the interpreter can execute that method even quickly.
This is how the JVM gains the efficiency back, losing by running Java code on the virtual machine. In this article, I have talked about the process of the JIT compiler in brief. There are a lot more things that happen inside the JIT compiler and the JVM. I guess I will be able to write about those things soon. In this article, there may be a lot of mistakes that I have made. So I truly appreciate your comments.
