Quarkus — JVM vs GraalVM: Hello world case

Quarkus tailors applications for GraalVM and HotSpot. Amazingly fast boot time, incredibly low RSS memory (not just heap size!) offering near instant scale up and high density memory utilization in container orchestration platforms like Kubernetes. To optimize, they use a technique we call compile time boot.

Quarkus has the capability to operate on both JVM (using bytecode) and GraalVM (employing native machine code). Within the scope of this article, our focus revolves around gauging the performance of a straightforward Hello World scenario for Quarkus, comparing its execution in the JVM against GraalVM. Intuitively, one might anticipate that machine code would outshine JVM in terms of performance. However, the outcome might just be unexpected! Let’s get started.

Setup

All the tests are executed on MacBook Pro M2 with 16G RAM. The software versions are:

• Quarkus 3.5.1
• Java v21.0.1

For load testing, we’ve used Bombardier test tool.

The simple application code is:

package org.acme;

import jakarta.ws.rs.GET;
import jakarta.ws.rs.Path;
import jakarta.ws.rs.Produces;
import jakarta.ws.rs.core.MediaType;
import io.smallrye.common.annotation.NonBlocking;

@Path("/")
public class HelloWorldApplication {

    @GET
    @NonBlocking
    @Produces(MediaType.TEXT_PLAIN)
    public String hello() {
        return "Hello World!";
    }
}

To create a native binary, we’ve used the following maven plugin:

<profiles>
    <profile>
      <id>native</id>
      <activation>
        <property>
          <name>native</name>
        </property>
      </activation>
      <properties>
        <skipITs>false</skipITs>
        <quarkus.package.type>native</quarkus.package.type>
      </properties>
    </profile>
</profiles>

Results

All tests are executed for 10 million requests using 50, 100, and 300 concurrent connections. The results in chart form are as follows. A quick analysis is presented under each chart.

Time taken to finish off 10M requests

At lower levels of simultaneous processing, the native application requires approximately 15 seconds more to complete 10 million requests. As concurrency increases, this time difference decreases to around 4 seconds. However, at all concurrency levels, it’s worth noting that the native code remains slower compared to the Java Virtual Machine (JVM).

Requests per second

As anticipated, the Quarkus JVM demonstrates a consistently higher Requests Per Second (RPS) compared to the native application. Furthermore, it’s noteworthy that the variance becomes inconsequential when the concurrency levels are elevated.

First quartile latency

In the first quartile region, across various levels of concurrency, the Java Virtual Machine (JVM) demonstrates superior latency when contrasted with the performance of the native application.

Mean & Median latency

The JVM maintains its winning streak when it comes to both mean and median latencies, consistently outperforming native code.

Third quartile latency

In the area corresponding to the third quartile, the Java Virtual Machine (JVM) continues to provide superior latency when dealing with low and medium levels of concurrency. Nevertheless, it is noteworthy that at elevated levels of concurrency, the native code exhibits marginally improved performance, marking a notable shift in results (a first-time occurrence).

Maximum latency

For the first time, the native application outperforms the JVM across all concurrency levels, consistently showcasing lower maximum latency in comparison with native code.

Average CPU usage

Following a triumphant phase characterized by maximum latency, the native application once more finds itself surpassed by the JVM in terms of CPU usage. It is worth noting that the native application consistently exhibits elevated CPU utilization levels.

Average Memory usage

Here’s where native applications really outshine JVM applications. Native apps use much less memory compared to JVM apps, several times less, in fact.

Conclusion

Surprisingly, the results turned out differently than we anticipated. We thought the native application would be faster than the JVM application, but it was the opposite. Except for very low memory usage, the JVM application performed better than the native one. This might be because the task we tested was straightforward. It’s possible that the native application could perform better in more complicated situations. We’ll explore that possibility soon.

Thanks for reading!