Java 21 Virtual threads — The Hype is real?

Java 21 introduced virtual threads. Before this, every thread created in JVM (called platform thread) was tied to the OS thread. The platform thread holds the OS thread for entire lifetime of a platform thread. This means the number of platform threads application can create is limited to the number of OS threads.

Virtual threads are not tied to a specific OS thread. Virtual thread still runs a code on OS thread. However, when code running in a virtual thread calls a blocking I/O operation, the Java runtime suspends the virtual thread. The OS thread associated with the suspended virtual thread is free to perform operations for other virtual threads.

In the world of micro-services, a service typically provides features through an api endpoint. A service is dependent on other services, data-stores, etc. to read/write the data and performs business logic to provide a required functionality. Meaning, there are blocking calls to other services, data-stores, etc.

Additionally, number of requests a service can serve on an api endpoint (throughput) is limited by the number of threads it can create. Virtual threads can be useful in such scenarios since we can create, in theory, unlimited virtual threads at the cost of less or equivalent memory.

But these are just words, is there any proof?

Let’s do a performance comparison on the web application with and without using virtual threads with the aim to validate the claims about high throughput and less or equivalent memory utilisation.

Before going further, it is worth mentioning that SpringBoot 3.2 already supports virtual threads.

To use virtual threads, you need to run on Java 21 and set the property spring.threads.virtual.enabled to true.

When virtual threads are enabled, Tomcat and Jetty will use virtual threads for request processing. This means that your application code that is handling a web request, such as a method in a controller, will run on a virtual thread.
- Spring boot 3.2 release notes

The performance testing model is simple.

• Run a spring boot application with embedded tomcat which has a slow HTTP GET endpoint and takes 4 seconds to respond.
• Client (Gatling) hits the endpoint at a constant rate of 400 requests/second for 1 minute.

The more requests application can process successfully, better the performance.

Without virtual threads:

On a Springboot application with embedded tomcat, for each request received it allocates a thread from a pool to handle the request. The thread is blocked until a response is generated.

Maximum number of threads in default tomcat thread pool is 200, meaning we can expect 200 requests to be handled concurrently.

Let’s look at the results using the setup above.

The results show that, due to the limit on number of threads in tomcat thread pool, most of the requests fail even before they can find a thread to accept them on the server.

Let’s see what virtual threads can do.

Using virtual threads (i.e. spring.threads.virtual.enabled=true)

Virtual threads are suspended on a blocking operation allowing an OS thread associated with the suspended virtual thread to perform operations for other virtual threads. This means, in theory, an application using virtual thread executor can handle unlimited requests in parallel. So, we can expect hight throughout and lower memory footprint when using virtual threads.

Let’s look at the results using virtual threads.

All 24k requests were served by the application. Virtual threads clearly show more throughput and less memory utilisation on threads, but more total memory used.

Let’s do the side-by-side comparison for more detailed analysis.

With the virtual threads, an application was able to process 347 more requests per second and memory utilisation by threads is 384mb less as compared to the application not using virtual threads.

Even though the less memory used by threads, the heap utilisation has gone up by 451mb. It is because,

1. The application was serving a significantly a greater number of requests (347 req/sec more) which means more objects on the heap to support these requests.
2. Virtual threads use Java objects within the garbage collected heap to represent stack frames. Hence, more virtual threads mean more stack frames and more heap utilisation.

Conclusion

Based on the results, it can be concluded that using virtual threads:

1. High throughput can be achieved for the applications which perform blocking operations.
2. Lifts the cap on number of threads spawned by the application.
3. Does not yield low latency.

Digging Deeper

If you find this article useful and want to understand about virtual threads in more depth, then articles below are a great resource to start with.

• Java 21 virtual thread docs
• Design of Java’s virtual threads