Introduction to Virtual Threads

Virtual threads got introduced in Java 19 and got finalized in Java 21 as part of Project Loom. In this article, lets try to delve deeper and understand more about virtual threads.

Why virtual threads?

In Java a thread is the basic unit of concurrency. So a Java application would be a set of threads running concurrently but independently. Each thread maintains its own thread stack in memory which contains the local variables as well as the methods.

Traditionally JDK implemented the threads as wrappers around OS threads. However an OS thread is an expensive system resource — takes ~1ms to launch and 20MB of memory is preserved for the stack. Hence the total number of threads which can be launched in an application is limited by the total number of OS threads we could have.

Platform threads are a wrapper around OS threads (1:1 correspondence)

The server applications usually employ a thread-per-request model, i.e. each request is served by dedicating a thread throughout the lifetime of the request. This implies that if the request does some blocking time-consuming operations, the corresponding OS thread is blocked.

As shown below, if there are a lot of requests which are waiting on blocking operations,

• The hardware utilization will be sub-optimal, since the threads are all in a waiting state most of the time, and not able to serve other requests.
• Further the number of threads which can be launched to serve new requests is also limited since there we cannot launch too many OS threads.

Blocking requests result in sub-optimal hardware utilization

Virtual threads try to address this challenge.

Basic Overview

A virtual thread is an instance of java.lang.Thread that is not tied to a particular OS thread. It is very cheap, much more light weight than regular platform threads and hence available in plenty.

As seen in the picture, there is no 1–1 correspondence between a virtual thread and an OS thread.

How do virtual threads work?

A virtual thread uses a platform thread as a carrier. The virtual thread is executed in the following way,

While executing a virtual thread, it is mounted on a platform thread.

When there is a blocking operation in the virtual thread,

• The thread stack is copied into the heap.
• The virtual thread is unmounted from the platform thread.
• The platform thread will now be released and ready to take on other tasks.

Once the blocking operation is ready to complete,

• The thread stack is copied back into memory from the heap
• The virtual thread is mounted on a platform thread(not necessarily the same) and execution resumes.

A few key considerations

There are a few instances when the virtual thread holds onto the carrier thread as following,

• When executing inside a synchronized block.
• When executing a native method.

This hinders the application from leveraging the advantage virtual threads bring to the table, since the platform thread itself will be blocked.

Summary

Virtual threads are light weight and cheap than regular platform threads which enables us to create more of them with very little overhead.

We will be delving deeper into more of virtual threads with coding examples and structured concurrency in upcoming articles.

Happy learning!

References

JEP 444: Virtual Threads