The Issue With Java Virtual Threads

Phoenix Rising
4 min readDec 29, 2023

--

What are Virtual Threads?

A lot has been written already on the ground breaking aspects of Java Virtual Threads , with Java 21 release this feature is no longer preview and we are seeing some adoption already. As I understand it, Java virtual threads provide the means to create more number of threads, than one can using the traditional mechanism of using platform threads.

Platform threads are essentially wrapper threads running on top of OS threads and hence expensive because OS threads are meant to be general purpose and each thread initialisation takes up considerable memory (~2–10 MB). Therefore, at some point we will run out of memory and we can forget about running millions of threads in parallel in some sort of Utopian concurrency dream.

Virtual threads on the other hand, shift the burden of the context switching on the JVM. A concept of “carrier” threads is introduced which has a 1:1 mapping with OS threads. The key magic is that virtual threads can be scheduled on these “carrier” threads and whenever a virtual thread blocks on a blocking operation, the “carrier” thread is released and is free to take on other virtual threads. The stack context is copied over to the heap by the JVM and once the blocking operation is complete the virtual becomes eligible for scheduling on any of the carrier threads. Creating these virtual threads is cheap and hence we achieve the promised land of millions of threads running concurrently and in peace. So what is the catch?

The issue of pinning

What happens if the virtual thread that is riding the “carrier” thread, creates a scenario where the “carrier” cannot be released ? We know if this happens, the other virtual threads (millions of them apparently) will be waiting for a long time to be scheduled. Therefore, we basically lose all the scalability benefits of virtual threads (even though the application itself maybe running “correctly”). This phenomenon is called “pinning” or we can say that virtual thread is pinned to the carrier thread (which is not good).

According to Oracle pinning can happen in two scenarios:

  • Virtual thread runs code encapsulated with the synchronized keyword.
  • Virtual thread runs a native method of foreign function.

Depending on what domain we work in, it is possible that we encounter the first case more than the second. I have not seen a good solution to address the second problem but the suggested solution to solve the first problem is to replace the synchronized block with Reentrant lock. For example, if we had something like the following been used by some thread:

public void survivingDinosaurMethod(){
synchronized(myLockObject){
// take your sweet time here
}
}

To get the value out of using virtual threads, we would need to replace it with:

public void noLongerADinaosaurMethod(){
lock.lock();
try{
// still taking its sweet time
} finally{
lock.unlock();
}
}

The documentation mentions that this needs to be done only if the method is long lived or frequently used, which makes some sense.

So what really is the issue if we have a way to get around it? Well all modern applications use tons of libraries , some of which are buried deep in a dependency tree, especially if it is a cloud based one. So one fine day , you may migrate your application to Java 21 and someone has the bright idea to use virtual threads and now you have something like:

Thread
.startVirtualThread(() -> {//old stuff in new bottle})
.join();

You make sure that you have gotten rid of all the synchronized keywords and replaced them with locks. However, you do not see any scalability benefits at all because you are using some library that does some fancy stuff that you need and that library has not yet decided to replace the synchronized blocks or methods in its own code base. If it is an open source library , you could probably try to do the replacement of the synchronized code with locks yourself (it would not be straight forward always) but if the library is proprietary , you are basically stuck until the maintainers decide to do the damn change. Alternatively, you may hope that the next release of Java solves this. Is it easy to find out of any dependent library will be a roadblock to the virtual threads journey?

Detecting Pinning

The documentation mentions a cool JVM parameter, that you can use:

-Djdk.tracePinnedThreads=full or -Djdk.tracePinnedThreads=short

A quick way to simulate this is to execute this code with the -Djdk.tracePinnedThreads=full parameter:

package com.sunny.java.virtual.threads;

public class VirtualThreadsSimulation {

public synchronized void pinningSimulation() {
try {
Thread.sleep(3000);
System.out.println(STR."Work done and isVirtual \{Thread.currentThread().isVirtual()}");
} catch (InterruptedException e) {
throw new RuntimeException(e);
}
}

public static void main(String[] args) throws InterruptedException {
VirtualThreadsSimulation virtualThreadsSimulation = new VirtualThreadsSimulation();
Thread.startVirtualThread(virtualThreadsSimulation::pinningSimulation).join();
}

}

and we can get an output like:

detecting pinned virtual threads

The detection is based on the logic that if an operation blocks, it triggers an onPinned event which does a printStackTrace.

Alternatively, one can also use the Java Flight Recorder to detect the jdk.VirtualThreadPinned event as detailed in the documentation.

Conclusion

I believe that many libraries have already made their codebase virtual threads friendly but there are others who are catching up. Ultimately we will need the Java ecosystem to migrate or the Java framework needs to provide us with means to use virtual threads with legacy code (using Java 21).

P.S: Do you see anything that I misunderstood? or do you have any suggestions/comments ? Please feel free to suggest in the comments section.

👍🙌😀🚀

--

--