Virtual Threads. First impressions (part.1)
Overview
Since was officially unveiled, as preview in JDK 19, Project Loom drain attention exactly why promise to deal with concurrent programming in a completely different manner, at least in Java world.
Definitely, one of the most important part of this project are Virtual Threads (JEP-425 / JEP-436) where will going to focus here.
It primary goal is manage the structured concurrency, allowing run, pause and resume tasks while keep them in more readable operations based on imperative paradigm.
Since I have started to take a little deep into Java Virtual Threads, I have found some points that I thought would be interesting to share. So, let’s go!
The problem we want to solve
Nowadays, modern applications constantly challenge engineers towards looking for manners to tackle performance bottlenecks, scalability and high throughput issues.
Currently, web servers work in a “thread per request” model and increase throughput meaning increase the number of available platform threads (~1 MB of memory), just to start to play. In fact, platform thread is a quite expensive resource.
As a workaround, some patterns have been used to handle most common cases. Let’s take a quick look at each of them, pros and cons.
Most frequent approaches
Reactive programming
- Code should be divided into small operations, in the form of Lambda functions.
- It’s necessary to “dirty the code” by adding annotations and special types such as Mono, Flux, Flowables, Observables, and others.
- In this approach we need specialized frameworks (i.e. Spring Webflux, RX Java) that orchestrates lambda execution passing computation output as input to another operation (pipeline).
- It’s hard to debug and write tests. In some cases, even a special mechanism is needed to make tests feasible.
@Service
@Slf4j
@RequiredArgsConstructor
@Transactional
public class UserService {
public Mono<User> deleteUser(String userId){
return userRepository.findById(userId)
.flatMap(existingUser -> userRepository.delete(existingUser)
.then(Mono.just(existingUser)));
}
public Flux<User> fetchUsers(String name) {
Query query = new Query()
.with(Sort
.by(Collections.singletonList(Sort.Order.asc("age")))
);
query.addCriteria(Criteria
.where("name")
.regex(name)
);
return reactiveMongoTemplate
.find(query, User.class);
}
}Asynchronous code based on Completable Futures
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.ExecutionException;
public class CompletableFutureCallbackHell {
public static void main(String[] args) throws ExecutionException, InterruptedException {
CompletableFuture<Void> future = CompletableFuture.supplyAsync(() -> {
System.out.println("Step 1");
return "Result from step 1";
}).thenApplyAsync(result -> {
System.out.println(result);
System.out.println("Step 2");
return "Result from step 2";
}).thenApplyAsync(result -> {
System.out.println(result);
System.out.println("Step 3");
return "Result from step 3";
}).thenAcceptAsync(result -> {
System.out.println(result);
System.out.println("Step 4");
});
future.get();
}
}And then, Virtual Threads arrived!
OK! We’re aware there is no magic. How does it works, tho?
To summarize…
- A special pool (ForkJoinPool running in asynchronous mode) provides platform threads where Virtual Thread will be mounted.
- A Task (Runnable) is executed by run() method of a special object called Continuation, and Virtual Threads is mounted on top of platform thread.
- When Task reaches a blocking call (by I/O operation, for example), a call for Continuation yield() method has been made and Virtual Thread is unmounted and stored in heap memory.
- Meanwhile, the virtual threads remain waiting for the IO call to be completed while the platform thread is completely free to perform another task.
- Once IO call are done, OS emits a signal that calls run() method of Continuation and Virtual Thread stack is moved from heap and scheduled to be mounted again by ForkJoinPool.
- It’s important to notice a Virtual Thread might be mounted either on top of the same platform thread or another one that be available.
And finally an example of Virtual Threads approach:
public List<String> searchTickets() throws InterruptedException, ExecutionException {
List<String> results = new ArrayList<>();
Callable<String> task1 = () -> "Tickets: 01,02,03";
Callable<String> task2 = () -> "Tickets: 04,05,06";
Callable<String> task3 = () -> "Tickets: 10,11,12";
try (ExecutorService executor = Executors.newVirtualThreadPerTaskExecutor()) { // create an executor
List<Future<String>> futures = executor.invokeAll(List.of(task1, task2, task3)); // dispatch tasks executions
for (Future<String> future : futures) {
results.add(future.get()); // get results
}
return results;
}
}- More readable code
- Imperative approach becomes simple and clear to express business logic
- Debug the same way as you’ve already known following execution step by step, introspecting methods and variables.
- Although we still have a blocking code due Virtual Threads nature our throughput increases tremendously.
- Virtual Threads are lightweight structures, so the memory footprint is extremely low, in comparison to platform threads.
- We can still rely on the same concepts regarding parallelism and concurrency (i.e. race condition, mutual exclusion, lock, etc). as well structures and controls in java.util.concurrent structures (i.e. ConcurrentHashMap, ReentrantLock, etc).
Last comments
I was badly excited to take a look closer in Virtual Threads. The manner how engineering has been applied to tackle concurrency and high throughput demand in Java was pretty well thought.
OK! Everything looks great, at least in theory.
I have been working on code examples, close to real scenarios. Then, wait for the second part soon… ;)
See you!
