Performance benefits of virtual threads: Hello world HTTP server

From v19, Java has introduced virtual threads as a preview feature. These remain preview in v20 too. In their own words (credit: openjdk):

Virtual threads are cheap and plentiful, and thus should never be pooled: A new virtual thread should be created for every application task. Most virtual threads will thus be short-lived and have shallow call stacks, performing as little as a single HTTP client call or a single JDBC query. Platform threads, by contrast, are heavyweight and expensive, and thus often must be pooled. They tend to be long-lived, have deep call stacks, and be shared among many tasks.

In summary, virtual threads preserve the reliable thread-per-request style that is harmonious with the design of the Java Platform while utilizing the available hardware optimally. Using virtual threads does not require learning new concepts, though it may require unlearning habits developed to cope with today’s high cost of threads. Virtual threads will not only help application developers — they will also help framework designers provide easy-to-use APIs that are compatible with the platform’s design without compromising on scalability.

Virtual threads are generally considered one of the biggest changes that happened in Java.

In this article, I’m trying to find out the performance benefits brought in by virtual threads. I’m going to run a simple hello world HTTP server with — native thread pool & virtual threads. Let’s see what do we get out of the new, promising technology.

Test setup

The tests are executed on MacBook Pro M1 with 16G of RAM. The java version is v20. I’ve used — enable-preview to get virtual threads.

The base code is as follows:

import java.io.IOException;
import java.io.OutputStream;
import java.net.InetSocketAddress;
import java.util.concurrent.Executors;

import com.sun.net.httpserver.HttpExchange;
import com.sun.net.httpserver.HttpHandler;
import com.sun.net.httpserver.HttpServer;

public class NativeHelloWorldServer {

    public static void main(String[] args) throws Exception {
        HttpServer server = HttpServer.create(new InetSocketAddress(3000), 0);
        server.createContext("/", new MyHandler());
        server.setExecutor(Executors.newCachedThreadPool());
        server.start();
    }

    static class MyHandler implements HttpHandler {
        @Override
        public void handle(HttpExchange t) throws IOException {
            String response = "Hello world!";
            t.sendResponseHeaders(200, response.length());
            OutputStream os = t.getResponseBody();
            os.write(response.getBytes());
            os.close();
        }
    }

}

For using virtual threads, I’ve changed one line:

//server.setExecutor(Executors.newCachedThreadPool()); -- CHANGED TO FOLLOWING
server.setExecutor(Executors.newVirtualThreadPerTaskExecutor());

Execution

Each test is executed for 5M (5 million) requests. The tests are executed for 10, 100, and 300 concurrent connections. The load test is carried out using Bombardier HTTP testing tool. A warm-up of 1000 requests is given before taking readings.

The following are the tables showing results for each concurrency level:

Analysis

First, virtual threads don’t look as groundbreaking as I thought.

As concurrent connections increase, the benefits of virtual threads decrease. At low concurrency, virtual threads finishes 5M requests in 55s, while platform threads take 68s. At high concurrency, the difference reduces to 5s only.

In terms of resource usage, the CPU usage is about the same for both. The memory usage of virtual threads is always lower than platform threads.

Am I missing something here? Please comment if this is not the right use case for virtual threads.

For a comparison of virtual threads with physical threads for a more realistic file server case, you can visit the article here.