Java 21-Spring-boot virtual thread performance evaluations

Virtual Threads support has been around for a while since the Java 21 and Springboot 3.2 release.
Virtual threads are considered a game-changer for applications that involve synchronous (blocking) implementation of external API calls and blocking database queries. The Java runtime optimizes utilization of blocked platform (OS) threads, providing near-optimal throughput/concurrency without necessitating the implementation of reactive programming code.
I was excited and have been seeking performance evaluations/benchmarking with the Springboot-JVM ecosystem (Kotlin, Springboot, Postgres) for most common blocking implementation in Spring-based web app:

1. Inter-service (Service-to-Service) synchronous HTTP calls.
2. APIs with synchronous calls to a database (using Spring JDBC).

However, I couldn’t find any existing evaluations out on the internet that demonstrate these specific test cases with desired Implantation. Most evaluations /benchmarking , just confined within a single API call which fails to adequately explain or ensure the benefits of virtual threads in inter-service blocking calls and when a backing database has synchronous (blocking) implementation.

Therefore, decided to conduct the performance evaluations on the above specified test scenarios by enabling virtual threads and without virtual thread. Would like to share the findings with you all as many of our products are powered by Spring -JVM ecosystem.

Library and tools versions used:

• Spring-boot 3.2.3
• Java 21 (JDK)
• Kotlin 1.9.x
• Postgres 16.1
• Jmeter 5.6 (for the load generation)

Concurrency:

The maximum number of worker threads has been set to 50 to simulate some level of load (resource contention) on the web server ( spring boot default is 200 workers threads)

Load Simulation in Jmeter : 100 users (threads) with each thread 150 loops counts = 15k req per iteration.

Performance Evaluation:

The performance analysis is conducted, comparing scenarios with and without virtual threads enabled on most common two test scenarios.

1. Service-to-Service blocking (synchronous) HTTP calls
2. HTTP API with blocking calls to DB ( Spring JDBC)

The performance benchmarking represented in the two key performance measurement units: throughput and turnaround time.

This evaluation provides valuable insights into the application’s responsiveness and efficiency under simulated thread configurations, shedding light on the efficacy of virtual threads measured in the turnaround time and throughputs.

Evaluation case 1 — Service-to-Service blocking (synchronous) HTTP calls

The scenario involves Service A initiating a blocking HTTP (REST) call to another web service, Service B for data exchange. Service B takes 20ms to return the response ( simulated with thread.sleep in Service B API) which means Service A thread is in waiting state for a minimum of 20ms for each request. Service B runs within a container, thereby ensuring isolation and execution within separate JVMs. The objective was to assess the impact of enabling or disabling virtual thread settings under identical configurations.

Two iterations, each comprising 15,000 requests, were conducted, resulting in a total of 30,000 requests for comparison. The following results were obtained.

Test 1 — Performance Overview

So virtual thread appears a clear winner here with over 3x performance improvement in the blocking inter services HTTP calls(*the result may vary depending on the workload/blocking). Overall , the finding is, higher the blocking, the higher performance gain with the virtual threads enabled.

Evaluation case 2 — HTTP API with blocking calls to DB (Spring JDBC)

Conducted an assessment to ascertain the potential advantages of employing virtual threads within a web application having blocking long running queries to the database (SQL). The evaluation involved a test web application featuring an exposed HTTP API, which initiates a blocking JDBC query to the database. Notably, the database query intentionally undergoes a 20 ms block (using pg_sleep) to simulate protracted database operations.

Test 2 — Performance Overview

This benchmarking reveals no discernible benefits associated with the implementation of virtual threads in applications grappling with blocking database calls.

Conclusion

Virtual threads significantly enhanced performance in inter-service blocking HTTP calls, yielding a 3x improvement in the current evaluation. the actual performance numbers may vary depending on the server workload and the thread block/wait time, but relative improvements are likely. However, virtual threads did not demonstrate notable benefits in blocking database calls, with similar turnaround times and throughputs observed with and without virtual threads. I think it might require some more time for JDBC drivers to take advantage of the virtual threads.In conclusion, it’s advisable to enable virtual threads for applications that involve substantial external API/IO calls and can leverage blocked/waiting threads to serve other incoming requests thereby improving throughput.