Open in app

Sign in

Write

Sign in

Helidon Níma — Helidon on Virtual Threads

Tomas Langer
Helidon
Published in
6 min readSep 12, 2022

Helidon 4.0.0-ALPHA1 is now released with our brand new Helidon Níma, providing a virtual threads-based web server. This is an early access release for those of you interested in the latest Java technology, but it is not yet suitable for production use!

To try out Helidon that is production-ready, check out our latest release, Helidon 3.0.0.

To learn more about Níma, we happily invite you to our JavaOne talk — “Helidon, present and (faster) future — LRN1405”

What is Níma?

Níma is a Java 19 (currently early access) based implementation of a server designed for Java Virtual Threads (product of Project Loom).

With the Alpha release, we provide implementation of the following protocols:

  • HTTP/1.1 with pipelining — server and client
  • HTTP/2 server (prototype, known issues)
  • gRPC server (prototype, known issues)
  • WebSocket server (prototype)

Threads

The implementation uses virtual threads and is designed and implemented to provide a stellar, low-overhead, highly concurrent server, while maintaining a blocking thread model. This lets you write code that isn’t complicated by issues commonly encountered in reactive programming, for example.

Socket listeners:

  • Socket listeners are platform threads (there is a very small number of these — one for each opened server socket)

HTTP/1.1:

  • 1 virtual thread to handle connection (including routing)
  • 1 virtual thread for writes on that connection (can be disabled so writes happen on connection handler thread)
  • All requests for a single connection are handled by the connection handler

HTTP/2.2:

  • 1 virtual thread to handle connection
  • 1 virtual thread for writes on that connection (can be disabled so writes happen on connection handler thread)
  • 1 virtual thread per HTTP/2 stream (including routing)

The virtual thread executor services use unbounded executors.

Protocols

The following protocols are implemented in our Alpha release:

  • HTTP/1.1 with extensible upgrade mechanism
  • HTTP/1.1 WebSocket upgrade implementation
  • HTTP/1.1 to HTTP/2 plaintext (h2c) upgrade implementation
  • HTTP/2 with extensible “sub-protocol” mechanism
  • HTTP/2 gRPC sub-protocol implementation
  • Extensibility for other TCP protocols (including non-HTTP)
  • Server side TLS support for any protocol
  • Mutual TLS support for any protocol
  • Extensible application layer protocol negotiation (ALPN), used by HTTP/2 (h2)

Routing

The same web server can be used to route to multiple protocols (e.g. you can have a single port that serves HTTP/1.1, HTTP/2, WebSocket and gRPC). HTTP/1.1 routing is implemented by default (as other protocols upgrade from it); other protocols are separate modules that can be added.

  • HTTP routing is version agnostic — the same routes can be used for both HTTP/1.1 and HTTP/2
  • Support for protocol-specific routes — a route that only serves HTTP/2 is possible
  • gRPC routing (unary, server streaming, client streaming, bidirectional)
  • WebSocket routing (currently does not implement sub protocols)

Features

The following features are implemented and can be tried:

  • Tracing support — using existing Helidon tracing implementations, such as Jaeger or Zipkin
  • Static content support — from classpath or file system
  • CORS support
  • Access Log support
  • Observability endpoints (health, application information, config)
  • Fault Tolerance (Bulkhead, Circuit Breaker, Retry, and Timeout features)
  • HTTP/1.1 client
  • Testing support

Getting Started

There are a few prerequisites:

  1. git installation (needed to clone the repository); workaround: download a zip of the repository from GitHub and unpack it
  2. Apache Maven 3.8 installation which can be downloaded from https://maven.apache.org/download.cgi
  3. Java 19 with virtual thread support, now released and available at: https://www.oracle.com/java/technologies/downloads/#java19

To run Níma:

  1. Clone https://github.com/tomas-langer/helidon-nima-example repository
git clone https://github.com/tomas-langer/helidon-nima-example
cd helidon-nima-example

2. Run Maven from the root of the repository

# Linux
./mvnw clean package# Windows
./mvnw.com clean package

3. Run the Níma service (requires preview, as virtual threads are a preview feature in Java 19)

java --enable-preview -jar nima/target/example-nima-blocking.jar

4. Exercise one of the endpoints

  • Single response
curl -i http://localhost:8080/one
  • Sequential execution of multiple endpoints
curl -i http://localhost:8080/sequence
curl -i http://localhost:8080/sequence?count=5
  • Parallel execution of multiple endpoints
curl -i http://localhost:8080/parallel
curl -i http://localhost:8080/parallel?count=5

Sources of the example may be found in the checked out project in module nima:

  • NimaMain — the main class that configures routing and starts the server
  • BlockingService — HTTP service with endpoints excercised above
  • resources/application.yaml — configuration of the server

Blocking vs. Reactive

Let’s compare the implementations between Níma (blocking) and Helidon SE (reactive) for the same task.

We need to establish the basic rules for each framework:

  • Reactive — you cannot block the thread of the request, this is supported through reactive stream APIs, such as in Helidon’s Multiand Single
  • Blocking — you must not finish the response asynchronously (e.g. the response must be sent from the same thread to which the request was issued)

Note: In either case, you should not “obstruct” the thread. Obstruction is a long-term, full utilization of the thread. In a reactive framework this would consume one of the event loop threads effectively stopping the server. In blocking (Níma) this may cause an issue with the “pinned thread”. In both cases this can be resolved by off-loading the heavy load to a dedicated executor service using platform threads.

Simple Case

In this example we will call another service and write the response to our client. When an exception happens, it returns an internal server error.

Blocking:

Reactive:

We can see that with blocking code, we can just get the response and send it. If anything happens, the default exception handler will correctly handle the exception, or send an internal server error.

On the other hand, with reactive, we must work with a reactive stream and handle exceptionally, as otherwise the exception is lost.

Advantages of blocking code:

  • Straightforward exception handling
  • Meaningful stack traces (including thread dumps)
  • Easy debugging
  • Less scaffolding

This becomes more visible with a slightly more complex use case…

Complex Case

In this example, we will call another service in parallel, combining the results into a single response.

We expect to get the number of parallel requests from a query parameter; we will have it in variable “count”. Also, handling of InterruptedException and ExecutionException is not shown here, even though it must be done (in the next Alpha release, handlers will allow throwing a checked exception).

Blocking:

Reactive:

Although we have achieved the same with reactive code, the code is much less readable (and is difficult to write correctly).

Performance

Our primary focus was performance. Below we show the current numbers (ALPHA-1 release) compared to a non-blocking implementation on pure Netty (version 4.1.36.Final — no extra features, just HTTP).

These are quite simple benchmarks carried out on a single machine using loopback interface (e.g. there is a known interference of client and server processes, as they share the CPU). Nevertheless it provides us with a quick comparison in performance, to see if we can compare to a non-blocking implementation. As with any performance testing, the results will differ depending on a lot of factors (specifically how performance optimizations are done for Linux environments).

Testing machine:

Intel Core i9–9900K @ 3.6 GHz x 16
32 GiB memory
Ubuntu Linux

Tools:

HTTP/1.1 benchmark done using wrk

HTTP/2 benchmark done using h2load

Note: Níma was used as designed — with fully featured routing, allowing full blocking and simple-to-use routes; Netty test is the “smallest” possible example that is designed to handle a single type of request; Dropwizard is a framework with inversion of control and runtime dependency injection.

Results:

Performance results (requests/second)
Performance results (requests/second), HTTP/1.1 with pipelining

Note: What we can see from these numbers (and what is our goal with Níma) is that we can achieve performance comparable to a minimalist Netty server, while maintaining a simple, easy to use programming model.

To repeat these tests:

HTTP/1.1 using wrk:

HTTP/2 using h2load:

h2load -n 50000000 -t 5 -c 5 -m 100 https://localhost:8081/plaintext
  • Netty: customized TechEmpower benchmark (as mentioned above) with HTTP/2

What’s Next

To learn more about Helidon, and specifically Helidon Níma, please join us at the following conferences:

To get answers, contribute and stay informed:

Helidon
Nima
Loom
Virtual Threads

--

--

Tomas Langer
Helidon

Written by Tomas Langer

78 Followers
Editor for

Help

Status

About

Careers

Press

Blog

Privacy

Terms

Text to speech

Teams