Spring WebFlux — Under the hood
An overview of how Spring WebFlux works with Netty
Reactive frameworks have become increasingly popular in recent years due to their ability to create resilient and scalable applications that are able to handle large volumes of data without compromising performance and scalability. In this article, I’ll explain how the WebFlux framework, using the Netty server, works under the hood.
Some concepts about Spring WebFlux
Spring Webflux is a framework for building applications following the functional programming paradigm, in a reactive and non-blocking manner. Not familiar with such concepts? I will explain them in more detail.
Functional programming: It is a programming paradigm where we use pure and immutable functions, composing them to create a program. Below is an example of non-functional and functional code in Java:
Non-functional code:
import java.util.ArrayList;
import java.util.List;
public class NonFunctionalExample {
public static void main(String[] args) {
List<String> words = new ArrayList<>();
words.add("cat");
words.add("dog");
words.add("bird");
words.add("fish");
List<String> filteredWords = new ArrayList<>();
for (String word : words) {
if (word.length() <= 3) {
filteredWords.add(word.toUpperCase());
}
}
System.out.println(filteredWords);
}
}Functional code:
import java.util.Arrays;
import java.util.List;
import java.util.stream.Collectors;
public class FunctionalExample {
public static void main(String[] args) {
List<String> words = Arrays.asList("cat", "dog", "bird", "fish");
List<String> filteredWords = words.stream()
.filter(word -> word.length() <= 3)
.map(String::toUpperCase)
.collect(Collectors.toList());
System.out.println(filteredWords);
}
}Non-blocking code: It is a programming model that prevents a task from blocking the execution of other tasks in the same thread, enabling asynchronous execution of tasks.
Reactive programming: The term “reactive” refers to a programming model that reacts to events asynchronously rather than blocking the flow of execution as in a traditional application. Operations in this model are performed asynchronously, generating events in response that are observed by other interested processes. According to the reactive manifesto, there are four fundamental principles that a reactive application must follow:
- Responsive: The system consistently responds to events considering fluctuations in workload or network conditions.
- Resilient: The system remains responsive in case of failure. Failures must be handled within each component or application, without compromising the overall solution.
- Elastic: The system must be able to adapt to demand. Reactive systems generate performance metrics that can be used for scaling policies.
- Message driven: The system is built using asynchronous event flow that does not unnecessarily block any operation.
In Spring WebFlux, these concepts were combined, allowing the creation of applications capable of handling a large number of simultaneous requests in a scalable, efficient and high-performance manner. This is done using fewer threads and less memory, making applications more resilient under load.
By default, Spring Boot with WebFlux uses the Netty server, which operates on the Event Loop Model, leveraging non-blocking and asynchronous processing with the reactor pattern and multiple worker threads. This allows Netty to handle a large number of simultaneous requests with a single thread per event loop, without blocking the execution of other tasks. But how does it work? First, let’s understand the traditional Thread Per Request Model, and then I’ll explain more about Netty’s Event Loop Model.
Thread Per Request Model
Thread Per Request Model is the traditional model used in servlet containers like Tomcat. In this model, each request is synchronously serviced by a thread that is responsible for processing it and returning the response. To handle multiple requests in parallel, multiple threads are used.
When a new request is received, the Tomcat server allocates a thread from its thread pool to process it. As soon as the request completes, the thread is returned to the pool and can be used to serve other requests.
If the application needs to wait for an external response, such as a call to another service, the thread enters a waiting state, waiting for the response. This scenario can result in blocking the request and inactivity of the CPU in the meantime.
However, creating multiple threads has scalability limitations and can lead to excessive resource usage.
While Spring WebFlux can also be used with Tomcat in Servlet 3.1+ implementation that allows non-blocking background processing, it is not recommended.
Event Loop Model
This model uses a single non-blocking thread for processing requests. To better understand, let’s get to know its main components:
Channel: It represents a client server connection. When a request arrives, a Channel is created to handle it. A Channel can also be created to handle I/O operations (eg writing to a database).
EventLoop: It is the main component of Netty, responsible for processing data input and output events. Each EventLoop is associated with a single, never-changing thread. Tasks can be created in EventLoop for immediate or scheduled processing. To make better use of hardware capacity, it is possible to create multiple EventLoops according to the number of processor cores. Each EventLoop can serve several channels simultaneously, ensuring high performance and efficiency in data processing.
Event Queue: Netty uses a queue of events, known as Event Queue. In this queue, tasks wait in order of arrival (first-in-first-out) to be asynchronously processed by the EventLoop. This allows Netty to process tasks efficiently and without blocking the execution of other tasks in the application.
Here is an example of the mechanism used by Netty:
- A request is sent to a Spring WebFlux application with Netty, generating task 3 (in orange in the picture). A Channel is created to handle the request. Note that there are already two tasks (task 1 and task 2) to be executed in Netty’s execution queue (Event Queue).
2. Task 3 goes to the Event Queue to wait for its turn in EventLoop processing. At this point, task 1 is being processed and task 2 is next in line.
3. Meanwhile, another request is sent to the application (task 4 in red) and a new Channel is created to handle it.
4. Task 4 is also added to the Event Queue and waits for its turn to be processed by the EventLoop. Note that at this point, task 2 has already been processed by EventLoop.
5. The EventLoop selects task 3 for processing. This task requires a more time-consuming I/O operation. In order for the EventLoop not to be blocked, this task is sent to another thread to process it. This way, the EventLoop is available to execute the next Event Queue tasks.
6. Task 4 is selected for processing. This task does not require any I/O processing, so it is terminated by the EventLoop and the response is sent back to the client.
7. Task 3 is finished by a parallel thread and goes back to the Event Queue.
8. The EventLoop now executes the next task that was in the Event Queue, which is task 3. The processing completes and the response is returned to the client.
Finally, the complete execution represented above would look like this:
Conclusion
Spring WebFlux is a good choice for projects that require high scalability and efficiency. However, its learning curve may require a bit more effort and time. For projects that don’t need as much scalability or don’t involve processing a large load simultaneously, a simpler choice like Spring MVC might make more sense. For those who want to go deeper into the subject, below are some references that I used to improve my knowledge on the subject:
https://www.digitalocean.com/community/tutorials/node-js-architecture-single-threaded-event-loop