Understanding Nodejs(V8) and Spring Webflux(Netty) — based on Event Loop design
For the last five months, I have been using Spring Webflux in my current project, and I like how easy it is to get started writing functional-style code which is asynchronous and reactive.
I have also written a good number of Nodejs applications, and that also works pretty much the same way using Promise style coding (async/await) while Webflux uses Mono/Flux.
On one side Nodejs is popular, and with time more folks are creating awesome applications using it. On another side, Spring Webflux is also gaining popularity with time, and new projects are using Spring Webflux to create non-blocking asynchronous reactive Applications.
I started googling, what distinguishes between Nodejs and Spring Webflux as both are solving the same problem. While reading online articles, I realized that there are a few blogs/articles which does a fair comparison, and others have just done a comparison between Spring boot (Tomcat) with Nodejs which according to me doesn’t seem fair as both are using different Thread Model and request handling.
If you have a long-running task, then Nodejs is not appropriate, and it is always good to go for servers designed for such use cases. Even If we use Spring Webflux, it is also not designed to handle the long-running task, so it doesn’t make sense to compare Nodejs with Spring boot (Tomcat).
In this article, I discuss different components used when a request reaches Nodejs(V8 JavaScript runtime engine) and Spring Webflux(Netty) applications. It is focused on event loops as both Nodejs and Spring Webflux use this design approach to handle the I/O request asynchronously in the application.
Short Answer:
They both are the same and give the same performance for the basic CRUD application. It is interesting to see that, though they are using the event loop approach, the internal implementation for both the server/framework is different. If we are looking for JVM based framework, then Webflux is one of the possible choices to go for.
Long Answer:
Let’s see a comparison based on the below points that will help us understand what is different in both the design approach and What is happening internally?.
Thread Model
It is basically about thread management. How created threads have an impact on application performances. We need to understand the thread modeling before deciding on any framework. It also means, How code is going to execute. Multithreading can be complex, as our application evolves, there will be other thread-related problems that can arise.
Nodejs:
- Nodejs application runs on a single thread, and it uses an event loop that runs on the same thread that is nothing but the main thread that allows running one event/process at a time.
- It leverages the multithreading feature of the Modern OS kernel that does those execution and callback things for us with the help of Libuv, which also has its thread pool for I/O tasks to use if required.
- Per the thread, there is one event loop.
Webflux:
- A task that runs continuously and checks for available events that can be picked up and executed can be referred to as an event loop. The basic idea behind the event loop can be simplified as below,
- loop until not terminated:
= block until events are ready to be picked up
= for every event which is an instance of Runnable got in the previous step, start executing/processing it. - The event loop uses a single thread and never changes, which uses executor service.
- asks can be sent directly to the event loop for immediate or scheduled execution.
- In Netty, we can configure multiple event loops, and a single event loop can handle multiple channels. (Channel is a component providing users a way to process I/O operations, such as read and write)
- Events and tasks are executed in FIFO order, so it guarantees to order and avoids data corruption.
- Per the thread, we can have multiple event loops controlled by the parent EventLoopGroup.
Asynchronous Operations
Nodejs:
- Nodejs internally uses Libuv for handling asynchronous tasks. Here it will try to offload as much to the OS kernel as the modern Kernel is capable of.
- If Libuv is unable to delegate the task to the kernel then it uses its created thread pool(default 4 thread) to handle the work.
- Whenever possible, Libuv will use those asynchronous interfaces, avoiding usage of the thread pool.
Webflux:
- In Netty 4, all I/O operations and events are handled by the same thread which is assigned to the event loop.
- Whereas in Netty 3, there is a separate event loop for the inbound event which is handled in the I/O thread pool, the outbound event might be in the I/O thread pool or another pool.
Event Loop Structure
Nodejs:
- An event loop is like a process that has a set of phases with specific tasks. It cycles through it in a round-robin fashion to handle different events.
- While there are queue-like structures involved, the event loop does not run through and process a stack. Here each phase has its queue.
- Event Loop has multiple phases to handle the events, and they are a timer, pending callback, idle and prepare, poll, check and close callbacks.
- To understand more, refer to the below link,
Webflux:
- Event Loop has its task queue.
- Whenever a new request comes to the application, it is stored in the java heap, one of the event loops will pick it from the java heap and do processing, and add the result back to memory.
Task Scheduling in Event Loop
Nodejs:
- Everything that was scheduled via setTimeout() or setInterval() will be processed in the timer phase of the event loop.
Webflux(Netty):
- We can schedule tasks using an event loop. Here event loop extends ScheduledExecutorService that does thread pool management.
- If we use ScheduledExecutorService directly, then in heavy load this has performance costs and can become a bottleneck if tasks are aggressively scheduled.
- Netty addresses this problem by using the channels event loop.
- After 60 seconds, our code will be executed by the event loop assigned to the channel. Since the event loop uses ScheduledExecutorService so we can access the same set of APIs that JDK provides to handle different cases.
CPU Utilization
Nodejs:
- A Node.js application runs on a single thread. On multicore machines that means that the load isn’t distributed over all cores.
- Using the cluster module that comes with Node, it’s easy to spawn a child process per CPU.
- Each child process maintains its event loop, and the master process transparently distributes the load between all children.
Webflux:
- We understood that a single thread is used by the event loop so it means if there is any long-running task that got scheduled to run, then it will block the current thread from picking a new task, and indirectly we can see a big performance impact.
- If we need to run a long-running task then it is better to create a separate thread executor pool and handle it there. The returned result can be later picked by the event loop. This allows the event loop to unblock itself from the long task.
- We can also increase the event loop instance to increase CPU utilization.
Tuning the Thread Pool
Nodejs:
- Libuv will create a thread pool with the size of 4.
- The default size of the pool can be overridden by setting the environment variable UV_THREADPOOL_SIZE.
Webflux(Netty):
- The default size of the event loop thread pool is two times the available Processor.
- The pool size may be modified as per need.
Handling Backpressure Problem
Backpressure in software systems is the capability to overload traffic communication. In other words, emitters of information overwhelm consumers with data they are not able to process.
Eventually, people also apply this term as a mechanism to control and handle it. It is the protective actions taken by systems to control downstream forces. In other words, control how many elements the recipient can consume.
Nodejs:
- The HTTP server that is called returns data after 1s to simulate a slow backend. It may cause backpressure as requests waiting for the backend to return pile up inside Node.
- It is interesting to see, Event loop duration and frequency are dynamically adapted, which means that the metrics under no-load are similar (low frequency, high duration) to an application that talks to a slow backend under high load.
- To implement backpressure in the stream, we can use a Readable and Writeable Stream with a highwatermark to efficiently handle the backpressure between producer and consumer of the data.
- To understand more, refer to the below link,
Webflux:
- The responsibility for backpressure is managed by the Project Reactor. It internally uses Flux functionalities to apply the mechanisms to control the events produced by the emitter.
- Webflux uses TCP flow control to regulate the backpressure. But it does not handle the logical elements the consumer can receive.
- There are three ways available in Flux, using which we can control backpressure.
- Option 1: Using request(), the consumer has the control to let the publisher wait until it gets the request for new events. In short, consumer subscribes to the events and process them based on demand.
- Option 2: Using limitRange(), We are setting the number of items to prefetch at once. The limit applies even when the consumer requests more events to process. The publisher splits the events into chunks avoiding consuming more than the limit on each request.
- Option 3: Using cancel(), the consumer can cancel the events to receive at any moment. we can cancel the subscription and later subscribe again to continue receiving the next events.
- To handle backpressure between client and server, we can Channel.isWritable() to check whether to wait or send the next events by calling Channel.write() or we can also list to fireChannelWritabilityChanged event to decide when to send more data to the channel.
- To understand at the code level, refer to this link, https://www.baeldung.com/spring-webflux-backpressure
Wrapping-Up
Based on the above points, I don’t see one is better than the other as both have some pros and cons. But on the majority side, they are the same performance-wise. So the decision can be made based on skills availability, How willing the team is ready to adopt change, the project ecosystem, etc.
As time goes on, both Nodejs and Webflux are improving and adding what is lagging so they both are converging to provide the equivalent feature. I feel this is nice, and it helps to pick up new projects/frameworks fast as the learning curve is reduced.
To understand more, refer to the below References:
