Shifting from Java to Node.js: Duel to the Death
It was the beginning of this winter when they told me the bitter news. I arrived very early in the morning (11 am — programmer morning) to the office so it was mostly empty.
I was drinking my cup of java, with soy milk of course, and Alice (my boss) approached me. I suspected something was wrong, because no one more than my boss, knows that my 15 minutes of morning java are precious.
She said that I need to sit down before I hear it, although I was already sitting. She told me that due to a reorganization of our division, I will now own a new Node.js service, and the list of features is endless.
Alice knew it was bad news for me because I was not a fan of Node.js, and knew I had a bad experience with it in the past.
She tried to convince me things have changed: “Listen Eliran, it’s not how it used to be! Take TypeScript for example — certain errors that you could only detect on runtime, you can now detect during compilation, just like in Java”.
I immediately replied: “Why write code like Java when we have real Java?”.
“Node.js scales much better!” she said.
Not knowing how to react, I suddenly stood up. The headphones wire (yeap, it wasn’t Bluetooth) was still connected to my laptop and my coffee spilled all over the new t-shirt I got at the last Java conference.
As I ran towards the ping-pong area to relax, I shouted “Numbers don’t lie! Numbers don’t lie!”. That’s because I already knew what I was about to do — I was going to have a midday duel between her inferior Node.js and my dear Java.
Midday duel rules
- The duel has 2 participants — a Java-based service (a.k.a “Snake Charmer”) and a Node.js-based service (a.k.a “Black Mamba”)
- Each participant is basically a small service that invokes a third-party service
- The third-party service:
- It is assumed to have all the resources to scale as needed, meaning this service won’t be our bottleneck.
- Its performance is assumed to be given (average response time is around 3 seconds).
- It is executed by invoking
GET http://localhost:8090/task
4. Using Gatling I prepared this simulation that invokes 100 client requests per second, for 60 seconds. I will use it to measure the average response time for each participant.
5. The winner is the one that has the lowest average response time.
Let the games begin!
Duel #1
Snake Charmer (Java 8) armed with the following weapon:
- Spring Boot 2.2.6
- Tomcat embedded 9.0.29
Black Mamba (Node 12) armed with the following weapon:
- Express 4.16.1
Snake Charmer was ready to fight within a minute (using Spring Initializer), but Black Mamba (to my surprise), was even faster and was locked and loaded after just a few seconds (via command line: $express <app name>).
Snake Charmer’s code:
Black Mamba’s code:
Snake Charmer, my favorite, drew first:
The 95-percentile response time (32 seconds) was not even close to 3 seconds.
Well… 100 requests per second are quite a lot, so I assumed these are pretty good results.
Black Mamba was next:
The 95-percentile response time was around 3 seconds. I assumed I made a mistake, so I verified the simulation parameters (100 requests per second, for 60 seconds) and tried again.
I got the same results — 3 seconds average response time!
So Black Mamba was the clear winner — Node.js performed 10 times better than java in this use case. W**?!
While staring at my monitors and trying to understand what I did wrong, Alice got back from her lunch. I have minimized all related apps on my desktop, so she won’t notice (open-space, you know…) and switched to an online book (Node.js design patterns).
I decided to deal with it when I get back home.
Feeling less embarrassed at home, I have reopened all related apps and started checking why Node.js performs better.
I was already experienced with Java and by monitoring the logs and resources (jvisualvm) I identified the bottleneck — it was Tomcat threads.
Every request is executed on a separate thread, and if there are no available threads (because all threads are waiting for a response from the third-party service, i.e blocked by I/O), then requests are starting to pile up.
Therefore, the average response time increases…. make sense.
Here’s a screenshot from jvisualvm — you can see:
- Almost immediately, all Tomcat threads (200 by default) are started
- Even so, the CPU utilization remains extremely low
Next, I wanted to understand how the Node.js process utilizes the resources. Using the top command, I’ve noticed that the number of threads within the Node.js process does not increase as the number of incoming requests increases.
How come the fixed number of threads is not a bottleneck?
Something should change to accommodate the increasing number of incoming requests. I’ve done some research, and discovered clinic.js Doctor, which is a common tool for monitoring Node.js servers. So, I ran the simulation again, and this time used clinic.js Doctor for monitoring:
clinic doctor — node appI examined the results, and one specific metric, called “Active Handles”, caught my eye:
As it turns out, Node.js delegates the TCP operation (i.e invoking the third-party service) to some kind of entity called libuv, and the “Active Handles” metric indicates the number of delegated tasks that have not yet been reported as “complete”.
By checking libuv documentation (see diagram below), we can see that Node.js does not start a new thread per TCP operation. In fact, it doesn’t use its own threads at all for TCP operations, so those threads can’t be the bottleneck.
Instead, libuv utilizes the OS I/O event notifications by using the epoll API (for linux, or kqueue for MacOS), which allows the application to continue its execution, and to get a signal from the OS once the TCP operation completes.
Duel #1 result — Node.js wins!
Revenge was my only medicine…
These dramatic results took me back to a confusing time — July 2011:
Kevin James was talking to animals in “Zookeeper” (the movie, not the apache service), and at the same time, it was announced that the latest Java version can communicate with the OS. It can instruct the OS to perform I/O operations in the background, and wake up a thread when the I/O operation has finished.
Going back to 2020, even though this concept of non-blocking I/O (NIO.2) was introduced in JDK 1.7 (almost 9 years ago), it is still complex to use, while in Node.js it’s much more natural.
Anyway, to get the same scaling capabilities in Java (as we can get in Node.js), having the ability to delegate tasks to the OS is not enough. I need some kind of a single-threaded entity that will :
- Continuously poll for new events sent from the OS (e.g. incoming data from network sockets).
- Pass those new events to the appropriate event handler.
What I described is actually a special case of “event loop” (or “message dispatcher”) design pattern.
Googling “Java NIO event loop”, one of the first results would probably be Netty. Netty is a “NIO client server framework which enables quick and easy development of network applications such as protocol servers and clients”, and it even has an event loop.
Here you can see examples of using Netty event loop, but even without reviewing the code examples, the problem is clear. As a Node.js developer, things are simpler (as opposed to Java), because I don’t have to be aware of the event loop (I mean — explicitly in the code; knowing the concept is vital).
Hence, my dream is to have another abstraction layer on top of Netty, which will allow me to write code that’s more declarative, instead of taking care of non-blocking capabilities.
Introducing Spring WebFlux framework — when dreams come true!
Duel #2
Since Snake Charmer lost the first duel and was taken out of the game, Pai Mei (Java 8) stepped-up, armed with the following weapon:
- Spring Boot 2.2.6
- WebFlux
Black Mamba (Node 12):
- Same Node.js service from the previous duel
To arm Pai Mei, I used Spring Initializer (similar to the first duel), but this time I added the “Spring Reactive Web” dependency (see below).
Pai Mei’s code:
*Black Mamba’s code (Node.js) hasn’t changed since the first duel.
Time for payback!
It was time for Pai Mei to demonstrate his strength, and for me to keep my toes crossed (crossing my fingers would have blocked operations that are more useful, like typing…):
Except for the understanding that toes-crossing is hard to uncross after some time, I was very satisfied — the 95-percentile response time was similar to Node.js — about 3 seconds.
Also, monitoring the java process using jvisualvm, I have noticed that the number of threads is fixed during the entire time and lower than the previous duel (19 vs 218).
WebFlux does a great job abstracting the non-blocking capabilities, so we need a lot fewer threads to handle the same amount of incoming requests.
Epilogue
I arrived at the office earlier than usual and was ready to share the good news. After about an hour, Alice showed up in our cubicle with her green smoothie (she doesn’t drink coffee of course).
Excited, I showed Alice the results. Her first response was: “By your test results, it seems that Java has no advantage over Node.js, as they have the same response time. What about the heap size?”.
“Take a look at the jvisualvm and clinic.js screenshots, Alice”, I said.
“Well, Eliran… it seems that Java uses 280 MB and Node.js uses only 38MB!”
Hovering with her skateboard and green smoothie toward the ping-pong area, she shouted: “Numbers don’t lie! Numbers don’t lie!
@elirandav
