Eventloop in NodeJS: Ways to avoid blocking EventLoop

A documentation of my learning journey.

Contents:
∘ What is Event Loop?
∘ What is Blocking EventLoop?
∘ Let’s block EventLoop
∘ Now, Let’s unblock the event loop
∘ How to avoid event-loop blocking?

What is the Event Loop?

The event loop is what allows Node.js to perform non-blocking I/O operations — despite the fact that JavaScript is single-threaded — by offloading operations to the system kernel whenever possible.

Since most modern kernels are multi-threaded, they can handle multiple operations executing in the background. When one of these operations completes, the kernel tells Node.js so that the appropriate callback may be added to the poll queue to eventually be executed. We’ll explain this in further detail later in this topic.

This below diagram shows the order of execution of different phase in event Loop

Analogy:

In a coffee shop, like a Barista(read EventLoop) takes orders of every customers queued(read IOs) and the preparation of coffee, handing out coffee, or packing the take away is done by different different clerks (read workers aka c++ apis in different thread) is how an EventLoop (Single Thread — single cashier) works.

Phases in EventLoop:

• timers: this phase executes callbacks scheduled by setTimeout() and setInterval().
• pending callbacks: executes I/O callbacks deferred to the next loop iteration.
• idle, prepare: only used internally.
• poll: retrieve new I/O events; execute I/O related callbacks (almost all with the exception of close callbacks, the ones scheduled by timers, and setImmediate()); node will block here when appropriate.
• check: setImmediate() callbacks are invoked here.
• close callbacks: some close callbacks, e.g. socket.on('close', ...)

Event Loop Interpretation (Left: JS in browsers, Right: nodejs)

What is Blocking EventLoop?

If a thread is taking a long time to execute a callback (Event Loop) or a task (Worker), it’s called “blocked”. While a thread is blocked working on behalf of one client, it cannot handle requests from any other clients.

Let’s block EventLoop

Let’s start with an example on ways to block the IO:

// lets prepare requisites 
touch blockEventLoop.js && npm init -y && npm i fastify -s 

blockEventLoop.js

Run the server…

node blockEventLoop.js

Do a ping, and check for a pong!

curl 127.0.0.1:3000/ping

Let’s add another route with an expensive task….

Let's run …

// in Terminal 1
node blocking_event_loop.js
// in Terminal 2
 while  true; do date; curl 127.0.0.1:3000/ping; sleep 1; echo \n; done;
// in Terminal 3
date && curl 127.0.0.1:3000/block-event-loop && date

As soon as /block-event-loop is called… /ping call stalls. Also check the CPU usage of the node process

If example we are using /ping route as simple healthCheck in k8 environment as liveness probe, this will seem like an unhealthy server and pod is restarted.

What really happened here?

Long-running IO (here crypto function) blocked the event loop

Now, Let’s unblock the event loop:

Let’s add route /async-block-event …

and run

// in Terminal 1
node blocking_event_loop.js
// in Terminal 2
 while  true; do date; curl 127.0.0.1:3000/ping; sleep 1; echo \n; done;
// in Terminal 3
date && curl 127.0.0.1:3000/block-event-loop && date
// in Terminal 3
date && curl 127.0.0.1:3000/async-block-event-loop && date

Nothing much changes though even though we are calling the function wrapped in async-await paradigm.

Why?

We are still blocking event loop by creating too many micro-tasks.

Let’s add route /unblock-event-loop …

Let's run …

// in Terminal 1
node blocking_event_loop.js
// in Terminal 2
 while  true; do date; curl 127.0.0.1:3000/ping; sleep 1; echo \n; done;
// in Terminal 3
date && curl 127.0.0.1:3000/unblock-event-loop && date

Now /ping call succeeds without stalling and we are good to handle incoming requests. CPU utilization is also relatively low.

But there is a caveat…. it takes longer to complete /unblock-event-loop call. That’s the tradeoff we pay but we didn’t block other calls.

Why did calling setTimeout (/ setImmedidate/ process.nextTick) unblock the event loop?

Because of order through which setTimeout, setImmedidate, Promise, callbacks, etc are called in event loop.

Medium

In our microtask example (hash.update function) added more and more tasks to queue, with this event loop didn’t get a chance to handle incoming request (network IO). By calling setTimeout we kind of gave breathing space to execute macro tasks like (network IO call) to execute with completing event loop for each hash.update call made.

How to avoid event-loop blocking?

By avoiding these:

• Long async ops (http, db, file ops. Etc).
  Ex: 1. Sync file reads of large files, blocking db calls
• Timers usage (setImmediate, setTimeout, process.nextTick ) Ex: Timers Getting created but not getting destroyed.
• Open sockets/ too many sockets creation.
  Ex:1 Creating new connection for every API call for intra MS communication => fix keep-alive header usage with max connections
  2. Db connection pools
• DNS queries. Ex: DNS TTL is too low and we need afresh resolve very frequently
• Crypto functions. Ex: Long awaiting crypto functions with large data
• Compressing functions
• Large Sync operations like JSON.parse(…large data…)
• Lengthy regex ops

Additional reads:

• https://nodejs.org/en/docs/guides/dont-block-the-event-loop/

Eventloop in NodeJS: setTimeout, setImmediate vs process.nextTick

Eventloop in NodeJS: MacroTasks and MicroTasks

NodeJS Architecture: Relationship between libUV, V8 and JS

Like to tinker and learn through trials and errors? We might use your quirky mind in our team! Join us in making the next life-centric digital solutions