Blocking Code Blues: Frontend Server’s Worst Nightmare

Picture this: It’s a sunny afternoon, and your website is buzzing with activity. Users are happily clicking away, eagerly awaiting the lightning-fast responses they’ve come to expect. But suddenly, on some pages, API requests time out, and even temporary downtime for a few minutes is experienced with 503 error and your once-happy users are left staring. What’s going on? Welcome to the world of synchronous code — a seemingly harmless addition that turned into a front-end server’s worst nightmare.

We’ll dive deep into how a few lines of blocking code can choke the front-end server. Buckle up, as we navigate through the synchronous execution and uncover the path to a more resilient, responsive frontend.

So, here’s the scoop: A while back, we rolled out this slick new feature letting users rate vendors on our platform. Sounds simple enough, right? Well, we needed an API to fetch the top-performing vendors for a client company. In an ideal world, we’d whip up a single API on the backend to get the
whole enchilada in one go. But guess what? The backend query seemed heavy, and we were racing against the clock.

So, in our desperation, we rigged up an API on our Next.js server. This API made a bunch of network calls to a few backend APIs, mashed up all those responses, and spit out the final list of top-notch vendors. It was kind of a makeshift, but it worked… until it didn’t.

Little did we know, our makeshift solution was a ticking time bomb, ready to wreak havoc as soon as the user traffic spiked. And boy, did it spike! The server got choked, and our users were left high and dry. Lesson learned: shortcuts and quick fixes can come back to bite you hard.

Fun Time

But before we dive into the nitty-gritty code, let’s tackle a burning question. We all love sprinkling async/await in our JavaScript, but does it block the main thread of execution?

By “main thread,” we’re talking about the event loop. (If you’re scratching your head wondering what the heck the event loop is, do check out this talk — it’s a game-changer.)

Now, let’s have some fun and see if we can figure out the output of this code snippet:

The output will be:

You might notice that the statement “Task ends” prints first, even though we used await in the fn method.

So what’s the deal?

The reason is that async/await doesn’t block the main thread (the event loop). It pauses the execution within the async function (in this case, fn) until the next promise is resolved. So, the code for msg2 will only run after the promise for msg1 is resolved, and so on. These steps are picked up again from the callback queue and moved into the call stack later, following the usual event loop mechanics.

In JavaScript, code execution is non-blocking, and asynchronous operations are handled by the event loop.
The event loop continuously checks if any tasks need to be processed on the task queue.

When an async operation is initiated just like the fn method, it’s handed off to a system outside of the JavaScript runtime (like the web browser or Node.js APIs), which processes it in the background. Once that operation is complete, a callback for that operation is queued on the task queue, allowing the rest of the code outside fn to continue running in the call stack. That’s why the control isn’t blocked, and “Task ends” gets printed first.

In a nutshell, async/await makes our async code look synchronous, but under the hood, it’s all about keeping the event loop-free and clear. Neat, huh?

What happened in our case?

Now let’s examine what the culprit code looked like that went live

Now here the API handler method is also async and inside for…of loop await is being used for FetchDetails and CheckIfAlreadyRated, both these methods make network requests to the backend server.
When using await inside a for…of loop, each iteration of the loop will wait for the asynchronous operation to complete before moving on to the next iteration. This can have a significant impact on performance, particularly when dealing with API calls or database queries.

Here’s why this happens:

Now, when you use await, you’re telling JavaScript to “pause” the execution of the enclosing async functions and wait for the Promise to resolve before continuing. But this “pause” doesn’t block the entire thread; it only pauses the current async function. While the function is waiting for the Promise to resolve, other tasks on the event loop can still be executed.

However, inside a for…of loop, the use of await means that these pauses happen back-to-back. Each loop iteration must finish its asynchronous work before the next iteration starts. As such, none of these operations overlap in time, and they don’t take advantage of possible concurrency. The event loop waits for the promise from one loop iteration to settle before proceeding to the next iteration. This serial processing is often not desirable when the order of completion of the tasks does not matter.

This will look something like this:

Now why is this a problem?

In a single user’s session, if the FetchDetails operation takes a long time to complete, this approach can lead to inefficient processing. This is because one slow API call will hold up the processing of subsequent vendors. If there are many such calls to make, they are all delayed by waiting for each one to finish before starting the next, hence slowing down the user experience and timeouts.

And this is exactly what happened in our case. In fact, in our case, we had two API calls in a single iteration of the loop one for FetchDetails and one for CheckIfAlreadyRated, Although we were normalizing the top vendors list and were setting an upper bound to limit the size to 15 vendors maximum, still maximum of thirty network requests could have been made in a single API call to the next js server.

These thirty network requests were getting resolved synchronously due to the blocking nature of the code that was implemented.

How this can be solved?

To improve efficiency and allow concurrent operations, instead of using await inside a loop initiate all asynchronous operations at once and then use Promise.all() to wait for them all to complete. With Promise.all(), the event loop can manage multiple ongoing operations simultaneously, effectively utilizing parallelism.