Async/Await Essentials for Production: Loops, Control Flows & Limits
util.promisify in Node.js’ core to allow for easy conversion of callbacks to promises.
Before we get into running in parallel and limiting items in a loop let’s get back to the basics. A common pattern with async await is to simply wait for a promise, get its value and continue the function, let’s assume we want to get a user and based on the user object fetch the particular feed, this can be done easily by tagging our getUser function with async and using await before each promise-returning function call:
a simple use-case that would work with any promise based function/library, but what if we wanted to use good old callbacks with promises? If you have used petkaantonov/bluebird before perhaps you would be familiar with the promisify method, fortunately if you are using Node 8.0 and above you can save a couple of minutes of your time by not installing the module and instead using a similar functionality which is available under Node’s core as
Util#promisify that converts error first callbacks to promises:
Note that as of v9.4.0 returning multiple arguments with
Util#promisify is only available to Node internally and if you are outside of a Node.js environment you can fallback to Bluebird’s
Promise#promisify as mentioned earlier.
*One fact that you should not forget unlike some misconceptions being spread around, async/await is not simply syntactic sugar over Promises, but it carries properties from Promises that can greatly simplify debugging. One of its biggest advantages around debugging and error handling capabilities would be the all powerful yet hardly ever loved try/catch block. Catching synchronous errors and asynchronous errors within the same block of code has never been easier.
To demonstrate this let’s write a promise that throws at around a 100 milliseconds at all times:
We can catch errors from this promise in an async function using the traditional try/catch block, although the line following will throw a synchronous error it will never be reached:
Much like catch in promise you can catch an error occurring within the flow:
Whether you are running tasks in parallel, scheduling a loop or creating a cascading structure or a pipeline async/await can simplify what your process translates to in code with efficient and readable control flows. Let’s go through some of the common patterns:
1. Parallel execution
There is no particular syntax to parallel execution with async/await but we can utilize
Promise#all against an array (or any iterable) of promises to get the expected results:
Promise#all combines a list of promises into a single promise that will return all values resolved by those promises in an array when the promises have all been resolved. This happens in parallel and we don’t need any other trickery beyond this simple and elegant function.
Perhaps the least sung hero of async/await is the promisified timeout. It is effective and essential, especially when used within the context of a loop (see delayed loop below), we can wrap traditional timers (
setImmediate) in promises like below:
We can then put these functions into action by creating a async pause (non-process-blocking) between two functions regardless of whether or not they are synchronous or asynchronous:
This way we never have to leave the function’s context, this can lead to more elegant code when used appropriately.
Control flows are dead simple with async/await but my personal favorite practice of async/await is loops, a simple async loop can be represented in multiple ways and of course we will follow this up with parallel execution.
1. Series loop
A loop through a given number of items that performs a number of asynchronous actions, in our example below stop the loop on each step to fetch some data from the database and log the results and then continue to the next item, going one by one, thus creating a series loop:
2. Delayed loop
We can utilize the concept of timeouts within our loop, for example if we wanted to create a method that would add a random number to an array once a second for a total of 10 seconds we could use either
setImmediate with a counter or a for loop awaiting the
timeoutPromise we implemented earlier:
and we can even go as far as implementing a conditional
setInterval with a while loop:
3. Parallel loop
If it runs in parallel, parallelize. Parallel loops can be created by pushing a promise into an array that will later resolve into a value thus all promises start at the exact same time and can take their own time to finish, the final results will be ordered automatically by
This can be more elegantly represented with
Array#map, and it helps that we get a little more functional with our implementation for real-world use-cases, by mapping the array of items into an array of promises and awaiting the values:
Promise Races and Limits
Another hardly explored topic is setting limits to control the total number of tasks executing in parallel with async/await. If you are an avid user of caolan/async you are most likely making use of
Async#eachLimit but fear not, setting limits is possible. We’ll go back to our Promise magic and start a race!
Promise#race will return a promise that will resolve when the first item in a given list of promises resolves.
1. The basic race
A simple race can be created by passing a list of promises which in the following case would be the return of an async function that resolves in a random amount of time:
The first promise to resolve will win the race and get collected as our result.
2. Setting limits
We can utilize nearly all that we have covered so far to build a function that executes other async functions in parallel with a given limit. A real-life example of this would be to process screenshots of a list of webpages only 5 at a time.
To achieve this with promises and pure async/await we’ll need to have a way to store promises which are currently in-flight or in other words have not yet been resolved. Unfortunately this is not available as part of spec-grade promises so we’ll instead use a
Set to store and delete promises that are in-flight:
We can then utilize
Set.size to check the total number of in-flight promises, allowing us to determine how many more iterations of our loop we can continue to schedule.
Next we’ll use
Promise#race as part of our control-flow arsenal. What we need is a way to stop iteration of the loop (in this case
Array#map) until the next promise has been resolved (we’ll use a race for this) and check if the total number of in-flight promises is less than the limit that we are looking for, if it isn’t then we continue back with another race. This is easy to achieve with the following while loop:
Combining the two together and we’ll end up with
parallelLimit (Go ahead you can run this last bit) :
A Paradigm shift or just nicer syntax?