Making array iteration easy when using async/await
In this article I will introduce the npm module I created, and why I decided to create it.
Node.js version 8 is out, and we’re finally able to use async/await natively in the server side. Currently, I’m trying to move our production code to Node 8, but in some ways, the transition is not as straightforward as I would like.
async/await allows us to write asynchronous code in a synchronous-looking way, so trying to use the built-in Array iteration methods introduced in ES5 is one of the first challenges you may face. Actually, there are many related questions in Stack Overflow. Doing an await inside these methods iteratees is not going to work as one would expect.
Problem
In order to be able to use await, the innermost function that surrounds it must be an async function. This means that an async function has to be passed as iteratee if we are to use await, otherwise a syntax error will be raised. So then, let's just use an async function as iteratee!, is the next thing we could try, but that's not going to work either.
To illustrate the problem, I adapted a couple of examples used by Dr. Rauch in his article about async functions.
Let’s start with the Array#map() method:
async function getUsers (userIds) {
const users = userIds.map(async userId => {
const response = await fetch(`/api/users/${userId}`);
return response.json();
});
// ... do some stuff
return users;
}This is how the function will behave:
- The result is not going to be an Array of plain user objects, but an Array of Promises.
map()execution is going to finish before all the iteratees are executed completely, because Array iteration methods execute the iteratees synchronously, but these iteratees are asynchronous.
Putting an await before map() is not going to work either, because in case something besides a Promise is passed to await, it just returns the value as-is. In this case we are passing an Array of Promises instead of a Promise, so that is what we get.
This can be solved by doing the following:
async function getUsers (userIds) {
const pArray = userIds.map(async userId => {
const response = await fetch(`/api/users/${userId}`);
return response.json();
});
const users = await Promise.all(pArray);
// ... do some stuff
return users;
}Passing the array of Promises to Promise.all() is the way to go, as it will return a single Promise that gets resolved when all the promises in users have resolved.
Now let’s take a look at Array#forEach():
async function logUsers (userIds) {
userIds.forEach(async userId => {
const response = await fetch(`/api/users/${userId}`);
const user = await response.json();
console.log(user);
});
}In this case all the iteratees are going to be executed, but forEach() is going to return before all of them finish execution, which is not the desirable behaviour in many cases. Using a for-of loop instead of forEach() is going to work, but as you can guess, it is going to be executed in series.
We could make the execution concurrent using map() and Promise.all():
async function logUsers (userIds) {
await Promise.all(userIds.map(async userId => {
const response = await fetch(`/api/users/${userId}`);
const user = await response.json();
console.log(user);
}));
}This has been solved in a similar way as we did above in the getUsers() function, with the difference that we don't need the values returned by map() (an Array of Promises again). This can be misleading and a bit verbose.
How about the other Array iteration methods? Emulating their behaviour gets more complex and dirtier if you want to execute asynchronous calls concurrently. I did not find a clean way to emulate find(), every(), reduce() and friends using async/await in plain Javascript. Personally I would not like to have complex and difficult to understand code all over the place.
Introducing p-iteration utilities
I wanted to keep using the Array methods without making my code dirty, so I decided to create the module p-iteration, to simplify theses cases.
Let’s try again the examples we had above:
const { map } = require('p-iteration');async function getUsers (userIds) {
const users = await map(userIds, async userId => {
const response = await fetch(`/api/users/${userId}`);
return response.json();
});
// ... do some stuff
return users;
}const { forEach } = require('p-iteration');function logUsers (userIds) {
return forEach(userIds, async userId => {
const response = await fetch(`/api/users/${userId}`);
const user = await response.json();
console.log(user);
});
}
This time it looks more straightforward. Also, these utilities are compliant with the built-in Array iteration methods, so there is no need to learn some new syntax. If you don’t mind using a module instead of plain Javascript, give it a try.
Also Promises can be included in the Array, as they will be unwrapped by the module before their corresponding iteratee is invoked:
const { forEach } = require('p-iteration');
const fetchJSON = require('nonexistent-module');async function logUsers () {
const users = [
fetchJSON('/api/users/125'), // returns a Promise
{ userId: 123, name: 'Jolyne', age: 19 },
{ userId: 156, name: 'Caesar', age: 20 }
];
return forEach(users, (user) => {
console.log(user);
});
}
Each iteratee will be invoked as soon as its corresponding Promise in the array is resolved, so it will cover other use cases as well:
const { forEach } = require('p-iteration');// function that returns a Promise resolved after 'ms' passed
const delay = (ms) => new Promise(resolve => setTimeout(() => resolve(ms), ms));// 100, 200, 300 and 500 will be logged in this order
async function logNumbers () {
const numbers = [
delay(500),
delay(200),
delay(300),
100
];
await forEach(numbers, (number) => {
console.log(number);
});
}
The following methods have been implemented as well: find(), findIndex(), some(), every(), filter() and reduce(). With the exception of reduce(), all of them run concurrently, but just in case you might want them to run in series, a series version of each one is also provided by the module, named as ${methodName}Series (eg. filterSeries() or everySeries()).
Any feedback would be appreciated!