HOWTO : Async Generators In NodeJS
(Also published on my blog)
At the time of writing, NodeJS v10 was just released in the wild, which comes with some neat ES2018 features. The new features I am excited to talk about are the async generator functions and for-await-of loops which complement each other. Lots of bling bling, but how does it work? Is there even a valid use case for these things? Let’s find out!
The Generator Function
Let’s refresh our memory for a moment here about generator functions. How did that thing, called a generator, work again? I won’t dive too deep, but to give you an idea:
A generator function is declared like function* functionName , hence, the asterisk that comes after the function keyword. Such a function can generate multiple return values. You do this by first, calling the generator function, this function will return a generator object. This generator object can be used to iterate over all the return values. We can do this the explicit way :
Or, we can do it more implicit. As our returned generator object is iterable, we can use the for-of loop. Notice that the for-of loop will return the value itself and not an object, like sequence.next() does.
We can generate multiple values with a generator function. This can be useful to generate a number sequence like in the example above. A more concrete example would be to generate a Fibonacci sequence. Generator functions were also temporarily used for mimicking the behaviour of async functions when they were not supported yet.
Then it got asynchronous
By now,async functions and promises have become popular concepts in JS. An async function allows us to write asynchronous code, in a synchronous fashion. An async function is declared by prefixing the function keyword with the async function.
Now we’re good to go and we can use the await keyword to await any Promise. Don’t forget, any value that a async function returns, will be wrapped in a promise.
Putting it together
We did a refresher about thegenerator function and theasync function, what if, we want to combine those two? The folks at TC39 added this possibility in ES2018, and it was implemented in NodeJS v10.
As you might expect, the definition of a async generator goes like this:
We’re now able to yield and await in our function. Let’s take our first example of the sequence generator and make it async.
We’ll get back to using the await later. First we need to rewrite how we use our generator in the explicit way. We need to await every next() call because an async function returns a promise, therefore an async generator will return also every time a promise when we ask for the next value.
The same for the implicit approach, using the for-of loop…
… woops! This one throws TypeError : sequence is not iterable ! Since we’re not explicitly calling next() , how are we supposed to await the next value? Remember the for-await-loop that I mentioned earlier? Remember I told you it was complementary to async generators ? Behold why:
Because we can iterate over a generator object with a for-of loop, it was crucial that, if a generator object could return promises, we should be able to use the for-of loop properly to handle such a use case. Therefore, the for-await-of loop was born.
The for-await-of loop will await every time the next value is requested from our generator. So this await will happen before the start of every loop.
Are there valid use cases for this?
There definitely are! I have created an example were I use an async generator for paging through the Google Books API. Paging through the API results that is. In the code example I will get from each API request, 3 books and I page till there are no more items OR I have reached a max page index. The max page index is just a safety precaution.
The getBooksPaged async generator function encapsulates neatly the paging implementation details that I don’t want to worry about. Have a look at the code and notice that you could use the same approach for reading a file, line by line.
Conclusion
I can definitely see valid use cases where we can putasync generators to work, like paging an API endpoint, read files, handle streams and such. Be aware of the for-await-of loop which really reduces the boilerplate code else required for iterating over the generator. Worth checking out!
If you enjoyed this article, give me a 👏 , comment your opinions, feedback or criticism down below and follow me on Twitter @SegersIan.
Other articles by me:
