JavaScript iterators, generators, and async iterators
Iterating over lists of objects is a very common (if not daily) problem that developers confront at work. With the introduction of ES6, iterators and generators have officially been added to JavaScript to help developers solve this particular issue.
Iterators give the developers the superpower of consuming a list of items, one at a time and proceeding to the next item only after finishing consuming and processing it.
Generators are a special kind of function that gives the developer the superpower of producing data and pausing execution until called again.
They might seem to be the same thing, but they are not. They are strongly connected with each other, but not the same.
Although iterators are very powerful, their definition and the maintenance of their internal logic can be very challenging. Generators then come in place to provide a less error-prone and more efficient way of using iterators, hiding the complexity of using iterators behind the hood.
After reading this article about iterators and generators, you should be able to answer the following questions:
- What are generators? How are they connected to iterators?
2. How does the ”for … of” Loop work behind the scenes?
3. How does the ”for … of” Loop know how to go through an Array, Map, Set, or any ITERABLE Data Structure?
Tip: It does not know ANYTHING about the type of the data structure that it iterates over.
Answer: ”for … of” Loops only work on ITERABLES. Iterables have ITERATORS. “for … of” Loops use these iterators. To better understand what these are, let’s define an iterable and iterate over it using the “for… of” Loop.
Our iterable object will give access to 5 random numbers, through its iterator.
Stepwise explanation:
1. We define our iterable as an object, that has a special property, accessible through the “[Symbol.iterator]” key — defined using Symbols, makes the property unique and generic. You can read more related to Symbols here: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Symbol"
2. This special key gives access to a method that returns the actual ITERATOR object.
3. This iterator object is used to access our ITERABLE’s items one by one, using its inner method “next()”.
Once you understand the Explanation, you will be ready to write your own “for… of” Loop using both basic loops: for and while.
Note that the first value is created only on the first call of the next() method.
Actually, the [Symbol.iterator] key of the iterable gives access to a generator function — a function that returns an iterator that spits values each time its next() method is called. You can view this as a PAUSABLE FUNCTION THAT RETURNS MULTIPLE VALUES, one at a time. By “pausable” I mean that it returns values only when the user wants it to do so. Until that, it waits.
JavaScript provided generator functions
Note 1: asterisk (“*”)” symbol after the function keyword (or before function name)
Note 2: yield keyword, used for “returning” the desired next value.
This generator's functions are actually syntactic sugar, using iterators behind the hood. They are as well PAUSABLE FUNCTIONS.
Therefore, we can replace the iterator generator method from our iterable with the generator function:
JavaScript Async Iterators
Our iterable now gives access to its values through an async generator function. In the above example, you cannot immediately see its value because we are yielding random numbers, but imagine that we are yielding different API call results. But we cannot use a basic “for … of” loop, because async iterators return promises of the yielded values/objects. We instead have to use the “for await … of” loop like this:
Note that we have to use this inside an async function because we are using await.
CONCLUSION
We can now write our own practical and more advanced iterables that give us sequential access to the desired data. In one of the next blog posts, I intend to get more practical and simulate a sequential data streaming service using async iterators.
The problem I am trying to solve is the following: we want to create a car selling website and we have an API that contains hundreds of millions of records that we need to display on our website. Due to the huge size of the records, we can not get them through a single request and we need to create a mechanism to access them sequentially.
We can use our function in the following way:
Explanation: we are iterating over our iterable returned by the searchCars() function using async iterators.
How does the searchCars() method work? The async generator function of the returned iterable waits on the first yield call being prepared to request the first page of cars to our API, yielding the first page after the API responds.
After that, it automatically increments the pageNumber and makes another request, and yields its response. In the above example, while(true) means that we are incrementing pageNumber an infinite number of times, meaning that we are supposing that our API has an infinite number of pages. Of course, it’s impossible, but for the sake of understanding, let’s pretend it’s not. The code will still run because we are not able to consume all pages of the API so quickly, but at some point, they will end.
In this article, we learned what iterators, async iterators, and generators are and how and when we can use them. We also learned how to make any object iterable and how to write our own data streaming service using async iterators. Pretty cool, right?
