Asynchronous iterators in python
How to write async for loops in python
We have all worked with
python by now as a developer. This post is not another difference between or comparison between iterators and iterables in
python. Let me the spare that. This post gives an overview of how
asynchronous iterators work and how to write our own
To get an introduction to
asyncio, please refer here.
At the end of this post, you should be able to
- Write an asynchronous iterator.
- Understand the internals of
- An introduction to regular
- How are non async iterators different from async iterators.
Now, let’s get started. Shall we?
Iterators in python:
Before discussing the
async iterators, let's quickly go over the regular iterators in python.
- An iterator must implement the
__iter__special method should return an
iterable.i.e. Any object that implements the
__next__special method. This could be its own class(
self) or any other class object.
__next__method has the logic to run the
iteratoruntil a condition is satisfied.
- Once the condition is satisfied,
StopIterationerror message should be raised.
Now, let’s quickly write an
OddCounter which iterates only through the odd numbers. I know this doesn't make more sense now. But it's fun to play around with the capability of
iterators. You could try implementing an even counter or fibonacci counter.
The above example was just to illustrate the working of regular
python.Now that we have learnt it, let's jump to what we are here for.
In the regular iterators, what if there is a scenario where you have to do some I/O tasks to get data inside the
__next__() method. That's exactly what
async iterators do.
async iterator typically contains,
__aiter__()method instead of
__aiter()__method must return an object that implements a
async def __anext__().
__anext__()method must return a value for every single iteration and raise
StopAsyncIterationat the end instead of
Now, let’s try rewriting our
OddCounter iterator using async iterator.
Though this is syntactically right and works, we don’t have any compelling reason to use an
async iterator here. One such scenario would be, if we do some operation inside the
__anext__() method that requires fetching data from elsewhere.For instance, if we fetch data from a remote database or another coroutine function one at a time, then it would be the best candidate for writing asynchronous iterators.
In the above example,
KeyTaker is an asynchronous iterator that extracts keys from a
coroutine. Now you could argue this could be done with a simple class and a function. Of course we could. The point here is,
all_keys() method could be performing an I/O operation like fetching information from a database or calling an API. To keep things simple, I have just created a dictionary and returned their values for the key queried.
main() function just iterates through
KeyTaker and prints the values. We could also call another coroutine inside the
async for loop. That would make this code complete.
- async iterators have __aiter__ and __anext__ special methods
- StopAsyncIteration should be raised at the end of iteration.
- async iterators could be used if we perform an I/O operation on demand where the iterator in turn is performing another I/O operation.
Originally published at https://dock2learn.com on March 10, 2022.