The Absolute Bare Minimum You Need to Know Before Using Asyncio in Python
Written By: Alden Page
Asyncio is one of the most powerful but frequently misunderstood parts of the Python standard library.
After conducting hundreds of code reviews and responding to dozens of production incidents throughout my career, I have learned that a significant number of quality and performance issues can be traced back to confusion about the asyncio library. Further, after interviewing a number of senior developers over the past several years, I have found that only a small percentage of otherwise well-qualified engineers have any practical experience with asyncio.
Personal anecdotes aside, a review of a search of the top comments on Hacker News related to asyncio reveals that developer sentiment towards the library tilts negative. To protect the innocent, I’ll summarize instead of using direct quotes:
“I don’t understand when to use asyncio.”
“I recommend avoiding asyncio if you can.”
“You should just use threads.”
Clearly, a significant portion of the developer community struggles to identify when and how to use asyncio.
This amounts to a huge missed opportunity: when used correctly, asyncio can greatly improve the performance of networked applications, enabling superior ergonomics and readability compared to multithreading. This is doubly true for Python applications, where limitations imposed by the global interpreter lock (GIL) restrict the usefulness of multithreading and force us to use heavyweight solutions like multiprocessing, interprocess communication, and complex synchronization strategies. Often, these result in more lines of code, memory overhead, and bugs than for an equivalent program using asyncio.
At the same time, numerous pitfalls and common misconceptions create challenges for novice and experienced developers alike. One of the most common (and costly) mistakes is using asyncio in circumstances where it is not called for. Even when asyncio is applied in the appropriate context, developers often struggle to implement their applications, as the official documentation offers few practical examples of the architecture of concurrent programs.
To help you navigate these challenges, I’ll briefly review what asyncio is, scenarios where it is most useful, and, most importantly, summarize the key subset of the asyncio API you need to know to write an effective concurrent application. Lastly, I’ll offer advice for asyncio learners on safely developing production applications.
The basics
Many others have written comprehensive introductions to asyncio, so I will keep this one short.
What is asyncio?
Asyncio is a library of the Python programming language for executing code concurrently as opposed to sequentially. “Concurrent” execution means that two tasks can interleave with each other. Crucially, concurrency does not necessarily imply parallelism, where two functions execute at the exact same time on different CPU cores. While all parallel programs are concurrent, not all concurrent programs are parallel.
The most critical thing to understand is that asyncio offers the most benefit in networked programs that do a lot of waiting.
A simple example scenario
Consider a program that makes 5 HTTP requests. If I am making 5 synchronous HTTP requests, I have to send a request, wait for a response, and do something with the response (e.g., stick it in a database, print it to the console, send a follow-up request, etc.), 5 times in a row. Assuming a round trip HTTP call takes 1 second, that means my program takes 5 seconds to run. The vast majority of that second is spent waiting for a response from the remote server to arrive over the wire.
A smarter program would spend this waiting time productively. Instead of blocking the entire application while waiting for the response, it can start sending the remaining HTTP requests and then wait for them all to arrive simultaneously. The result is that we only wait slightly more than 1 second to send these 5 requests.
The beauty of the concurrent solution is that it achieves parallel-like performance without requiring the developer to use multithreading or multiprocessing.
Do you actually need asyncio?
It is often said that asyncio can be used to optimize any I/O bound application. The official documentation more accurately states:
“asyncio is often a perfect fit for IO-bound and high-level structured network code.” [source, emphasis mine]
The fact that a task is I/O bound does not necessarily indicate that asyncio is the best tool for the job. The I/O also needs to take place over a network. Even then, using asyncio may not actually improve the application.
Certain I/O-bound applications, and even some network-bound applications, have nothing to gain from asyncio, as we’ll cover below. I cannot emphasize enough that only protocols that involve a lot of waiting (making a request and then waiting a significant time for a response, repeatedly) benefit from concurrency. Moreover, you need to make a large number of requests to justify the additional complexity of using asyncio. Your throwaway script making 10 HTTP calls probably doesn’t need asyncio, but an hourly job that makes 500,000 HTTP calls almost certainly does.
So when do you need asyncio?
In most cases, asyncio offers the most benefit when your application spends a lot of time making requests and waiting for responses over a network.
Some examples:
- A script that needs to make hundreds of thousands of requests to an HTTP API in a reasonably short period of time.
- A server backend that needs to maintain open connections with a large number of clients and internal services simultaneously.
- Complex multitasking systems that aren’t CPU bound, such as a web crawler.
In almost every other case, using asyncio is worse than writing everything synchronously, as you will only achieve equal or worse performance but with considerable additional complexity.
When not to use asyncio
To further dispel the myth that asyncio is useful in any I/O bound scenario, we can explore a few scenarios where using asyncio is not beneficial.
Streaming protocols
A good streaming protocol works like a firehose: the application opens a connection, exchanges a small amount of metadata with a broker, and in return, gets blasted with data. Very little time is spent waiting. Consequently, if streaming is the bottleneck in your application, it will not be made faster by introducing concurrency. Moreover, if the order of streamed events is important, as it often is, you wouldn’t even necessarily want to make this a concurrent process because concurrent processes interleave with each other and execute out of order.
It may still be desirable to introduce concurrency to multitask as streaming is ongoing, but the cumulative execution time of the application will not be reduced.
Disk I/O
Solid state drives are fast these days. Your application is probably not bound by disk I/O. But, even if it is, asyncio cannot help. The disk is already saturated with writes. Adding more concurrent writes will not circumvent the physical limitations of disk write speed. In this scenario, you will need to explore more complex optimizations like sharding.
Memory I/O
Writing to local memory is fast. Adding asyncio to a process that spends most of its time moving data around in memory will result in decreased performance and unnecessary complexity.
CPU-bound applications
If your application is CPU-bound, using asyncio can have the opposite impact on performance than intended. There is no waiting involved in a CPU-bound task, so concurrency has no benefits. In this scenario, multiprocessing is likely to be a better solution.
In the select circumstances where asyncio applications need to perform both high-volume network IO and heavy computation, you can use executors to prevent the CPU-intensive work from blocking the event loop. You can think of it as an asynchronous wrapper for process pools.
The bare minimum part of the asyncio API you really need to know
There’s no question that the asyncio API is large. As of today, there is extensive documentation of the API, but few concise resources summarizing how to use it. This can make using asyncio for the first time a daunting task, especially when relying on the official documentation as your starting point.
It turns out that you can write a fully featured concurrent application after learning just three basic concepts:
async and await keywords
The async keyword marks a function as a coroutine. Coroutines can only be called with the await keyword. Calling await indicates that you are returning control to the event loop, which allows the event loop to interleave other tasks with your code.
Tasks, TaskGroups, and gather
A task is an instance of a coroutine run as a background job. Unlike a regular function call, the application does not block when you execute a task and the program moves on to the next instruction.
Often, we want to schedule many tasks at once and then wait until they are complete. Consider an application that needs to make 100,000 HTTP requests. We would most likely accomplish this by launching enough concurrent HTTP tasks to saturate our bandwidth or rate limits (let’s say 50 of them), wait for all of them to finish, and then repeat until all 100,000 requests have been made. The “waiting” part is important; if we don’t synchronize our program to wait for the requests to finish, the number of concurrent requests launched will be unbounded, and we will trigger far more requests than intended.
TaskGroups are the way that we block execution of the program until all subtasks are complete. If any of the subtasks fail, the scheduler cancels all remaining subtasks and raises an exception group.
In circumstances where canceling other concurrent subtasks after a single failure is not desirable, use asyncio.gather.
asyncio.run
Asynchronous execution in Python is made possible by using a scheduler. The scheduler is an event loop that controls the execution of concurrent tasks. You start the event loop by calling asyncio.run(task). All Python applications that use await or async must have a call to asyncio.run somewhere.
Safety tips
While I argue that asyncio is often a better alternative to threads or multiprocessing, using it still introduces additional complexity to programs. The control flow of a concurrent program is considerably different than a synchronous one, and there are a number of counterintuitive “gotchas” to look out for.
Since listing all possible risks or errors is impossible, I will review a few engineering best practices that can help you easily detect and fix the most common mistakes made when writing any non-trivial concurrent application.
Handle error responses
Task groups and asyncio.gather(*tasks) return the response of the functions they wrap. They can also return exceptions. Even if you’re “firing and forgetting” a bunch of tasks and don’t really need to do anything with the results, inspecting each response object and handling errors is critical. When making hundreds of thousands of requests over an unreliable network, some will fail and you have to decide how your application will handle that. Ignoring failed responses is usually not the desired behavior and will all but guarantee eventual problems in production.
Profile your code
Using asyncio is usually a performance-driven decision. It’s not good enough to choose some “fast” APIs like multiprocessing or asyncio and assume that your application will be fast; you still have to carefully measure the time it takes to execute the critical path. With or without asyncio, minute oversights can have dramatic effects on run time performance. The good news is that it’s easy to shake out these oversights with a few calls to time.monotonic() and some basic arithmetic.
Emit detailed logs
All backend applications should emit tons of logs, but it’s even more important to log when writing an asynchronous program. You should have both info-level logs showing high-level details of application behavior and debug-level logs revealing details about every network call. Without detailed logs, it’s extremely difficult to confirm whether your application is behaving as designed or whether there are issues in production that need to be fixed.
Write unit tests
Testing IO-heavy applications is hard. It almost always involves mocking out external APIs, which can be time-consuming. Developers will often skip this step and run the application “live” manually when possible, in hopes of shortening time to delivery. Ultimately, manual testing is time-consuming and error-prone, which results in slower feedback loops and longer development cycles. If and when the original developer moves on to their next project, their successor will be in a major bind. On balance, investing the time to write a unit test suite, even if only partially complete, will pay off many times over.
Takeaways
Asyncio belongs in every Python developer’s toolbox. When used correctly, asyncio can offer developers a way to build sophisticated concurrent applications on a single CPU core while avoiding all the baggage and dangers of multiprocessing. With knowledge of just a few basic APIs and meticulous adherence to engineering best practices, you can ship succinct, high-performing, and most importantly, correct applications to production.
