Concurrency with FastAPI
In one of my earlier tutorials, we have seen how we can create a Python application using FastAPI. To know more you can refer to Building REST APIs using FastAPI. One of the main reasons to choose FastAPI over other frameworks is due to its very high performance, which is on par with NodeJS and Go.
In this tutorial, we will see how we can optimize the performance of our FastAPI application using concurrency and parallelism.
Understanding Concurrency and Parallelism
Concurrency
Concurrency is when two or more tasks can start, run, and complete in overlapping time periods. It doesn’t necessarily mean they’ll be running at the same instant, such as multitasking on a single-core machine.
Threading is a concurrent execution model whereby multiple threads take turns executing tasks. One process can contain multiple threads. It uses pre-emptive multitasking.
Async IO is a style of concurrent programming that is single-threaded, single-process design. It uses co-operative multitasking. We will be covering Async IO in Python in detail as we go ahead.
Parallelism
Parallelism is when tasks literally run at the same time, wherein we can split tasks up into smaller subtasks, which can be processed in parallel using multiple CPUs at the exact same time, such as, on a multicore processor.
Multiprocessing is a means to effect parallelism, and it entails spreading tasks over a computer’s central processing units (CPUs, or cores). The processes all run at the same time, essentially multiprocessing on different processors.
Concurrency and parallelism both relate to “different things happening more or less at the same time”.
Introduction to Async IO
Async IO is a style of concurrent programming introduced in Python 3.4. Asynchronous I/O is a form of input/output processing. While Synchronous I/O waits for an operation to complete before moving on to the next one, Asynchronous I/O permits other processing to continue before the transmission of one operation has finished.
Synchronous I/O is easier to code as it follows all the steps in a program to complete a task before moving on to the next. To simplify asynchronous code and make it as simple as synchronous code, Async IO uses coroutines and futures as there are no callbacks. It uses async and await keywords to manage and execute asynchronous code.
Coroutines are special functions that work similar to Python generators. It is a function that can suspend its execution before reaching return, and it can indirectly pass control to another coroutine for some time. On await, they release the flow of control back to the event loop. A coroutine needs to be scheduled to run on the event loop, once scheduled coroutines are wrapped in Tasks which is a type of Future.
Futures represent the result of a task that may or may not have been executed. This result may be an exception.
With FastAPI we can take the advantage of concurrency that is very commonly used for web development. We don’t have to worry about the Event loop or async/await management. What we need is to just do the following :
- Declare the first path functions as coroutines via
async defwherever appropriate. If we do this wrong, FastAPI will still be able to handle it, we just won’t get the performance benefits. - Declare particular points as awaitable via the
awaitkeyword within the coroutines.
We will see this in practice below when we come to the coding part.
In most of the applications, we perform some Input-output (I/O) operations which are slow in nature. To demonstrate the slow nature of Input-output (I/O) operations, in this tutorial we will create a FastAPI application that will internally call some external APIs to get some data. When we call external API we are performing some network (I/O). Then we will see how to utilize Async IO to improve the performance of our application.
The following steps illustrate, how we can use Async IO in FastAPI to improve performance:
- Setup and Installation
- Add a Sync EntryPoint
- Add Middleware
- Start the Application
- Test the Application
- Add an Async EntryPoint
- Compare the Performance
Prerequisites
We require Python 3.9 with Pipenv and Git installed. Pipenv is a package and a virtual environment manager, which uses PIP under the hood. It provides more advanced features like version locking and dependency isolation between projects.
1. Setup and Installation
Once the prerequisites are in place we can begin creating our application.
To begin with, our application, create a folder called python-sample-fastapi-application in any directory on the disk for our project.
Navigate to the project folder.
b) Activate Virtual Environment
Once we are inside the project folder, execute the following commands to activate the VirtualEnv.
The virtual environment will now be activated, which will provide the required project isolation and version locking.
c) Install Dependencies
Next, install all the required dependencies using Pipenv as shown.
After we execute the above commands, the required dependencies will be installed.
We can see now two files, which have been created inside our project folder, namely, Pipfile and Pipfile.lock.
Pipfilecontains all the names of the dependencies we just installed.Pipfile.lockis intended to specify, based on the dependencies present inPipfile, which specific version of those should be used, avoiding the risks of automatically upgrading dependencies that depend upon each other and breaking your project dependency tree.
Note: Here, we have installed all the dependencies with specific versions, which worked on my machine while writing this tutorial. If we don’t specify any version then the latest version of that dependency will be installed, which might not be compatible with other dependencies.
The next step is to add the Entrypoint for our application.
2. Add a Sync EntryPoint
In the root directory of the project, let’s create a file named main.py . Now, let’s add an entry point. The entry point will call some external APIs and return the results to the client. Here, we are using one of the free API listed in Apipheny for demonstration purposes. We will use the Universities List API to get a list of universities in a specified country. For this tutorial, we are fetching the list of universities in turkey, india and australia. We will not be validating the response, as we are just simulating the scenario where in our application we may need to call multiple APIs, do certain processing on top of that and then return the results.
Let us have a look at the code for adding the non-async endpoint:
In the above code from line 7 to 9 we are calling the method get_all_universities_for_country for fetching the list of all universities from three different countries and storing the result in the dictionary. After collecting the results for all the countries we are returning the results to the caller.
Next, let’s see how we are getting the data. We will add a file named universities.py in the root directory. This file will contain the function get_all_universities_for_country that will be used to fetch the universities for the given country.
Let us have a look at the code which fetches the universities:
In line 5 we are using the HTTP client named httpx to send an HTTP request. HTTPX is a fully-featured HTTP client for Python 3, which provides sync and async APIs, and support for both HTTP/1.1 and HTTP/2.
Here, we are using the sync API to get the universities data. Later on, we will use the async API.
In the above code in line 6 we are calling the universities API to get the results. In line 7 we convert the response to JSON format. Then from line 9 to 11 , we map the JSON object to the Pydantic model University.
And finally, we return the dictionary, which contains the country name as key and the list of universities as value.
Now our sync endpoint is ready. To check the response time of our API let’s add a middleware.
3. Add middleware
A middleware is a function that works with every request before it is processed by any specific path operation and also with every response before returning it.
Add the below middleware code in main.py
As we can see in the above code, the middleware function uses the decorator @app.middleware("http") at the top. It will intercept our entrypoint request and then forward the request to the corresponding path operation. We will use this middleware to calculate the time taken by our entrypoint and return the response time to the caller. Here, we are adding a response header X-Process-Time in line 6 which will contain the response time.
4. Start the Application
Till now we have written all the code required for our application to run. We just need to add the below code to main.py file.
FastAPI is the framework that we have used to build our API, and Uvicorn is the server that we will use to serve the requests. We have already installed Uvicorn. That will be our server. Our application is ready now. We can start the application by executing the below command:
python main.pyOnce the application starts successfully, we can navigate to http://localhost:9000/docs. The system will bring up a page that looks something like this. We can see the endpoint in the Swagger-UI which was defined earlier.
5. Test the application
Let us test our universities endpoint and see, how much time does it take to complete the request.
As we can see in the response that the sync endpoint took around 2.0514 sec to get the list of universities for three countries.
6. Add an Async EntryPoint
Let’s now see how we can improve the performance of the REST API using Async IO.
We will add an async equivalent of the above endpoint. There is no functional difference between these two endpoints. It is just a different way of handling the request.
Let us have a look at the code for the async endpoint:
When we define a function with the async keyword, it is a special function called a coroutine. The reason why coroutines are special is that they can be paused internally, allowing the program to execute them in increments via multiple entry points for suspending and resuming execution. This is in contrast to normal functions which only have one entry point for execution.
When we use the await keyword, inside async function, it tells the program that this is a “suspendable point” in the coroutine. It will suspend execution at this point and can perform some other tasks. It will come back later once the results are available.
In the above code in line 7 we can see that, we are using asyncio.gather to run a sequence of awaitable objects (i.e. our coroutines) concurrently. Once we gather the response of all the coroutines, we are returning the response to the caller. The main difference here is that we are not waiting for the response of the first API to finish before calling the next API. It’s like we are calling all the APIs concurrently and waiting for the results.
Next, we will add one more function in universities.py called get_all_universities_for_country_async to get the universities in async mode.
The above function is declared with the async keyword, defining it as a coroutine. In the above code in line 3 we can see that here we are using the async API of httpx . async with httpx.AsyncClient() is the httpx context managed client for making async HTTP calls. Here, GET request is made with the await keyword, telling Python that this is a point where it can suspend execution and do something else. Once we receive the response then the code remains the same as sync call.
7. Compare the Performance
Now our async API is ready to be tested. Let us re-run the application. We can now see two endpoints for our application.
Let us run the async endpoint and compare the results.
As we can see that using Async IO we are able to get the response in 0.58 sec which earlier took around 2 sec
Conclusion
In this tutorial, we saw how we can take advantage of the Async IO style of programming in FastAPI to improve the performance of our REST APIs. If we have a scope in our application to use Async IO, then we should give it a try.
If you would like to refer to the full code, do check:
