FastAPI: API Framework to Serve Machine Learning Data Products
These days most of the machine learning models are served in production as a service. This method decouples the client applications from the machine learning models via Application Programming Interface (API).
In this article, I will walk you through the API concept, FastAPI game changer capabilities, and serve our machine learning data product with FastAPI.
What is an API?
An application programming interface (API) is a software interface for two or more applications to communicate with each other.
The easiest way to understand what is an API is to think about yourself in a restaurant. You enter the restaurant, find your table, and receive a menu. On the menu, there are names, descriptions, and costs of the food. You choose your combination and you want to give an order. But how your request is going to be delivered to the chef? The chef can’t come to the dining area and you can’t go to the kitchen. You need someone to get your request and deliver it to the kitchen and later when the meal is ready bring it to your table. Yes, you need a waiter or waitress!
The same analogy applies to the applications. In order to make applications talk to each other or get their requests and post their outputs APIs serves as the middle person.
Examples of APIs
Google Maps: Google Maps serve various APIs in the application. When you search for a city, area, or neighborhood it populates the weather, air quality, and even during the covid time the latest infection results. In the same manner, when you search for a place to eat Google Maps shows you the restaurants, cafes, their menus, opening hours, and even if it is available online reservation options through third-party applications.
Travel applications: Booking.com, Trivago, Expedia, and others offer you end-to-end traveling experiences these days. When you check their applications you can book a hotel, buy airplane tickets, rent a car, or make a reservation at a wine-tasting event. None of these items are directly owned and managed by these companies. They get your request and make connections to the other applications which serve this information.
FastAPI
FastAPI is a Web framework for developing RESTful APIs in Python. FastAPI is growing its user base every day. Not only software engineers but also data scientists, machine learning engineers, and data engineers have started to use FastAPI. The movement was revolutionary that pushed all the data people to migrate from Flask and Django to FastAPI in a short period of time.
GAME Changer Capabilities of FastAPI
Synchronous vs asynchronous requests
Before going into technical definitions let’s turn back to our restaurant example.
You give your order to the waiter and he tells you that there is only one cook in the kitchen and he already has 5 requests in the queue! You start to wait for your order to be delivered. After 30 minutes with an apology your meal arrives. You learn that a single person how fast he can be can’t handle 10 tables at the same time. This is a synchronous request.
Next time you want to try another restaurant. Before giving your order, you ask the waiter how many cooks they have. The waiter laughs at you and tells you that “We have the 5 cooks in the kitchen and they are waiting to serve you, sir. We are not amateurs!” You give your order and 10 minutes later you get your request. You look around and see that other tables are also being served as well. This is an asynchronous request.
A synchronous API call is a design pattern where the call site is blocked while waiting for the called code to finish. With an asynchronous API call, the call site is not blocked while waiting for the called code to finish, instead, the calling thread is notified when the reply arrives.
In heavier load conditions, it can be more efficient to submit multiple async calls and periodically check the status than to wait for each call to complete before submitting the next one.
from fastapi import FastAPI
app = FastAPI()
async def get_restaurant_working_hours():
# some operations
pass
@app.get("/")
async def get_resturant_reservation_details():
results = await restaurant_working_hours() # waits for the defined function
# some operations
return resultsParameter validations
FastAPI supports string and numeric parameter validations.
String validation example
In the example below, I have added a string constraint with a minimum 5 and maximum 100 character length. When I call this endpoint through Postman and pass a text with a size of 3, it throws an exception as below.
from typing import Optional
from fastapi import FastAPI, Query
import uvicorn
app = FastAPI()
@app.get("/orders/")
async def read_orders(query: Optional[str] = Query(None, min_length=5, max_length=100)): #defining the constraints
results = {"order": [{"order_id": "Order_1"}, {"order_id": "Order_2"}]}
return results
if query:
results.update({"query": query})
if __name__ == "__main__":
uvicorn.run("parameter-validations:app")
Numeric validation example
In the example below, I have added a numeric constraint with a minimum value of 1 and a maximum of 100. When I call this endpoint through Postman and pass 0, it throws an exception as below.
from typing import Optional
from fastapi import FastAPI, Query
import uvicorn
app = FastAPI()
@app.get("/orders/")
async def read_items(order_count: int = Query(gt=1, lt=100)): # defining the constraints
results = {"order_count": order_count}
return results
if __name__ == "__main__":
uvicorn.run("parameter-validations:app")
Integration with Pydantic
Pydantic is a Python library used to perform data serialization and validation.
FastAPI has integration with Pydantic. This ensures at runtime to parse, evaluate and inform the user through the IDE about the type-related errors.
Below you can find a sample error message;
fastapi.exceptions.FastAPIError: Invalid args for response field! Hint: check that False is a valid Pydantic field type. If you are using a return type annotation that is not a valid Pydantic field (e.g. Union[Response, dict, None]) you can disable generating the response model from the type annotation with the path operation decorator parameter response_model=None. Read more: https://fastapi.tiangolo.com/tutorial/response-model/Custom error handling
Building data products require many different teams to work together; e.g. data science, machine learning, software engineering, etc. In this cross-functional collaboration alignment on error handling and messaging becomes crucial. FastAPI supports custom error handling with message and status codes.
from fastapi import FastAPI, HTTPException
import uvicorn
app = FastAPI()
cities = {1: "Berlin", 2: "Hamburg", 3: "München"}
@app.get("/cities/{city_id}")
async def read_city_id(city_id: int):
if city_id not in cities:
raise HTTPException(status_code=404, detail="City id is not found") # status code and error message is defined
results = {"city": cities[city_id]}
return results
if __name__ == "__main__":
uvicorn.run("error-message:app")
OpenAPI standards automatic doc generation
FastAPI supports the developers with automatically generated interactive and easy-to-test endpoints in the browser. By following the link below, you will be directed to the routes you generated.
http://localhost:8000/docs 
When you click the arrow on the blue panel, you will get more details.
The documentation allows users to test the created endpoints. You just need to click the “Try it out” button.
To test the numeric validations, I passed 0 to the order count and the API returned the error message and 422 status automatically.
Machine learning model serving with FastAPI
As we have learned the basics of FastAPI, we are ready to use it to serve our machine-learning models.
In this example, I used a used car data set to predict the value of the car. The data set contains characteristics about the cars from the brand to how many kilometers were driven with the car. In order to simplify the example, I will not share the data cleaning, data preparation, and model-building steps but serve everything in a pickle file.
The Python code loads the pickle file, creates a prediction endpoint, and returns the predicted price of the car.
from fastapi import FastAPI
import joblib
import numpy as np
import pandas as pd
import uvicorn
# passing title, version and description parameters to the class object
app = FastAPI(
title="Car Price Prediction",
version="1.0",
description="Model type: Linear Regression",
)
# loading previously saved pickle file
model = joblib.load("./LinearRegressionModel.pkl")
# creating predict end point
@app.get("/api/predict")
async def predict(
name: str, company: str, year: int, kms_driven: float, fuel_type: str
):
result = model.predict(
pd.DataFrame(
columns=["name", "company", "year", "kms_driven", "fuel_type"],
data=np.array([name, company, year, kms_driven, fuel_type]).reshape(1, 5),
)
)[0]
return result
if __name__ == "__main__":
uvicorn.run("main:app", host="127.0.0.1", port=8000, reload=True)
When we pass the parameters to the prediction endpoint, it returns the car prediction at the response body. Next time you can even dockerize this and serve it with a beautiful web application!
Conclusion
FastAPI has already become the go-to framework for API-serving purposes in the data domain. Every day I read yet another article that data teams migrated to FastAPI and how much they benefit from this change. I am looking forward to seeing more updates in the upcoming months and years.
Kudos to Sebastián Ramírez, his team, and all the contributors!
Thanks a lot for reading 🙏
For my Data Product series, you can check the links below;
Data Product Manager: The Most Crucial Job of the 21st Century
Product Thinking for Data Teams
Would You Like to See Our “Data Product” Menu?
You can find me on Linkedin and Mentoring Club
