How to serve Machine Learning model using FastAPI+MLFlow+MINIO+MySQL
This is the easy way to serve ML models with FastAPI
It will be a long journey, if you are ready let’s start.
Let’s talk about What are these technologies ?
First of all, get to know FastAPI
FastAPI is a Python-based web framework. It has attracted a lot of attention in recent years. Its development structure is similar to Flask, which was the most popular choice for those who wanted to learn Python web development.
FastAPI is simple to use, and its documentation is clear and easy to follow. FastAPI is used by tech giants such as Microsoft, Netflix, and Uber.
FastAPI has all the features you would expect from an API builder, but it’s not limited to these. You can use it to mount backward WSGI, which most users don’t know about.
And Uvicorn ?
Uvicorn is an ASGI web server implementation for Python.
Until recently Python has lacked a minimal low-level server/application interface for async frameworks. The ASGI specification fills this gap, and means we’re now able to start building a common set of tooling usable across all async frameworks.
Secondly, What is the MLFlow and What does it do ?
MLflow is an open source platform for managing the end-to-end machine learning lifecycle. It has the following primary components:
- Tracking: Allows you to track experiments to record and compare parameters and results.
- Models: Allow you to manage and deploy models from a variety of ML libraries to a variety of model serving and inference platforms.
- Projects: Allow you to package ML code in a reusable, reproducible form to share with other data scientists or transfer to production.
- Model Registry: Allows you to centralize a model store for managing models’ full lifecycle stage transitions: from staging to production, with capabilities for versioning and annotating.
- Model Serving: Allows you to host MLflow Models as REST endpoints.
First-time users should begin with the quickstart, which demonstrates the basic MLflow tracking APIs. The subsequent articles introduce each MLflow component with example notebooks and describe how these components are hosted within Databricks.
MLflow supports Java, Python, R, and REST APIs.
Why we use MinIO ?
MinIO is a High Performance Object Storage released under GNU Affero General Public License v3.0.
It is API compatible with the Amazon S3 cloud storage service. It can handle unstructured data such as photos, videos, log files, backups, and container images with a current maximum supported object size of 5TB.
Anyone interested in data must have heard of Scikit-learn, but let’s not go without defining it again.
Scikit-learn (Sklearn) is the most useful and robust library for machine learning in Python.
It provides a selection of efficient tools for machine learning and statistical modeling including classification, regression, clustering and dimensionality reduction via a consistence interface in Python. This library, which is largely written in Python, is built upon NumPy, SciPy and Matplotlib.
Okay, last but not least MySQL ?
MySQL is the world’s most popular open source database. According to DB-Engines, MySQL ranks as the second-most-popular database, behind Oracle Database. MySQL powers many of the most accessed applications, including Facebook, Twitter, Netflix, Uber, Airbnb, Shopify, and Booking.com.
Since MySQL is open source, it includes numerous features developed in close cooperation with users over more than 25 years. So it’s very likely that your favorite application or programming language is supported by MySQL Database.
Let’s Start
First of all , we must talk about Why we need Machine Learning model serving ?
May be some of them thinking like that;
“I developed the machine learning model in the best way, isn’t it enough ? ”
“My model predicts very well, isn’t it enough ? ”
Because the model that you are training must serve in a production enviroment or in a running state
The next step after the model development is finished is that the model is in service.
In this text, we will focus on this point.
Let’s move on to the application part
1- First I development environment will be a Centos Linux.
I will use “Conda” as management system and environment management system
Let’s start the application starter by setting up an environment that name is “myenv”
now let’s activate this environment
conda activate myenvLet’s install the “requirements.txt” file, which you can access from the Github Repository, in the environment we created, via “pip”.
pip install -r requirements.txt
2- Let’s unpack the “docker-compose.yml” file in the “Docker” folder
And I have port forwarding for fastAPI,minıo,mlflow
3- Now, Talk about What will our machine learning model be about ? and train it.
The model can predict Electricity Price Prediction
Since the subject we want to draw attention to here is not Model development, I will not go into too much detail. But those who are interested can find it here.
There are four important files, thesee are main,models, databases and trainModelwithMLflow.py
NOTE : The explanations I will make will be for important places. You can access the public codes from the Github repo.
Saved model on MLFlow
>Review the trainModelwithMLflow.py
Our MLFLOW uri and urls that we determined in docker yaml file
# Determine the Urls
os.environ['MLFLOW_TRACKING_URI'] = 'http://localhost:5001/'
os.environ['MLFLOW_S3_ENDPOINT_URL'] = 'http://localhost:9000/'our code to register the trained model on MLFlow
# Set MLflow experiment
experiment_name = "Deploy Model using FastAPI-MLflow-MINIO"
mlflow.set_experiment(experiment_name)
registered_model_name="RFElectricityPricePrediction"
# Train model and register mlflow
with mlflow.start_run(run_name="with-reg-rf-sklearn") as run:
estimator = RandomForestRegressor(n_estimators=number_of_trees)
estimator.fit(X_train, y_train)
y_pred = estimator.predict(X_test)
(rmse, mae, r2) = eval_metrics(y_test, y_pred)
print(f"Random Forest model number of trees: {number_of_trees}")
print(" RMSE: %s" % rmse)
print(" MAE: %s" % mae)
print(" R2: %s" % r2)
mlflow.log_param("n_estimators", number_of_trees)
mlflow.log_metric("rmse", rmse)
mlflow.log_metric("r2", r2)
mlflow.log_metric("mae", mae)
tracking_url_type_store = urlparse(mlflow.get_tracking_uri()).scheme
# Model registry does not work with file store
if tracking_url_type_store != "file":
# Register the model
mlflow.sklearn.log_model(estimator, "model", registered_model_name=registered_model_name)
else:
mlflow.sklearn.log_model(estimator, "model")4- We will talk about FastAPI implementations
Firstly, look at the “models.py” file;
from typing import Optional
from datetime import datetime
from sqlmodel import SQLModel, Field
class PricePredictions(SQLModel,table=True):
id: Optional[int] = Field(default=None, primary_key=True)
Day: int
Month: int
ForecastWindProduction: float
SystemLoadEA: float
SMPEA: float
ORKTemperature: float
ORKWindspeed: float
CO2Intensity: float
ActualWindProduction: float
SystemLoadEP2: float
prediction: float
prediction_time: datetime = Field(default_factory=datetime.utcnow, nullable=False)
client_ip: str
class Price(SQLModel):
Day: int
Month: int
ForecastWindProduction: float
SystemLoadEA: float
SMPEA: float
ORKTemperature: float
ORKWindspeed: float
CO2Intensity: float
ActualWindProduction: float
SystemLoadEP2: float
class Config:
schema_extra = {
"example": {
"Day": 10,
"Month": 12,
"ForecastWindProduction": 54.10,
"SystemLoadEA": 4241.05,
"SMPEA":49.56,
"ORKTemperature":9.0,
"ORKWindspeed":14.8,
"CO2Intensity":491.32,
"ActualWindProduction":54.0,
"SystemLoadEP2":4426.84
}
}These are will be our models. “PricePredictions” model created for saving the our predictions info on mySQL and these will be create as a table in mySQL table. “Price” model help us for prediction features.
Let’s move on to the “main.py” file.
Important part is that, load ML model from MLFlow
from mlflow.sklearn import load_model
# Learn, decide and get model from mlflow model registry
model_name = "RFElectricityPricePrediction"
model_version = 1
model = load_model(
model_uri=f"models:/{model_name}/{model_version}"
)After model loading from MLFlow, you can use normal model methods.
There is a another point is that, Create a end point which make prediction and save the predict informations on mySQL
# Electirical Price Prediction endpoint
@app.post("/prediction/priceprediction")
async def predictPrice(request: Price, fastapi_req: Request, db: Session = Depends(get_db)):
prediction = makePrediction(model, request.dict())
db_insert_record = insertPrice(request=request.dict(), prediction=prediction,
client_ip=fastapi_req.client.host,
db=db)
return {"prediction": prediction, "db_record": db_insert_record}5- Speaking of mySQL, let’s take a look at what’s going on in “database.py” .
Create a methot that start the Session
import os
from dotenv import load_dotenv
from sqlmodel import create_engine, SQLModel, Session
load_dotenv() # take environment variables from .env.
SQLALCHEMY_DATABASE_URL = os.getenv('SQLALCHEMY_DATABASE_URL')
# Create engine
engine = create_engine(SQLALCHEMY_DATABASE_URL, echo=True)
# Create table
def create_db_and_tables():
SQLModel.metadata.create_all(engine)
# Connet to database
def get_db():
db = Session(engine)
try:
yield db
finally:
db.close()We will call the “create_db_and_tables” and “get_db” methods in manin.py.
Like this;
from database import engine, get_db, create_db_and_tables
# Creates all the tables defined in models module
create_db_and_tables()
# This is a not part of the flow code, this is quotation from endpoint part of FastAPI code method
# This is about demostrations.
async def predictPrice(request: Price, fastapi_req: Request, db: Session = Depends(get_db)):Now that we understand all our files, it’s time to get the project up and running.
Note : You must pull the project from Github repo, if you haven’t
Firstly, you must change directory
cd trainModel/After that run the “trainModelwithMlflow.py”
When we check MLFlow after the model training is finished, we will be able to see all the files in the artifact section.
Detail;
This control is valid in MINIO
Before the start fastAPI we must create database and database user for store prediction info.
Detarmine mySQL user and database.
NOTE: You have to determine this information in “.env” files
Run mySQL as root
create databese that name “traindatabase”
Create user that name is “trainUser”
CREATE USER 'trainUser'@'%' IDENTIFIED BY 'Password';And give all permissions
Lastly, clean up the internal caches
Chech user and database
Now, Run the FastAPI via Uvicorn,
We had set our port number in the “docker-compose” file.
Later you will see that the “priceprediction” table will be created.
And run “localhost:8002/docs” and will see the our end point
Open it and see the default information
Try it out and predict !
Thesee are our outputs
{
"Day": 10,
"Month": 12,
"ForecastWindProduction": 54.1,
"SystemLoadEA": 4241.05,
"SMPEA": 49.56,
"ORKTemperature": 9,
"ORKWindspeed": 14.8,
"CO2Intensity": 491.32,
"ActualWindProduction": 54,
"SystemLoadEP2": 4426.84
}And check the mySQL databases
You can see the model prediction and other information saved in “priceprediction” table
Okay, it’s that simple
With these stages, you will now be able to make a model serviceable after you have developed it.
Again Github repo is : https://github.com/duzgunilaslan/Deploy-ML-Model-FastAPI-MLFlow-MINIO-MySQL
