Machine Learning Model Deployment in Docker using Flask
This article explain how machine learning model can be deployed in a docker environment using flask, gunicorn and uwsgi in python environment
In a real-world setting, testing and training machine learning models is one phase of machine learning model development lifecycle. These models need to be deployed in real-world application to utilize it’s benefits. In this article I will discuss on how machine learning model can be deployed as a microservice in a plain Docker environment. Now a days Kubernetes becoming more popular which act as a management layer for the Docker runtime. It provide necessary container management functionalities which suitable for production grade deployment. However choosing the deployment strategy is upto the organization that building the applications that utilize the machine learning models.
What is a microservice?
Couple of years ago, microservice became a buzzword in tech world which accompanied by rise of container technologies. Prior to microservice architecture, applications built using monolothic architecture where every service sits in a single large application. This has it’s own pros and cons. Main issue of monolithic architecture is the scalability of single component without scaling whole application. With the rise of containers, independence and scalability of single component became more easy and fundamental concept of application development. Microservice can be defined as a small service unit of business functionality which are independent and communicate with other units across standard interfaces such as APIs and REST interfaces like HTTP.
What is a container?
A container is a standard unit of software that packages up code and all its dependencies, so the application runs quickly and reliably across diverse computing environments. You may think container as a virtual machine but container share the resources of host operating system which contrast to the virtual machine. This make container more lightweight compared to virtual machine. A Docker container image is a lightweight, standalone, executable package of software that includes everything needed to run an application: code, runtime, system tools, system libraries and settings.
What is Docker?
Docker is a tool that is used to ease the deployment of the containers. Docker allow users to create, deploy and run services using containers. You might have worked with virtual machines. Docker is like a virtual machine, but it can share the underline operating system without need of all dedicated operating system.
Use case overview
We are going to use Boston house price prediction dataset to build a machine learning model and deployed it in a docker container and expose it as REST API which will be access over HTTP. Let’s consider the house price prediction machine learning model will be useful for both sellers and buyers in the market to understand the price of a house. Application developer may develop useful web applications using this model.
Flask python package used to expose our machine learning model as a REST API. Flask is not a complete production ready server. Hence we are going to use gunicorn which is python HTTP server to expose our REST API.
Github repository : https://github.com/harsha89/ml-model-tutorial
Building the machine learning model
Machine learning model build using Pandas and sklearn python modules then exported the model via joblib module. MLPRegressor used to train the model which can be found in here. In this phase, you will try different algorithms to build a robust machine learning model to predict the house price. This model usually expose to a joblib or pickle which can be reuse when predicting the house price. As you aware, training a machine learning model may take couple of minutes to days. Once the model trained, you should export the model to reuse it in an application.
Building the HTTP REST API
We are going to use our exported machine learning model to build a simple REST API. We are using python Flask to expose our API through web interface. In this python logic, it will accept parameters that are required to predict the target price of the house using our trained model stored in ‘rf_model.pkl’ file. You can find the code in here. I defined simple HTTP resource called ‘predict’ to accept parameters to predict house price which come as a json format. This may include number of rooms in a house, distance from main city and etc.
Docker file of the service
Once you build the service, the first step would be to define your docker image. The most important part of building the docker image is to define Dockerfile content which include all the required dependencies along with copying the content of your application to the container. I have explain the meaning of each steps in below sample Dockerfile content. You can find the relevant Dockerfile here. When you building the container, be mindful on cleaning resources and size of your docker image. You should reduce the size of a docker image as much as possible. If you define ‘requirements.txt’ file for your python application, I recommended to run python pip installation by giving the requirement file. You can refer to this article.
#Using the base image with python 3.7
FROM python:3.7
#Set our working directory as app
WORKDIR /app #Installing python packages pandas, scikit-learn and gunicorn
RUN pip install pandas scikit-learn flask gunicorn
# Copy the models directory and server.py files
ADD ./models ./models
ADD server.py server.py
#Exposing the port 5000 from the container
EXPOSE 5000 #Starting the python application
CMD ["gunicorn", "--bind", "0.0.0.0:5000", "server:app"]
Build and run the docker container
In this section, I’m going to build the docker container which will create a container with our service. I assume you have installed docker in the environment. This article include useful resources for install docker and further readings at the end of the article.
#Cloning the project repository in to you environment
git clone https://github.com/harsha89/ml-model-tutorial.git#Build the docker image with name ml-model
docker build -t ml-model .
Once you have build the docker container, you may decide to push it into a docker repository which you can pull from anywhere and run the application. In this example, I assume you trying it in your environment. I’m starting the docker container using ‘docker run’ command and expose port 5000 to access our service.
docker run -d -p 5000:5000 ml-modelNow you can successfully invoke the machine learning model service using curl or any REST client as follow.
curl --location --request POST 'http://localhost:5000/predict' --header 'Content-Type: application/json' --data-raw '{
"crim": 0.85204,"zn":0,"indus": 8.14,"chas":0,"nox": 0.538,"rm":5.965,"age": 89.2,"dis":4.0123,"rad": 4,"tax":307,"ptratio": 21,"lstat":13.83
}'Production grade docker image with NGINX, UWSGI and Flask
Gunicorn is a production grade python server which use with Flask. I’m going to explain on how docker image can be construct with NGINX proxy with UWSGI which is another popular python server runtime. You may follow the same approach with Gunicorn with appropriate NGINX configurations. We are going to use NGINX with UWSGI which is a application server to forward requests coming to our machine learning model service. Below image shows our request flow.
Github repository: https://github.com/harsha89/ml-model-tutorial-nginx
The steps of building and running the docker contain is same as the previous step. Below shows the changes in the Dockerfile to build the container with the requird components.
#Using the base image with python 3.7
FROM python:3.7#Installing the NGINX
RUN apt-get -y update && apt-get install -y --no-install-recommends \ nginx \ && rm -rf /var/lib/apt/lists/* #Copy the nginx.conf to the configuration file location
COPY conf/nginx.conf /etc/nginx/sites-enabled/default #Installing python packages pandas, scikit-learn, flask and uwsgi
RUN pip install pandas scikit-learn flask uwsgi #Set the working directory as app and copy the required files
WORKDIR /app
ADD ./model ./model
ADD server.py server.py#Starting the nginx service and python application
CMD service nginx start && uwsgi -s /tmp/uwsgi.sock --chmod-socket=666 --manage-script-name --mount /=server:app
In a production grade deployment, you need to decide which python package going to act as the python HTTP server for your application. I prefer combination of NGINX, UWSGI and Flask for my deployment as NGINX can provide more functionalities for the server.
Useful resources:
[1] https://pythonise.com/series/learning-flask/python-flask-uwsgi-introduction
[2] https://hackersandslackers.com/deploy-flask-uwsgi-nginx/
[3] https://vsupalov.com/flask-web-server-in-production/
[4] https://flask.palletsprojects.com/en/1.1.x/deploying/wsgi-standalone/
Docker Installation resources
[1] Windows Pro/Enterprise/Education: Follow Docker Desktop Installation guidelines https://docs.docker.com/docker-for-windows/
[2] MacOS: Follow docker installation guidelines in https://docs.docker.com/docker-for-mac/install/
[3] Linux: Follow docker installation guidelines in https://docs.docker.com/install/linux/docker-ce/ubuntu/
Done!
