Deploying your ML model using Flask and Docker
While a lot of blogs and online courses exist around teaching you how to build and train a simple machine learning model or a deep neural network, this mostly ends with a model file on your local computer. Let’s however not forget that this isn’t the end of the story: the model still needs to be deployed in order to be used and evaluated by its end users.
Webservice using Flask
Flask is a micro web framework for Python. It is called a micro framework because it does not require particular tools or libraries. While you would not guess this looking at its (simplistic) overview page, it is for example the main API technology of Pinterest. You can use a Flask webservice to create an API call which can be used by the front-end, or even a full-on web application.
A very simple flask app would look like this:
from flask import Flask
app=Flask(__name__)#code to load model@app.route('/ml-model')
def run_model():
#run model
return resultif __name__ == '__main__':
app.run()
Here we will load the model files when starting the service, this way no useless time is lost when a call is made. Of course, you still need to define the necessary input parameters or files which need to be added to the API call.
Important to note is that the build-in Werkzeug server of Flask is not production-ready and will not be able to handle multiple request in a scalable manner. You will need to run your Flask app using a WSGI server like gunicorn. A performance analysis of different WSGI server can be found here.
This webservice will be deployed in an independent package: a Docker container.
Containerized deployment using Docker
Never heard of Docker? This is the definition from opensource.com:
Docker is a tool designed to make it easier to create, deploy, and run applications by using containers. Containers allow a developer to package up an application with all of the parts it needs, such as libraries and other dependencies, and ship it all out as one package.
You think of Docker as some kind of Virtual Machine (VM), but unlike a VM it will not contain a full operating system. By only packaging the application and necessary libraries, using Docker is a lot more efficient than using VM’s. We will create a Docker image consisting of our flask ML-webservice, which makes it possible to deploy the model on any system. A runnable instance of this Docker image, is called a container.
You can create your own lightweight Docker image, or start by using one from ufoym/deepo (containing base Docker images for research or production with supported GPU acceleration and theano,keras, pytorch or tensorflow pre-installed). For a more lightweight base-image you could take a look at alpine-python.
Finally, these are some tips and tricks we found out when using Docker for our machine learning deployment:
- If you are using a lot of libraries to train your model, building the Docker image takes time. This might cause a timeout error in your deployment process (e.g. Bitbucket pipelines). We solved this by building a separate Docker image containing most libraries, which is used as a base to create the final image much faster.
- When your Docker container stops unexpectedly, it is difficult to find out what’s going wrong. The best way to figure this out, is by going into the container in command mode. If you run test scripts from there, you will see all error logs and for example, find out that a library is missing.
- Like in git, you can use a .dockerignore file to avoid adding unnecessary files into the build directory (e.g. your .git folder won’t be needed there). This will save you some build time and results in a smaller docker image.
