Deploying an ML-based containerised (Docker) Web application to a local Kubernetes cluster
1. Introduction
With the evolution of machine learning Model as a service, a critical aspect is the ability to package this service within a container and deploy the same to ensure high portability, availability & scalability. One of the widely accepted solutions is a Docker container which in turn can be deployed on a container orchestration platform like Kubernetes.
This article is primarily targeted at the Data Science audience who are conversant with Python/R programming and want to get the initial headwinds of deploying a model as a service using a containerised approach which is an integral part of the MLOps framework.
Within the data science team, we use an internal platform called Walmart Cloud-Native Platform (WCNP), based on the Kubernetes clusters, to deploy the respective container configuration & code files to the repository. This in turn would trigger the pipeline to deploy this container on a Kubernetes cluster.
As seen in Figure 1, during the initial developmental stages we tend to directly deploy these to the Kubernetes platform but we might end up spending more time just to identify and debug many ML code-related or docker configuration-related issues.
A straightforward solution might be to develop the entire pipeline to the point where a functional containerized solution exists on the local machine, akin to developing a basic ML algorithm on your local machine/like a playground — Refer to Figure 2.
This makes testing various functions, exploring configurations and experimentations much easier post which we can simply port these files to the above WCNP related pipeline (with a few additional cluster configurations). Overall, thus avoiding & reducing any major code-related, docker configuration related or cluster-related issues.
Lo & behold, the solution to this is a simple desktop-based platform tool — Docker Desktop.
While researching and executing the coding aspect of a container & Kubernetes deployment on the local machine, I found a behemoth of materials (mentioned in below references along with numerous Q & A sites) in terms of various codes, commands, systems/cluster configuration related files, etc. This involved multiple hours of reading & testing just to get a basic web app container up and running.
Hence keeping in mind, the perspective of a Data Scientist, I planned on preparing this tutorial which would list the very basic and essential steps required to get an ML model deployment running on a Kubernetes cluster within a matter of minutes. Most of the basic commands would remain the same when deployed on the cloud services with the added complexity of configurations coming into play.
Below, we list the steps to build a basic ML Spam detection model & focus more to create a web application, package the same into a Docker container & deploy the container onto the Kubernetes cluster in local (running minikube single node cluster on the backend).
Objectives
- Package a sample ML-based web application
- Expose the sample app using REST API
- Create, Build & Run a Docker image
- Use minikube for cluster creation on local
- Deploy the container to the Kubernetes
- Test the service
2. Installation:
Docker Desktop can be downloaded from the website which would help spin up the Kubernetes container orchestration system using minikube (helps run a single-node Kubernetes cluster for daily development work). Other than this we would require a working Python editor such as Jupyter.
Post-installation of Docker Desktop, go to Preferences -> Enable Kubernetes.
3. Developing the ML Model:
We use the UCI SMS Spam Collection Dataset to develop a spam detection model which is built using a basic Count Vectoriser for feature creation and a Random Forest model for prediction. More sophisticated models can further be explored using tf-idf, context-based vectors, etc.
Any interim outputs need to be saved to a storage space, which would be required in the Prediction phase such as the vocabulary and model object in this case.
#Save vectorizer vocabulary to storagepickle.dump(cv.vocabulary_,open("feature.pkl","wb"))#Save the model to storagefilename = 'model.pkl'pickle.dump(rfc, open(filename, 'wb'))#Consumables:· feature.pkl· Model.pkl
Model Output: comma-separated values as 0 — ham, 1 — spam
4. Deploying the ML Model as a Flask Application:
Flask package is a WSGI-based web application framework and is frequently used to expose the model as a service using REST API and we would use the same.
As per Figure 4, we load the previously saved objects for the prediction and ensure the creation of the default IP address (0.0.0.0) and port 5000 to be used for the API call. Debug mode allows for easy detection of errors. This Python notebook is converted to the *py format for deployment (app.py).
The default “Hello world” at the base address can be checked to ensure that the service is up & running, further, we would use the “/predict” extension for predictions.
To quickly test our application, we use the below command in the same working directory terminal ensuring app.py is present. You could even use the URL and predict using the code we describe further on.
$ python3 -m flask run
5. Building & Running the Docker container image:
Navigate to the same directory as the app.py and ensure the creation of the below files in the same format.
Dockerfile — In this example we use the base image of python:3.6 (specified in the first line). The EXPOSE keyword allows the container to listen on the specified port(s) during runtime.
Please refer to https://hub.docker.com/ for any specific docker image as per the needs of the application.
requirements.txt — This is a collection of packaged libraries, which makes it easier to download. Typically, the pip freeze command is used, but the pipreqs packages work better in terms of capturing only the dependencies of the current project.
$ pip freeze > requirements.txt
$ pipreqs /home/project/location
Successfully saved requirements file in /home/project/location/requirements.txt
Now, navigate to the directory with the app.py, *.pkl, Dockerfile & requirements.txt and use the below commands in your terminal window.
Application Name: flaskft
Version Name: develop1
docker build -t flaskft:develop1 .
We can now run our image as a container using the below command.
docker run -it -d -p 5000:5000 flaskft:develop1
Docker can run the container in detached mode using -d.
To announce a port for our container, we use the -p flag on the run command in the format [host port]: [container port]. So, if we wanted to expose port 4000 inside the container to port 2000 outside the container, we would pass 2000:4000 to the -p flag.
To check the container details, we can use the ps command.
docker ps
Thus, every time you want to change this app, you could change the version name and redeploy.
6. Deploying the Docker container on a Kubernetes cluster
Kubernetes, which is an open-source platform for managing containerized workloads and services is then used for deploying this container created. While there are multiple Kubernetes versions in the public domain, we use the local desktop deployment which would require the use of minikube to run a single-node Kubernetes cluster.
We first look at the deliverables that are required.
yml file: Specifies the configuration for the pod is to be created. In this configuration we have a single container running inside the pod.
name: Deployment name of the Container.
imagePullPolicy: Kubernetes generally tries to pull the specified image from a registry, either in the minikube docker registry or in the public Docker registry but in this case, we have not registered the image with any of these and hence need to state this is ‘Never’ to prevent an error.
image: Use the same image name as the docker image.
restartPolicy: The restartPolicy applies to all containers in the pod with possible values of Always, OnFailure, and Never. While the default is active, we set it to never as this is only used for testing purpose.
containerPort: This is the port number to access the container belonging to the pod when targeted by the service.
Please ensure correct indentation and removing extra white spaces which might otherwise lead to deployment failure. Ensure the files are present in the same directory.
Now, navigate to the directory with the above files & use the below commands in your terminal window.
minikube start — Starts the minikube service
eval $(minikube -p minikube docker-env) — configure your local environment to re-use the Docker daemon inside the minikube instance.
docker build -t flaskft . — We need to rebuild with the above environment variable configurations, ensure the same app name as specified during container creation.
kubectl create -f spamcheck.yml — will create the pod as demov1 and get an output like pod/demov1 created.
kubectl get pods — View the pods
kubectl port-forward demo 5000:5000 — Port-forward from the local machine to the pods.
7. Testing the deployment
To test this deployment, we use the localhost URL to test two strings of being a “ham” or “spam”.
As above, we have seen how to deploy our machine learning solutions using a container and Kubernetes using our local machine which can be used for developmental/experimentation purposes during the initial stages of new deployments.
Overall, the Docker Desktop enables us to deploy and interact with various containers on the local machine. It provides a user-friendly GUI interface to perform common actions around container deployment which is very beneficial to new users. Moreover, it helps validate that the deployed code is working as expected before delving deeper into the container-based configuration and issues.
References:
https://kubernetes.io/docs/concepts/workloads/controllers/deployment/
https://docs.docker.com/engine/
