Machine Learning Prediction as a RESTful Web Service

ML Prediction as RESTful Web Service

Interested in such topics or need some help with them?

Get in touch — https://linktr.ee/pranaychandekar

Please refer only to the Demo section(included). This article is old and the components after the Demo section are obsolete. The master branch in the GitHub repository is updated with the latest framework — FastAPI. The link to the updated repository is available in the Introduction section. As the project is containerized, you can still host the web service with the same steps.

Introduction

Often we spend a lot of time in data processing and fine-tuning the machine learning model. But let’s say we satisfactorily trained an ML model. What next? Then we would like to use this model for prediction in production. How?

We can do so in two ways:

1. We can integrate the model in all the downstream services which need predictions from this model.
2. We can host the ML model in a web service and expose a REST API which can be called by all the downstream services for predictions.

Note: There is no ideal way to do it. Depending on the requirements one method can be better over the other.

In this article, we will get our hands dirty with the second approach, the ML Prediction Web Service.

As an open-source contribution, I recently created a project on GitHub which can be used as a template.

pranaychandekar/ml-prediction-web-service

Instead of writing the entire web service from scratch, we will be using this template and only focus on the components which one needs to tweak to customize this project as per the individual need.

Demo

Usually, we implement then we do the demo. But I feel we should go through a demo to get a feel. Then we can dive into the implementation details. So at this point, I would recommend you to try building and running this project using a Docker Engine.

To start with the demo, please refer to the home page documentation of this repository.

Implementation

Requirements: Docker Engine and a Text Editor

The project includes a Dockerfile and is built using the Sanic Framework. The following is the directory structure of the project:

Project Directory Structure

Some details on the directories:

1. conf: This directory contains two files i) app.conf — which contains the service configurations and ii) logging .conf — which contains the logger configurations.
2. docs: This directory contains all the documentation related to the project.
3. logs: The service is configured to write all the log files to this directory.
4. resources: This directory contains all the resources required by our service such as model, data etc.
5. src: This directory contains the entire source code of our web service which includes fetching configurations, parsing the request, prediction, preparing the response, writing the logs etc.
6. test: This folder contains the test cases of our web service.

For more details on individual files, please refer to the source code documentation within them.

Whenever we start the docker container it will run the start-server.sh script which internally runs train.py(to train the model for prediction) and prediction.py(which starts our web service).

Note: The file train.py is optional. Instead of including the model, which was big, I included the data and a training script to train the model for running a demo service. Please delete the line calling train.py in start-server.sh while creating your own custom web service and include your own model in resource->model.

The prediction.py is our controller where we define all the end-points and the corresponding logic.

prediction.py

Running the above script will expose two web-based API. As we have already experienced in the demo, the sanic-openapi integration allows us to access those APIs directly from the browser.

Swagger UI

Files to be changed to use this template for hosting your custom model:

1. app.conf: Add the configurations as per your service need such as DB connection details etc.
2. classifier.py: Add a singleton class to load your ML model.
3. prediction_service.py: Call the classifier instance and perform the prediction task. Prepare the response to be sent back.
4. request_parser.py: Add the logic to parse your custom request.
5. test_prediction.py: Add the test cases corresponding to your service logic.
6. start-server.sh: Remove the 2nd line calling train.py and remove it from the project as well. To host the web service we only need to run the prediction.py.
7. Dockerfile: Update it if required.
8. Update the code documentation.

With these minimal changes, we are now ready to host our custom prediction service.

Takeaways

An ML model once trained can be used for prediction by directly integrating the model with the downstream services or we can host it as a RESTful web service. In this tutorial, we explored one open-source project which we can use as a template and with minimal tweaks can host our custom ML model as a web service in a docker container.

To keep things interesting, I would recommend integrating your custom model by tweaking the project. Do share your experience and feedback.

Interested in such topics or need some help with them?

Get in touch — https://linktr.ee/pranaychandekar

Citations

1. ml-prediction-web-service project.
2. Introduction to API
3. REST API
4. Docker
5. Model training