Serve Machine Learning models with a scalable deep learning API powered by IBM
Deploy machine learning models in minutes into production at scale with IBM Machine Learning.
In this article I discuss how you can run different machine and deep learning frameworks like Tensorflow, Pytorch or Scikit-Learn on a scalable REST API. In the first section, I discuss what decisions you have to make before you starting to code your own API. In the second section I explain how you can use the IBM Machine Learning service to do the same in minutes.
What problem does this article address: Data scientists struggling to serve & deploy their machine learning models via REST API.
Who should read this article: People that looking for a simple way to serve & deploy machine or deep learning models via REST API (Online) or a batch job.
Provided Solution: The IBM Machine learning service is designed for performance, scalability and high availability. It allows data scientists to focus on data science work instead of maintaining and coding a REST API.
Let’s start by considering what goes into creating your own scalable deep learning REST API.
- What Machine Learning frameworks should your REST API support?
There are different frameworks like: Tensorflow, Scikit Learn, Pytorch, SPSS, PML, Onyx, Keras and more. Most of them deliver their own way to serve the models. For example in PyTorch you can use TORCHSERVE.
Nice - but what happens if you have different models? One option is to use a Python Flask server that allows you to create you own API. But, that means everything is up to you. You are responsible for maintaining and testing the code required to build the API.
Another method is to convert all models to the same framework using PMML or ONNX. This is probably one of the best options, but you have to rely on the dependencies of the AI framework and ONNX.
2. What is necessary for a scalable API?
If you answered the first questions above, your next step is to think about the architecture of your REST API. Things to consider are:
- Performance (Space, Memory, CPU)
- Scalability (Vertical, Horizontal)
- High Availability (Load Balancer, Backups, Recovery)
- Hosting (Cloud, Bare Metal)
- Security (SSL, OAuth, API-Key)
All of these points are important and should be defined at the beginning of each project. Changing the architecture at a later stage is expensive and time consuming.
3. What kind of API do you need for you model?
Online: Some models need to be available online as a web service, or REST API, that generate predictions in real-time. For example, you create an application for a real estate company that uses a regression model to predict house prices. The model would called every time you create or edit a property listing.
Batch: Some models are used as part of a batch job that processes input data from a file, data connection, or connected data in a storage bucket, and writes the output to a file. For example, if you create a scoring model for movies and you want to calculate the score for additional movies, you might use a batch job to process bulk requests.
4. Pros and Cons of creating your own REST-API.
- control of the code and the environment
- high customization
- independence
- high development, test and maintenance effort
- need of developers, admins, architects and site-reliability engineers
IBM Machine Learning Service:
IBM’s machine learning service helps you to deploy and serve your machine and deep learning models in minutes. All the questions and considerations discussed so far are covered by IBM’s Machine Learning Service. It provides an easy to use API endpoint for machine learning models. The Machine Learning Service handles all the aforementioned points for you allowing you to focus on data science work.
What machine learning frameworks are supported?
IBM machine learning supports the most common frameworks and allows you to deploy them in various ways. Use the link and table below to view the supported frameworks.
If IBM doesn’t support your preferred framework, you can use custom packages and define your own software specification. The ability to use custom packages, allows you greater flexibility in your software stack.
What is needed?
- Create an IBM Cloud Pak for Data account (Try it for free)
Try it out!
Step 1: Signing up for Cloud Pak for Data as a Service
Step 2: Create a project
Step 3: Deploy and Test your Model
If you prefer written documentation, check out these links:
- Signing up for Cloud Pak for Data as a Service
- Create a project deploy and test your model
- Deploying models with Watson Machine Learning
Additional documents related to IBM Machine Learning:
What else should you know about IBM’s machine learning service?
Watson Machine Learning provides a full range of tools and services you can use to build, train, and deploy machine learning models. Choose from tools that fully automate the training process for rapid prototyping to tools that give you complete control to create a model that matches your needs.
Congratulation — that’s it!
You have learned how to register for IBM’s Cloud Pak for Data and how to use the Machine Learning Service to serve your models.
Disclaimer: Opinions are my own and not the views of my employer.
