We come across lot of interesting deep learning projects on HackerNews and Reddit and one which caught my eye was this neat “image background removal app” developed by Alon Burg and Gidi Superber. Both Alon and Gidi have been kind enough to open source thier work for others to experiment. Thanks guys.
Having run an e-commerce startup, I could clearly see the value of such an application. It’s still early days, but deep learning will go a long way in automating many image processing workflows.
While the image background removal application is amazing, what I found most interesting Alon Burg’s post on deploying the app’s Keras model. They’ve used the following tools to get this up and running (1) Flask to wrap the model and serve it as an API (2) Heroku for deployment (3) GitLab to manage the CI/CD pipeline (4) GitLFS to manage model files/weights (5) Docker, one of the key pieces of software in data science, for reproducible environments.
Both Alon and Gidi come from the app development space so most of these technologies are known to them. However, this may not be the case for every data scientist or machine learning engineer. Agreed, that you can learn most of these technologies but it’s just too time consuming and in a business setting, as James Lindebaum puts it, developer productivity is the most important resource in any company. In our case, it is productivity of the data scientist!
Most of these tools have matured a lot over the years but I feel they are still too low-level for a data scientist/machine learning engineer. It is hard to productionize your models efficiently. To address this, we built a Platform-as-a-Service (PaaS) for data science that abstracts away all the complex configuration and plumbing to make deployment frictionless. This post is about how we deployed Alon and Gidi’s model into production using rorodata.
Our workflow and deployment approach looks something like this
- Each deployment is encapsulated in a project. The project contains all the related files which are locally version controlled with git. Maintaining a github repo is completely optional
- Write a small predict function that loads the model and returns the desired response. We’re also working on making this step obsolete, more on this in the future.
- Composes a simple YAML file (similar to Procfile in Heroku-land) that specifies runtime (environment), hardware you want to use (it may CPU or GPU) and specify your predict function. Like in case of any python project you can also have a requirements.txt for additional libraries
- That’s it. With one command your entire model and app is deployed and you get a unique RESTful API endpoint
All you code remains local, however the entire deployment happens on cloud.
Setting up your project
You will need create an account on rorodata. We will soon have a public sign-up page, but until then just write to us and we can create an account for you. Then install the rorodata CLI on your machine.
$ pip install roro
...
$ roro login
Email: rb@rorodata.com
Password:Login successful
$
Now just create a new project in your working folder, like so.
$ mkdir background-removal
$ cd background-removal
$ roro create background-removalCreated project: background-removal
Managing model storage
Every project on rorodata gets a dedicated storage volumes /volumes/data and /volumes/notebooks In our case stored the model into /volumes/data volume.
Writing the predict function
The predict function is just few lines of python code that loads the model and runs the prediction. Please note there is not Flask code or routes anywhere, just pure python function.
Getting ready for deployment
Before we deploy all we need to do is setup the simple configuration file that provides details of (1) runtime (2) predict function to be deployed as an API and (3) hardware
project: background-removal
runtime: rorodata/python3-keras services:
- name: api
function: background_removal.predict
size: S1
Runtimes are reproducible environments composed using docker. We support four runtimes python3 python3-keras python2 sandbox. We will be using the python3-keras runtime for our project. We also have a runtime sandbox that provides support for all popular python ML/DL frameworks. For advanced users, we offer the flexibility of adding more runtimes or extending existing ones. More on this in a following post.
We simply specify the prediction in the format <file-name>.<function-name> format. Finally specify the hardware you want to run this on. Typically, for inference a small CPU is good enough (these are called dynos in Heroku-land). We support four CPU sizes (S1, S2, M1, M2)and one GPU (G1). For more details, please refer user guides.
Now, we’re ready to deploy.
$ roro deploy
Deploying project background-removal. This may take a few moments ...
Building docker image ... done
Updating scheduled jobs ... done
Restarting services ...
api: https://background-removal--api.rorocloud.io/That’s it! Your service is live and you can consume this API via any REST client. That said, we prefer that you use a client we’ve optimized, we call it firefly. More things are in pipeline which will let you build and manage data products (and APIs) easily. We would love to hear your views, suggestions and feedback.
Conclusion
As engineers, we are most productive if we work with right tools and at right level of abstraction. Large internet companies have realised this and built platforms that allow thier data scientists/machine learning engineers to build, test, deploy and monitor their models in production. We can see many examples from Uber, Instacart etc. but these are just 1% of the companies. Check this talk by sharath rao from Instacart. This is just one example there are many you can find in the wild.
I believe that a lot of innovation using ML/DL will be happening in very interesting verticals and domain experts are going to need tools to get their ideas to market. We’re committed to making it easy to build and deploy data products.
Bonus
Do you want to deploy models on your own server? Please try rorolite, a small open source version of our API. Here is short video tutorial on using rorolite with Digital Ocean.
Author: Raghavendra Badaskar
Please do share your comments below or reach us via twitter @rorodata or Slack
