Deploying your Keras model

Our tools

  1. Flask — A python micro-framework which is easy to setup. Please note that if you need to wrap your service in more logic and better web toolbox (like user login, asset pipeline, CSS frameworks, database, ORM…) you might want to consider separating the model serving from the actual user-facing Keras model serving, and interact with Keras through the server. In this example both of these functions will be in under the same server.
  2. GitLab — Gitlab offers free private repositories for individuals and groups, as well as shared CI (Continuous Integration) workers which will help us prepare and build our server docker image.
  3. Heroku — Heroku is a PaaS (Platform as a service) solution which takes away most of the DevOps work needed to set up a server. This, of course, is a compromise since Heroku does not offer GPU instances. Ideally we would like to serve our model from an AWS GPU instance which will require more DevOps work and would be a topic for a separate blog post. Nonetheless, considering how easy it is to use Heroku, it might be worth checking out as a first step, and compare the performance of your model. Heroku offers few different levels of server compute resources.
  4. Docker — is a containerization solution which will allow us to set up easily reproducible environments that can run the same both locally on our dev machine as well as on production/Heroku. As a side-note, Nvidia has its own fork of Docker which allows for GPU accelerated containers, so that once you’re ready to move your model into an AWS GPU instance, this should be an easy switch. Since this article is directed mostly to data scientists, you could think of Docker as a parallel to virtualenv at the OS level, allowing you to install all dependencies, servers, services and configurations into a container and launch it easily on any machine, scaling it up/down.
  5. Git LFS (large file storage)— as most of the model weights data is more than a few MB, it is advised not to store it in regular git, but rather use a separate git tool to store your weights. Follow the instructions to install git lfs locally on your machine.
  1. Every code change is committed to the local (development machine) git repository and pushed to the remote git repository which in this case will be hosted by GitLab
  2. Once a change has been pushed to GitLab, GitLab’s CI system will pick up the changes and run a build process taking a Dockerfile to create a docker image
  3. The docker image will be pushed to Heroku’s container registry and be used to serve your server

Setting up your local Flask server

curl -OL
mv python-miniconda-master background-removal-server
cd background-removal-server
git init
git add .
git commit -m 'initial commit'

Setting up Git LFS

git lfs track webapp/model/* # This will create a .gitattributes file
git add .
git commit -m 'model'
git remote add origin
git push -u origin master # Watch your models upload with Git LFS

Writing the server logic
cd webapp
FLASK_DEBUG=1 flask run
curl -oout.jpg -F”file=@/Users/---/Downloads/camvid1.jpg” http://localhost:5000/predict
git add . 
git commit -m 'working model prediction'

Deploying to Heroku containers using Gitlab-CI

heroku create background-removal
Gitlab CI log pushing your Keras model server to Heroku using a docker image

Option 2: Run model on the client-side with KerasJS




Ruby (Rails), Javascript, Python, Deep Learning, and mobile apps engineer

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

Review List

Locating gaps in enumerated rows using the SQL lag window function (MySQL edition)

Modular Power-Ups in Unity

Ultimate & Easiest way to install geopandas on windows & add to jupyter notebook which will…

Value Objects in Java with AutoValue and Lombok



Automated backups of your AWS resources with AWS Backup

Getting ING to notify you when your salary is received

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Alon Burg

Alon Burg

Ruby (Rails), Javascript, Python, Deep Learning, and mobile apps engineer

More from Medium

Key takeaways from building multiple machine learning models

Phishing Detection using Machine Learning (ML)

How to use Cosine Similarity and the Tanimoto Coefficient

Performance Analysis of Text-Summary Generation Models Using ROUGE Score