Deploying your Keras model using Keras.JS

Demo | Sourcecode

After we explored our first option of running a deep learning model on the server side, let’s consider running it in the browser.

What are the upsides:

1. Saves on server costs. Static HTML can be run for free.
2. Saves user bandwidth and time by avoiding upload times.

The downsides and how we can avoid them:

1. Slow — For the model presented in this example, KerasJS was running almost ~50 seconds per image prediction, comparing to a ~4 seconds prediction per image on a CPU only server side.
2. The browser environment is not as controlled as the server side, and things tend to break. For example, KerasJS will throw an exception when running on IOS with GPU on. Additionally iterating on the code in Javascript and building a pipeline can be quite time-consuming as some of the Javascript tools for dealing with images is not as easy to use as Python, e.g: having to move between Image and Canvas, and the lack of good Arrays/Matrix data-structures which are used across libraries.
3. CPU intensive causing the UI thread to halt completely. In order to address the UI thread getting stuck, one could use WebWorkers. However, this currently comes at the cost of using CPU only as WebWorkers do not have access to the GPU. A more naive relief could be the KerasJS option for layerCallPauses which will pause for a tick after each layer. New libraries such as WebDNN offer WebAssembly and GPU which should speed up things significantly.
4. Memory consumption for the browser process can reach 8GB easily for a model such as the 100-layer Tiramisu.

The Chatbot Conference

Our tools

1. Vue — A client-side framework (somewhat similar to React), which has an easy an easy start.
2. KerasJS — Is a port of Keras for the browser, allowing you to load your model and weight, run predict().
3. Gitlab CI and pages — We will use GitlabCI to build our project each time it is pushed and publish it to Gitlab Pages

Setting up a skeleton

We will be using a default template to bootstrap our app, and tweak it a little to support KerasJS. In your terminal run:

npm install -g vue-cl # Install vue-cli
vue init background-removal-vue # Create our project from a template
cd background-removal-vue
npm install --save keras-js blueimp-load-image fs lodash ndarray ndarray-ops 
npm install --save-dev raw-loader glslify-loader # These are required for KerasJS shaders to load

Tweak your build/webpack.base.conf.js to use the glslify-loader and stub node.fs

webpack.base.conf

Now that we’re ready to start development, run npm run dev to start your local dev server. A new browser tab will open showing your app. Webpack Hot Module Reload (HMR) will automatically refresh the browser each time you save a file to reflect the changes.

Setting up Git LFS

Make sure you have Git LFS installed. Use KerasJS helper utility to encode your model for the browser and then commit those files using Git LFS.

mkdir static/model
cp $YOUR_MODEL_DIRECTORY/model_wigh_weights.h5 static/model/
python ./node_modules/keras-js/encoder.py static/model/model_with_weights.h5
git lfs track static/model/* # This will create a .gitattributes file
git add . 
git commit -m 'model'

Setting up Gitlab CI + Pages

In order to publish our app, we could use Gitlab Pages which will host our model and HTML pages, and run a build process each time we push the project.

In order to do that, let’s add a .gitlab-ci.yml file

.gitlab-ci.yml

Once you commit this file, gitlab will run a build process and our new app will be available automatically at https://username.gitlab.io/project-name/ You can add a custom domain alias by going to your Gitlab project Pages setting.

Running the model

This part is rather specific to the model and task we had at hand and so I won’t dive too deep into it:

1. In the first part of runModel() we convert our image into a (224,224) input size, and create an Input tensor of 3 RGB channels from our canvas 4 channel image.
2. Before running the model, we set the input dictionary, and set callbacks for the progress bar.
3. When the predict() ends, we process the output so that each pixel which was determined to be background, will have alpha=0, while the rest will have alpha relative to their activation. The main for-loop acts similarly to an argmax() function.

One pitfall which you might want to note when dealing with images is the EXIF metadata which requires to rotate the image according to the metadata. Blueimp.loadImage() can do it when setting orientation: True.