Deploying a Keras Model on Android

(Using TensorFlow on Android)

Even with the large number of tutorials about deploying Keras models on Android, I had to spend quite some time to sort things out. So, like this amazing article by Yoni, I decided to dump my experience here.

Existing Guides

Assuming that you have your Keras model trained and ready to go, you should convert freeze the graph to a .pb or protobuf file. This is as easy as doing the following:

After that you should follow the steps in Yoni’s article (which I mentioned previously), with a few changes when writing the Android code.

Deploying a TensorFlow model to Android

These changes reflect the change in the Tensorflow Java API. Check out the diff with the code in Yoni’s article here.

Issues I faced

1. Obtaining a frozen version of the model using Keras

Do check that the names of the input nodes and the output nodes that you use on android are the same as that in the model. To check the names, do the following:

This function takes in the name of the file, and prints out the name of the input and the output nodes. It is similar to the one in Yoni’s tutorial, and it also helps you with the Keras Learning Phase error, which happens when you run your model on android. This is the error: java.lang.IllegalArgumentException: You must feed a value for the placeholder tensor ‘ls1/keras_learning_phase' with dtype bool.

If the error occurs for you, then a node with the name ls1/keras_learning_phase will be printed when you use the print_graph_nodes function. To solve it, when you are training, add this just before the line where you feed other inputs to your model: K.set_learning_phase(1). (Note that K is the same as from Keras import backend as K.)

This happens because when you use the model on android, it’ll ask you to feed data to the learning phase node, as it is also an input node. It is for the same reason that we do K.set_learning_phase(0) when we freeze the model.

2. Building from Scratch using NDK and Bazel

This is not really necessary, and takes a lot of time. If you do not need to modify the tensorflow android code to customize it for your use case, you don’t need to do this!

Instead, you can download the files from http://ci.tensorflow.org/view/Nightly/job/nightly-android/lastSuccessfulBuild/artifact/out/ , and directly savelibandroid_tensorflow_inference_java.jar, and all files in libtensorflow_inference.soto the libs folder.

But in case you need to modify the file (I needed to add a method to feed boolean input values as input to the neural net — solution here), then you have to follow the instructions in the tensorflow android repo.

3. Debugging Errors in Android

There is no documentation about the errors you get while running the model on an Android app. This is the most helpful thing that I found by guojia-git:

I guess understanding the error messages has been the biggest obstacle for me throughout the process. It is especially hard for people who are not working with the C++ API (I don’t know if it’s possible to debug and see what’s happening in the C code below JNI). It’s tricky to figure out that You must feed a value for placeholder tensor XXX happens when some placeholder exists in the graph but was not fed with a value, while FeedInputs: unable to find feed output happens when some placeholder does not exist in the graph but you are feeding value to it.

This, coupled with the article by Yoni, was all the info I needed to make my app.

Have fun making the android app!

Note: This article is also present on my blog at https://thepulkitagarwal.github.io/deploying-a-keras-model-on-android