From Keras to Android with TensorFlow Lite

Photo by Mika Baumeister on Unsplash

Do you want to find out how to create a machine learning model and run it on the phone? The first part of the post shows how to easily create a Keras model that is using TensorFlow. The next part covers how to train the model and convert it to TensorFlow Lite. The last part covers building a simple Android app that runs the model.

It is easy to find materials on how to create only a model or only a mobile app. There are a few tutorials that describe how to run the prepared specific model in the mobile app. Nevertheless, the real challenge comes when you would like to create your own model or someone asks you to run their brand new super awesome model in the app.

Keras setup

You will need Python in order to use Keras. I would recommend using Python3 but in general, everything that we are going to do might work even with Python2.

brew install python3

Once you have Python installed I would recommend to setup pipenv which simplifies handling of the Python virtual environments.

pip install pipenv

The next step is to create a library where our experiments will happen.

mkdir keras_sample
cd keras_sample
pipenv --three
pipenv shell

If everything went well you should see in your command line output similar to this one.

Creating a Pipfile for this project…
Launching subshell in virtual environment…
bash-3.2$  . ~/.local/share/virtualenvs/keras_sample-uJxZiIeS/bin/activate
(keras_sample) bash-3.2$

There is a need to install a few more libraries before we continue. I use the Jupyter notebook for the following examples. Nevertheless, the examples should be easily runnable in a command line or PyCharm.

pip install jupyter
pip install pandas
pip install matplotlib
pip install sklearn
pip install keras
pip install tensorflow
pip install toco

Just in case that you would like to use the Jupyter notebook too here is command how to start it.

jupyter notebook &

The ampersand at the end causes the process to start in the background. Thus you can go to the command line anytime and use it for installation of the other libraries.

TensorFlow Lite operations restrictions

The dark side of aiming for usage of TensorFlow lite is that you need to use limited a subset of Tensorflow operations. Which might need you to rewrite your models in such a way that they will not use RNN or LSTM. The full compatibility list is available here. If you would like to have your model accelerated by GPU on both iOS and Android then there are unfortunately even more restrictions to usable operations. On the other hand, if you full-fill the restrictions for GPU the outcome will be 20–50% faster inference execution on both Android and iOS.

Keras model

This part will be hopefully very straightforward as the example model that I share here is just sort of “Hello World!” one. I was originally thinking about the usage of some well-known data-sets but I realized that that would take away some initial tweaking fun.

We can create a simple model with the generated data. That will predict results only based on this data thus “real” world ones might not work.

This model has some room for improvement but it shows enough for what I want to demonstrate. Despite the inaccuracy of a model I believe that it will provide you quite some space for playing with it. You can use it to get more understanding of how small changes can affect the model. Sample tasks that you can try to see how the model accuracy can be influenced:

• reduce the size of the generated training set from 50K to 10K
• increase amount of units from 16 to 128
• add another layer to model

Conversion of the model

We set up our environment, created a data generation tool and created a model in the previous chapters. The save model file ‘trained_model.h5’ should be available in your folder. I encourage you to have a look at the file in spite of it being a binary file as you will see that the graph definition is normally readable. This might be useful if you will want to find out anything about the model that you obtained from some other team e.g. whether the input is float or int.

The command to convert the Keras model to TensorFlow lite one:

toco \
  --output_file=trained_model.tflite\
  --keras_model_file=trained_model.h5

I am using toco but you can obtain the same results with ‘tflite_convert’ tool or you can use python API. I also believe that you will be very surprised by the size of the converted model. The original Keras simple model has 14KB and the tflite version has only 1.4KB.

I would highly recommend checking also the other toco/tflite_convert_tool params. The one that could be especially interesting is the quantization that, simply said, swaps FLOAT weights for UINT one. This step will decrease the accuracy on average between 1–8%. At the same time, it will shrink your model by about 50–75%. This might be especially useful on the mobile phone where you load the whole model to memory.

Android App

Android Studio(AS) installation guide can be found here: https://developer.android.com/studio/install .

• Once you have AS installed, open it and select create a new project and select “Empty Activity”
• Select “Empty Activity”

Once the project is opened go to app/build.gradle and the following dependency. You can eventually replace version “1.13.1” with “0.0.0-nightly” which will result in the download of the latest available nightly build.

implementation 'org.tensorflow:tensorflow-lite:0.0.0-nightly'

The next step is the creation of the “assets” folder where we will place our trained_model.tflite model file.

<project folder>/app/src/main/assets

When we have the model in our assets directory then all that is left is to add code that will actually run it. In order to achieve it just go to the layout of your main activity which is “activity_main.xml” in my case and make sure that you have there Textview with id android:id=”@+id/result”. The next step is activity code itself which you can take from here:

The calling setup is straightforward one if you know what is the input shape and if quantization was applied. The input shape means how many rows, columns, dimensions (e.g. RGB for pictures) or types. The basic output comes in the form of prediction value and prediction score. The sample app is available at my GitHub repo. Once you are done with the setup and the app is running you should see on your screen exactly the same result as you saw in the python just before saving the model.

I showed you, in simplified form, how to define the ML model and processes needed in order to prepare the model to run it on the mobile phone. I made sure to keep testing examples as similar as possible so that it is easy for you to recognize the Keras model mapping to an Android app setup. Yes, there are some restrictions/compromises applied in order to make the flow simple. Nevertheless, if you can fit your model into those restrictions then you should enjoy running the models in your app. I am going to show you how to handle the situations when the TF Lite is too restrictive in the next post.