On-Device ML using Flutter and TensorFlow Lite (pt.1): Model Training

This is a two-part article on On-Device ML using Flutter; this article deals with the model-training part of the tutorial; see part 2 of this series in which I develop a Flutter app that consumes this model in an on-device fashion. I invite you to visit other related articles from the community, such as this one and this one. This is my take on how I’d approach it, with my own spin to it.

AI is all the rage — Generative AI, ChatGPT, Google Bard, xAI — due to all the advances and strides that have been made in this field as of late.

Adding ML capabilities to your mobile apps — powered by your device itself — opens up a wide range of possibilities and use cases ranging from entertaining your users to solving complex problems — right at their fingertips, without relying on the cloud — fully offline.

This is where OnDevice ML comes in — The ability to use your device (in this case your mobile app or web browser) to run machine learning algorithms locally, feeding input data — like images, text, or audio — to a locally running ML pertained model instead of sending that data to a server and doing the processing there.

In this tutorial we’ll achieve the following:

• Build and training a simple ML model that converts a value from celsius to Fahrenheit (from scratch) using a Google Colab notebook.
• Export our trained model out of Google Colab as a TensorFlow Lite model for on-device consumption
• Build a simple Flutter app that consumes our created ML model and displays the converted temperature, using the tflite_flutter package

Google offers a range of solutions to use on-device ML to unlock new experiences in your apps, as well as easy-to-use turn-key APIs for more common challenges, such as the ML Kit SDK, containing a wide range of high quality pre trained models that you can easily integrate in your apps.

Benefits of On-Device ML:

• Low Latency: By having the ML model right on your device, you can feed it user-generated data (text, audio, video) in real-time.
• Keep data on-device: Great for user privacy — the data doesn’t leave the device and you can perform inference locally.
• Works offline: Lack of reliance on a network connection or having to run a service in the cloud makes this attractive — all processing happens on the device.

Limitations:

• ML models may be too large, making your app bundle bigger.
• Some ml models require some processing power — limited by your device’s capabilities

Let’s start!

Building your ML Model in Google Colab

Google Colab is free to use, and. you should use a valid Gmail account to access it.

The problem we will solve is to convert from Celsius to Fahrenheit, where the approximate formula is:

𝑓=𝑐×1.8+32

Of course, it would be simple enough to create a conventional Dart function that directly performs this calculation, but that wouldn’t be machine learning.

Instead, we will give TensorFlow some sample Celsius values (0, 5, 10, 15, 20) and their corresponding Fahrenheit values (32, 41, 50, 59, 68). Then, we will train a model that figures out the above formula through the training process.

Google Colaboratory, or “Colab” for short, is a Google product that allows anybody to write and execute arbitrary python code through the browser, and is especially well suited to machine learning, data analysis and education.

Google Colab Landing Page

Create a New Notebook by clicking on the New Notebook button.

Start by adding a code cell as below:

As instructed, in each code cell, you’ll be adding the code provided in the steps below.

Import Dependencies

We need to import Tensorflow; use an alias — tf for short- as well as libraries such as numpy (commonly used in machine learning) and logging.

import tensorflow as tf
import numpy as np

Code cell should look like below. Click on the “Run Cell” button on the left of the cell to execute. Let’s do this for every code block moving forward:

Define the training data

We’ll be performing Supervised Machine Learning — which is all about figuring out an algorithm given a set of inputs and outputs. We want a model that given the degree in Celsius, it yields the degree in Fahrenheit, so we will create two lists — one called celsius_features and fahrenheit_labels, type float — that we will use to train our model.

Add a new code cell block with the following code — make sure to Run the code cell:

celsius_features = np.array([0, 5,  10, 15, 20, 25, 30, 35, 40,   45, 50,  55,  60, 65,  70, 75,   80,  85, 90, 95, 100],  dtype=float)
fahrenheit_labels = np.array([32, 41, 50, 59, 68, 77, 86, 95, 104, 113, 122, 131, 140, 149, 158, 167, 176, 185, 194, 203, 212],  dtype=float)

for i,c in enumerate(celsius_features):
  print("{} degrees Celsius = {} degrees Fahrenheit".format(c, fahrenheit_labels[i]))

Some Machine Learning terminology

• Feature — The input(s) to our model. In this case, a single value — the degrees in Celsius.
• Labels — The output our model predicts. In this case, a single value — the degrees in Fahrenheit.

Create the model

The next step is to create the model. A Dense network is the simplest TensorFlow model we can use since our problem is pretty is straightforward. This will be a network with a single layer, and a single neuron.

Build a layer

We’ll call the layer layer_0 by using the tf.keras.layers.Dense and the following parameters:

• input_shape=[1]: this means the input to this layer is a single value — the shape of a one-dimensional array with one member. Since this is the first (and only) layer, the input shape is the input shape of the entire model. The single value is a floating point number, which represents the degrees Celsius.
• units=1: the number of neurons in the layer, or what defines how many internal variables the layer has to try to learn to solve the problem. Since this is the final layer, it is also the size of the model’s output — a single floating point number representing degrees Fahrenheit.

Add the following code block:

layer_0 = tf.keras.layers.Dense(units=1, input_shape=[1])

Add the layer to a Sequential model, appropriate for a plain stack of layers where each layer has exactly one input tensor and one output tensor:

model = tf.keras.Sequential([layer_0])

Compile the model, with loss and optimizer functions

You must compile the model before training occurs. During compilation of the model, you must provide the following to the model:

• Loss function — A way of measuring how far off predictions are from the desired outcome. (The measured difference is called the “loss”). In our case, the “mean_squared_error”.
• Optimizer function — A way of adjusting internal values in order to reduce the loss; in our case, an optimizer that implements the Adam algorithm.

Add the following code block:

model.compile(loss='mean_squared_error',
              optimizer=tf.keras.optimizers.Adam(0.1))

Train the model

Train the model by calling fit on the model.

During training, the model takes in Celsius values, performs a calculation using the current internal variables (called “weights”) and outputs values which are meant to be the Fahrenheit equivalent. Since the weights are initially set randomly, the output will not be close to the correct value.

The difference between the actual output and the desired output is calculated using the loss function, and the optimizer function directs how the weights should be adjusted.

This cycle of calculate, compare, adjust is controlled by the fit method. The first argument is the inputs, the second argument is the desired outputs. The epochs argument specifies how many times this cycle should be run, and the verbose argument controls how much output the method produces.

Add the following code block:

model.fit(celsius_features, fahrenheit_labels, epochs=2000, verbose=False)
print("Finished training the model")

Use the model to predict values

Congrats! You’ve just trained a model that has been trained to learn the relationship between celsius_features and fahrenheit_labels. With that done, you can proceed and use the predict method in order to have it calculate the Fahrenheit degrees for a previously unknown Celsius degrees.

So, for example, if the Celsius value is 0, what do you think the Fahrenheit result will be? Take a guess before you run this code.

print(model.predict([0.0]))

# output should be 32!

Yes, it should be 32, but initially, it may not yield the correct result (you may get something like 30.897123 or something like that; it may give you an approximation to the actual expected value — this means only one thing. We must train our model a bit more! Go back previous code cells (where the model.fit execution block is) and run it one more time. Then come back down to this one and run it again.

With one more run of the model.fit, the model improves — just like anything, with more training, your model gets “fit”. Run it a couple of times until the value of 0.0 becomes 32. Then try a couple of other values as well. For us, we brought it to 31.999968 (pretty close, don’t you think?).

Save and Export your Trained Model

Let’s get our model out of Google Colab for consumption.

But first let’s do some configuration to accomplish this.

Save the model in the Tensorflow format (tf). The saved model contains:

• the model’s configuration (architecture)
• the model’s weights
• the model’s optimizer state (if any)

Store the name of the path of our model in an environment variable called KERAS_MODEL_PATH, as such:

KERAS_MODEL_PATH = 'tf_keras_c2f.tf'

Save the model using the provided path in the tf format, as such:

model.save(KERAS_MODEL_PATH, save_format='tf')

With the saved model, now let’s proceed to convert our Tensorflow model to a Tensorflow Lite format, for on-device consumption, using the TFLiteConverter utility; save it to your local Google Colab environment, from where you can later download it.

# convert the TF model to TFLite
converter = tf.lite.TFLiteConverter.from_saved_model(KERAS_MODEL_PATH)
tflite_model = converter.convert()

# Save the TFLITE model
with open('modelc2f.tflite', 'wb') as f:
    f.write(tflite_model)

After the last code block execution, you should get a file called modelc2f.tflite in the Files section on the left of your colab notebook, as shown below:

Download it by right-clicking on it and selecting Download.

Now you have a fully-trained, simple, yet fully-functional ML model that you can use anywhere you can consume a Tensorflow Lite model; all it takes is finding the proper packages and SDK to read the .tflite file, feed it the expected input and just wait for magic to happen!

In part 2 of this tutorial series, I’ll be developing a Flutter app from which we’ll consume this Tensorflow Lite pre-trained model, and build a simple, yet cool looking interface to feed Celsius values to it, and display the corresponding Fahrenheit values it yields.

Full Code to the Project (including Google Colab File (.ipynb file), trained model file (.tflite) and corresponding Flutter app inside this Github Repo.

See you on the next tutorial!