ML in Android -3 : Deploy Tflite on Android using Java

Learn how to deploy tflite model on android app using Java. We’ll also solve “tflite model can give same output for different output on android” issue .

In the last article of the tutorial series we discussed about various concepts of tensorflow-lite and built a model that predict x+1 on giving an input of x.

Scope of the article

• In this article, we’ll deploy the model on android app and thus built first Deep Learning android app.
• In this article, you’ll learn how to pass input to .tflite model using java and also how to handle the output.
• We’ll also go through the concepts behind the code and a possible bug.

Integration
Since, we have model built from last tutorial, we’ll look how to deploy add to android studio.

• To import a TensorFlow Lite (TFLite) model, click on File, then New > Other > TensorFlow Lite Model

Fig . showing 1st step

• Select the location of your TFLite file and select “add auto build feature…” to automatically add dependencies into your Android module’s build.gradle file, and click finish.
• After the import is complete, the following screen will be displayed. To use the model, go to the Sample Code section and copy and paste the code in Kotlin or Java. Double-click the TFLite model in the ml directory in Android Studio to return to this screen.

Fig . Successful integration & model name is model1a.tflite

Developing UI/UX

We’ll build interface to take input from the user and display the output on the screen. We’ll have one activity , so the acrivity_main.xml is given below.

<?xml version="1.0" encoding="utf-8"?>
<ScrollView  xmlns:android="http://schemas.android.com/apk/res/android"
    xmlns:app="http://schemas.android.com/apk/res-auto"
    xmlns:tools="http://schemas.android.com/tools"
    android:background = "@color/white"
    android:layout_width="match_parent"
    android:layout_height="match_parent"
    tools:context=".MainActivity">
<LinearLayout
        android:orientation="vertical"
        android:weightSum="10"
        android:layout_width="match_parent"
        android:layout_height="match_parent"
        android:layout_gravity="center_horizontal|center_vertical"
        android:background="@color/white"
        >
<TextView
            android:layout_gravity="center_horizontal|center_vertical"
            android:id="@+id/textView2"
            android:layout_width="212dp"
            android:layout_height="81dp"
            android:text="Aim : X to X+1"
            android:textSize="30dp"
            android:textColor="@color/black"
            android:elevation="2dp"
            />
        <androidx.cardview.widget.CardView
            android:layout_width="match_parent"
            android:layout_height="wrap_content"
            app:cardElevation="10dp"
            app:cardCornerRadius="20dp"
            android:layout_margin="10dp"
            app:cardMaxElevation="12dp"
            app:cardPreventCornerOverlap="true"
            app:cardUseCompatPadding="true"
            >
            <LinearLayout
                android:layout_gravity="center_horizontal|center_vertical"
                android:layout_margin="0dp"
                android:orientation="vertical"
                android:layout_width="match_parent"
                android:layout_height="match_parent"
                android:background="@color/black"
                >
<EditText
        android:layout_gravity="center_horizontal|center_vertical"
        android:id="@+id/editTextNumberDecimal"
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
        android:ems="10"
        android:inputType="numberDecimal"
        android:hint="Input Data"
        android:textColor="@color/white"
        />
<Button
        android:layout_gravity="center_horizontal|center_vertical"
        android:id="@+id/button"
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
        android:text="Predict"
         />
<TextView
        android:layout_gravity="center_horizontal|center_vertical"
        android:id="@+id/textView"
        android:layout_width="212dp"
        android:layout_height="81dp"
        android:hint="Prediction Area"
        android:textColor="@color/white"
        android:elevation="2dp"
        />
            </LinearLayout>
        </androidx.cardview.widget.CardView>
    </LinearLayout>
</ScrollView>

Output Screen for user:

Fig: App running on Android Phone (Samsung Galaxy)

Things used in xml:

• ScrollView : A view group that is used to make vertically scrollable view.
• LinearLayout : A layout that arranges other views either horizontally in a single column or vertically in a single row. Here , we have mentioned oreintation as vertical (android:orientation=”vertical”),so elements like TextView ,Buttons, EditText are arranged vertically .
• TextView : TextView is an UI element to displays text .
• EditText : To make UI element user-editable text, we use EditText. (android:hint=”Input Data”) The text Input Data remains till the output comes.
• android:layout_gravity: It specifies how an object should position its content on both X and Y axis. Possible values :top, bottom, left, right, center, center_vertical, center_horizontal etc.
• Setting height and width : match_parent is used when we want the height or width of a view to be as big as its parent view. wrap_content is used when we want the view to occupy only as much space as required by it.

Java Backend managing Input and Output

Java Code Main Activity

• We declare all the variables needed to use xml element and put a “setOnClickListener” to start a event whenever the button is pressed.
• It is good practice to code an event in try…. catch… block so that the app does not crashes whenever some error occurs , and Toast can be used to notify the user about the issue/bug , if any. Basically copy the code shown in android studio when we click on the tflite file.

• We can see that we need a bytebuffer array to store one input data. So ,first we link the source of data in xml file to java variable,

EditText inputEditText;
inputEditText = findViewById(R.id.editTextNumberDecimal);

• Now, we use .getText() to get text data from “inputEditText” variable . We convert this data to string using .toString()
• We notice that we need a bytebuffer array to store and the data type is FLOAT32 ,so we convert the string data to float and store it as “data”.

Float data= Float.parseFloat(inputEditText.getText().toString());

• We create a bytebuffer array to store input data of datatype FLOAT32, and allocate the space as 1*4. 4 is used for FLOAT32 .

ByteBuffer byteBuffer= ByteBuffer.allocateDirect(1*4);
byteBuffer.order(ByteOrder.nativeOrder()); // order in little endian format
byteBuffer.putFloat(data);
byteBuffer.rewind();

• When we create a bytebuffer array , it is important that the byte order of the buffer is in LITTLE_ENDIAN format because by default Tflite used this format . To ensure the correct format we use this line:

byteBuffer.order(ByteOrder.nativeOrder())

If this line is not written , java will use BIG_ENDIAN format to store.

• On getting the output from the model we display the result in TextView by using setText(<string>). We’ll convert the output to string before passing it to setText()

TextView tv= findViewById(R.id.textView);
float[] data1=outputFeature0.getFloatArray();
tv.setText(String.valueOf(data1[0]));

• Release the model after use.

model.close();

Checking gradle

Make sure to have these dependencies to use cardview and tensorflow. Open build.gradle(Module:<app_name>).

dependencies {

    implementation 'androidx.appcompat:appcompat:1.4.0'
    implementation 'com.google.android.material:material:1.4.0'
    implementation 'androidx.constraintlayout:constraintlayout:2.1.2'
    implementation 'org.tensorflow:tensorflow-lite-support:0.1.0'
    implementation 'org.tensorflow:tensorflow-lite-metadata:0.1.0'
    implementation 'org.tensorflow:tensorflow-lite-gpu:2.3.0'
    testImplementation 'junit:junit:4.+'
    androidTestImplementation 'androidx.test.ext:junit:1.1.3'
    androidTestImplementation 'androidx.test.espresso:espresso-core:3.4.0'
}

Possible Bug : TfLite Model is giving different output on Android app and in Python . For most inputs , tflite model gives same output on android.

Solved by using :

byteBuffer.order(ByteOrder.nativeOrder())

Ensure that bytes are returned in LITTLE_ENDIAN. By default, the order of a ByteBuffer object is BIG_ENDIAN. Finally, the order method in is invoked to modify the byte order. The ByteOrder.nativeOrder() method returns the LITTLE_ENDIAN byte order. The order method creates a new buffer modifiedBuffer, and sets the byte order to LITTLE_ENDIAN. TfLite saved models only support Little Endian format by default.

So, as promised in the last article we have exemplified ‘Build in Python and deploy in Java/kotlin’.

What’s next?

In next article, end to end deploy Iris classifier using Kotlin. The model will be built Python will be deployed using android app.