ML Kit Android: Implementing Text Recognition — Firebase.

Machine Learning has started to reshape how we live, everything around us nowadays has a touch of ML in it, so has mobile applications. Google has re-introduced Firebase with the support of some ready to use machine learning models like — Text recognition, Face detection, Image labeling, and more and support to implement custom models using TensorFlow Lite for iOS, Android and Web apps.

In this article, we’ll see how to implement Firebase Text Recognition ML model in android applications. Let’s get started.

Firebase provides us with two Text Recognition APIs- on-device and Cloud. On-device is free-to-use and less accurate on the other hand Cloud API is more accurate and isn’t free to use. Detailed comparing can be found here,

In this article, we will see how to implement on-device API only. There are few prerequisites:

A device running:

• Android 4.1 (API level 14, Jelly Bean) or later
• Google Play services 15.0.0 or later

First, we create an Android project with an empty activity. We then need to add our Android project to Firebase, which can be done in two ways — Manually or by using Firebase Assistant (requires Android Studio version 2.2 or later.)

Adding Firebase manually:

Step 1: Go to Firebase console, click on ‘Add project.’

Step 2: Give a project name, project id(optional), accept the terms and conditions and click on ‘Create project’.

Step 3: Once the project is created, click on ‘Continue’.

Step 4: Click on the settings icon (beside Project Overview) -> Project settings and select ‘Add Firebase to your Android app’.

Step 5: Add the package name of your app and click on ‘Register app’ button.

Step 6: Download google-services.json file and move it to the Android app module root directory. Once done, click on ‘Next’.

Step 7: Now we need to add a few plugins and dependencies. We modify our build.gradle files as follows -

Project-level build.gradle ( <project>/build.gradle):

buildscript {
  dependencies {
    // Add this line
    classpath 'com.google.gms:google-services:4.0.1'
  }
}

App-level build.gradle (<project>/<app-module>/build.gradle):

dependencies {
  // Add this line
  implementation 'com.google.firebase:firebase-core:16.0.1'
}
...
// Add to the bottom of the file
apply plugin: 'com.google.gms.google-services'

and sync the project, then click on ‘Next’.

Step 8: Now we can verify if our app is connected to Firebase by simply running it.

Adding Firebase using Firebase Assistant (requires Android Studio version 2.2 or later) :

• Click Tools > Firebase to open the Assistant window.
• Click to expand one of the listed features (for example, Analytics), then click the provided tutorial link (for example, Log an Analytics event).
• Click the Connect to Firebase button to connect to Firebase and add the necessary code to your app.

Firebase is now set up, we can now start building our Text Recognition app.

We need Firebase ML Vision dependency, we add it in our app-level build.grade file:

dependencies {
  // ...

  implementation 'com.google.firebase:firebase-ml-vision:18.0.2'
}

we also add a declaration in our AndroidManifest.xml file, that automatically downloads the ML model once the app is installed. It’s optional but recommended.

<application ...>
  ...
  <meta-data
      android:name="com.google.firebase.ml.vision.DEPENDENCIES"
      android:value="ocr" />
  <!-- To use multiple models: android:value="ocr,model2,model3" -->
</application>

Now, we’ll add an ImageView, a TextView and two Button/ImageButtons to our activity_main.xml file :

We’ll bind the views in MainActivity.java file

In MainActivity.java, we’ll create two methods- openCamera() and detectImg().

After capturing the image from the camera, we’ll set the image into the ImageView as:

Don’t forget to add this in your AndroidManifest.xml

<uses-feature android:name="android.hardware.camera"
    android:required="true" />

That’s it! Our app is ready to use. Run the app and click on the camera icon to launch the camera on your Android Device. Click a picture of some text, then click on tick icon and watch Firebase do the magic for you.

Reference

Firebase Documentation