Android: App that scans barcodes and recognizes text!
In this post I’ll be talking about how we can leverage the amazing Firebase ML kit to scan barcodes of products. We will be making a simple android app to achieve our goal.
So let’s get started
Step 1
Before we can start using the ML kit we need to connect our android app to a firebase project. So to do that, head over to your firebase console and create a brand new project, you can name it whatever you want. After creating the project, You will be taken to your project screen. From the project overview click ‘ Add Firebase to your Android app’ and enter your app’s package name.
After entering the package name and clicking continue, the browser will automatically download the ‘google-services.json’ file. This file contains all the necessary firebase metadata for your app. Copy-paste the file inside the ‘app’ directory of your project.
Step 2
Now that you have successfully connected your app to firebase we can start by adding the required dependencies. Add the following dependencies in your app-level build.gradle file.
dependencies {
// ...
implementation 'com.google.firebase:firebase-ml-vision:16.0.0'
implementation 'com.google.firebase:firebase-core:16.0.0'}
apply plugin: 'com.google.gms.google-services'
Sync your project to reflect the changes.
Step 3
In this final step I will explain how you can use the ML Kit to extract meaningful information from images. Copy-paste this XML code into your activity_main.xml file.
Now on line number 19 you will notice that we have implemented a custom view. We will be adding the GraphicOverlay class in the next step but for now create an empty class named ‘GraphicOverlay’ and copy-paste its absolute path in the XML file instead of mine.
Create the following classes — GraphicOverlay, TextGraphic, and CloudTextGraphic. Copy-paste the following blocks of code into these classes respectively.
GraphicOverlay.java
TextGraphic.java
CloudTextGraphic.java
These three classes are responsible for drawing ‘boxes’ over the text in images to show that the texts have been identified. We will not explore the code in these classes in-depth. However you are encouraged to do so yourself. You could use these classes or simply extract the raw value from Firebase’s model for your own use.
Now finally, copy this code into your MainActivity file.
MainActivity.java
We supply images to our app in two ways — from the Gallery and Camera.
In both ways the image is scaled down and converted into a bitmap before being displayed on the screen. I will not discuss much about it but feel free to explore the code for your understanding.
In this file we will mainly explore the following methods:
- runTextrecognition()
- processTextRecognitionResult()
runTextRecognition()
In this method, We first create a FirebaseVisionImage object passing in the required bitmap as the paramter.
Next we create a FirebaseVisionTextDetector object. Then we call ‘detectInImage()’ method on the detector object passing in the FirebaseVisionImage object as the parameter. We tack on a listener to this method and we are provided with two callback methods. If all goes well we receive our texts in the form of ‘FirebaseVisionText’ as a parameter in the ‘onSuccess()’ callback method.
Finally, we pass these results into the ‘processTextRecognitionResult’ to extract meaningful texts from the result.
processTextRecognitionResult()
We create a List of FirebaseVisionText blocks and assign it to the value returned by calling ‘getBlocks()’ on the FirebaseVisionText object.
We then loop through the individual blocks, line, and finally the individual elements which are essentially the words. Voila! you can see red boxes appearing around the text in images.
Note: you can use the raw getRawValue() method to simply extract the text.
The complete source code for the app is available on my github.
Thanks so much for reading this, I hope you enjoyed reading as much as I did writing it. Feel free to reach out in the comments if you have any doubts. As always, happy coding!
