Discovery of Firebase ML Kit image labeling

Maxime Dupierreux
Aug 10, 2018 · 3 min read

Since it’s announcement at IO back in May, I wanted to try the new Firebase ML Kit tools.

I had the opportunity to try it during my summer holidays.

This is my first try with the label detection provided by ML Kit. One of the coolest thing is that you can use it both online and offline.

The app I made basically takes an image URL and labels it online or offline. Here’s how I created the app.


Create your project in Firebase & activate Google Cloud Vision API.

Go to the Firebase console create a new project and follow the steps to add it to Android.

Image for post
Image for post
Firebase project dashboard

Change the pricing to blaze, this is needed in order to use Cloud Recognition (you have 1000 uses free per month).

Image for post
Image for post
Firebase pricings

Activate Cloud Vision API for your project by following this link.

Image for post
Image for post
Cloud Vision API

Add ML Kit vision API to your project

In the build.gradle of the app in Android Studio, add the following dependencies.

implementation ‘com.google.firebase:firebase-core:16.0.1’ implementation ‘com.google.firebase:firebase-ml-vision:16.0.0’ implementation ‘com.google.firebase:firebase-ml-vision-image-label-model:15.0.0’

Create the layout

Next, we create a layout with :

  • An TextInputLayout in which we’ll copy an URL
  • An ImageView, used to show the image
  • A Button to load the image at the given URL
  • A Button for local labeling
  • A Button for cloud labeling
Image for post
Image for post
The layout

Add the method for local labeling

fun runImageLabelingOnDevice(bitmap : Bitmap){        
val image = FirebaseVisionImage.fromBitmap(bitmap)
val labelDetector = FirebaseVision.getInstance().visionLabelDetector
labelDetector.detectInImage(image)
.addOnSuccessListener {
processImageLabelingFromDevice(it)
}.addOnFailureListener{
Log.d("LABELFAILURE",it.toString())
}
}

First, we create a FirebaseVisionImage object with the bitmap we want to label.

Then, we create a VisionLabelDetector and we use its detectInImage() method to find labels in the image


Add the method for cloud labeling

fun runImageLabelingCloud(bitmap : Bitmap){        
val image = FirebaseVisionImage.fromBitmap(bitmap)
val options = FirebaseVisionCloudDetectorOptions.Builder() .setModelType(FirebaseVisionCloudDetectorOptions.LATEST_MODEL) .setMaxResults(15)
.build()
val labelDetector = FirebaseVision.getInstance().getVisionCloudLabelDetector(options)
labelDetector.detectInImage(image).addOnSuccessListener{
processImageLabelingFromCloud(it)
}
.addOnFailureListener{
Log.d("LABELFAILURE",it.toString())
}
}

The cloud method is nearly identical except for these two things :

  • We need a VisionCloudLabelDetector instead of a VisionLabelDetector
  • We need to pass that detector some options, how much results we want to display and what kind of model we want to use (in this case : FirebaseVisionCloudDetectorOptions.LATEST_MODEL)

Test the app !

The app running !


That’s it for my first try with ML Kit (and first story on Medium). I hope you guys liked this article. Comments and suggestions are welcome.

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store