Firebase Image Labelling: Extract Objects and Label it using Firebase Cloud API ML model.

Courtesy: Pixabay

In this blog, we are going to discuss cloud-based image labelling in Android using Firebase ML kit. I previously explained how the objects can be extracted from image and categorize using on-device ML kit model. Image labelling can be considered as an extended version of object detection. The only difference is that it happens not on the device but rather it depends on the cloud API of Firebase to label objects in the image.

Just to highlight, for using Firebase cloud API you have to convert your Firebase project to blaze plan. It gives you around 1000 free hits per month and if you want to implement it on a production app, check out the pricing plans which suits your usage.

To do with image labelling using ML kit follow the below steps

1. Get the cloud blaze plan for the project.
2. Take the picture and convert to firebase vision image as shown in the below blog.
3. Create a cloudImageLabeller detector and Pass image to the detector.
4. Retrieve the details of the image in the success method.

Get the cloud blaze plan for the project.

Plans available in Firebase

Once the project is converted to blaze plan

Take the picture and convert to Firebase vision image as shown in the below blog.

Firebase Object Detection and Labelling: Detect Objects and Categorize from Images Or videos.

To get the camera image, start camera using intent use the bitmap that’s returned to the onActivityResult. The first part of the above link explains how the camera image can be generated using default intent.

Create a cloudImageLabeller detector and Pass image to the detector.

Create a firebase vision image from the generated camera bitmap and that firebase vision should be passed to the cloud image detector. The detector should be of the type cloudImageLabeler when you initialise the detector. ML kit detector won’t take default bitmap as input, so creating a firebase vision image is inevitable.

Retrieve the details of the image in the success method.

Once the cloud API detects all the labels in the image, the list of labels will be returned to onSuccessListener.

The Cloud image labeller stands very different from on-device object detection. Using cloud API has its own advantages. Apart from categorizing and knowing just the category of the object; Name, Confidence and Entity id can also be retrieved using cloud API. Activating blaze account in your project is also mandatory otherwise, it will end up getting an error.

This is how you can integrate image labelling into your app and to conclude, check out this link to explore and understand more about labels available in the cloud API.

You can find the whole project here in Github. This is a public repo which has many demo projects. You can use GitZip to download a specific project from this repo. Paste this link in GitZip and you have it.

Follow me on Twitter and medium. Feel free to contribute to Repo — AndroidAppSamples. Let’s grow together 💚