Identifying Places in a provided Image using Firebase MLKit

This story is fourth in the part of the series, MLKit for Mobile Developers.
If you’re not quite caught up, you can start here:

Creating a Barcode Scanner using Firebase MLKit

Series Pit Stops

• Creating a Google Lens clone using Firebase ML Kit
• Creating a Credit Card Scanner using Firebase ML Kit
• Creating a Barcode Scanner using Firebase ML Kit
• Identifying Places in a provided Image using Firebase ML Kit (You are here)
• Building a “Pokédex” clone using Firebase ML Kit and TensorFlow Lite
• Implementing Smart Replies in an app using Firebase ML Kit
• Recreating Google Translate using Firebase’s ML Kit and Cloud Translate API
• Building a real-time object detection app using Firebase ML Kit
• Creating an offline translation app using Firebase ML Kit
• Blink detection in Android using Firebase ML Kit

Following up from the earlier articles on Image Labelling, Text Recognition and the Bar Code Scanner APIs, in this post I’ll be covering the fourth API, i.e. the Landscape Detection API and how you can use this to add powerful machine learning features in your app!

The Landscape DetectorAPI, allows us to tag and identify some of the common locations in the images provided by the user.
For example, using this API you can sort the photos provided by the user by the location where they were taken.

Neat, isn’t it?

Before we get started, here are some screenshots from the app which showcase the end result :

Yes, I travel a lot! 😐

The API can detect multiple landmarks in a given image, along with the accuracy, latitude, longitude and the name of that location.

Contrary to the API we covered earlier, there is no on device variant of the API available, and you need to upgrade to the Blaze plan of Firebase in order to use it.

But worry not, you can feel free to grab an apk of this sample project if you want to try this API out and don’t want to pay for it ;)

The code for the sample app above can be found over here, feel free to fork it and follow along :

the-dagger/MLKitAndroid

As mentioned earlier, this API only has the Cloud variant which runs on Google Cloud.
When you pass an image to this API, you get the landmarks that were recognized in it, along with each landmark’s geographic coordinates and the region of the image the landmark was found.
It’s free for first 1000 requests which is good enough if you just want to play around.

Without any delay, let’s get started with the implementation!

• Setup Firebase in your project and add the vision dependency
  This is a simple one, simply setup firebase in your project. You can find a good tutorial here.
  In order to use this API, you also need to add the following dependencies.

• Add image picker functionality in your app.
  The Vision API needs an image to extract the data from, so you can feel free to go ahead and implement the image picker functionality in the app.
  I found this library to be pretty handy and easy to use instead of implementing image picker manually, so you can feel free to take a look at it.
  The library returns a File which makes it easier to convert the file to Bitmap and pass it on to the Firebase APIs.
• Convert the picked file to a Bitmap
  If you used the library above, it provides you with a File object for the captured image which you easily convert to a Bitmap using BitmapFactory.decodeFile() method.

As you can see in the code above, I checked and requested runtime permission for accessing external storage and only when I had the permission, I added an onClickListener to my floatingActionButton in line 30.

The Image Picker is started in line 16 and the picked image is converted to a Bitmap in line number 43.

• Use the Bitmap to make an API call
  Making the API call is super simple, just pass the Bitmap to a FirebaseVisionCloudLandmarkDetector object and you’re all set!

In the above code snippet, we first create a FirebaseVisionImage from the bitmap.

Followed by that, we create an instance of FirebaseVisionCloudLandmarkDetector which is used to recognize landmarks in a supplied Image.

Lastly we pass the Image to the detectInImage() method and the API do its magic! ✨

We have a success and a failure callback which contains a list of FirebaseVisionCloudLandmark objects containing all the detected Landmarks and their information.

Further, we can loop over the above list and get information contained in each and every Landmark that was detected.
For each landmark, you can also get its bounding coordinates in the input image in case you want to highlight the Landmark separately.

And that’ll be all for this one, I’ll be off to exploring the remaining MLKit APIs so expect articles for them soon enough!

If you want to play around with the app shown in the screenshots, you can build it from the GitHub repository I linked above and it should work well after adding it to your Firebase Project.

P.S. You also need to setup the Billing for your Firebase Project by upgrading it to the Blaze plan and enabling Google Vision API in your Google Cloud Console for the Landmark functionality to work properly.

Thanks for reading! If you enjoyed this story, please click the 👏 button and share to help others find it! Feel free to leave a comment 💬 below.
Have feedback? Let’s connect on Twitter.