Text recognition with Firebase MLKit

The official getting-started guide to text recognition with MLKit is pretty straightforward. At least, when you’re processing local images:

I found it a little harder to take photos with your camera and process these, so this is what I’m going to describe.

The official developer guide to take photos looks simple at first, start an intent and get the photo in onActivityResult(), but if you just quickly fly over it like me you may miss that the returned Bitmap is just a thumbnail and completely inappropriate to be processed with the MLKit.

What you rather want is caching the taken photo in full resolution in your internal storage. For this we need to work with FileProvider.

Adding the FileProvider to your app

First, add the FileProvider, which is actually a ContentProvider, to your Manifest in the <application></application> block:

Then, create the provider_paths.xml and define that we require URIs for our internal storage (Context.filesDir):

Afterwards we’re able to create anIntent and point to a URI where want so save that photo internally:

Retrieve the taken photo

Now, we have to read the photo when onActivityResult() is called from the camera. Sadly, we cannot simply use BitmapFactory.decodeFile(outputFile), as it results in an error. Rather, we have to get the Bitmap like this:

val bitmap = MediaStore.Images.Media.getBitmap(contentResolver, imageUri)

With this Bitmap we could now callFirebaseVisionImage.fromBitmap(bitmap) again. But there’s more!

Process the taken photo

If you just pass the Bitmap to the MLKit now it will hardly detect anything. This is because photos taken by the camera are normally rotated, so you have to tell MLKit what the rotation is.

However, MLKit only accepts a rotation parameter if you pass it a media.Image via

val image = FirebaseVisionImage.fromMediaImage(mediaImage, rotation)

I didn’t figure out how to actually get this type from the camera, so I couldn’t use their suggested way to detect the rotation.

I ended up with just rotating the Bitmap by 90°, because I knew this is the rotation:

And this is the final Bitmap you can pass to the MLKit for text recognition!