Text recognition with Firebase MLKit
The official getting-started guide to text recognition with MLKit is pretty straightforward. At least, when you’re processing local images:
I found it a little harder to take photos with your camera and process these, so this is what I’m going to describe.
The official developer guide to take photos looks simple at first, start an intent and get the photo in
onActivityResult(), but if you just quickly fly over it like me you may miss that the returned
Bitmap is just a thumbnail and completely inappropriate to be processed with the MLKit.
What you rather want is caching the taken photo in full resolution in your internal storage. For this we need to work with FileProvider.
FileProvider to your app
First, add the
FileProvider, which is actually a
ContentProvider, to your Manifest in the
Then, create the
provider_paths.xml and define that we require URIs for our internal storage (
Afterwards we’re able to create an
Intent and point to a URI where want so save that photo internally:
Retrieve the taken photo
Now, we have to read the photo when
onActivityResult() is called from the camera. Sadly, we cannot simply use
BitmapFactory.decodeFile(outputFile), as it results in an error. Rather, we have to get the
Bitmap like this:
val bitmap = MediaStore.Images.Media.getBitmap(contentResolver, imageUri)
Bitmap we could now call
FirebaseVisionImage.fromBitmap(bitmap) again. But there’s more!
Process the taken photo
If you just pass the
Bitmap to the MLKit now it will hardly detect anything. This is because photos taken by the camera are normally rotated, so you have to tell MLKit what the rotation is.
However, MLKit only accepts a rotation parameter if you pass it a
val image = FirebaseVisionImage.fromMediaImage(mediaImage, rotation)
I didn’t figure out how to actually get this type from the camera, so I couldn’t use their suggested way to detect the rotation.
I ended up with just rotating the
Bitmap by 90°, because I knew this is the rotation:
And this is the final
Bitmap you can pass to the MLKit for text recognition!