Flutter +MLKit = ❤

Stefan Blos
Feb 9, 2019 · 7 min read
Image for post
Image for post

Text Recognition

The first topic I want to introduce to you is text recognition. This generally means that you have an image with some text in it. You want to get the text out of it because you want to extract information. If you’re not a computer vision expert or like to spice up things with a little Convolutional Neural Network (CNN) for your own deep-learning pipeline (hopefully the topic of a future blog post) you might think you’re lost now, right?

Image for post
Image for post
Flow Chart of our pipeline (I think it’s obvious where most of the work is happening)

Business-Card-Mail-Extractor

This is where our example app (with that short, catchy name) will come in. We want to be able to take a photo of a business card and directly get the email out of it. Why, you might ask? Because we are lazy and why should we type a few letters if we can build a full-blown app for that!? So we simply snap a photo and then we want an output that will look like this:

Image for post
Image for post
Example for Mail extraction from an example image from: https://businesscardtemplate.info/wp-content/uploads/2018/05/3Kings-Image3.jpg
  • use MLKit to extract the text from the photo
  • search for the email in the text

1. Take a photo

We can use a wonderful little Flutter package called “camera” to get access to the camera (it is important to follow the steps described in the “Installation” part of the docs of the package). First we need to add the package in our pubspec.yaml file. Then we can integrate this into our starter app in order to use it to take a photo and save it to disk.

2. Use MLKit to extract text from the image

Another task, another package. There is one from the Flutter team themselves, that provides access to MLKit called firebase_ml_vision. Again, you can check out the link on how to configure it (especially that it runs locally).

3. Search for mail address

Lastly we need to extract the address from all the text we received from MLKit. One cool thing of these packages is that you can access the source code of them (at least for all packages I have ever used). So we can look at the code of the VisionText type here. We see that it consists of a list of TextBlock objects and these contain lists of TextLine objects. The following image shows the structure of a VisionText object:

Image for post
Image for post
How a VisionText object is structured.

We did it!

That’s it! We have done everything we wanted to do. You can check the Github Repo how to do the other stuff like visualization. But this is an example how you can use MLKit and their text recognition capabilities in your app.

  • You could go further than simply detecting mail addresses and could directly add those to your contacts or send a “Nice to meet you” message to them.
  • You could use the live-feed of the camera to consistently search for mail addresses without the need to snap a photo.
Image for post
Image for post
Photo by rawpixel on Unsplash

Flutter Community

Articles and Stories from the Flutter Community

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store