Flutter Text & Barcode Scanner App with Firebase ML Kit
ML Kit is a collection of powerful machine learning API released to the public by Google at IO 18 under the Firebase brand. It comes with a set of ready to use APIs such as text recognition, barcode scanning, face detection, image labelling, landmark recognition that can be used simply by developers without knowledge of Machine Learning.
In this article, we will use text recognition and barcode scanning to build a Flutter app where user can pick image from camera or gallery and then uses MLKit to detect the text or barcode from the image. The app will then display the results of the image detection and provide the result as a text or metadata displayed to the user.
What we will build:
- Main Widget is the screen where user can select the type of scanner to use and image picker from camera or gallery.
- Detail Widget is the screen where the image detection result is performed and displayed to user.
Third Party Dependencies Integration
Firebase does not have official SDK to use ML Kit for Flutter. Luckily we can use the mlkit library by azihsoyn that provide platform channel implementation for iOS and Android ML Kit SDK. To use it we need to create Firebase Project for Android and iOS with the same applicationId and bundle identifier as our app, then import the json and plist into the Flutter Android and iOS project directory.
To get image from the Camera we will use the official image_picker library. This plugin provides support to take image from camera or select image from gallery for both iOS and Android.
Main Screen Widget
The main screen widget tree diagram:
Row containing RadioListTitle is used so user can select whether to use text recognition and barcode scanner, the value is stored in _selectedScanner instance variable.
Row containing FloatingButton provides the interaction for user to select whether to pick image by taking picture using the platform specified camera app or by selecting image from user photo library. After user picked the image, we store the gile and use the Navigation class to push the DetailScreen widget passing the file as a constructor.
Image Detection with ML Kit and Display Result in Detail Widget
The detail screen widget tree diagram:
DetailScreen widget accepts 2 parameters, File of the image and String based on scanner type constant so the widget knows whether to use TextDetector or BarcodeDetector to detect the image from the filepath. Singleton of FirebaseVisionTextDetector and FirebaseVisonBarcodeDetector from mlkit library is stored as instance variables. Both the result array for text and barcode is also stored as properties use to build the widget and the ListView.
When Detail widget is initialized, we call the FirebaseVisionDetector to detect the image from the filepath using async await, then we store the result of the detection respectively inside the instance variable based on the type of scanner to use (Text Recognition or Barcode Scanner).
We use FutureBuilder widget that builds itself based on latest snapshot from a Future object. In this case we will invoke the _getImageSize method passing the file of the image that will return the size of the image as Future. In the builder block, we check if the snapshot passed has the data of the image size, then we render Image widget and add ForegroundDecorator widget that will draw red rectangle of detected text or barcode inside the Image widget detected using BoxPainter.
To display the result of the detected text or barcode, ListView.Builder is used to build the item dynamically. The data will be populated based on the result from the array that is updated when the FirebaseVisionDetector detect from the filepath and stored in property. Each item will output the VisionResult text.
We have build Flutter Application that is able to use the Firebase ML Kit into our project to detect text and barcode in image. Flutter provides flexibility for developer to call native Android and iOS library and SDK using the Platform Specific Channel API and use it in Flutter App. While Firebase ML Kit is still in beta, the result of the image detection already look very good and promising. While in this project we use on local device image detection, Firebase also provides on cloud image detection to detect images. Stay tuned and happy Fluttering!.