Labelling pictures in Flutter using Firebase Machine Learning

Photo by Alex Knight on Unsplash

Hi! In today’s article I would like to cover topic of image labelling using Firebase ML in Flutter.

Firebase ML is a library that uses machine learning for solving a variety of problems, including language identification, text recognition, face detection, or — main topic of this article — image labelling. With Firebase ML we can use great potential of machine learning without having expert knowledge on how neural networks work. In this article you’ll learn how to implement this API in your Flutter project.

Ready? Let’s start!

Example

In this article we’re gonna build a simple app that will allow us to choose photo from the gallery and label it with Firebase ML.

Flutter Image Labelling with Firebase ML

When working with Firebase ML, you can use either cloud or on-device API. In this article we will use on-device version. For more information about which version you should use, check out the documentation.

Installing

To setup Firebase ML in the Flutter app, we’re gonna use firebase_ml_vision library. First step is adding it to your pubspec.yaml file:

dependencies:   
  firebase_ml_vision: ^0.10.0

Note: At the time of writing this article, current version was 0.10.0. You may want to check current version on the firebase_ml_vision website.

Next step is creating Firebase project and doing platform-specific configuration (for Android and iOS). To do that, you can follow steps from this tutorial: Codelabs — Firebase for Flutter.

The last step is making setup for using on-device image labeler:

Android

To setup Android, add this piece of code to your build.gradle in android/app folder:

android {
    dependencies {
        // ...

        api 'com.google.firebase:firebase-ml-vision-image-label-model:17.0.2'
    }
}

iOS

To setup iOS, add this line to your Podfile in ios folder:

pod 'Firebase/MLVisionLabelModel'

And run cd ios && pod update

Usage

To use image labels detection, we have to create FirebaseVisionImage object from an image file. For getting image file you can use many different libraries, for example image_picker.

Note: To see the full code of Firebase ML with image_picker, visit my GitHub repository.

Once you get a file, you can use it to create FirebaseVisionImage object. I’ll do it in the separate method that will be later called by tapping on a button:

Future<void> _getImageAndDetectLabels() async {
  final pickedFile = await picker
    .getImage(source:ImageSource.gallery);

  if (pickedFile != null) {
    File image = File(pickedFile.path);

    final FirebaseVisionImage visionImage =
      FirebaseVisionImage.fromFile(image);
    ...
  }
}

With this object, we can call imageLabeler() to get a labeler and call processImage() to detect image labels:

Future<void> _getImageAndDetectLabels() async {
  ...
  final ImageLabeler labeler =
      FirebaseVision.instance.imageLabeler();
  final List<ImageLabel> labels = await labeler
      .processImage(visionImage);
  }
}

ImageLabel class contains information about labels — its name and confidence of the result:

for (ImageLabel label in labels) {
  final String name = label.text;
  final double confidence = label.confidence;
}

And we can use that to show results on the screen:

Text(
  labels
      .map((label) => '${label.text} with confidence ${label.confidence.toStringAsFixed(2)}')
      .join('\n'),
),

Ta-da! That’s all. We learned how to use Firebase ML in Flutter to detect image labels.

To see the full source code of this example visit my GitHub repository:

pszklarska/flutter_image_labelling

Thanks, that’s it for today! I hope you like this article. If you do, don’t forget to clap! ✌️

Follow Flutter Community on Twitter: https://www.twitter.com/FlutterComm