ML Text Recognition with Flutter
In 2023, we witnessed the rise of Generative AI, with the proliferation of tools like ChatGPT, Midjourney, and many others made available to the general public.
From here on, we can only imagine how many more tools will emerge to assist us in our daily lives and work. That’s why it’s worth understanding these technologies as soon as possible to harness them to the fullest!
Today, we will integrate a pre-trained model to recognize text from an image within a Flutter application.
The model is made available with Google ML Kit and works entirely offline 🚀
💻 The complete project is available on GitHub here.
Flutter Project Setup
⚠️ Make sure you have Flutter installed on your system and that your development environment is correctly configured.
You can verify the Flutter installation by running the command:
flutter doctorIf the command returns warnings or errors, follow the instructions provided to resolve them.
Start a new Flutter project by running the command:
flutter create project_namein your command line, replacing project_name with the desired name for your project.
In this guide, I named the project ml_text_recognition, so in the code blocks you will find file imports following this pattern: import 'package:ml_text_recognition/...' .
Remember to rename the import if your project_name is different!
Package Integration
Within our project, we will use google_ml_kit_text_recognition for text recognition and image_picker to select images from the gallery or take pictures with the camera.
Let’s implement the packages within the pubspec.yaml file:
...
dependencies:
flutter:
sdk: flutter
cupertino_icons: ^1.0.2
google_mlkit_text_recognition: ^0.11.0
image_picker: ^1.0.7
...Package versions may change over time, so make sure to check for any updates.
Also, make sure you have followed all procedures for integrating image_picker for both Android and iOS as described in its guide (see here).
Let’s Build the App!
Now that we have integrated the necessary packages, let’s create our application!
We’ll start by creating the ImagePreview widget to help us display the image or a starting message.
Let’s create the image_preview.dart file:
import 'dart:io';
import 'package:flutter/material.dart';
class ImagePreview extends StatelessWidget {
const ImagePreview({
Key? key,
required this.imagePath,
}) : super(key: key);
final String? imagePath;
@override
Widget build(BuildContext context) {
return Container(
width: double.infinity,
height: 300,
decoration: BoxDecoration(
color: Colors.white,
borderRadius: BorderRadius.circular(10),
boxShadow: const [
BoxShadow(
color: Color.fromARGB(255, 203, 203, 203),
blurRadius: 10,
spreadRadius: 2,
offset: Offset(0, 3),
),
],
),
child: imagePath == null
? const Center(
child: Text(
"No image selected",
style: TextStyle(
fontWeight: FontWeight.bold,
),
),
)
: Image.file(
File(imagePath!),
fit: BoxFit.contain,
),
);
}
}This widget will help improve readability within the HomeScreen later on.
Let’s now create an initial version of the HomeScreen where we will encapsulate the logic for text recognition from an image.
Create the home_screen.dart file and define our graphical interface:
import 'package:flutter/material.dart';
import 'package:ml_text_recognition/image_preview.dart';
class HomeScreen extends StatefulWidget {
const HomeScreen({Key? key}) : super(key: key);
@override
State<HomeScreen> createState() => _HomeScreenState();
}
class _HomeScreenState extends State<HomeScreen> {
String? pickedImagePath;
String recognizedText = "";
bool isRecognizing = false;
@override
Widget build(BuildContext context) {
return Scaffold(
appBar: AppBar(
title: const Text('ML Text Recognition'),
),
body: SafeArea(
child: Column(
children: <Widget>[
Padding(
padding: const EdgeInsets.all(16.0),
child: ImagePreview(imagePath: pickedImagePath),
),
ElevatedButton(
onPressed: isRecognizing ? null : () {},
child: Row(
mainAxisSize: MainAxisSize.min,
children: [
const Text('Pick an image'),
if (isRecognizing) ...[
const SizedBox(width: 20),
const SizedBox(
width: 16,
height: 16,
child: CircularProgressIndicator(
strokeWidth: 1.5,
),
),
],
],
),
),
const Divider(),
Padding(
padding: const EdgeInsets.only(
left: 16,
right: 16,
bottom: 16,
),
child: Row(
mainAxisAlignment: MainAxisAlignment.spaceBetween,
children: [
const Text(
"Recognized Text",
style: TextStyle(
fontWeight: FontWeight.bold,
),
),
IconButton(
icon: const Icon(
Icons.copy,
size: 16,
),
onPressed: () {},
),
],
),
),
if (!isRecognizing) ...[
Expanded(
child: Scrollbar(
child: SingleChildScrollView(
padding: const EdgeInsets.all(16),
child: Row(
children: [
Flexible(
child: SelectableText(
recognizedText.isEmpty
? "No text recognized"
: recognizedText,
),
),
],
),
),
),
),
],
],
),
),
);
}
}Let’s now add the necessary variables and methods for text recognition within _HomeScreenState:
class _HomeScreenState extends State<HomeScreen> {
late TextRecognizer textRecognizer;
late ImagePicker imagePicker;
String? pickedImagePath;
String recognizedText = "";
bool isRecognizing = false;
@override
void initState() {
super.initState();
textRecognizer = TextRecognizer();
imagePicker = ImagePicker();
}
void _pickImageAndProcess() async {
final pickedImage = await imagePicker.getImage(source: ImageSource.gallery);
if (pickedImage == null) {
return;
}
setState(() {
pickedImagePath = pickedImage.path;
isRecognizing = true;
});
final inputImage = InputImage.fromFilePath(pickedImage.path);
final RecognizedText recognizedText = await textRecognizer.processImage(inputImage);
setState(() {
recognizedText = recognizedText.text;
isRecognizing = false;
});
}
@override
Widget build(BuildContext context) {
// Build method remains the same
}
}With the _pickImageAndProcess method, we prompt the user to choose how they want to acquire an image (from the gallery or by taking a picture with the camera).
We then call the _pickImageAndProcess method passing the chosen ImageSource. This is where the actual interpretation of the image occurs, extracting the text with the ML model.
Lastly, we add a method _copyTextToClipboard for the user (but not mandatory) to quickly copy the text recognized by the Machine Learning model.
The complete home_screen.dart file will look like this:
import 'package:flutter/material.dart';
import 'package:flutter/services.dart';
import 'package:google_mlkit_text_recognition/google_mlkit_text_recognition.dart';
import 'package:image_picker/image_picker.dart';
import 'package:ml_text_recognition/image_preview.dart';
class HomeScreen extends StatefulWidget {
const HomeScreen({super.key});
@override
State<HomeScreen> createState() => _HomeScreenState();
}
class _HomeScreenState extends State<HomeScreen> {
late TextRecognizer textRecognizer;
late ImagePicker imagePicker;
String? pickedImagePath;
String recognizedText = "";
bool isRecognizing = false;
@override
void initState() {
super.initState();
textRecognizer = TextRecognizer(script: TextRecognitionScript.latin);
imagePicker = ImagePicker();
}
void _pickImageAndProcess({required ImageSource source}) async {
final pickedImage = await imagePicker.pickImage(source: source);
if (pickedImage == null) {
return;
}
setState(() {
pickedImagePath = pickedImage.path;
isRecognizing = true;
});
try {
final inputImage = InputImage.fromFilePath(pickedImage.path);
final RecognizedText recognisedText =
await textRecognizer.processImage(inputImage);
recognizedText = "";
for (TextBlock block in recognisedText.blocks) {
for (TextLine line in block.lines) {
recognizedText += "${line.text}\n";
}
}
} catch (e) {
if (!mounted) {
return;
}
ScaffoldMessenger.of(context).showSnackBar(
SnackBar(
content: Text('Error recognizing text: $e'),
),
);
} finally {
setState(() {
isRecognizing = false;
});
}
}
void _chooseImageSourceModal() {
showModalBottomSheet(
context: context,
builder: (context) {
return Padding(
padding: const EdgeInsets.symmetric(vertical: 20, horizontal: 16),
child: Column(
mainAxisSize: MainAxisSize.min,
children: <Widget>[
ListTile(
leading: const Icon(Icons.photo_library),
title: const Text('Choose from gallery'),
onTap: () {
Navigator.pop(context);
_pickImageAndProcess(source: ImageSource.gallery);
},
),
ListTile(
leading: const Icon(Icons.camera_alt),
title: const Text('Take a picture'),
onTap: () {
Navigator.pop(context);
_pickImageAndProcess(source: ImageSource.camera);
},
),
],
),
);
},
);
}
void _copyTextToClipboard() async {
if (recognizedText.isNotEmpty) {
await Clipboard.setData(ClipboardData(text: recognizedText));
if (!mounted) {
return;
}
ScaffoldMessenger.of(context).showSnackBar(
const SnackBar(
content: Text('Text copied to clipboard'),
),
);
}
}
@override
Widget build(BuildContext context) {
return Scaffold(
appBar: AppBar(
title: const Text('ML Text Recognition'),
),
body: SafeArea(
child: Column(
children: <Widget>[
Padding(
padding: const EdgeInsets.all(16.0),
child: ImagePreview(imagePath: pickedImagePath),
),
ElevatedButton(
onPressed: isRecognizing ? null : _chooseImageSourceModal,
child: Row(
mainAxisSize: MainAxisSize.min,
children: [
const Text('Pick an image'),
if (isRecognizing) ...[
const SizedBox(width: 20),
const SizedBox(
width: 16,
height: 16,
child: CircularProgressIndicator(
strokeWidth: 1.5,
),
),
],
],
),
),
const Divider(),
Padding(
padding: const EdgeInsets.only(
left: 16,
right: 16,
bottom: 16,
),
child: Row(
mainAxisAlignment: MainAxisAlignment.spaceBetween,
children: [
const Text(
"Recognized Text",
style: TextStyle(
fontWeight: FontWeight.bold,
),
),
IconButton(
icon: const Icon(
Icons.copy,
size: 16,
),
onPressed: _copyTextToClipboard,
),
],
),
),
if (!isRecognizing) ...[
Expanded(
child: Scrollbar(
child: SingleChildScrollView(
padding: const EdgeInsets.all(16),
child: Row(
children: [
Flexible(
child: SelectableText(
recognizedText.isEmpty
? "No text recognized"
: recognizedText,
),
),
],
),
),
),
),
],
],
),
),
);
}
}Let’s modify the main.dart file to call our HomeScreen:
import 'package:flutter/material.dart';
import 'package:ml_text_recognition/home_screen.dart';
void main() {
runApp(const MyApp());
}
class MyApp extends StatelessWidget {
const MyApp({super.key});
@override
Widget build(BuildContext context) {
return MaterialApp(
title: 'ML Text Recognition',
debugShowCheckedModeBanner: false,
theme: ThemeData(
colorScheme: ColorScheme.fromSeed(seedColor: Colors.deepPurple),
useMaterial3: true,
),
home: const HomeScreen(),
);
}
}Conclusion
In this guide, we explored how to integrate text recognition within a Flutter app using the machine learning models provided by Google ML Kit.
Thanks to the combination of these technologies, we can develop intelligent and innovative apps quickly and effectively.
Google ML Kit and Flutter prove to be powerful partners for cross-platform application development, enabling developers to create advanced solutions with ease and flexibility.
