Implementing Nanonets OCR in Flutter
In this article we are going to see how to implement Nanonets OCR in a Flutter application using flutter_nanonets_ocr package by creating a beginner friendly demo flutter project.
Before diving into the implementation we first need to understand briefly what is OCR and how it works behind the scenes.
What is OCR and how does it work?
OCR stands for Optical Character Recognition, It is a technology that allows computers to recognise and extract text from images or scanned documents.
The OCR process typically involves following steps:
- Image preprocessing: The input image is cleaned and enhanced to improve the quality of the text for better recognition.
- Text detection: OCR algorithms identify areas of the image that contain text, separating them from other non-text elements.
- Character segmentation: If the text is handwritten or contains multiple lines, individual characters are separated from each other.
- Character recognition: The separated characters are then recognized and converted into machine-readable text using pattern recognition and machine learning techniques.
- Post-processing: Additional steps may be taken to improve the accuracy of the recognised text, such as language modelling and error correction.
Now we got a brief idea about OCR in general, let’s begin with the actual implementation in Flutter.
Before the coding part there are few pre-requisites which we need to set up in order to use the OCR functionality in our mobile applications.
- Go to https://nanonets.com/ and register yourself.
2. Login using any of the following methods:
3. After successful login, create an API Key by clicking here , this key will come in handy when we will be using Nanonets APIs in our flutter application.
4. On the Home page of Nanonets, we will see a “New Model ”option in left navigation panel, after clicking that there will be many options for creating different kinds of OCR models, Nanonets by provides many pre-trained OCR models for various documents.
But in this example we are going to create our very own OCR model from scratch, so click on “Create Your Own”.
Note: Only three models can be created using a free account, and each model can be used for 100 calls/month only, if the usage exceeds this quota you will get error in creating a new model or fetching data from existing models.
4. Upload at least 10 unique images for training this model, make sure images are of high quality in order to increase accuracy of our model and click “Next”.
Remember the more training data we feed, the better the accuracy of the model will be.
5. Now enter the fields you need to fetch from the image documents, for this example I am taking two fields, “Owner Name” and “Total Amount”. Click Start Training.
6. Now this is the most crucial step, as we are going to train our OCR model by labelling the fields that we have selected as follows. Also remove any unnecessary data that nanonets is fetching in order to avoid noise.
As we mentioned above remove unnecessary data as follows.
7. Follow the same with all the other training images and click Train Model.
8. Once the model is trained and is ready to extract you will receive an email on the registered email
9. Test the model by passing a document to verify if the model is giving expected results or not.
The model we just trained is giving accurate data so now we are all set to integrate it into our flutter application.
The Nanonets set up is completed, now let’s create a new Flutter application by using following command:
flutter create nanonets_ocr_appOnce the Flutter project is created successfully, add “flutter_nanonets_ocr” package in your pubspec.yaml file.
Also add image_picker dependency to fetch the document from mobile device in case we want to predict information from a local image document. Also make sure to add platform specific changes as instructed in the documentation of image_picker.
Now that we are all done with project setup. Let’s get started with the coding part.
Add a file named HomePage in lib/views/screens/home_page.dart as follows and make it home route in main.dart:
// ignore_for_file: prefer_const_constructors
import 'dart:developer';
import 'dart:io';
import 'package:flutter/material.dart';
import 'package:flutter_nanonets_app/views/screens/prediction_page.dart';
import 'package:flutter_nanonets_app/views/widgets/update_document_url_textfield_widget.dart';
import 'package:image_picker/image_picker.dart';
import '../../constants/color_constant.dart';
import '../../constants/string_contants.dart';
class HomePage extends StatefulWidget {
const HomePage({super.key});
@override
State<HomePage> createState() => _HomePageState();
}
class _HomePageState extends State<HomePage> {
@override
void initState() {
super.initState();
}
File? pickedImage;
Future<File?> pickImage(ImageSource imageType) async {
try {
final photo =
await ImagePicker().pickImage(source: imageType, imageQuality: 100);
if (photo == null) return null;
File? tempImage = File(photo.path);
setState(() {
pickedImage = tempImage;
});
} catch (error) {
log(error.toString());
}
return pickedImage;
}
@override
Widget build(BuildContext context) {
return Scaffold(
body: SafeArea(
child: Container(
alignment: Alignment.center,
child: Column(
mainAxisAlignment: MainAxisAlignment.center,
crossAxisAlignment: CrossAxisAlignment.center,
children: [
Text(
homeOCRPredictor,
style: TextStyle(
color: kBlack, fontWeight: FontWeight.bold, fontSize: 20),
),
pickedImage == null
? Container()
: Container(
height: MediaQuery.of(context).size.height * 0.5,
padding: EdgeInsets.symmetric(
horizontal: 5,
),
decoration: BoxDecoration(
shape: BoxShape.rectangle,
),
child: ClipRRect(
borderRadius: BorderRadius.circular(8),
child: Image.file(
pickedImage!,
fit: BoxFit.contain,
),
),
),
SizedBox(
height: 40,
),
InkWell(
onTap: () async {
try {
await pickImage(ImageSource.gallery);
if (pickedImage != null) {
Navigator.of(context).push(
MaterialPageRoute(
builder: (context) => PredictionScreen(
image: pickedImage,
isUrl: false,
),
),
);
}
} catch (err) {
log(err.toString());
}
},
child: Container(
height: 70,
width: 300,
alignment: Alignment.center,
decoration: BoxDecoration(
color: kWhite,
border: Border.all(color: kBlack, width: 2),
borderRadius: BorderRadius.all(Radius.circular(30))),
child: Text(
pickedImage == null
? uploadDocumentImage
: editDocumentImage,
textAlign: TextAlign.center,
style: TextStyle(
color: kBlack,
fontSize: 20,
fontWeight: FontWeight.bold),
),
),
),
SizedBox(
height: 20,
),
/// Predict Document URL
UpdateDocumentUrlTextField()
],
),
),
),
);
}
}This screen will fetch document image or document url and navigate to a next screen where we will be using FutureBuilder to make the nanonets api call using “flutter_nanonets_ocr” package. Let’s create the PredictionScreen in lib/views/screens/ now.
Now will be needing the Nanonets API Key, that we created while the Nanonets setup initially, if you have not created the Nanonets API key yet, no worries, just click here and create a key.
Once the API key is created, copy the the id of ocr model that you want to use from nanonets console here.
Once you have the key and model id in hand, just replace “NANONETS API KEY HERE” from API key and “OCR MODEL ID HERE” from ocr model id in the following code snippet.
import 'dart:io';
import 'package:flutter/material.dart';
import 'package:flutter_nanonets_ocr/flutter_nanonets_ocr.dart';
import 'package:flutter_nanonets_ocr/models/ocr_predictor_response_model.dart';
import '../../constants/color_constant.dart';
import '../../constants/string_contants.dart';
class PredictionScreen extends StatefulWidget {
final File? image;
final String? documentUrl;
bool isUrl;
PredictionScreen(
{this.image, this.documentUrl, required this.isUrl, super.key});
@override
State<PredictionScreen> createState() => _PredictionScreenState();
}
class _PredictionScreenState extends State<PredictionScreen> {
String apiKey = "NANONETS API KEY HERE";
@override
Widget build(BuildContext context) {
NanonetsOCR nanonetsOCR = NanonetsOCR();
return Scaffold(
appBar: AppBar(
title: Text(homeOCRPredictor),
),
body: FutureBuilder(
future: widget.isUrl
? nanonetsOCR.predictDocumentURL(
apiKey, widget.documentUrl, "OCR MODEL ID HERE", context)
: nanonetsOCR.predictDocumentFile(
apiKey, widget.image, "OCR MODEL ID HERE", context),
builder: (context, snapshot) {
if (snapshot.connectionState == ConnectionState.active) {
return loadingWidget();
} else if (snapshot.connectionState == ConnectionState.done) {
return SingleChildScrollView(
child: predictedDataSection(context, snapshot));
} else {
return loadingWidget();
}
}));
}
Widget loadingWidget() {
return Center(
child: Stack(
alignment: AlignmentDirectional.center,
children: [
Container(
padding: EdgeInsets.symmetric(horizontal: 5),
decoration: BoxDecoration(
shape: BoxShape.rectangle,
),
child: ClipRRect(
borderRadius: BorderRadius.circular(8),
child: widget.image == null && widget.isUrl
? Image.network(widget.documentUrl ?? "")
: Image.file(
widget.image!,
fit: BoxFit.fitHeight,
),
),
),
Container(
height: MediaQuery.of(context).size.height,
padding: EdgeInsets.symmetric(horizontal: 5),
decoration: BoxDecoration(color: Colors.purple.withOpacity(0.2)),
child: Center(child: CircularProgressIndicator()),
)
],
),
);
}
Widget predictedDataSection(
BuildContext context, AsyncSnapshot<OcrPredictorResponseModel> snapshot) {
return Container(
alignment: Alignment.center,
padding: EdgeInsets.only(top: 50),
child: Column(
mainAxisAlignment: MainAxisAlignment.center,
crossAxisAlignment: CrossAxisAlignment.center,
children: [
Text(
"Document's Details",
textAlign: TextAlign.center,
style: TextStyle(fontSize: 20, fontWeight: FontWeight.bold),
),
SizedBox(
height: 20,
),
Padding(
padding: const EdgeInsets.symmetric(horizontal: 20),
child: snapshot.data!.result.isEmpty
? Container(
alignment: Alignment.center,
child: Text(
"Can't fetch data. Please check model id, api key or uploaded image and try again.",
style: TextStyle(
color: kRed,
fontWeight: FontWeight.bold,
fontSize: 16),
textAlign: TextAlign.center,
),
)
: ListView.builder(
physics: NeverScrollableScrollPhysics(),
shrinkWrap: true,
itemCount: snapshot.data!.result[0].prediction.length,
itemBuilder: (context, index) {
return snapshot.data!.result[0].prediction[index].label ==
"table"
? Container()
: Container(
alignment: Alignment.center,
padding: EdgeInsets.all(5),
color: kGreen,
child: Text(
"${snapshot.data!.result[0].prediction[index].label.toString()} : ${snapshot.data!.result[0].prediction[index].ocrText.toString()}",
style: TextStyle(
fontSize: 18,
fontWeight: FontWeight.w500,
),
));
}),
),
SizedBox(
height: 20,
),
Container(
padding: EdgeInsets.symmetric(horizontal: 5),
child: ClipRRect(
borderRadius: BorderRadius.circular(8),
child: widget.image == null && widget.isUrl
? Image.network(widget.documentUrl ?? "")
: Image.file(
widget.image!,
fit: BoxFit.fitHeight,
),
),
),
SizedBox(
height: 20,
),
],
),
);
}
}After this, connect with a simulator or physical device and run the above project using following command in root directory of the Flutter project and it will run the project in connected Simulator or physical device:
flutter runOnce the code is successfully compiled and launched in the connected device, we are all set to fetch the data from our documents.
Conclusion
We implemented Nanonets OCR in a Flutter application from beginning to end by creating a beginner friendly demo application from scratch, we also set up a custom nanonets model and trained it.
Please find full source code here .
If this blog helped you, don’t forget to give it a clap :)
Happy Reading!!!
