Text Recognition in React Native
Making images speak: A guide to text recognition in React Native
Imagine converting handwritten recipes or printed documents into digital text using your smartphone. Well, In the world of React Native, this is not just a dream; it’s a reality. Wait! What?
Our journey began with experimenting with various third-party React Native text detection libraries & several machine learning tools, including ML Kit, TensorFlow & Tesseract OCR.
After persistent exploration, we discovered that implementing this functionality directly in React Native wasn’t straightforward. Also, we did search for JavaScript libraries but did not find anything promising.
Ultimately, we decided to go with native modules for both iOS and Android and successfully crafted an effective and efficient way to implement text detection from images in React Native using ML Kit.
To know, how we can make pictures talk — take these steps with us:
- Implement image capture & image picker functionality
- Implement TextDetectionModule for Android
- Implement TextDetectionModule for iOS
- Access Modules in React Native
1. Implement image capture & image picker functionality
We’ve used the react-native-image-picker library for image-related tasks, and you may choose an alternative library for camera functionality based on your specific needs.
We will not go into detail for this part, as the setup is straightforward, and you can refer to the document for the same.
2. Implement TextDetectionModule for Android
a. In your project-level build.gradle file, add Google's Maven repository.
b. Add the dependencies for the ML Kit Android libraries to your module’s app-level gradle file, which is usually located in the app/build.gradle.
c. Create a native module in Android and declare a method that will extract text from images.
To begin, supply an image URL to your native method, which can be called from JavaScript code.
In this example, we’re using the file URI method, but you could also opt for media.Image, ByteArray, or Bitmap.
Here, we’re passing both the application context and the file URI as arguments to the inputImage.fromFilePath() method.
Next, initialize the TextRecognizer object from ML Kit.
Pass the image to the process method:
Finally, ML Kit provides you with the extracted text. When text recognition succeeds, it creates a structured response containing recognized text blocks.
To summarize everything in Android, here is the full method recognizeImage that we can utilize.
3. Implement TextDetectionModule for iOS
a. Add the specified ML Kit dependencies to your Podfile.
pod 'GoogleMLKit/TextRecognition', '3.2.0'b. Similar to the Android approach, provide an image URL as input to your native method, which can be invoked from JavaScript. Then, create an MLKVisionImage object using an UIImage or CMSampleBuffer, ensuring that you set the .orientation.
Following that, initialize a TextRecognizer instance with the options related to the SDK, which we previously declared as a dependency.
Then, process the image by passing it to the process(_:completion:) method:
Ultimately, extract text from blocks of recognized text. If the text recognition operation succeeds, it returns Text as an object.
With ML Kit, similarly to Android, I’ve implemented the recognizeImage native method for text detection in images for iOS.
4. Access Modules in React Native
Now that we've finished implementing native methods, we can access and utilize them in our React Native application with NativeModules.
Tada! Here’s the output:
Conclusion
In this blog, we uncovered how React Native and ML Kit can unlock the hidden potential of text recognition from images, exploring how apps are made with simple yet strong tools.
For the complete source code, check out the GitHub repository.
For more updates on the latest tools and technologies, follow the Simform Engineering blog.
