Recognizing the Forms by Huawei Form Recognition Feature in ML Kit

pexels.com/@punchbrandstock

Hello everyone!

Today, I want to tell and show the benefits and the implementation of Form Recognition Service of Huawei. As It is understood from the name obviously, this service is used for recognition of forms.

Form recognition uses Ai technologies to recognize the form and return structure information and texts in this form.

We can use this service for all daily needing. Converting physical form documents to digital documents can be achieved by it. This reduces the effort who would do it manually :)

There are some conditions which is defined in the guide of the service which is shown below:

While using images for recognition which contain forms, these conditions should be met.

As first step, integration developing application to the HMS Core should be done. You can follow this guide to integrate:

Android | Integrating Your Apps With Huawei HMS Core

Additionally, ML kit should be enabled from AppGallery Connect because form recognition is a feature under ML kit. If you are already familiar with this step, we can go to the next step.

Scenario

Lets start the logic and implementation of our scenario. We have an exam which includes 4 questions. We will recognize form table which includes the answers of student. There is a button which will open the gallery. When we pick a from image which is shown below from gallery, It will be sent to form recognition analyzer. Then, we will parse the recognition result and show the points that user has from the exam.

Lets start the implementation!

Add form recognition and its model dependencies into the app gradle:

implementation 'com.huawei.hms:ml-computer-vision-formrecognition:2.1.0.300'
implementation 'com.huawei.hms:ml-computer-vision-formrecognition-model:2.1.0.300'

For interactions, we will have a layout;

activity_mail.xml

Then, we need to create form recognition analyzer in MainActivity.kt;

By the code which is below, we implemented the logic for picking image from gallery;

Now, we have the bitmap which can be used for recognition. We will have a a method (recognizeForm) to recognize it;

recognizeForm method is sending mlFrame which includes bitmap information converted to object which taken as parameter to analyzer. Now, I want to show that the returned result in here. For this, I used this table to recognize;

We are seeing in here the questions and answers of the students. The result from analyzer; (I used parser to understand data better)

Bingo! We got the data of recognized form. Of course, we need to parse it programmatically.

First property in json is retCode. This field shows if recognition is successful or not. In service, there is an enum to check the result is successful or not; MLFormRecognitionConstant (It has SUCCESS(0) and FAILED(1)).

Lets implement recognizeForm method deeply;

We implemented data classes for response. By using Gson library, we converted response JsonObject file to RecognizeResponse object. We needed to implement 5 classes;

There should be exam object which stores correct answers of questions.

In onCreate, we will define the results;

Now, we will calculate the students point with a method;

Final status after we picked image;

I am adding the final status of MainActivity.kt file;

I wanted to show the benefits and and implementation of possible usage scenerios for form recognition service. There may be a lot of usage field for it. I hope you found useful knowledge from my article.

Thanks for reading. If you have question, please feel free to contact me by writing comment!

References

Huawei ML Kit Official Guide: https://developer.huawei.com/consumer/en/doc/development/HMSCore-Guides/service-introduction-0000001050040017

Huawei Form Recognition Official Guide: https://developer.huawei.com/consumer/en/doc/development/HMSCore-Guides/form-recognition-0000001058920154