Integration of Huawei Machine Learning Kit into Ionic Application ( Cross Platform Integration )

Introduction

Machine learning is an application of artificial intelligence that provides systems the ability to automatically learn and improve from experience without being explicitly programmed. Machine learning focuses on the development of computer programs that can access data and use it learn.

HMS ML Kit allow us to support diverse artificial intelligence applications throughout a wide range of industries.

HMS ML Kit Services

1. Text-related Services
2. Language/Voice-related Services
3. Image-related Services
4. Face/Body-related Services
5. Custom Model

In this article we will focus on text related services provided by HMS ML Kit.

Ionic Framework

Ionic Framework is an open source UI toolkit for building performant, high-quality mobile and desktop apps using web technologies such as HTML, CSS, and JavaScript with integrations for popular frameworks like Angular and React.

Think of Ionic as the front-end UI framework that handles all of the look and feel and UI interactions your app needs in order to be compelling. Unlike a responsive framework, Ionic comes with very native-styled mobile UI elements and layouts that you should get with a native SDK on Android or iOS but didn’t really exist before on the web.

Since Ionic is an HTML5 framework, it needs a native wrapper like Cordova or Capacitor in order to run as a native app.

Here we will use Ionic framework with Angular and Capacitor as native wrapper.

Text-related Services

There are four type of services which comes under text related service:

1. Text Recognition
2. Document Recognition
3. Bank Card Recognition
4. General Card Recognition

Here we will use three services that is Text Recognition, Bank Card Recognition and General Card Recognition to showcase the power HMS ML Kit in Ionic application.

Demo

Prerequisite

1. Must have a Huawei Developer Account.
2. Must have a Huawei phone with HMS 4.0.0.300 or later.
3. Must have a laptop or desktop with Android Studio, Jdk 1.8, SDK platform 26 and Gradle 4.6 installed.
4. Must install node in the system
5. Must install Ionic in the system using below command:

npm install -g @ionic/cli

Things need to be done

1. Generating a Signing Certificate Fingerprint. For generating the SHA key, refer this article.
2. Create an app in the Huawei AppGallery connect and enable ML Kit in Manage Api section.
3. Provide the SHA Key in App Information Section.
4. Provide storage location.
5. Download the agconnect-services.json and store it somewhere on our computer.
6. Create a blank Ionic Project using below command:

ionic start Your_Application_Name blank --type=angular

7. Download the Cordova Ml Kit Plugin. Run the following command in the root directory of your Ionic project to install it through npm.

npm install <CORDOVA_MLKIT_PLUGIN_PATH>

8. If you want full Ionic support with code completion etc., install @ionic-native/core in your project.

npm install @ionic-native/core --save-dev

9. Run the following command to copy the “ionic/dist/hms-ml” folder from library to “node_modules/@ionic-native” folder under your Ionic project.

cp node_modules/@hmscore/cordova-plugin-hms-ml/ionic/dist/hms-ml node_modules/@ionic-native/ -r

10. Run the following command to compile the project:

ionic build

npx cap init [appName] [appId]

Where appName is the name of your app, and appId is package_name in your agconnect-services.json file (ex: com.example.app).

11. Run the following command to add android platform to your project:

ionic capacitor add android

12. Make sure your project has a build.gradle file with a maven repository address and agconnect service dependencies as shown below:

13. Add the Signing certificate configuration to the build.gradle file in the app directory as show below:

14. Add plugin to the build.gradle file in the app directory as show below:

15. Add ads service implementation into to dependencies section of build.gradle file in the app directory as show below:

16. Add agconnect-services.json and signing certificate jks file to the app directory in your Android project as show below:

17. To update dependencies, and copy any web assets to your project, run the following code:

npx capacitor sync

Let’s Code

Install FileChooser

We need to install FileChooser to select a file or in this case an image, returns a file URI. Run the following command on visual studio terminal:

npm install cordova-plugin-filechooser
npm install @ionic-native/file-chooser
ionic cap sync

Android Permission

We need android permission in order to allow our user to give their permission to the application like camera permission, storage permission etc. Run the following command on visual studio terminal:

npm install cordova-plugin-android-permissions
npm install @ionic-native/android-permissions
ionic cap sync

Avoid No Provider Error

To avoid No Provider Error, we need to add FileChooser, Android Permission and HMS ML kit in app.module.ts providers section and also make sure to import the same as shown below:

Import

Import the following code in your home.page.ts :

Check Permission

To check permission at the beginning of the application, we need to use AndroidPermission as shown below:

Select Image from gallery

We need to use FileChooser to select image from gallery as shown below:

Text Recognition

Basically it helps us to extract text from any given image. The image could be a receipt, business card or a document. It will be a very useful tool for industries such as printing, education or logistics. This service can run on both cloud and device. Here we will look into Local Text Analyser which is service running on device, runs a detection model on the device and returns the detection result.

In the above code ocrType decides whether the service is working on device or on the cloud. If ocrType is OCR_LOCAL_TYPE it means service is running on device and if it is OCR_REMOTE_TYPE it means service is running on cloud. The filePath is where we provide the path of our image after selecting the image from gallery using FileChooser.

Bank Card Recognition

Basically it helps us to extract information such as card number and validity period from bank cards in camera streams within an angle offset of 15 degrees. It provides two ways to extract information from bank card:

1. Bank Card Recognition with the SDK
2. Bank Card Recognition with the plugin

Here we will use Bank Card Recognition with the plugin to show case the use of this service.

Here the orientation can be set to three modes:

1. ORIENTATION_AUTO
2. ORIENTATION_LANDSCAPE
3. ORIENTATION_PORTRAIT

General Card Recognition

As the name suggest, it can analyse and extract any information from any given card.

Run application

Now we can run the application using the following command on visual studio terminal:

ionic capacitor run android

Conclusion

We learn how to integrate HMS ML Kit in Ionic Project. Feel free to comment, share and like the article. Also you can follow me to get awesome article like this every week.

For more reference

1. https://developer.huawei.com/consumer/en/doc/development/HMS-Plugin-Guides/introduction-0000001050765773
2. https://developer.huawei.com/consumer/en/doc/development/HMS-Plugin-Guides/integrating-cordova-plugin-0000001050765783