Huawei ML kit — Integration of Scene Detection
Huawei scene detection service can classify the scenario content of images and add labels
Introduction
To help understand the image content, the scene detection service can classify the scenario content of images and add labels, such as outdoor scenery, indoor places, and buildings. You can create more customized experiences for users based on the data detected from image. Currently Huawei supports detection of 102 scenarios. For details about the scenarios, refer to List of Scenario Identification Categories.
This service can be used to identify image sets by scenario and create intelligent album sets. You can also select various camera parameters based on the scene in your app, to help users to take better-looking photos.
Prerequisite
- The scene detection service supports integration with Android 6.0 and later versions.
- The scene detection needs READ_EXTERNAL_STORAGE and CAMERA in AndroidManifest.xml.
- Implementation of dynamic permission for camera and storage is not covered in this article. Please make sure to integrate dynamic permission feature.
Development
1. Register as a developer account in AppGallery Connect.
2. Create an application and enable ML kit from AppGallery connect.
3. Refer to Service Enabling. Integrate AppGallery connect SDK. Refer to AppGallery Connect Service Getting Started.
4. Add Huawei Scene detection dependencies in app-level build.gradle.
5. Sync the gradle.
We have an Activity (MainActivity.java) which has floating buttons to select static scene detection and live scene detection.
- Static scene detection is used to detect scene in static images. When we select a photo, the scene detection service returns the results.
- Camera stream (Live) scene detection can process camera streams, convert video frames into an MLFrame object, and detect scenarios using the static image detection method.
Implementation of Static scene detection
We can set the settings MLSceneDetectionAnalyzerSetting() and set the confidence level for scene detection. setConfidence() methods needs to get float value. After settings are fixed, we can create the analyzer with settings value. Then, we can set the frame with bitmap. Lastly, we have created a task for list of MLSceneDetection object. We have listener functions for success and failure. The service returns list of results. The results have two parameter which are result and confidence. We can set the response to textView tvResult.
Implementation of Camera Stream detection
We can process camera streams, convert video frames into an MLFrame object, and detect scenarios using the static image detection method. In this article, we will implement synchronous detection API which uses LensEngine class built in the SDK to detect scenarios in camera streams.
To create the instance of scene detection analyzer.
Create the SceneDetectionAnalyzerTransactor class to process detection results.
To Set the detection result processor.
To start the camera and read camera streams for detection.
Add surfaceview in layout xml file.
After the detection is complete, stop the analyzer to release detection resources.
Tips And Tricks
If you get error code 5. Its related to image size.
- Check the resolution of the image to be recognized. (The minimum resolution is 224 x 224 and the maximum resolution is 4096 x 4960.)
- Check whether the image format is RGBA8888 and whether the video stream format is NV21.
Conclusion
In this Article, we learnt how scene detection service can classify the scenario content of images and add labels, such as outdoor scenery, indoor places, and buildings, helps to understand the image content. Based on the detected information, you can create more personalized app experience for users.
