Smart Product Categorization System For Shopping Applications by Huawei Machine Learning Kit.
Hello everyone.
In this article, we will learn how to use the Huawei Machine Learning Kit Custom Model feature in an e-commerce application. The purpose of using this feature will be to categorize products with machine learning in e-commerce applications. We will make a demo project about it and test the results. Especially in e-commerce applications, it is very important to classify the products into the right categories. The reason for this is that e-commerce applications analyze the categories of interest to the user and want to make more sales by showing us the products in similar categories on different screens of the application.
With the Huawei ML Kit Custom Model, you can easily use Convolutional Neural Network (CNN), one of the important algorithms of Deep Learning. With the CNN algorithm, pictures can be classified with a high success rate. For example, when you train pictures of cats and dogs with CNN, if you have used enough data, they will be able to distinguish between a new picture is a cat or a dog.
With the HMS Custom model, we will train the model without using code. Then, we will develop a project using the Kotlin language. I will not use any third party library and architecture other than Picasso in the project. By writing the codes at the simplest level, I will try to ensure that you can easily use them in accordance with the architecture you want.
MindSpore Lite is a framework built for custom models provided by the HMS ML Kit to simplify integration and development. With MindSpore Lite, we can create and train our own model.
HMS ML Kit provides Transfer Learning functionality. With deep learning and model training, you can quickly train and create new models in your application. Currently only image classification is supported.
After making these definitions, let’s start the steps we will do step by step.
1- Register on the Huawei Developers website
2- Download the HMS ToolKit plug-in
This IDE plug-in provides all the tools you need to develop your own HMS Core-integrated app (creation, coding, debugging, and testing) and release it in HUAWEI AppGallery.
I opened Android Studio settings and clicked the plugins tab. Then I installed the plug-in that came out when I typed HMS Toolkit in the marketplace. After that, I have to restart the android studio.
3- If not installed Python 3.7.5 before, download and install Python on your computer. (Currently only 3.7.5 version is supported)
Official download link.
If you get this output when you run the command I wrote below. You have installed Python correctly.
4- Before proceeding with the actual processes we need to do, we prepare the picture set that will train machine learning. I will train machine learning with pictures from the coat, t-shirt, jean and shoes categories.
For a better understanding of how my dataset looks like, you can see the screenshot below. You can also use this link if you want to download my dataset.
After these steps, we will be ready to use the AI Create feature of HMS ToolKit. With AI Create, we will be able to classify images without using any code.
In order to classify the pictures:
Images must be of high quality and divided into respective classes. Pictures’ path and class names contain only letters, numbers, or the underscore (_) character. Spaces and other symbols are not allowed.
The lower limit of the classification number of the training data set is 2 and the upper limit is 1000.
Each class in the training data set should contain at least 10 pictures.
Supported image formats: .bmp, .jpg, .jpeg, .png or.gif.
You can download the picture sets I used here.
Let’s start building the model now.
5- We can do this step when any project is open in Android Studio. In the Android studio, we select the Codding Assistant from the HMS tab as in the picture below.
It can make the authentication process on the screen that appears. After this process, we must select the AI Create option as in the picture.
Then we continue the steps by selecting image classification and clicking confirm on the next screen. If a popup appears with a warning message about php after clicking confirm, it may be because you cannot install PHP 3.7.5.
After doing these steps and clicking confirm, we will see a screen like the one below.
In this step, we have to select our dataset folder by clicking “Please select train image folder”.
You can use the Advanced section if you want. In this section you can check the number and speed of the learning process. If you choose too much, the learning process will take longer if your data set is large. You may also encounter an overfitting situation.
After selecting the Training Set, you can click for Create Model. When you press this button, the learning process starts.
While processing is done, a screen like the one below appears. This process may take a while depending on the size of the dataset.
After the training process is finished, the MindSpore file will be created in the path we chose in the previous step. In the example, we see that accuracy is 91.67%. We predict that our model will work successfully. You can increase this rate much more depending on the quality and clarity of the image used.
6- In the “Please select test image folder” section, I selected the test data that was ready before. My model, 26 of the 28 images knew which category it belongs to.
After this stage, you can continue the process by using our MindSpore file in a project you have created. If you want, you can create a demo application with our MindSpore file by clicking the “Generate Demo” button at the bottom of the screen. I will create the project for both possibilities. So you can continue with the steps you want.
Here, only an important point is that if you want to create your project and test your model with the “Generate Demo”, you must put your test data under the assets folder instead of the “cat” and “dog” images.
7- After this stage, we will continue by creating a new project. First, let’s start by making the necessary additions to our gradle and manifest files. Our manifest and gradle file will be as follows. We only need to add internet permission to Manifest. We have added the libraries we will use and the necessary code block to Gradle since we will use the file in .ms format.
8- We add the MindSpore and labels.txt to the assets file that we created earlier. The assets files & labels.txt should look like the following.
9- Let’s start coding now. First, let’s start by generating our necessary xml files.
We used only one recycler view in the activity_main.xml file.
I created the RecyclerView’s items as follows.
10- I created the product model that we will use in the application.
11- First of all, I created a object class called LabelUtils. In this class, let’s write a function where we read the label where the categories we use in our model are written. Then let’s send a request to our model and write another function that we will get the result.
With the parameter we sent to the readLabels function, we created a piece of code where we read the label content.
In the processResult function, we send the probability and list to which category it belongs. As a result, we return the most likely category. If you want to return the whole list and possibilities, you need to activate the part with the comment line.
We will see where to call these functions while writing the next classes.
12- I created an abstract class called ModelOperator. This includes functions such as input output functions. We will use them when requesting our model and getting and managing results.
Then we created a class called ImageLabelModel. In this class, we manage the input type, inputs and output result. The important point is that modelName and modelLabelFile must have the same names as in your assets file.
13- The class we created with the name of InterpreterManager is very important. We created the class where we made all the requests and settings made to ML.
In this class, we first get all the information of our model with the class I wrote earlier. Then we create the MLCustomLocalModel and MLModelExecutorSettings object. Then we set the inputs. After this process, we finally send a request to our MLCustomModel, set result to hashmap and return hashmap to our adapter -we will write next step- with a callback.
14- We created a recyclerview adapter. After each product image was uploaded with Picasso, we convert image to bitmap and send this bitmap to our model for analysis. After getting the result with hashmap, we set it to the product_category(TextView).
15- Finally, let’s code the MainActivity. We manually set 4 products to the recyclerview adapter. As you may have noticed, the category field is “” for now. Because this information we will get by ML after the picture loaded.
Now our application is ready at the end. Now, what kind of an application awaits us with a few screenshots. Let’s check it out.
As you can see in the video, the category sections do not appear before sending a request to ML. Then, when the result is returned from the request, we set the Textview. You can see the category results of 4 different products on the screen.
As a result, we have successfully used the Huawei ML Kit Custom Model to automatically categorize the products for e-commerce application. We have focused on this use case only in this article. However, the Huawei ML Kit Custom Model feature is a successful feature that can be used for many different use cases. It can use this feature with a very high success percentage.
You can find the source codes of the project here.
Thank you very much for reading my article!
