How to Search any Product Price by Taking a Picture of It
Welcome to the multi-part series of making a product searcher application by image classification. We want to prove that if we have an image of a product, we can decide what it is. We can find the cheapest price. We can even use this technology in mobile devices. These series will contain training a TFLite model for a few product types, creating a web service to search the price of products on the market and a mobile client. Each part will be simple to demonstrate how to do without any knowledge. Each part will contain a basic explanation of every state. Feel free to comment on anything that isn’t clear and I’ll be happy to edit.
Expectations
Learning the basics of Tensorflow and training your model with your images. Also, we are going to use this model to test prediction accuracy. Then, we are going to create a web service that will respond to our request based on the product name. If we sent a product name, this service will return the cheapest sellers on the internet. The final step will be about creating our client to show how we combine these features on a user-friendly application.
Part I
Fast Introduction to TensorFlow
It is one of the most popular open-source deep learning tools. First of all, you should know what is a machine learning model.
A model in Machine learning means a function with learnable parameters that maps an input to a desired output. For more information check TensorFlow models page.
This tool is generally used for classification, perception, and prediction. Also, you can train your model on Google Colab under the hardware features of Google. We are going to use these features to train our image classification model.
Collection of sample inputs
In our example, we are going to train our model to decide the right shoe model on the given input image. I am going to use a chrome extension to download sample images from google search. I will use the “Image Downloader” extension to download images from google images. There are many examples. You can use any of them. The next step is just typing the name of the product and downloading images. For our tests, we will download 150 images per product. You can download already prepared image folders from my Github.
Training Model
In this article, you are going to train the image classification model for our project.
Result of this part
We are going to be able to classify the model name of any shoes by a picture of it. In the end, we are going to use the name of the shoes to search on the internet and list the similars with prices. Therefore, keep following the next parts!
PART II
Searching Market
In this part, we are going to create a Spring Boot web API to search product prices on the web with the name of the product. If you don’t know anything about web services or Spring Boot, you can just run this service on your computer with some basic modifications.
We are going to use cimri.com to search for our product prices. This site is working as a price provider for popular online sellers. In our service, we will only do get requests to this web site. Then, we will parse HTML responses.
Requirements
- Java — 1.8.x
- Maven — 3.x.x
- MongoDB — 4.x.x
Explore Rest API
There are many controllers in this project, but all of them are not completed yet. For our project, we can just check how to search for a product with this service.
GET
------ "/product" -> returns all products
------ "/query/{product_name}" -> Search a product and save it.Example with Results:
Query a product with the name of it
GET →http://localhost:8080/product/query/md-runner
List whole queried products
GET →http://localhost:8080/product/
Source Code
Currently, I had no time to create a Medium article. you can find the source code in this Github repo. Besides, It is not fully completed, but working as expected for basic requests for this project.
PART III
Creating a Mobile Client
The final part of this series is about creating a mobile client with React-Native to detect our shoe model and search the prices with our web service. For this part, you don’t need to know anything about react-native to test this app. Just download the project folder from my Github repo. Then, go into the project directory and run “npm i” to complete the installation in the windows console or a terminal. The last part is going to be running the project. Same console we are going to run “react-native run-android” to test it on an emulator. Soon, I am planning to create a medium article for this part too.
Requirements
- NodeJS
- Node package manager
npm i react-native-tensorflow-lite --save
Note
Before you run your react-native client, you should modify the rest service address in the code. We are not able to connect web service with the localhost domain name in the react-native project. Set it with your local IP.
Example: “http://xxx.xxx.xxx.xxx:8080/product/query/”
In addition, Can’t use quantized smaller model yet, due to limitation in react-native-tensorflow-lite, and the project is currently Android ONLY with aspirations for iOS support.
Final product view
This is the final view of our product. In the main menu, there are three example images that are not used in the training sessions to test the prediction.
Note: the last image on the main menu is my original shoe photo :)
Useful links
Here you will see source codes of all parts.
- Creating an Image Classification model is explained in this repo with a Jupyter Notebook.
- Searching a product price on the internet with Spring Boot is in this repository.
- Mobile Project source code is in this Github repo.
