Giving the Best Product Recommendation with Category-based Similar Product Engine
What is an Similar Product Engine(SPE) and why do we need it
Customers like to compare for a better purchases. Maximisers want to pick the best from a set of selections, while satisfiers settle with good enough products [1]. They all have a fundamental needs: a list of products for them to compare and that is the reason why we will need a similar product engine (SPE). The following is an example of an SPE output for a ring:
Once a customer has found an interested item, such as a ring, the system will show similar rings. Without SPE, the customer has to think of some keywords, or look for the items in the same categories, or even browse the social media to find enough products for comparison. It is time consuming, and painful, especially for maximisers. Seeing more products not only wastes a lot of time, but also it gives the customers a hard time to choice from.
However, SPE is not a recommender. Although one of the goal of SPE is to increase the purchases, it aims to help customers to compare and pick the right products. Comparing to recommender that the customers may not looking for any items, the customers know what they need before using SPE. For example, a recommender would suggest a computer before you know that you look for a computer, while an SPE will help you to compare different computers when you look for one. Then the SPE will help you to compare the CPU, ram and the GPU on the computers. On the other hand, if you are comparing a ring, you would like to compare the design, the size of diamond, and the materials. Given such a big difference between different items, it is desirable to build a framework for SPE.
How to develop a SPE (and feature engineering), and why by category
Similar to recommendation, there is no magic behind SPE. It is conducted based on different information from the products and the similarities among them. The assumption is that customers like to compare products with similar category, such as, rings with a similar design and materials. The similarity in category can be captured by some similarity measurement. One of the most popular similarity calculation is Jaccard similarity. Given the sets of categorical information of 2 items, it is the interception of 2 sets divided by the union of 2 sets. For example, for these 3 fashion products:
Product A: {blue, skirts, wing collar}
Product B: {black, skirts, bib collar}
Product C: {orange, skirts, wing collar}
The similarity of products A and B is:
{skirts} / {blue, black, skirts, wing collar, bib collar} = 1/5
The similarity of products A and C is:
{skirts, wing collar} / {blue, orange, skirts, wing collar} = 1/2
Hence, products A is more similar to C than B. Jaccard similarity offers an easy method to compute the similarity, but there are some drawbacks too. The quality of the category will affect the quality of SPE a lot.
Another question arise. It is unclear that how to select the right features. Some features are very important in SPE for some products (e.g., the color of a ring) while it is less for others (e.g., the color of a computer). It is important to select the right features for different categories and objects have different definitions of similarity. Besides the similarity, there are other considerations also. There could be personalization, to rank some products higher according to user preference. As well, the providers may not want to show products that are much cheaper, as it may reduce their profit.
In the next section, we will discuss how we can do a better job with machine learning.
Relationship between Machine learning and SPE
It is not a surprise that machine learning also play a critical role in SPE. There are several techniques that are particularly useful for it. The first one is Auto-encoder [2]: a type of artificial neural network than can learn a representation in an unsupervised manner, that is, without the needs of labelled data.
In categorical similarity calculation, the category data is first encoded into one-hot vector, and then they are used as the input of the auto-encoder to learn a representation of each data point. The following is an example of encoding a number image. The representation have the advantage of dimensionality reduction and noise reduction:
In this example, the category is first encoded into 1-hot vector with 1-hot encoding. Each 1 represents that a ring belongs to that category. A representation is learned using the 1-hot vector, with the goal is to re-create the 1 hot vector using the representation, given the input as the the 1-hot vector.
In this way, the encoder learns the similarity among the data points, and hence the representation can make a better similarity calculation. One of the advantages is that even 2 products do not share any category in Jaccard similarity, it is still possible to compute the similarity of these 2 products.
If there are labels on the products, we can employ a similar technique, Product encoder, or siamese network [3]. It tries to learn a representation of each product in a way that if 2 products are similar, they will have a higher similarity and vice versa. Hence, recommendation is to find products that have similar representation.
Reference
[1] Schwartz, Barry, and Barry Schwartz. “The paradox of choice: Why more is less.” New York: Ecco, 2004.
[2] Tschannen, Michael, Olivier Bachem, and Mario Lucic. “Recent advances in autoencoder-based representation learning.” arXiv preprint arXiv:1812.05069 (2018).
[3] Lee, Hanbit, Jinseok Seol, and Sang-goo Lee. “Style2vec: Representation learning for fashion items from style sets.” arXiv preprint arXiv:1708.04014 (2017).
[4] Deng, Zhi-Hong, et al. “Deepcf: A unified framework of representation learning and matching function learning in recommender system.” Proceedings of the AAAI conference on artificial intelligence. Vol. 33. №01. 2019.
