Modern RecSys

The future of visual RecSys is an exciting one. Let us explore some of the most cutting edge techniques and ideas that we can incorporate into our recommenders.

Style2Vec (2017) — Combining Multiple Convolutional Neural Networks

Lee, H., Seol, J., & Lee, S. (2017). Style2Vec: Representation Learning for Fashion Items from Style Sets. Open Access From https://arxiv.org/abs/1708.04014

The authors “propose Style2Vec, a vector representation model for fashion items. Based on the intuition of distributional semantics used in word embeddings, Style2Vec learns the representation of a fashion item using other items in matching outfits as context. Two different convolutional neural networks are trained to maximize the probability of…

Outfits selected by our RecSys. Left: “Anything with Shorts”. Right top: “Urban Lifestyle”. Bottom Right: “Flowery Dreams”. Outfits from DeepFashion, open-source by Liu Z. et al.

Modern RecSys

We will make use of transfer learning, approximate nearest neighbors, and embeddings centroid detection in PyTorch to build our recommender.

I have worked in the data industry for over seven years and had the privilege of designing, building, and deploying two recommender systems (RecSys) that went on to serve millions of customers. …

Modern RecSys

We will cluster COVID-19 X-ray images based on severity with our CNN RecSys flow using transfer learning, Spotify’s Annoy, and PyTorch

This work is meant as a proof-of-concept of how we can apply the same framework we set up in the previous CNN chapter onto a completely different domain.

We will swap out the training data and employ a more powerful pre-trained model (Resnet152); the rest of the code remains identical…

Target image on left, recommendations generated by our model on the right. Outfits from DeepFashion, open-source by Liu Z. et al.

Modern RecSys

We will build a recommender with transfer learning, Spotify’s Annoy, PyTorch, and return visually similar products across 240K images in 2ms

This is part of my Modern Visual RecSys series; feel free to check out the rest of the series at the end of the article.

The Data

We will be using a subset of DeepFashion data open-sourced by Liu Z. et al., The Chinese University of Hong Kong. Our data consists of…

Modern RecSys

In this chapter, we will explore the “hello world” data for visual models, the FashionMNIST dataset from Zalando with PyTorch, Tensorboard and Colab.

This is part of my Modern Visual RecSys series; feel free to check out the rest of the series at the end of the article.

FashionMNIST and the visual challenge

Source: FashionMNIST by by Kashif Rasul & Han Xiao

The data consists of:

  • Training set of 60,000 images and a test set of 10,000 images.
  • Each image is 28x28 grayscale, across 10 classes: T-shirt/top…

Modern RecSys

For this chapter, I will introduce the RecSys Design Framework with a case study of Amazon.

This is part of my Modern Visual RecSys series; feel free to check out the rest of the series at the end of the article.

RecSys Framework — Amazon case study

Recommendations on my Amazon homepage

An eCommerce website like Amazon is heavily reliant on having a good RecSys. After all, users cannot be expected to browse through millions of products…

Modern RecSys

In this series of articles, I will introduce modern approaches to visual recommender systems. We begin with a case study of Spotify.

I have worked in the data industry for over seven years and had the privilege of designing, building, and deploying two recommender systems (RecSys) that went on to serve millions of customers. For each chapter, I will walk through case studies and share my experience designing RecSys.

This is part…

Kai Xin Thia

Snr Data Scientist at Refinitiv Labs, M.S. CS Georgia Tech. 9+ years in data, found ❤️ in RecSys, NLP, Computer Vision, Applied R&D. linkedin.com/in/thiakx

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store