Modern RecSys

The future of visual RecSys is an exciting one. Let us explore some of the most cutting edge techniques and ideas that we can incorporate into our recommenders.

Style2Vec (2017) — Combining Multiple Convolutional Neural Networks

Lee, H., Seol, J., & Lee, S. (2017). Style2Vec: Representation Learning for Fashion Items from Style Sets. Open Access From https://arxiv.org/abs/1708.04014


Modern RecSys

Building a Recommender That Evolves with Seasons

The Business Problem


Outfits selected by our RecSys. Left: “Anything with Shorts”. Right top: “Urban Lifestyle”. Bottom Right: “Flowery Dreams”. Outfits from DeepFashion, open-source by Liu Z. et al.

Modern RecSys

We will make use of transfer learning, approximate nearest neighbors, and embeddings centroid detection in PyTorch to build our recommender.

The Business Problem


Modern RecSys

We will cluster COVID-19 X-ray images based on severity with our CNN RecSys flow using transfer learning, Spotify’s Annoy, and PyTorch

This work is not intended as medical research nor representative of how we can use CNN to detect COVID-19.


Target image on left, recommendations generated by our model on the right. Outfits from DeepFashion, open-source by Liu Z. et al.

Modern RecSys

We will build a recommender with transfer learning, Spotify’s Annoy, PyTorch, and return visually similar products across 240K images in 2ms

The Data

What is Convolution?


Modern RecSys

In this chapter, we will explore the “hello world” data for visual models, the FashionMNIST dataset from Zalando with PyTorch, Tensorboard and Colab.

FashionMNIST and the visual challenge

Source: FashionMNIST by by Kashif Rasul & Han Xiao

What are embeddings and why we need them?


Modern RecSys

For this chapter, I will introduce the RecSys Design Framework with a case study of Amazon.

RecSys Framework — Amazon case study

Recommendations on my Amazon homepage


Modern RecSys

In this series of articles, I will introduce modern approaches to visual recommender systems. We begin with a case study of Spotify.

RecSys Basics — Spotify Case Study


Kai Xin Thia

Snr Data Scientist at Refinitiv Labs, M.S. CS Georgia Tech. 9+ years in data, found ❤️ in RecSys, NLP, Computer Vision, Applied R&D. linkedin.com/in/thiakx

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store