A Glimpse into Deep Learning for Recommender Systems
Dr. Ernesto Diaz-Aviles, Chief Scientist at Libre AI.
Deep Learning (DL) is one of the next big things in Recommender Systems (RecSys). During the past few years deep neural networks have shown tremendous success in computer vision, speech recognition, and natural language processing (NLP). Deep learning methods are also becoming a powerful tool to tackle Recommender Systems tasks such as music, news, fashion articles, and mobile apps recommendation.
This post takes a glimpse of current research in this particular field. The aim is to help identify new opportunities for research and industrial application of Deep Learning to enhance the recommendation experience.
Why Recommender Systems?
Recommender Sytems help us to deal with choice overload, a cognitive process in which people have a difficult time making a decision when faced with many options.
RecSys are everywhere and we use them every day, directly or indirectly, in our digital interactions: when we buy that book that Amazon recommends us based on our previous history, when we listen to that playlists tailored to our taste in Spotify, or when we watch with the family that that movie recommended in Netflix, and discover new friend connections in Facebook, or read that news articles we care about that Twitter offers, or when we apply to that ideal job hinted by LinkedIn … Recommender Systems help us finding that valuable item among an ocean of choices.
Why Deep?
Deep Learning is a field of Machine Learning that allows computational models that are composed of multiple processing layers of representation and abstraction that help to make sense of data such as images, sound, and text.
These methods have dramatically improved the state-of-the-art in computer vision, speech recognition, natural language processing (NLP) and many other domains such as drug discovery and cancer cell detection. [Deep learning. Nature 521, 436–444 (28 May 2015)]
Is deeper better? For image classification deeper models with hundreds of layers and novel architecture shave shown impressive improvements reducing the classification error more that 24 percentage points in the last few years.
What about DL for RecSys? are such improvement in recommendation performance possible?
Does Deep Learning work for Recommender Systems?
Let’s have a look how RecSys have benefit from DL. In particular, we will explore examples on how Convolutional Newural Networks (CNNs), Recurrent Neural Networks (RNNs), Deep Neural Networks (DNNs) and Auto Encoders (AE) habe been used to enhance the power of RecSys.
Convolutional Neural Networks (CNNs)
CNNs consist of multiple layers of receptive fields that model animal visual perception. They are variations of multilayer perceptrons designed to use minimal amounts of preprocessing. CNNs receptive filters process portions of the input image. Their outputs of these are then tiled so that their input regions overlap, to obtain a higher-resolution representation of the original image; this is repeated for every such layer. The initial layers, close to the input image, learn basic and low-level features, such as edges, followed by layers that seem to learn more higher order features like shapes. The very last layers seem to learn semantic attributes such as eyes or wheels, which are crucial parts in several categories. The machine learns to represent the objects without the need of hand-crafted features.
CNNs for RecSys: Feature Learning to enhance Collaborative Filtering
Items’ content features, e.g., visual features, are very valuable in multiple domains such as fashion, retail, or entertainment, and they can complement Collaborative Filtering algorithms. CNNs can be used to extract such visual features automatically. For example, He et al. introduce VBPR, an approach that re-use a pretrained CNNs to extract a visual features vector that is then embedded and combined with item factors learned by a pairwise matrix factorization using BPR.
The visual features indeed give a boost in performance measured by AUC, the metric optimized by BPR.
The visual embeddings discover semantic associations automatically and are useful for discovering similar items or to tackle cold-start problems, where new items could be paired with existing ones in similar item regions.
Deep content-based music recommendation
Pioneer work from Spotify also uses CNNs to extract audio features from music tracks. The content features could then used to cluster similar tracks and to produce personalized playlists. [Deep content-based music recommendation]
Recurrent Neural Networks (RNNs)
A recurrent neural network (RNN) is a class of artificial neural network where connections between units form a directed cycle. This creates an internal state of the network which allows it to exhibit dynamic temporal behavior. Unlike feedforward neural networks, RNNs can use their internal memory to process arbitrary sequences of inputs. This makes them applicable to tasks such as unsegmented connected handwriting recognition, speech recognition, or neural conversational models.
RNN for RecSys: Session-based Recommendation
RNN for RecSys: Multi-task Recommendation
DNNs and Auto Encoders
DNN for RecSys: Google’s Wide & Deep Models
Auto Encoders for RecSys
Conclusion
- DL (DNN, AE, CNN, RNN) boosts recommendation performance
- CNNs ideal for content-based feature learning: ameliorate cold-start problem
- RNNs very powerful for sequence based recommendation, multi-modal learning, and order-aware distributed representations, trends forecast …
- Is deeper better for RecSys ?
Beyond
- Model understanding: Explanation
- Exploration embedding semantics:
- Specific Architectures for RecSys
- Recommendation Algorithmic Bias
- DL+RecSys beyond traditional tasks → Innovative Applications
- Machine as reviewers and critics
- Personalized content generation: news articles, art, movies, songs, design, fashion, …
- Different verticals
- …
Workshop on Deep Learning for Recommender Systems @ ACM RecSys 2017
http://dlrs-workshop.org/
Submission deadline: 22 June 2017
Credits: Material of this presentation is taken from excellent presentations and papers of great Deep Learning researchers and practitioners. All copyrights belong to their respective owners.
