Back From RecSys 2021
RecSys is the biggest conference on recommender systems and just like last year, we came back with an enormous amount of new ideas, thoughts and questions about our future data architecture and next strategic steps. Research on recommender systems is very active. Seeing how many industry leaders made the effort not only to show up but also present their work indicates how firmly they put this field at the heart of their technological progress.
At Groover, we aim to build the best recommendation system for artists to meet music industry professionals. Since we are dealing with data from various sources and have enough interaction history to explore machine learning solutions, we must follow state of the art research on recommender systems.
What Were The Main Research Topics of Recsys 2021?
Companies such as Google, Microsoft, Facebook, Netflix, NVIDIA and Rakuten presented new achievements for exploration and training strategies, recommender-specific OPS best practices and session-based recommendations. World leading research labs from Carnegie Mellon University, University of Illinois, University of Antwerp and EPFL introduced new ideas on bandits algorithms, cold start-specific algorithms and variational autoencoders. After much brainstorming and discussions amongst our data team, here are what we consider to be the highlights of this year’s edition of RecSys. This summary does not address every topic covered at the conference, yet it provides a broad overview of what has been presented.
Bandits
They were one of the hottest topic of RecSys 2020, and were a key subject of discussion this year as well. Bandits were not the main topic of many research papers, yet they seem to be part of much recommender architecture. Nike, Google and Deezer mentioned them as the exploration inducer of their recommendation pipeline.
Bandits remain one of the most widely used mechanisms to induce exploration, and there are many ways to apply it. The notion of exposure equity is assuring that the exploration strategy remains fair to each curator or item. Making sure that exposure is measured remains an essential topic in recommender systems, because it prevents them from being static. Stationarity can naturally be erased when user preferences have enough momentum to change item popularity, and TU Kaiserslautern were one of the participants dealing with this event. They introduced Burst-Induced Multi Arm Bandits (BMAB), which seem to be handling popularity shifts pretty smoothly.
On the OPS side, Nike’s recommendation pipeline is including bandits, among many other items:
Even when they are not a main research topic they remain a key component of recommender systems. There is no one fits all recommender algorithm, rather a modular structure with each subpart optimized for its task.
There is no one fits all recommender algorithm, rather a modular structure with each subpart optimized for its task.
Autoencoders
Autoencoders were at heart of numerous state of the art algorithms of the past 5 years. The idea that data can be projected into a lower dimensional space to grasp a latent representation is very powerful, and it is not surprising to find out that breakthroughs in Recommender Systems were exploiting the power of autoencoders. Leveraging transfer learning, cross-domain autoencoders are trained on data from a source domain and then used in a target domain. Singapore University and Rakuten researchers co-authored a paper in which two strategies are applied: hard and soft. The former is using a Variational Auto Encoder (VAE) to directly learn a generative model in the target domain. The second approach is encouraging the target model to be similar to its source counterpart. Both seem to be a promising approaches.
Exploration Strategies
Both Google papers are emphasizing the need for exploration, to increase user characterization and reduce sparsity in training sets. By exploring uncovered areas, the interaction matrix becomes less sparse and it allows for better exploitation because it re-calibrates the training set. The second Google paper also emphasizes that user characterization benefits from an exploration setting. Since most of their users do not have self declared preferences, using their preference as a proxy is a meaningful way to characterize them.
The Jheronimus paper shows a negative correlation between the desire to explore and musical expertise in a streaming context. Our problem setting differs from this, but a similar behavior is observed in our case when we consider the fact that artists tend to pick the highest ranked curators. Exploration is rarely user driven when it comes to people who are convinced of their expertise, which is why it is our role to bring it in a subtle way.
OPS: The Other Side of The Coin
COVEO came to RecSys with a bold statement: Recommendations do not need to reinvent the wheel. They showed that one can take standard tools at every pipeline step and have pretty decent recos. Here is Coveo’s end-to-end architecture:
Netflix was also presented some OPS considerations for recommender systems. Here are key quotes, in no particular order:
- OPS reduces firefighting time and it should reduced to the minimum
- Prerequisites to good OPS are: unit tests, integration tests and an MLOPS mindset
- Never rely only on your partner team’s audit, OPS architectures are meant to be challenged
- Low rank picks in RS should be one of the key learning sources
- ML can be integrated into debugging to predict issues ⇒ to be taken carefully.
- ML hot-fixes are always sub-optimal, the best strategy is to unplug and redeploy.
Session-Based Recommendations
Analyzing how users interact with a platform on a short time-frame turns out to be very impactful when it comes to refining existing recommendations.
Using information coming from logs of user sessions can greatly help understanding what they want. Being it an existing basket embedding or the entire user history on a short time-frame, one is able to grasp additional information because it brings all the time spent hesitating and comparing items into a quantitative setting. The transformer framework provided by NVIDIA brings a hand-on solution that needs to be looked at carefully because it could provide an easy way to prototype session based solutions.
Training Strategies: Data Get Old
Robert Mercer once said that there is no data like more data, and many people believed this statement was a neat way to summarize why big data infrastructure would bring a technological advantage to many companies. This is still accurate, however it does not imply that everything must be used for all use cases.
In recommender systems, trends are an essential aspect to consider. In our case this is especially true because some of our curators are no longer on the platform. Questioning whether training should be applied on only visible curators is a question that was discussed at Groover in the past, yet it highlights that our training strategy could benefit from further questioning. The Grubhub presentation highlighted the need for smarter training strategies accounting for the periodicity of user behavior, and the Würzburg paper showed how over-sampling and under-sampling subsets of a dataset will have a great impact on recommendations.
Conclusion
Large conferences such as RecSys are made to open your research horizon, and this year’s edition made no exception. RecSys helped challenging our research roadmap, both on the short and long term. Processing this amount of information takes time and resources, however it helps us making informed decisions.
Sources
All papers were published at RECSYS 2021
Autoencoders
- Towards Source Aligned Variational Models for Cross-Domain Recommendation (Rakuten & Singapore University)
Bandits
- Top-K Contextual Bandits with Equity of Exposure (University of Antwerp)
- Burst-Induced Multi-Armed Bandit for Learning Recommendation (TU Kaiserslautern)
Exploration Strategies
- Exploration in Recommender Systems (Google)
- The role of preference consistency, defaults and musical expertise in users’ exploration behavior in a genre exploration (Jheronimus Academy of Data Science & Eindhoven University)
- Values of User Exploration in Recommender Systems (Google)
OPS
- AIR: Personalized Product Recommender System for Nike’s Digital Transformation (Nike)
- cDLRM: Look Ahead Caching for Scalable Training of Recommendation Models (University of Southern California)
- Jointly Optimize Capacity, Latency and Engagement in Large-scale Recommendation Systems (Facebook)
- Local Factor Models for Large-Scale Inductive Recommendation (Microsoft)
- RecSysOps: Best Practives for Operating a Large-Scale Recommender System (Netflix)
- You Do Not Need a Bigger Boat: Recommendations at Reasonable Scale in a (Mostly) Serverless and Open Stack (Coveo)
Session-Based Recommendations
- Accordion: A Trainable Simulator for Long Term Interactive Systems (Netflix)
- Neural Basket Embedding for Sequential Recommendation (Czech Technical University)
- Transformers4Rec: Bridging the Gap between NLP and Sequential / Session-based Recommendation (NVIDIA)
Training Strategies
- A Case Study on Sampling Strategies for Evaluating Neural Sequential Item Recommendation Models (University of Würzburg)
- Online Learning for Recommendations at Grubhub (Grubhub)
