The 42nd edition of ACM SIGIR took place this week in Paris, the homeplace of Criteo AI Lab, and we couldn’t miss it.
The full conference consisted of five days focused on information retrieval and recommendations. At Criteo AI Lab, our research teams constantly push the limits of the current state-of-the-art in recommender systems, click prediction or reinforcement learning to deliver cutting-edge solutions to the challenges in online advertising. This conference is thus relevant to our applications and research.
In this post, we would like to highlight a few works that we found particularly interesting.
Best paper award
The Best Paper Award was attributed to Variance reduction in gradient reduction in online learning to rank, by Huazheng Wang, Sonwoo Kim, Eric McCord-Snook, Qingyun Wu, Hongning Wang. In a context of online learning to rank, the approach reduces the variance of the gradient estimation by projecting the selected updating direction into space spanned by the feature vectors from examined documents under the current query. The paper proves that this method provides an unbiased estimate of the gradient and illustrate the benefits with significant improvements compared to several state-of-the-art models.
Keynote on “Automatic Understanding of the Visual Word” by Cordelia Schmid
Cordelia Schmid gave a keynote on “Automatic Understanding of the Visual Word”. The keynote covered a variety of work on data (manual, synthetic, and weakly supervised), video understanding and demonstrations on robotic arms interacting with the real world. An interesting takeaway was that using novel data such as the SURREAL dataset (synthetic humans for real tasks) could significantly improve algorithms performance. Also, fusing text and video with architectures such as VideoBERT can be useful for zero-shot classification of videos. A structured model for action detection shows the benefits of modeling temporal dependency. Overall, the focus was given on the importance of spatial and temporal information into the design of models, with excellent results for weakly supervised learning tasks. Cross-modal interaction with text (and sound?) is very valuable as well.
Collaborative Filtering Did you know that in Criteo, a collaborative filtering method based on R-SVD is one of the algorithms fueling our recommendations? We are aware that dealing with product popularity biases is crucial to accurately embed less popular items. It was interesting to see recent work on Noise Contrastive Estimation for One-Class Collaborative Filtering. The authors presented an efficient method based on de-popularization of the implicit matrix by re-weighting with respect to the noise contrastive objective (similar to word2vec), and SVD. Check out this intriguing analysis of the ability to deal with popularity bias for the state-of-the-art methods:
Interpretability and Explainability At Criteo, we are aware that interpretability of recommended products is important to build users’ trust in the system. For this reason, our attention was caught by work from Google AI about Transparent, Scrutable and Explainable User Models for Personalized Recommendation
In general, there are three aims for recommendation explanations: transparency helps users to understand how the system works, justification provides an explication of individual recommendations, and scrutability allows users to tell the system if it is wrong. This paper suggests using a set-based recommendation method. Instead of explaining to the user why a given item was recommended, the system provides a textual description that summarizes the model’s understanding of the user’s preferences. The user can scrutinize this summary and change her user model.
Are you asking yourself “How much accuracy do I sacrifice by making my recommender system both transparent and scrutable?” The authors show that their explainable method achieves a quality comparable to that of state-of-the-art recommendation algorithms. Finally, they demonstrate how the model can be explicitly scrutinized by users, leading to much-improved recommendations.
Sequential Recommendations Is your recommender system able to distinguish products with a recurring need (such as toilet paper) from products typically only bought once (such as a toilet seat)? If not, you risk recommending irrelevant products after a buy (See for example this viral tweet ). At Criteo, we call this problem ‘post-sale recommendation’. To address this problem, Ting Bai and its co-authors propose to model the probability a user will buy a product over time as a Hawkes process parameterized by historical product features. The weight learned for each product represents its typical time of repurchase. Check out the full paper for more details: CTRec: A Long-Short Demands Evolution Model for Continuous-Time Recommendation.
Learning to Rank
A great work on To Model or to Intervene: A Comparison of Counterfactual and Online Learning to Rank from User Interactions provided a comparison of counterfactual and online methods, two common strategies to deal with bias in the field of Learning to Rank (LTR for short). Counterfactual methods learn a ranking model offline from historical data and use a re-weighting strategy to debias the interaction data. Online methods optimize and update a ranking model after every interaction, combating the bias by displaying slightly modified rankings, i.e. interventions. The counterfactual methods have the advantage of avoiding the risk of showing untested rankings. On the flip side, this means the possible rankings are limited. The online learning allows to explore novel rankings, and applies the learned behavior immediately, but which also increases the risk of potentially hurting the user experience. In practice, the choice between these two approaches has direct impact performance and user satisfaction. This paper provides interesting guidelines on how LTR practitioners should choose which method to apply.
Networking events at SIGIR — Women in IR
Women in IR session was opened by Mounia Lalmas (Research director at Spotify) giving an inspirational talk about her career as a researcher. Afterward, a lively discussion was opened about actions that can be taken to attract and keep more women in IR: there is a need for individual and collective action to expand the applicant pool by encouraging more women to apply. Women in IR organized mentor-mentee matching, a great way to network and motivate young women to pursue their careers.
Artificial Intelligence in Industry Meetup
On Wednesday, Criteo hosted a meetup Artificial Intelligence in Industry. Four invited speakers shared their knowledge with the AI community. Ricardo Baeza-Yates, CTO at Ntent, spoke about Semantic Mobile search and how to use contextual visual information for NLP. Mounia Lalmas, Director of Research at Spotify, talked about personalization of recommendation and evaluation of user satisfaction at Spotify. Patrick Gallinari, Distinguished Researcher at Criteo, demonstrated the importance of visual context in his talk “Learning visual context representations”. Diego Saez Trumper (Research Scientist at Wikimedia) presented openly available Wikimedia Public Research Resources. The evening finished with networking and views over the Paris skyline from the rooftop of our headquarters.
Finally, SIGIR 2019 was a great success, we would like to thank the organizers, we will be back the next year!
Authors: Zofia Trstanova, Marc Tchiboukdjian, Anne-Marie Tousch, Olivier Koch