RecSys 2021, a Veepee’s Perspective

This article has been written by Benoit Leguay, Oriol Roqué, Shihe Long and Till from the Customer eXperience Optimization team, from the vpTech community.

Introduction

Being in charge of Veepee’s recommender system, the Customer eXperience Optimisation (CXO) team had the opportunity to attend the RecSys 21 event. This conference gathers the research in recommender system, including contributions to algorithms (from collaborative filtering to knowledge-based reasoning or deep learning), contributions to systems (practical issues of scale and deployment), and contributions through applications (bringing forward the lessons of innovative applications across various domains from e-commerce to education to social connections).

Hence, this conference is a good yearly meeting for our team to catch up with the latest improvements in this field. On one hand, paper presentations are useful to have a good overview on the future of research in recommendation. On the other hand, the meetups with the partner companies (Netflix, Facebook, Google, Nvidia, Twitter etc ..) allow us to ask more practical questions, and to discuss with people working on concrete applications.

In this article, we would like to highlight some of the topics we found interesting and how they can help us to provide better solutions for both our business stakeholders and our customers. The rest of the article is structured as follows: first, an introduction to the echo chamber phenomena on recommender systems and possible impacts; second, how to provide better recommendations for a business distributed among different organizations; and last, a recap on the business side concerns that might arise due to recommender systems.

Echo chambers

Recommender systems aim at benefiting both users and the company. To do so, algorithms are trained to maximize metrics based on user’s interaction with the platform: impressions (e.g. social media), orders (e.g. e-commerce), watch time (e.g. video provider), etc. These approaches demonstrated having a high impact in increasing users’ fidelity (RS survey). This benefits companies by maximizing profit and, on paper, it should also improve user experience by providing personalized content. While the enterprises’ benefit is real, the users’ benefit is uncertain. Namely, researchers have theorized a negative recommender system impact: echo chambers.

The original use of echo chambers comes from media, it refers to situations in which “beliefs are amplified or reinforced by communication and repetition inside a closed system and insulated from rebuttal [Wikipedia]”. As the media, recommender systems are biased toward one objective: they choose information to be displayed and thus, shall be subject to echo chambers creation.

In a recommendation scenario, an echo chamber “is an environment in which a person encounters only beliefs or opinions that coincide with their own, so that their existing views are reinforced and alternative ideas are not considered” [Oxford Langage] . Another concept named filter bubble describes a very close phenomenon. Many resources on the internet try to distinguish these 2, we will not focus on this here and consider they refer to the same thing. We will use the term echo chambers, as it pictures the imprison concept as well as the echo intensifying across time. It is essential to mention that, without deep analysis, the boundary between a good recommendation system and one that creates echo chambers can be hard to gauge.

As mentioned earlier, usually, echo chambers are created because of the algorithm maximization goal. Intuitively, we could expect our system to provide, continuously, diversified contents that would satisfy all user’s interest facets, and thus, maximizing the number of interactions. In practice, current recommender system algorithms may lack a long term view and are susceptible to provide a small subset of user’s interest content. Reinforcement learning framework tackles this problem by introducing a whole independent exploration module.

At Veepee, we identify 2 scenarios where echo chambers could lead to suboptimal recommendation:

• The member loses interest for the platform because it always provides same type of products (we cannot expect a member to always purchase the same products)
• We satisfy ourselves with a subset of the products a member can buy while, ultimately, he could be interested and purchase more products we don’t recommend.

Because of these 2 reasons at least, we want to avoid this phenomenon. However, in our scenario, there is no real moral or ethical threat if our recommender system creates echo chambers. This is not true for social media.

In the RecSys 21 conference, 3 groups of researchers presented their work on echo chambers.

The first one, named “I want to break free! “ aims at lowering the assumption coming out of the homophily principle (used for user-based recommendation) in their recommender system. This principle, where similar users tend to be interested in similar items, has been proven to be a working proxy to recommend content. Though, it might also introduce echo chambers phenomenon. The solution proposed is called FRediECH (A Friend RecommenDer for breakIng Echo CHambers), and is a system that learns users preferences as well as their echo chambers affiliation, so it allows personalized recommendation, combining direct users’ interest with out-of-echo chambers’ interest.

The second work raises a new perspective concerning the echo chambers. They split the echo chambers into 2 types, the ones created by native affiliation: members gather and amplify their proximity only because they like similar content, the recommender system being the main actor of the snowball effect. The other type is more troubling, members gather and amplify their proximity by systematically excluding conflicting points of view. While the first issue can be solved with more diversified topics and points of view, the latter may not. They develop a complex graph-based solution to tackle these 2 issues. Beyond the technical solution, they want to prove that a single model with a single strategy may not be an appropriate way of solving echo chambers phenomenon.

The last presented work is an audit on YouTube recommendation system. The goal of this study is, first, to see if it is possible to get out of the filter bubble (or echo chamber), in other words “burst the bubble”. To do so, they created fake users on the platform that interact with misinformation content, entering the bubble, and then focusing on content that aims at debunking the concerned fake news. What they observed is that escaping the bubble and reaching diversified content is possible. The way of doing it may differ depending on the topic. Researchers also explain that for some specific topics it is very difficult to enter in the associated bubble, implying that the system can act very differently depending on the theme.

Echo chambers are both fascinating and a concerning phenomenon. It can lead to serious danger, such as polarisation involving both withdrawing and extremist behaviours. Moreover, because it is possible for an ill-intentioned company with large influence to create these echo chambers on purpose, this could become a powerful misinformation weapon in the future. For example, even though the case is not closed, the Facebook Files dossier leads us to believe that this could already be the case.

That is why it is very important to create tools, systems and metrics to detect and counterattack these echo chambers.

Boost recommendation system performance across organizations

In their paper Boosting Local Recommendations With Partially Trained Global Model submitted to RecSys 21, the authors discussed how to boost the performance of recommendation systems across different organizations. Indeed, in the era of deep learning, algorithms are often data hungry. It is natural to think that the algorithm should take advantage of all the possible data that it could use to get better performance. However, this could be very complex for companies operating in many countries since the data in their subsidiaries may vary substantially in both quality and quantity due to differences in the targeted audience and marketing strategies. The model trained on the combined data is susceptible to be biased towards the user behaviors of the organization with bigger user bases, which means it may be suboptimal when applied to the smaller organizations.

The scenario corresponds to the issue of performance gap that we are facing for home page personalization in Veepee. The global model trained on the consolidated dataset of all sites in Veepee’s data lake performs slightly worse in Spain and Italy if compared to France, where the business is more expanded. To tackle this problem, the authors propose a framework that can first partially train a global model with all data across organizations capturing global user behavior patterns, then boost it again with local data to learn specific characteristics in each of the sub organizations. We are expecting that the framework can be also applied to the case of Veepee with small adaptations to narrow down the gap between the big site and small site.

Recommenders in the business

Aside from the more technical presentations and workshops, we also found interesting discussions on the business side of the recommender systems. In particular, due to Veepee’s nature, the most related workshop was the Workshop on Recommender Systems in Fashion and Retail.

One of the most interesting challenges in Machine Learning is still the relation between business stakeholders and the solutions provided by Data Science teams. While it is true that machine learning is increasingly being used to solve business problems, there is still some lack of confidence in the algorithms. In turn, this can make business stakeholders to include some controlling parameters that might reduce the overall performance of the algorithm. Therefore, it is our mission to provide a transparent explanation to the business teams of how our recommender system algorithm works.

A possible solution to the aforementioned topic is to provide explainable predictions. In our case at Veepee, we are using our own personalisation algorithm to rank the available sales on the home page for each customer, as stated in the Learning to Rank previous post. In order to bring more transparency to the business and to the customer, we could complement the specific home page recommendation with a short explanation of why a sale is shown at a specific position. Maybe it is because the customer has previously purchased a product of the same or a similar brand, or it might be just because the sale is currently popular.

Another interesting topic that arose on the industry talks was how to handle long-term customer experience in the fashion industry. The fashion industry rapidly changes from season to season and at the same time, customers might change their preferences. Most of the recommender systems that are being used focus on short-term objectives, i.e., increase the number of purchases, turnover, etc. We assume that having steady short-term based recommendations will have a positive long-term effect on our user experience, although it is not the primary goal of the recommender algorithm. So, what if we focused instead on increasing the life-time value of our customers in Veepee?

The last topic we want to discuss is the problem of providing sufficient positioning opportunities. There is some concern on the business side when a sale, product or service does not perform as good as it was expected, which might result in blaming the recommendation system. Indeed, a powerful brand, with huge historical performance might have the upper hand on positioning if compared to a new, not so famous brand. This is especially a problem regarding ‘cold start’ situations: whenever the algorithm does not have enough data to provide reasonable predictions. In Veepee’s case, these situations could be either new brands and new customers. So far, our ability to ensure the performance of our recommender system on cold start situations has been done a posteriori, by analysing the results of A/B tests with different dimensions and granularity. But it is true that we could ensure more fairness in positioning in the model training phase, which we might consider doing in the future.

To conclude, in the Customer eXperience Optimisation team we believe that the RecSys 21 conference opened a lot of fields of improvement regarding our recommender system. Stay tuned to see the most interesting of them being implemented at Veepee!