Evolution of Recommender Systems

What has changed and how they are changing consumer marketing with them.

Recommendations are a crucial part of the personalized user experience for any digital brand. Leading companies, most notably Amazon, Youtube, and Netflix, have definitively demonstrated their value and have radically transformed what customers expect from any digital experience. The utility and RoI of recommendations are unquestionable. Amazon, for example, directly attributes an estimated 35% of sales to their recommender system. High quality recommendations generated by such systems can transform the user experience from annoying to delightful while also building long term trust and loyalty. Two decades of production deployment at Amazon reveals the key aspects behind engineering a successful recommender system. At a high level there are two important and critical themes that successful recommender systems embrace:

1. Understand users through the lens of behavior : A typical online user today is perpetually connected and on average spends 6–8 hours a day using 2–3 devices. With so much time spent online, users leave behind rich trails of behavioral activity. Compared to a static record of demographic attributes, a complete record of user’s dynamic activity can be 100 or even 1000 times in raw storage size. These rich profiles consist of of logged in and logged out activity, what pages they spend most of their time on, where they have shown active interest, along with historical activity. Analyzing a user’s actions in the context of their other actions reveals much more about the user than a simple static record. User profiles are transformed from rows and columns in a spreadsheet into a rich graph of interactions. Intelligent recommender systems can infer from this high dimensional data, and produce recommendations that tap into deeper interests and make every experience very personal.

Interaction Graph based on user behavior and attributes

2. Adapt recommendations to the user’s context : A rich user context has within it multiple intents at any point in time. Intents based on most recent activity, intents other users exhibit in similar situations, intents based on repetitive actions, intents based on tags and attributes are all examples of extracting value from different aspects of a particular user’s behavior. Such latent intents can only be amplified by recommendation systems that can both narrow down and broaden the scope of consumption and discovery. Each intent might require different sets of data inputs and algorithms. The learning techniques vary, but the system as a whole should support all of them. Instead of a single block of recommendations that try to narrow down and push users towards fewer options, this new generation of user experiences surprise and delight by telling stories that contain those magical moments of serendipitous discovery.

Let’s take a look at the evolution of Amazon’s home page, and how their systems have evolved to embrace those themes:

Marketing analyst Jay Shemenski provides a breakdown of many elements that dives into the latest Amazon home page (see below), which is an amalgamation many diverse, yet equally powerful, forms of recommendations. Each row showcases recommendations for different intents along with titles that the story behind that recommendation. Going through the list we can see how they come together:

1. Related items you’ve viewed: Exploits other users’ behavior through Collaborative Filtering, which expands discovery through cross user behavior on recent items.
2. More items to consider: Broadens the scope and allows for serendipitous discovery via the top trending items at category level.
3. Recommended for you in Kindle Books: Leverages the user’s previous interest in books against specific interests and device compatibility.
4. Inspired by your shopping trends: Incorporates co-buying behavior across users and their responses.
5. New for you: Showcases recent best sellers that correspond to the users’ potential interests.
6. More top picks for you: Some items never go out of style. Allow for serendipitous discovery items that may not be related to a user’s current history.

Amazon home page personalized with many rows of recommendations built on several behaviors

Amazon is not unique in their embrace of mapping their user experience to the different intents of a particular user. Netflix has transformed their homepage along similar lines. A single grid of products has evolved to fully personalized page with rows of video s along with a small story explaining each recommendation This juxtaposition of storytelling with recommendations creates a far more interesting and personal experience for the user, which has paid off dividends for both companies.

Recommender systems have evolved from generating simple rows of items or articles, to building dynamic pages that amplify different intents — powered by analyzing the sum total of user history and behavior. Leveraging rich behavioral stream of data that every user generates results in a far richer, exciting, and engaging experience.

This was the first post in our recommender system series. In subsequent posts we will take a deeper look at how marketing has adapted to these changes and how product experiences can now be embedded within marketing campaigns.