Ratings Are Passé: How We Built An Intuitive Recommendations System On LBB

Published in

From LBB

8 min readJan 17, 2020

NOTE: This article focuses on the algorithm that we used or how we got to it. It does not include the details of the technical implementation of the project.

In this article, we are going to illustrate how we at LBB built a recommender system in the absence of any ratings from users such as hearts, stars, likes, emojis, etc.

We don’t have any of these cool people on our platform, so sad :(

What is a recommender system?

A lame question in today’s world, but for those of you who don’t know, a recommender system is a simple algorithm whose aim is to recommend the most relevant items(post, product, song) by discovering patterns in the user’s past interaction with the items. The algorithm predicts the rating that the user may give to the items they haven’t already seen.

Below is an illustration of how a typical recommender system works in a context like e-commerce.

Where to start

While ratings may work on aggregators, we believe platforms should have systems that are more intuitive for customers. We based out recommendations system on more “passive” cues from our users, than expecting them to explicitly tell us what they like/dislike through their journey on LBB.

Below is how we derived ratings of users based on their simple interactions with items.

User Item Interaction Events

Post Impressions (I) — User scrolls through the post on feed or search results. We chose this particular event as we wanted to capture the intuition of a “dislike” button.

When a user searches for something, their intention is more-or-less clear. When they ignore some of these search results we give them (those search results) a negative score.

Post Views (V) — User opens the posting page from the feed, search results, notifications, etc. This event captures the intuition of an “average” rating.

Post Engagement (E)- User shares the post with somebody, bookmarks it or clicks an engagement button like “Order Here”, “Buy Now”, “Call Now”, etc. The main objective of these posts on LBB is for the user to connect with these businesses and when a user does the same we capture the intuition of a “like” button.

Engagement buttons highlighted on Post page on LBB

Implicit Ratings

So there are only the following three possibilities:

The user skips the posts from a search result or a feed. We give those posts the lowest rating i.e, 0.

User->Post := I= 0

2. The user clicks a post from a search result or a feed, to view it in detail. We give it the average rating i.e, 1.

User->Post := I + V = 1

3. The user clicks a post from a search result or a feed views it in detail and then engages with it (connects with the business). We give it the highest rating i.e, 2.

User->Post := I + V + E = 2

So by cumulating all these events, we derive a user- post rating matrix like this.

Implementation

So for the scope of this blog, we’re going to discuss how we implemented collaborative filtering over this user-item rating matrix.

Memory-Based Implementation

We considered two approaches…

User-User similarity or User-Based Collaborative Filtering(UBCF):

Consider a user X. Find a set N of other users whose ratings are “similar” to X’s ratings. Then estimate X’s ratings for remaining items based on ratings of users in N.

Item-Item similarity or Item-Based Collaborative Filtering(IBCF):

Find N items similar to an item I, i.e the items who got similar ratings from a set of common users between them say U. Then Estimate ratings of item I for users not in U based on ratings of items in N.

We chose Item- item approach based on the following observations:

The average no. of ratings by users is lower than the average no. of ratings for items. If we consider our dataset the most engaged users have rated ~80 items on an average. While items have an average of ~110 ratings.
The rate of increase in users is higher than the rate of increase in items. Thus calculating the group of similar users becomes costlier over time.
New users will have no or little information about them to compare with other users. As above, new items will also lack ratings to compare with others.
The prediction accuracy (RMSE) of the item-item implementation was lower with our data, which I’ll discuss below.

To implement item-item similarity:

We calculate the similarity among the items

Let’s consider two item pairs (Broom, Wand) and (Broom, Dog) from our user-item rating matrix. Then the item vectors R(i) can be written as…

R(B):= (2, ∅, ∅, 2, 0, ∅, ∅)R(W):= (2, 1, 2, ∅, ∅, ∅, ∅)R(D):= (∅, ∅, ∅, 1, 1, 2, ∅)

These vectors represent the ratings of items given by respective users. To think intuitively, similar items will be those who received roughly equal ratings from the same users. For that, we explored the following methods.

Jaccard Similarity

Jaccard Similarity calculates similarly as, no. of users who rated both the items out of no. of users who rated any of the items.

Sim(B, W) = 1/ 5, Sim(B, D) = 2/ 4Sim(B, W) < Sim (B , D)

The problem here is that it ignores rating values.

Cosine-Based Similarity

Cosine similarity is a measure of similarity between two item vectors in N-dimensional space (N being no.of users) that measures the cosine of the angle between them. Smaller the angle, the greater the cosine, thus greater the similarity between two items.

For this, we replace missing ratings with 0, Thus the item vectors become

R(B):= (2, 0, 0, 2, 0, 0, 0)R(W):= (2, 1, 2, 0, 0, 0, 0)R(D):= (0, 0, 0, 1, 1, 2, 0)Sim(B, W) = 0.471, Sim(B, D) = 0.289Sim(B, W) > Sim (B , D)

The problem here is that as the missing ratings are used as 0, thus they are being treated as the negative ratings.

Centered-Cosine Similarity

To avoid the problem of missing ratings, why want to consider them as average ratings. To do this, we normalize ratings by subtracting row mean (average rating of the item)

μ(B) = 0. 571, μ(W) = 0.714, μ(D) = 0. 571

Thus the item vectors come to be

R(B):= (1.429, 0, 0, 1.429, -0. 571, 0, 0)R(W):= (1.286, 0.286, 1.286, 0, 0, 0, 0)R(D):= (0, 0, 0, 0.429, 0.429, 1.429, 0)

Now the similarity is calculated same as cosine similarity

Sim(B, W) = 1.837, Sim(B, D) = 0.368Sim(B, W) > Sim (B , D)

The two ways this method works better are…

The missing ratings are treated as “average” ratings i.e, 0
As the ratings are averaged out it handles both “tough raters” and “easy raters”. This means that the ratings of the items which are very popular on the platform have similar average ratings with those who are less popular. Thus they become more comparable.

2. Calculation of rating

Now to predict ratings of each user for individual items we use the weighted sum method. Here ratings for a particular user x for an item i are calculated as the weighted average of the ratings by user x for the items similar to item i. The weights beings the similarity between items in N(i;x) to item i.

Sij… the similarity of items i and j rxi … rating of user x on item i N(i;x) set of items rated by user x similar to i

Model-Based Implementation

Here we used a well-known matrix factorization technique SVD (Singular Value Decomposition). Matrix factorization is a Dimensionality Reduction technique, which is used to derive the tastes and preferences from raw data.

Dimensionality Reduction is used mainly to remove redundant and noisy features that are not useful and discover hidden correlations in the raw data. It also helps in easier data storage and processing.

The goal of matrix factorization is to learn:

The latent preferences of users (Features that describe the characteristics of users based on which they choose some items).
The latent attributes of items from known ratings (Features that describe the characteristics of items).
Predict the unknown ratings through the dot product of the latent features of users and items.

The latest features could be thought of as typical preferences like food, shopping, travel, etc. or high-low-medium budget or less distance traveled or how old the post is, etc.

As an overview, SVD is an algorithm that decomposes a matrix A into a smaller/simpler approximation of the original matrix A. Mathematically, it decomposes A into two unitary matrices and a diagonal matrix:

In the above illustration, if we consider A as the user-item rating matrix, then U is the user latent features matrix, Sum is the diagonal matrix of weights of features, and V(transpose) is the item latent features matrix. U and V(transpose) are column orthonormal, and represent different things: U represents how much users “like” each feature and V(transpose) represents how relevant each feature is to each item.

The Evaluation

We used one of the most popular metrics to evaluate the accuracy of predicted ratings, Root Mean Squared Error (RMSE).

Root Mean Squared Error (RMSE) measures the average magnitude of error by taking the square root of the average of squared differences between predicted ratings and actual ratings.

And Voilà !!

We did a weighted average (80% MF, 20% IBCF) of the ratings from two implementations to fill the voids in our afro-mentioned User -Item Rating Matrix. Then we showed the unseen items to the users in the order of their predicted ratings.

The outcome of this system has been a tremendous growth in open rate on notifications, along with the most significant metric of success for us which is connect with business (is the user MORE likely to connect with a business basis the recommendations engine) growing by 50%.

Footnote…

We have explained only the collaborative filtering implementation here. We could have also used content-based filtering in hybrid to this approach.