Recommendation Engine — Based on latent feature

Recommendation Engine Overview:


Given set of items purchased or rated by a user, recommend or suggest set of additional items which user could be interested in.

Items could be movie, web pages, tv programs, grocery items, friends, advertisement etc.

Essentially we have to determine the utility of an item with a user, and then recommend items to a user with highest utility score.

Like all supervised machine learning, here as well we will have a training phase and then test phase.

The high level steps in training phase are:

1. Load the training/input transaction data (sparse matrix) into memory. Use efficient data structures. Lets call this matrix X.

2. Run algorithm to compute the model (2 low rank matrices). Has to be very efficient both in terms of space and time complexity. Lets call this matrices A and S.

3. Save model into persistent storage.

So essentially we need to split X into two low rank matrices such that X = A.S, the same is performed using probabilistic matrix factorization technique.

In test phase/ score phase.

1. Load the computed model into memory.

2. compute recommendation using model. This is essentially multiplying matrices A and S.

3. Save recommendation against each user. Remove items, which user already purchased, and take top N scored items.