Autoencoder is a type of directed neural network that has both encoding and decoding layers. By learning the latent set of features we can compress the input data in the mid layers, typically autoencoder is used in dimension reduction, recommendation system. Reconstruction process happens in the decoding layer, that’s also where we can perform inference for prediction. This post contains the key points from an Udemy course, and includes my extended implementations for autoencoder based recommender system. The full scripts for this project can be found here:
Structure of autoencoder
There are three typical components: visible input layer, hidden layer(s) and visible output layer. The input and output layers have the same number of units as we are compressing (encoding) and then reconstructing (decoding) the information. The encoding part shown in figure below (Udemy material) By manipulating the type of edge connections, we can indeed encode the input data into fewer units while retaining all the information such that we can decode it later. The example below shows that after decoding, the information are recovered after we apply a softmax layer onto the decoded values. Note that it’s not necessary that encoded units are less than input units, but if there are more units after encoding, the autoencoder will be prone to learn the equal relationship and “be lazy” to directly push the information through some units to output, resulting in learning nothing.
Implementation on autoencoder network
In the Encoder class, we have automated the creation of layers using nn.ModuleList, as we know that the input and output layers have the same number of units. Building an instance requires an array of integers representing the number of units in each layer. This can be a template to build simple autoencoder networks.
Data preparation, reindexing and prepare tensors
If it’s your first time working on recommender systems, you may wonder how to split the data into train and test sets to prevent overfitting. In this case, we split each user’s known ratings, by an approximation, we split from the raw records where each row is a triplet, comprising of user, item and rating. We use the classic Movielens 1M dataset in this project, and preprocess it such that all users and movies are encoded to intergers starting from 1. Some reindexing happens in the following functions. By executing the process below, you can obtain matrix form of user-item ratings and the torch tensors storing them.
Training autoencoder, test it and make top k recommendations
We first create an instance of Autoencoder, then to train it using train set, test it over test set. And finally we created a customised function to make top k recommendation for each user, using the function make_top_k_recommendations. One thing to note is after training the autoencoder instance (our model), we will use the full records containing all known user-item ratings as input to the network to get the output. The output dense matrix will be the sorted to obtain top k items recommended to each user. The logic has been implemented in the block of functions below.
Thus we have completed the procedures of building a basic recommender system based on Autoencoder.
