Principal Component Analysis (Autoencoder)
In machine learning, feature learning represents the techniques that any machine learning model uses to extract the representation of the underlying raw data. Feature learning replaces manual feature engineering and allows a machine to both learn the features and use them to perform a specific task. In deep learning, we train models by using large sets of labeled data and neural network architectures that learn features directly from the data.
A deep neural network can learn without the need for manual feature extraction, and this ability to perform automatic feature extraction from raw data also called feature learning. One of the significant challenges while working with large data is that there are too many dimensions. To improve the result of the machine learning model, we perform “dimensionality reduction,” which merely means reducing the dimension of the feature space.
There are many ways to perform dimensionality reduction, but most of these techniques fall into one of the two classes:
- Feature Elimination
- Feature Extraction
Principal component analysis (PCA) is used for dimension reduction, which comes under feature extraction/feature engineering. PCA combines our input variables in a specific way, and then we can drop the least essential variables while still retaining the insights of the variables in the process. As a result, each of the new variables after PCA are all independent of one another.
Import Libraries
Autoencoder
An autoencoder is a type of deep neural network which learns to compress data from the input layer into a compressed representation, and then uncompress that code into something that closely matches the raw input data. An autoencoder consisting of an encoder layer and a decoder layers which are constructed by stacking multiple layers of artificial neural network.
The encoder uses raw data as input and produces compressed features as output, and the decoder uses the compressed feature from the encoder as an input and tries to reconstructs the original raw input data as output. In autoencoder, the neural network learns the distribution of our input data, and since layer in the encoder is designed in such a way that it shrinks as they go deep, this forces the autoencoder to engage in dimensionality reduction.
Define Network
A neural network can be defined using TensorFlow fully connected API. A linear autoencoder has 3 layers (encoding, hidden and decoding), the encoding and decoding layers have ‘linear activations,’ and the hidden layer has two neurons. Substantially this structure approximates PCA by reducing the data from four features to two features in the hidden layer.
Loss Function
Since our target is to generate an output sample as similar to input raw data, we are going to use mean squared error.
Optimizer
Running the Session
Result
After passing the training data through the hidden layer, we get two new vectors, and by plotting them against each other, we can clearly view the blob and cluster formation similar to Principal component analysis (PCA).
Source Code: https://github.com/llabhishekll/pca-using-autoencoder
