Eigendecomposition of a Covariance Matrix with NumPy

For Principal Component Analysis (PCA)

Image by G Lopez from Pixabay

This article is for those who want to learn how Principal Component Analysis (PCA) works behind the scenes.

First, we manually apply PCA to a real-world dataset by decomposing the covariance matrix. Finally, the results of the manual implementation of PCA will be compared with the results of the automatic implementation of PCA.

Definition of eigendecomposition

Eigendecomposition is the process of decomposing (factorizing) a matrix into eigenvalues and eigenvectors. Here, we break down our initial square matrix into the product of its eigenvalues and eigenvectors as defined in the following mathematical equation.

Here, A is an n x n square matrix. λ is a scalar called the eigenvalue and x is a vector called the eigenvector with the corresponding λ value. The eigenvalues and eigenvectors come in pairs. Such a pair is known as an eigenpair. So, matrix A can have multiple such eigenpairs. The above equation shows the relationship between A and one of its eigenpairs.

Other key definitions

In addition to the definition of eigendecomposition, the following key definitions are also useful for understanding…