Eigenvalues and Eigenvectors
Eigenvector or characteristic vector of a linear transformation is a nonzero vector that changes at most by a scalar factor when that linear transformation is applied to it. The corresponding Eigenvalue, often denoted by lambda, is the factor by which the Eigenvector is scaled.
Geometrically, an Eigenvector, corresponding to a real nonzero Eigenvalue, points in a direction in which it is stretched by the transformation and the eigenvalue is the factor by which it is stretched. If the Eigenvalue is negative, the direction is reversed. Loosely speaking, in a multidimensional vector space, the eigenvector is not rotated.
An Eigenvector is a vector whose direction remains unchanged when a linear transformation is applied to it.
Consider the image below in which three vectors are shown.
Eigenvectors (red) do not change direction when a linear transformation (e.g. scaling) is applied to them. Other vectors (diagonal, in black) do.
The Eigenvectors and Eigenvalues of a covariance (or correlation) matrix represent the “core” of a PCA: The Eigenvectors (principal components) determine the directions of the new feature space, and the Eigenvalues determine their magnitude. In other words, the Eigenvalues explain the variance of the data along the new feature axes.
Determinant
We need to understand the formula for calculating a determinant of a matrix as it will be used for calculating Eigenvalues and Eigenvectors.
Steps
Now lets go thru the steps to calculate the Eigenvalues and Eigenvectors
To solve for the Eigenvalue, lambda, and the corresponding Eigenvector, X, of an n x n matrix A follow the following steps.
Step 1: Multiply a n x n identity matrix with the scalar lambda
Step 2: Subtract the result from Step 1 from the matrix A
Step 3: Find the determinant of the difference matrix from Step 2
Step 4: Solve for the value of lambda that satisfy the equation:
Step 5: Solve the correspondent vector for each values of lambda
Example
Let’s say we have a given matrix, A, below.
Step 1: Multiply a n x n identity matrix with the scalar lambda
Step 2: Subtract the result from Step 1 from the matrix A
Step 3: Find the determinant of the difference matrix from Step 2
Step 4: Solve for the value of lambda that satisfy the equation:
Step 5: Solve the correspondent vector for each values of lambda
For lambda = 8,
So,
For lambda = -2,
So,
So we get 2 Eigenvectors,
I hope this article provides you with a good understanding of some important concepts of Eigenvectors and Eigenvalues.
If you have any questions or if you find anything misrepresented please let me know.
Thanks!
