Matrix Eigendecomposition, its Importance, and the Applications
What is going to be the benefit of decomposing a matrix? What does that mean? When we decompose anything, we break it into its constituent elements. Assume we are going to disintegrate a tool (a car or a watch!). Such action helps us to understand the core particles and their tasks. Furthermore, it helps to have a better understanding of how that specific tool works and its characteristics! Assume that the tool is a matrix which we would like to decompose. There are different approaches to decompose a matrix. However, perhaps the most commonly used one is matrix eigendecomposition which is decomposing a matrix using its eigenvectors and eigenvalues.
This tutorial is dedicated to explaining the concept of matrix decomposition, its definition, and the process. You will learn how you can decompose a matrix to its constituent elements. You will learn the following:
- The definition of eigendecomposition
- The concepts of eigenvectors and eigenvalues
- The benefits of decomposing a matrix
- The important properties associated with matrix decomposition
- How to do it in Python and Numpy
Before You Move On
You may find the following resources helpful to better understand the concept of this article:
- Python Tutorials — A FREE Video Course: You will become familiar with Python and its syntax.
- Numpy — An Introduction to a Great Package for Linear Algebra: Numpy is one of the best scientific computing packages for Linear Algebra! You always need it for Machine Learning as you always need Linear Algebra for Machine Learning!
- The Remarkable Importance of Linear Algebra in Machine Learning: This article talks about why you should care about Linear Algebra if you want to master Machine Learning.
- Basic Linear Algebra Definitions that You Hear Every Day: Covers the primary and most frequently used Linear Algebra definitions in Machine Learning.
- Matrix Operations In Practice: Everything you need to know about matrices and their operations.
The Definition of Matrix Eigendecomposition
In this section, I am going to show you the definition of eigendecomposition and the subsequent concepts necessary to understand it.
Eigenvector and Eigenvalue
Before we move on, we should know the definition of eigenvector and eigenvalue. The definition of eigenvector and eigenvalue are somehow connected.
Due to that, it is customary to only work with eigenvectors that have unit norm. It is simple to construct an eigenvector with the unit norm. Assume v is our eigenvector. Then, the following vector is also an eigenvector with the unit norm:
The Process
Here, I want to explain how we decompose a matrix to its constituent elements and we call it the eigendecomposition of a matrix.
Discussion on Matrix Eigendecomposition
Not that we can only factorize diagonalizable matrices as above. But the question is what is a diagonalizable matrix?
A matrix is invertible if it is non-singular! Now, let’s have a more precise definition of a matrix being singular or non-singular.
Under some circumstances, we can calculate the matrix inverse using the decomposition.
Special Matrix and its Decomposition
We are interested to investigate a special kind of matrix: Real symmetric matrix. A real symmetric matrix is basically a symmetric matrix in which all elements belong to the space of real numbers R.
Useful Properties
Do it in Python and Numpy
Now, let’s do some practical work. I want to use Python and Numpy to compute eigenvalues and eigenvectors.
Conclusion
In this tutorial, you learned about decomposing a matrix to its constituent elements using its eigenvectors and eigenvalues. If I be honest with you, you may rarely need this concept in coding Machine Learning projects, BUT it does not mean it is NOT important! On the contrary, matrix decomposition is one of the most critical concepts in Linear Algebra, which is essential when you desire to dig into a Machine Learning problem. My goal is to cover whatever you may encounter in Machine Learning. If you want to know the applicable Linear Algebra in Machine Learning, trust me, you need to know matrix decomposition.
Originally published at https://www.machinelearningmindset.com/matrix-eigendecomposition/on December 17, 2019.
