Vector Norm in Machine Learning — An Introduction
“What is the vector norm, and why on earth do I need to know that?” Let’s assume you start with asking that question by seeing the glass half empty instead of half-full. So I am here to convince you. If this introduction could not convince you, then simply do not proceed as it would be a waste of time!
In Machine Learning, we are dealing with evaluations all the time. It is needless to say it usually involves Linear Algebra, vectors, and matrices. In evaluating an element, such as loss functions, often, you need to summarize all you need in one number. Well, you use mean, standard deviation, etc. BUT, how do you deal with vectors? How would you address the importance of a vector in terms of its elements? How would you evaluate the vector elements in terms of their contributions to the outcome? We need some metrics to get a sense of the magnitude of a vector to see how much it affects the algorithm optimization, evaluation performance, etc. Here are the norms that come to play.
In this article, I will explain the vector norms. You will realize why they are essential, and what is their interpretation? Furthermore, you will learn how to implement them. I start with the definitions and ends it with the matrix norms. By the end of this article, you should know:
- What is the definition of the norm?
- The norms’ properties
- Mostly used vector norms
- Definition of the matrix norm
- How to implement them in Python?
What is a Vector Norm?
The Norm Function Properties
Proving the Properties (Advanced)
Mostly Used Norms
In the previous section, I described what is the norm in general, and we implement it in Python. Here, I would like to discuss the norms that are mostly used in Machine Learning.
Norm of a Matrix
Conclusion
In this article, you learned what the norms are and how to implement them. I wish if I could tell you this is the end and that’s all you need! BUT, no! Learning is an active process. You will see many many tutorials and concepts regarding the vector norms. Here, I tried to address what I believed you would see more frequently. There is always MORE! You can start with what I just provided. Make sure to play with Python codes. I assure you it will help you to gain a better understanding. I hope you find it useful. Please don’t forget to share your thoughts with me.
