Scalars, Vectors and Matrices, oh my!

Day 3 and 4 was spent studying the foundational and math side of machine learning. Machine learning can be applied without mathematics. However, applying machine learning is more effective knowing what these systems are doing under the hood. It helps to work through introductory books to gain an understanding of the foundational side of machine learning.

On day day 3 and 4, I read Deep Learning Chapter 2: Linear Algebra written by Ian Goodfellow. Linear Algebra is everywhere in machine learning and can be seen in the basic materials. It’s widely used throughout engineering and science. Linear algebra is used to frame optimization algorithms and solving linear systems of constraints. So this blog post is my notes on topics needed to understand what will come next.

scalars, Vectors and Matrices

• A scalar is a single number
• A vector is an array of numbers.
• A matrix is a 2-D array
• A tensor is A tensor is a n-dimensional array with n >2
• scalars are written in lowercase and italics. For instance: n
• vectors are written in lowercase, italics and bold type. For instance: x
• matrices are written in uppercase, italics and bold. For instance: X

Code example

Matrix operations can be applied to linear algebra objects. These operations are addition, subtraction, transpose, multiplication, and inverse.

Addition : adding 2 vectors requires them to have the same number of elements and equal shape.

Subtraction: subtracting 2 vectors requires them to have the same number of elements and equal shape.

Transpose: Transpose of a Matrix is an operator which flips a matrix over its main diagonal like a mirror image. This can be done by calling numpy.transpose function or T method in numpy.

Multiplication: NumPy uses numpy.dot function for multiplication of both vectors and matrices. Matrix multiplication is not commutative.

Inverse: Inverse operation only applies to square matrices(matrix with same number of columns and rows). To compute inverse using numpy we need to use numpy.linalg.inv function.

Transpose: converting a row vector to a column vector and vice versa. It’s basically a mirror image. This can be done by calling numpy.transpose function or T method in numpy.

Summary

In this post, I discussed mathematical objects of Linear ALgebra that is used in ML. I briefly discussed addition, subtraction, transpose, multiplication, and inverse. Furthermore, you have learned important properties of matrices and how we can enable efficient computations. Day three and four of 100 day ML code challenge was spent reviewing the basic foundations. In the next few days I plan to complete Part I: Applied Math and Machine Learning Basics and document my progress.

Final Words

It was fun discussing part I of Linear Algebra. Stay tune for Part II where I discuss types of matrices. If any errors are found, please email me at dskswu@gmail.com. Meanwhile follow me on my twitter here. The log for journey can be found here.

The Wiz

Resources

@book{Goodfellow-et-al-2016,
    title={Deep Learning},
    author={Ian Goodfellow and Yoshua Bengio and Aaron Courville},
    publisher={MIT Press},
    note={\url{http://www.deeplearningbook.org}},
    year={2016}
}

Artificial Inteligence · GitBook