Mathematics for Machine Learning
Part — 1: Matrices and Matrix Operations
From the last few months, I’m studying Machine Learning (ML) and I think some basic mathematics is required to understand the concepts of ML and then you’re good to go. In this blog, I’ll be discussing matrix and I’ll try to cover most of the topics which are required for ML. In the coming days, I’ll be writing a few more blogs on linear algebra and calculus
Introduction
Matrix is a rectangular array of numbers. The above matrix has m rows and n columns. Each element of this matrix can be represented by a_ij where i = 1,2,3, ….m and j = 1,2,3, …n. We can say that the above matrix has an order of (m×n). If a matrix has only one column then it is called column-vector and if it has only one row then it is called a row-vector. So the order of column-vector will be (m×1) and the order of row-vector will be (1×n).
Special Types of Matrix
Equal Matrices
Two matrices A and B can be called equal matrices if the order of both matrices is the same and corresponding elements are equal. e.g. Let A = [a_ij], B = [b_ij] order of both matrices is (m×n) and a_ij = b_ij ; ∀ i ,j.
Zero Matrix
A matrix whose all entries are zeros is called zero-matrix.
Square Matrix
A matrix that has equal number of rows and columns is called a square matrix. In general order of a square matrix is represented by only one number instead of two numbers, e.g. Let A = [a_ij] be a square matrix and this matrix has an order on n which means the order of (n×n). a_11, a_22, a_33 …. are called the diagonal elements of a matrix (i.e. all elements a_ij where i = j are diagonal elements of matrix). These elements are also called primary diagonal elements. But I’ll be calling them diagonal elements only.
- Diagonal Matrix: A matrix whose all elements are zero except the diagonal elements. Such a matrix is called a diagonal matrix. e.g. If a_ij = 0 for i≠j.
Note: A matrix to be a diagonal matrix, it must have at least one non-zero element in diagonal elements and non-diagonal elements must be zero.
- Identity Matrix: A matrix whose all diagonal elements are 1 is called an identity matrix i.e. a_ij = 1 where i=j and a_ij = 0 where i≠j.
Triangular Matrix: A matrix that is upper triangular or lower triangular matrix is called a triangular matrix.
- Upper Triangular Matrix: A matrix whose all elements below diagonal elements are zeros (i.e. a_ij = 0 for i > j).
- Lower Triangular Matrix: A matrix whose all elements below diagonal elements are zeros (i.e. a_ij = 0 for i <j).
Operations on Matrix/Matrices
Transpose of a Matrix
Transpose of a matrix is defined when all entries of the matrix are real numbers. Let A = [a_ij] be a matrix then transpose of this matrix will be B = [b_ij] where b_ij = a_ji; ∀ i ,j.
Additions of Matrices
Matrices can be added only if they have the same order. Let A = [a_ij] and B = [b_ij] are two matrices, if we want to these two matrices then these matrices must have the same order and their addition will be C = A+ B = [c_ij] where c_ij = a_ij + b_ij; ∀ i ,j.
Multiplying scalar to a Matrix
Scalar multiplication to a matrix is multiplying each element by the scalar. If A = [a_ij] is a matrix then k*A = [k*a_ij]; ∀ i ,j.
Matrix Multiplication
Let A = [a_ij] and B = [b_ij] be two matrices. Multiplication of two matrices is denoted by AB which is also a matrix C = [c_ij].
Note: Multiplication of A and B is only possible if and only if number of columns in A= number of rows in B
If the order of matrix A is (m×n) then the order of matrix B will be (n×r). Here m and r can be any positive integer number. The order of matrix C will be (m×r).
Inverse of a Matrix
A square matrix A will have inverse if there exists a matrix B such that AB = BA = I (where I is denoting the identity matrix). Here B is called inverse of A. Order of A, B and I matrices is the same. But the inverse of a matrix (if exists) is unique. So we can write:
Let A and B be invertible matrices then
Some More Special Matrices
Let A be a square matrix with each element as a real number.
Symmetric Matrix
If transpose of A is equal to A then A is a symmetric matrix.
Skew-Symmetric Matrix
If transpose of A is equal to -A then A is a skew-symmetric matrix.
Orthogonal Matrix
If multiplication of A and transpose of A is equal to multiplication of transpose of A and A equal to an identity matrix I.
Transpose of an orthogonal matrix is the inverse of the orthogonal matrix. i.e. transpose of A is equal to the inverse of A if A is an orthogonal matrix.
Conclusion
In this blog, fundamental of matrices has been discussed. The concepts that have been discussed here are very important to understand other topics that are related to matrices like eigenvalues, eigenvectors, etc. To know more about matrices read this blog also.
