Matrices and vectors math for AI with Python examples
Read fixed and extended version of this article
Vectors and Matrices math is one of basic tools used for AI. There’s a lot you can do with vectors (and matrices, tensors and so on). Linear algebra is a good place to master vectors, but let’s take a look on popular basic stuff that can help in a lot of cases.
What are vectors and why we use them?
Vector is a point in (some) space, which we are “looking at” from (some) point (usually zero point). In simple 2-dimensional space, vectors look like:
We have 2 vectors here. Each is described by 2 coordinates — x and y. Vectors are usually written down as:
Vectors can have more than 2 dimensions. For example 3-dimensional vector will look like:
More dimensions will be harder to draw, but we basically can have any number of coordinates for vector:
Now let’s imagine, that we have a set of people that we want to analyze. Each person is described by several features - age, weight and height. We can, in the language of math, say that we have a set of vectors in 3-dimensional space:
So it’s easy to write down a list of observations (people in our case) with features (age, width, height in our case) as vectors:
And this means we can apply vector math to analyze our data. And vector math has a lot of powerful tools. But before looking at math operations, let’s discuss another structure, called matrix.
Matrices are also vectors
Matrix — is a set of vectors. Easy. Let’s imagine a vector, called M, which have V1 and V2 (two vectors from previous example) as coordinates:
This is called matrix. We can use another, more popular, form to write matrix down:
We say that this matrix has 2 dimensions — 3 colums and 2 rows, so the form of matrix is 3x2. Matrices can have any number of dimensions and be of any form. Example of 3-dimensional matrix 2x2x3:
So, 3-dimensional matrix is a vector of 2-dimensional matrices. And, as we remember 2-dimensional matrix — is a vector of vectors. So 3-dimensional matrix is a vector of vectors of vectors. And this means, everything comes down to vector math.
Let’s look at some basic vector/matrices operations.
Euclidean norm (distance)
Euclidean norm (or distance) is a way to calculate mathematical “length” of vector. It is calculated as:
In Python this can be calculated using Numpy package. Let’s find euclidean distance for a sample vector (1, 2, 3):
Adding vectors or matrices
In order to add 2 vectors, we have to add all corresponding elements of our vectors:
In Python you can just use standard + operator to add vectors:
Because, as we know, matrices are also vectors, in order to add 2 matrices we have to add each element of our matrices:
The same approach to add matrices in Python — just use + operator:
Which will give us exactly what we have calculated already:
Matrices multiplication
This is somewhat tricky. In order to multiply 2 matrices we have to calculate sum of row/column value products for each element of resulting matrix:
So each element of new matrix is a sum of products of corresponding elements of rows from left matrix and columns from right matrix. Note, that:
- You can only multiple matrices where number of left matrix columns is the same of number of right matrix rows.
- Resulting matrix can be of different form than source matrices (if they have different number of rows and columns).
There’s a special @ operator in Python to multiply matrices:
Which gives us:
Transpose matrix
Transposing matrix is a popular operation as well. To transpose a matrix we just have to change its columns to rows (and rows to columns). In other words — “mirror” it:
Each Numpy matrix in Python has .T property which returns transposed matrix:
Which prints transposed a matrix:
Matrix determinant
Matrix determinant is a tool to research system of equations based on our matrix. It’s calculated in multiple iterations as:
So we have to iterate down to 2x2 matrices from our bigger matrix. For 2x2 matrix determinant is simply calculated as:
In Python, we can use lingalg component to calculate matrix determinant:
Matrix rank
Matrix rank is the maximum number of linearly independent columns (or rows) of matrix and can be found in Python:
Summary
You can describe your data as vectors (matrices) and then apply math operations to analyze that data or prepare it for further processing. Many AI algorythms are based on vector/matrices operations.
