Summarizing the Essence of linear algebra
In this post I will write down the understanding that I get out of Essence of linear algebra series. Videos are great but sometimes we need some notes to refresh our understanding real quick. I hope this helps out other people too.
Video 1 — Essence of linear algebra preview
Linear Algebra is poorly understood. Students understand what they are doing but not what the things represent. The geometrical interpretation helps people
- decide which tool to use
- feel why they work
- know how to interpret the results
- Important as in these days we use computers to do the calculations and humans are the ones deciding what to use
This series will focuses on the geometrical interpretation of linear algebra to help people get an intuition on linear algebra.
Video 2 — Vectors, what even are they?
Mathematicians believe that vectors can be anything as long as there is a sensible notion of
- Adding them
- Multiplying them
In linear algebra a vector
- almost always sits in a coordinate system
- has length and direction
- is rooted at the origin
Convention is to write vectors as a pair of vertical numbers in square brackets.
Each vector represents a movement in space. When we add vectors we add how much they move us in the directions represented by the numbers. That is why vector addition can be considered to be starting from tail of vector A, and then considering that tail of vector B sits there and moving along that.
Multiplying by a number means that you scale the length of the vector. That is why numbers are called a scalars. If the number is negative we are also reversing the direction of the vector.
Mathematicians keep to an abstract understanding because the real value of the 2 interpretations — graphical and numerical is the ease with which people can switch between them.
Video 3 — Linear combinations, span, and basis vectors
Think of the 2 numbers representing a vector as 2 scalars — squishing the vector in 2 directions — x and y. These are known as i^ and j^. These 2 scaled vectors are added to create any vector e.g. 2i^ + 3j^. So all vectors scale these 2 basis vectors.
The choice of these basis vectors is arbitrary. All possible coordinate systems can be translated into each other. Except when your basis vectors overlap or are zero.
The span of two vectors v and w is the set of all their linear combinations i.e. av + bw. This is basically a way of asking what are all the set of vectors that you can reach by just using these 2 basic operations — addition and multiplication.
If we are thinking about individual vectors think of them as lines. If thinking about collection of vectors it is easier to think of them as points with their top sitting at the point and tail at origin.
- Span of two vectors that line up is a line.
- Span of two vectors that do not line up is a 2D plane. If thought of in terms of 2D coordinate system it is the plane. If thought of in terms of 3D coordinate system it is one of the planes.
- Span of 3 vectors in a 3D coordinate system is the 3D plane. Unless if one of those is sitting on the span of other 2. In that case the span does not change.
- Changing the 3rd vector can be thought of as moving a plane a little bit and getting additional set of values.
When removing one of the vectors does not affect the span then we can say that the 3rd vector is linearly dependent on other 2 vectors. This is same linear combination that was talked about earlier.
When removing one of the vectors does affect the span then we say they are linearly independent vectors.
Video 4 — Linear Transformation and matrices
We are going to understand a way to understand matrix multiplication which does not rely on multiplication.
The “transformation” in linear transformation is a fancy way of saying function. We take in input and get an output. But we don’t call it a function because transformation is suggestive of movement. Think of vector moving from one place to another. To understand the transformation think of every point in space moving to another point in space.
Transformation are linear transformations if
- all lines remain lines without curving
- origin does not move
A easy way to think is transformations that keep grid lines parallel and evenly spaced. But how do we know where do the new vectors corresponding to the all of the original vectors in the plane land after the transformation? Just keep note of where the basis vectors land. As long as it is a linear transformation all of the new vectors are still the same linear combination of the basis vectors.
So the 2D transformation can be described in terms of just 4 numbers — the new coordinates of i^ and j^. It is common to package them up into a 2 x 2 matrix where the columns can be interpreted as where i^ and j^ each land.
Matrix vector multiplication is simply a way of telling what the transformation does to a vector.
Video 5 — Matrix Multiplication as composition
Sometimes we want to have a way to describe a sequence of linear transformations. We represent individual transformations as matrices and use matrix multiplication to represent where the basis vectors will land after transformation.
Video 6— 3D Linear transformations
This was mainly visuals to show how the transformations described so far in 2D work in 3D.
Video 7–The determinant
Many of the transformations stretch or squish space. To understand these transformations it is useful to measure how much a given area has changed.
If we focus our attention on the square that sits on the basis vectors and see how much it changes we can tell how much any area in the coordinate space changes. This is because grid lines remain parallel and evenly spaced in a linear transformation.
The factor by which the area is changed is called the determinant of the transformation. Determinant can be negative. It would mean we flip space. That is called invert the orientation of space. In case of 3D space it is still orientation specific.
Video 8 –Inverse matrices, column space and null space
Matrices are useful for disciplines of Computer graphics and Robotics. But the reason linear algebra is important for technical disciplines generally is because they let us solve a system of linear equations.
The linear equations can be thought of as a matrix vector product producing another vector. That makes it easier to think about it. We can find the unknowns by simply playing the transformation in reverse. This inverse is called inverse of matrix.
In case determinant is non-zero then it is almost certain that there is a unique solution.
In case determinant is zero then it is possible to have an inverse but that is not something that a function can give you as you will have to map the squished vectors (which is now single vector/plane) to multiple vectors while function can do one-to-one mapping.
In case after transformation all points are on a line then it is rank 1. If everything falls on a plane then it is rank 2.
The set of all possible outputs of vector after the transformation is called column space of matrix (the one doing the transformation). Basically where the basis vector can be after the transformation. This is same as span of the matrix.
If the rank of matrix is same as number of columns/basis vectors then it is called full rank.
If matrix is full rank then only origin falls on origin. If it not full rank then whole line/plane of vectors gets squished to origin. This is called the null space/kernel of the matrix.
Video 9 –Non square matrices as transformations between dimensions
If we have a 3 X 2 matrix then it can be interpreted as a plane cutting through 3D space. So a non-square matrix is a transformation that squishes or stretches between dimensions.
Video 10 — Dot products and duality
Dot product of 2 vectors is the sum of corresponding elements of the vector. Geometrically it is product of the length of 1 vector and length of projection of 2nd vector on the 1st one. Due to this interpretation it is easy to understand why perpendicular vectors have zero dot product.
Video 11 — Cross Products
Numerically cross product is the determinant when v and w are the first and second column of the matrix. Graphically it is the vector parallel to the parallelogram with length equal to the area of the parallelogram formed. The direction is determined by the order of the vectors as shown above.
I only got as far as Video 11. Will update the below whenever I watch further.
Video 12 — Cross products in the light of linear transformations
Video 13 — Change of basis
Video 14 — Eigenvectors and Eigenvalues
Video 15 —Abstract vector spaces
