Why do we always have to compute the determinant

Since we have already covered the hardcore of Linear Algebra when we talked about Singular Value Decomposition, I have decided that it may be a good idea to take a step back and discuss what I assumed the reader already knows — the determinant. Reminder, we care about matrices because they are essentially tables of data, and data is the elephant on whose back the modern world is resting. But why would we care about det(A)? What does it represent?

Quick note on computing

Computing the determinant is pretty simple, especially if we have a 2x2 matrix.

Given the matrix above, to compute its determinant we just need to take the difference of the products of its diagonals: det(A) = ad-bc. It is a little trickier if we have a 3x3 matrix, but also very doable.

The exact formula for computing the determinant of a 3x3 matrix is given by:

But here is how to think about it without actually memorizing the formula. You write down all the entries of the first row (a, b, c), and after that you close the first column and first row with your finger, compute the determinant of the remaining 2x2 matrix and put the value next to a. Then, you close the second column and the first row with your fingers, compute the determinant of the remaining 2x2 matrix and put the value next to b. Repeat with the third column and the first row. The only thing left to do is to put alternating signs: start with positive before a, negative before b, and positive again before c.

This also works with rows. You can write down all the entries of the first column, then close the first column with your finger and start by looking at the remaining matrices after closing the first row, second row, etc. It is a consequence of the first property of the determinant: det(A)=det(A-transpose).

So, why determinant?

Matrices essentially represent linear transformations of the unit vectors (those that only contain zeros and ones). If we had a vector [0,1], it means that it points vertically and its length is equal to 1. But if we multiply this vector by 2, it will turn into [0, 2], and will now be still pointing vertically, but its length would be 2.

The determinant works pretty much in the same manner: it is a scaling factor by which a linear transformation changes any area (for 2D) or volume (for 3D).

If you look at the picture on top of this post (again, sorry for my poor artistic skills), on the left we see a 1x1 square, which is a matrix of unit vectors. One points in the x-direction and has a magnitude of 1 (1, 0), and the other points in the y-direction and has a magnitude of 1 (0, 1). This is the starting point. A linear transformation associated with the matrix ([2,0][0,3]) means that we stretch the first vector by the factor of 2 and the second vector by the factor of 3. If we compute the determinant of this matrix, (2*3–0*0), we get 6, which is exactly the area of the resulting rectangle.

What happens if the determinant is zero? In 2D case, when we only have a 2x2 matrix, it means that the resulting figure after the linear transformation has the area of zero. It is probably just a flat line (I say probably because it also may be a point). If it is a 3x3 matrix, then the resulting figure after the linear transformation has the volume of zero: it can be a flat surface, a line, or even a point.

Matrices with zero determinants are non-invertible, which opens the next chapter of the linear algebra discussion.