Linear Algebra 101 — Part 2

Sho Nakagome
Sep 1, 2018 · 6 min read

Before stepping into this article, please note that these stories are a series of “Towards understanding Linear Algebra” and I advise you to take a look at Part 1 first.

Like I wrote on Part 1, I believe it’s important that you don’t rely on just one source when studying any materials but rather use it as a starting point or a guide. Use multiple sources of information, articles, videos, people until you fully understand the concepts.

Alright, let’s begin!

Materials covered in Part 2:

In this article, I’m going to cover important fundamental concepts in Linear Algebra such as:

If you know all the above contents, you are good to go. I have a summary of these at the bottom for you to recap these key concepts quickly.

Recapping linear combinations

I talked a little about linear combinations in Part 1, but let’s make this concept stick.

Linear combination is any combination that can be expressed with

1) multiplying with constants

2) adding together

3) the combinations of 1 and 2

For example, let’s say you have 2 column vectors v and w. The linear combinations of the 2 vectors are something like the following.

It could be any combinations as long as they are linearly combined together. In other words, you are not transforming the vectors such as multiplying the vectors together. If you want to know a little bit more, check the link below.

Introduction to row and column space

So far, we studied vector space and subspace. Let’s talk about column and row space.

By now, you should know what’s rows and columns.

Long story short,

Let’s take a look at an easy example. Suppose you had a matrix like the one below. If that’s the case, we have 2 column vectors. The column space represented by such vectors are shown as below. It’s like a 45 degrees tilted plane on x-axis.

Same goes for the row space. I embedded a link to the wiki regarding the topic, so feel free to jump off and take a look below.

Introduction to nullspace

Now comes a very important concept in Linear Algebra called nullspace! Nullspace is a subspace represented by the following equation:

Notice that we are not solving the quadratic equations in the end? Let’s try that and see what happens.

This leads us nowhere. We want answers with non-zeros to understand what kind of x we are looking here.

So by choosing the arbitrary numbers, we have 2 resulting vectors. This will create a nullspace like the one below. Since we are looking at a 2D space, and there’s two vectors, the nullspace is basically the whole plane covering the 2D plane.

To summarize, nullspace is a subspace represented by the set of all vectors x that satisfies Ax = 0 (Given that A is a matrix, 0 is a zero vector).

Introduction to rank

Suppose you have a matrix like the one below.

Rank of a matrix is equivalent to the number of pivots. So let’s perform row elimination to find out pivots.

In this matrix, we have 2 pivots. The rank is the same as the number of pivots. Therefore, we have rank(A) = 2.

However, did you notice that the matrix A has 3 columns? So we have the matrix rank = 2, and the matrix dimension = 3. Let’s see where this leads us. First, we convert the row echelon form to reduced row echelon form like what we did in Part 1.

And let’s try to solve this based on the nullspace equation: Ax = 0

Now you can see there’s something odd here. Dimensionality of the matrix tells us how many variables x we need to identify, but since the rank of the matrix is 2, which is 1 less than the dimension, we only have 2 equations for 3 variables. If you remember back in the old days when you were solving quadratic equations, you know that you can’t solve the equations because we just don’t have enough information.

Let me give you another example to prove this point. Let’s think of another matrix that has the dimension of 3 and the rank of 3.

So now by looking at this comparison, I hope you understand that there’s no way to figure out the exact x_2 on the left example where the rank is 2 and 1 less than the dimension.

One last concept today is about free variables. In the left example above, we have no way to figure out the exact x_2 and if that’s the case, we can literally assign any values. It could be 1, 0.5, or 92842, etc. Usually we assign the most basic ones like 1. We call these variables, free variables. Important thing to remember is that the number of free variables corresponds to the dimension minus the rank. In the example above, we had dimension = 3 and rank = 2 which gives us 3–2= 1. That’s why we only had 1 free variable in the example.

Like I said in the Part 1, feel free to google and search for other explanations and materials if you still feel like you are missing something or the concepts are still hanging over your head. I’m trying to give you an initial trigger to know the material. To understand the concept, you need to study, process the information, until you get the concept right.

Summary

Let’s summarize what we have learned in this article:

Any combinations of 1) multiplying with constants and 2) summing the vectors are called linear combinations.

A subspace represented by a linear combinations of column vectors or row vectors.

A subspace represented by all the linear combinations of x, where x satisfies: Ax = 0.

The rank of a matrix is equivalent to the number of pivots of the matrix.

The sum of the number of pivots and free variables equals to the dimension of the matrix. In other words, the number of free variables = dimension of a matrix minus the rank.

I hope this helps! See you next time!

sho.jp

Let's make the cyberbrain system from Ghost in the Shell

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store