MIT 18.06 Linear Algebra- Gilbert strang- Lec1 notes

Thanks to Gilbert Strang for awesome series of lectures and MIT OCW for recording and making course accessible to everyone.

Lecture covers the following:

A) Goal of linear algebra
B)Row picture and Column picture
C) Why two perspectives- Row and Column ?
D)Can the set of linear equations be always solvable?
E) Multiplying matrix by a vector

A) Goal of linear algebra:

It is to solve a system of linear equations. For example : (2 equations and 2 unknowns) or (3 equations and 2 unknowns)

Lets consider an example below to be solved,

Eq1 : 2x - y = 0 ,Eq 2: -x+2y =3

we will try to solve above system by looking at it in two ways described below

B) Row picture and column picture:

Row picture : Most of us view equations in this perspective, drawing lines in x-y plane

For the Eq1 the , line can be formed by joining any two points satisfying the equation — (0,0),(1,2)and similarly for Eq 2 — (-3,0) &(1,2)

Eq1 — Red line, Eq2 — Green line met at a point (1,2)

As we can observe from graph two equations meet at a point & that point satisfies both the equations & is the solution for the current example

Column picture : This could be a new perspective for most of us, instead of looking at one equation at once lets try to look at them together vertically as shown below

Column picture

We want find the right numbers that multiply the Cyan coloured vectors to produce the sky-blue coloured [0,3] vector. This is column picture. Here is how we can visualize the vectors:

We need to find the right combination of blue and red vectors to create black vector

C) Why two perspectives- Row and Column ?

While it is easy to visualize the lines in when there are two dimensions its get really difficult to visualize when the dimensions get large. Following example will make it clear:

2x-y = 0 , -x+2y-z=-1 , -3y+4z=4

Row picture: We need to visualize the planes interacting with each other in x-y-z plane and meet at a point that solves all the three equations
Column picture: Whereas here we need to visualize three vectors that need to be combined in right way to generate [0,-1,4] vector.
We need to find the right combination of these 3 column vectors
Combining the vectors produces the right correct hand side.In current case it is 0,0,1

In column picture task is to figure out what is the right combination of vectors on left side of equation(in Cyan colour) generates the right hand side vector(in sky-blue in image above)

If we move to 4 dimensions it is the same, we need to visualize what is the right combination of vectors in 4 dimensional plane that generates the correct right hand side, but it is very difficult to do the visualize same in row picture — 3 such 4 dimensional planes interacting with each other and meeting at a point
Also we will be able to retain the same column space for the different right hand sides, it is just that they need to be combined in a different way

D) Can the set of linear equations be always solvable:

Lets represent the system of linear equations to be solved with Ax=b

Here A represent the matrix representation of the values on LHS and x is the solution set and b is the right hand side

Example : Eq1 : 2x-y = 0 ,Eq 2: -x+2y =3

Above equations can be represented as:

Ax=b representation of the equations

We are trying to find if vectors in the columns of matrix A can be combined by two numbers x, y to produce the b

Can we combine the two vectors above to produce any right hand side b?

Answer is yes , since the vectors are pointing in different directions we can combine them to fill up the entire 2- dimensional space, hence in this case the answer is yes

Lets consider another example:

2x+3y = 4, 4x+6y = 2 ; does this system of equations have a solution?
We have both the column vectors lying on the same blue line(red arrow mark [2,4],blue[4,6])

As we can observe from the plot , both the column vectors lie on the same line, and any combination of them lies on the same line. So we can only produce only those right hand sides that lie on the blue line.

In this case the combination of vectors on left hand side fills up the space on blue line through the origin. One of the inferences — the second vector doesn’t provide any new information, this way if one of the column is combination of other columns then combination of them wouldn’t be able to fill the entire space and thus wont be able to solve for all the b’s

Lets look at another example in 3 dimensions:

Can we solve this system of equations?

In the example above the 3rd column is combination of first 2 columns in matrix A. Forget about solving for given b, but lets think of a bigger picture,

Can we solve for all right hand side b?
No, we cannot. Third column is combination of first 2 and hence it lies in the same plane as first 2 are. (similar to the vectors lying in the same line in the 2 dim space in the plot above) . We will be able to solve for only those b’s that lie in the plane formed by the combination of 2 vectors.
Hence we will be able to solve only for the vectors lying in 2 dimensional plane inside the 3 dimensional space.
Can we think of the 9 dimensional vectors combining to produce some right hand side b, and if one of those vectors in left hand side is combination of all or some of the others , then we can only solve for the right hand sides that lie in that 8 dimensional space inside the 9 dimensions. This is how we can scale this concept of column way of looking at matrix

E) Multiplying matrix by a vector:

Lets consider the following example

How do we perform Ax?

Most of us do 1st row times col of x and 2nd row times col of x, which is correct.But can we get used to the column way of doing it?

Column way of looking at matrix — vector multiplication

End Notes: Ax is the linear combination of columns of A