Linear Algebra Next Stop: Vector Space

So I was studying this and it started to get difficult when you reach eigenvalues and eigenvectors (more on that on my upcoming blogs).

Vector Space:

If you have read about vectors then they follow two of the most basic properties 1. Additive (u+v) and 2. Multiplicative (cu) where u and v are vectors and c is a scalar(any number). Also there are eight more properties that vectors follow the above basic properties. Remember the previous post we were to solve linear equation Ax=b format. We found the solution in one dimension. Here we need to find in 2d, 3d and higher dimensions, we call it R1, R2,R3,………Rn. So the concept of vector space comes is how do we find the solution of a equation Ax=b for higher dimensions. What are the vectors in higher dimensions that satisfy the equation meaning they are in the vector space. The vectors will be number of column vectors (combination of vectors in that dimension) R1 one vector, R2 two vectors, Rn n vectors(components or linear combination of the vectors). So that it will always satisfy the above property. I think I already stated that.

Lets say a vector passing through origin (0,0). This vector will always satisfy Ax=b no matter any dimension. So an origin vector(0,0) is always in the vector space of Rn dimensional matrix for n =2. Here “b” becomes 0 as x passes through origin.

Vector Subspace:

A subspace is a closed set of values within a vector space. So if the scalar value is 0 and vector is in the subspace of the matrix A ,so we can say the smallest subspace (lets call it Z) for matrix of Rn dimension is 0 and satisfies the property 0+0 =0 and c*0 =0 where c is a scalar. So for 3D plane (R3) one of the vector space is the origin vector(0,0,0) and any line (lets say vector x ) passing through the origin and the line is on the plane is also within the subspace of R3.

3D plane

Column Space:

Column space is the linear combination of all the columns of A under subspace of any dimension of Rn. Considering a m*n matrix “A” (I am sorry I am not able to put examples here). By multiplying A with vector x we find all the combinations of A so that we find individual solution b1,b2,b3,b3. If b vector(b1,b2,b3,b4 =0) then Ax =0. Find the vectors x which will give this solution.

For matrix A
Finding combinations of A and x to give b vector. (Please forgive my handwriting)
Possible values of b so the equation satisfies

Null Space:

Null space of N(A) is trying solve Ax=0. Finding all the vectors of x so that the equation satisfies.Lets say for the matrix A

in the picture there are several solutions of x that will prove the equation.First is (0,0,0) vector. Try to find other solutions. After some calculation we can say every multiple of unit vector (1,1,-1) satisfies the equation.

Similarly null space for matrix of higher dimensions can be solved and all of them will follow the u+v and cv property of vectors.

The remaining properties that vectors follow are:

Here x,y,z are vectors and F is any dimension matrix

1.  Closure of addition: if x, y are in F, then x+y is in F.
2.  Closure of multiplication: if x, y are in F, then xy is in F.
3.  Associative Law of Addition: if x, y, z are in F, then (x+y)+z=x+(y+z)
4.  Associative Law of Multiplication: if x, y, z are in F, then (xy)z=x(yz)
5.  Distributive Law: if x, y, z are in F, then x(y+z)=xy+yz
6.  Existence of 0: an element of F satisfying x+0=x for all x in F
7.  Existence of 1: an element of F satisfying x.1=x for all x in F
8.  Existence of additive inverses (negatives):If x is in F, there exists y in F such that x+y=0
9.  Existence of multiplicative inverses (reciprocals), except for:  If x is in F is not the zero element, then there exists an element y in F such that xy=1.
10. Commutative Law of Addition: If x, y are in F, then x+y=y+x
11. Commutative Law of Multiplication: If x, y are in F, then xy=yx.

Solving equations of type Ax=0 and Ax=b

If you have solved some equations using Gaussian elimination you may be aware of pivot values (values that cannot be further divided in a matrix).

The matrix above will give some solution like this. So we find the pivot values for m*n matrix here 4*3, the pivot value is called the “Rank” of the matrix.

Here “C” is the scalar that can be multiplied with the vector are possible solution. (x1,x2,x3,x4) are members of matrix “x”

The last form of matrix that will come after elimination is called Reduced Row Echelon Form (rref).

From that every solution of the matrix can be found out.I may not be able to show the solution here(even with the screen shots). So the rref form of the equation and all its linear combination will give the solution to Ax=B. Follow the problem set of MIT open course ware (18.06) for detailed info.


I learnt a good application for this concept. Coding Theory

Satellites transmit a lot of data from outer space, but data is filled with noise so to encode the message in such a way that when its decoded on earth the noise factor is reduced. For that same data is repeatedly sent twice or thrice but we don’t have enough space to keep copies of data. So we follow the approach of error detecting and correcting.

Most of the data is binary format (still as its converted to binary even though we send data in text). We use Hamming code for error detection. Hamming code is itself an topic of discussion. May for the next blog.

For detailed process on usage of Hamming code and linear algebra. Click here.

Thank you for reading this mathematical post.I will be adding more concepts soon.