Part 15 : Orthogonality and four fundamental subspaces

Two vectors are said to be orthogonal if they are subtended on a right angle.

x and y are orthogonal to each other

Also, for two orthogonal vectors x and y

Transpose of x multiplied with y yields 0 as product

Their dot product also 0. As, cosine of 90 degrees is equal to 0.

Orthogonal Subspaces

If subspace U is orthogonal to subspace T, then every vector in S should be orthogonal to every vector in T.

Suppose, S = {s1, s2, s3, s4……..sn} and T = {t1, t2, t3, t4, ….. tn}

Then

Every possible pair of vectors from both subspaces are orthogonal to each other

Subspaces S and T will be called orthogonal complements and that could be represented as a figure

Orthogonality of Four Fundamental Subspaces

The four fundamental subspaces are :

1. Row Space
2. Column Space
3. Null Space
4. Null Space of transpose (Left Null Space)

Row Space and Null Space

Row Space and Null Space are orthogonal complements i.e. transpose of any vector in row space multiplied with any vector in null space will give 0 as product.

Proof

Consider a matrix A (of order m×n) having a solution matrix (Null Space) x (of order n×1). We can recall from Gaussian Elimination and definition of null space, that

Null space is a matrix, which on multiplication with A gives zero matrix as product.

m row vectors of A multiplied with n elements of x yield Zero vector as product

We can see that every row vector of A is orthogonal to every vector in null space (as the result is a zero matrix).

This could be represented as a figure

Column Space and Null Space of transpose

Taking transpose of the matrix A, we will get another matrix Aᵀ.

Suppose, the null space of transposed matrix is y (another matrix).

Such that

taking transpose of complete equation we get

The order of multiplication also change on taking transpose

y is called left null space of matrix A because it behaves like a null space but it is on the left side.

Expanding the transposed equation, we get

n column vectors of A (transposed) multiplied with m elements of y yield Zero vector as product

From the second equation, we can conclude that Column space of any matrix is orthogonal to left null space of that matrix.

This could be represented as

Orthogonality of Zero vector

Zero vector is orthogonal to every subspace.

Read Part 16 : Dimension and Basis

You can view the complete series here
I’m now publishing at avni.sh

Connect with me on LinkedIn.