What is Linear Algebra?
Prologue of MIT 18.06 linear algebra, spring 2005
Linear algebra is a pillar of the field of machine learning(ML), and many recommend it as a prerequisite subject before starting ML.
Before studying linear algebra, let’s figure out what linearity and algebra mean respectively.
Linearity
There are two conditions must be true for linearity: Homogeneity and Additivity. A system must be both homogeneity and additivity in order to be linear.
I am going to describe mathematically what those two things mean.
- Homogeneity
The first we’re interested in is homogeneity. Homogeneity can be described this way:
If there is a system f(x) and an input of the system is some constant k, an output is equal to some constant k times that of the system.
For example, there is a system f(x) = x. If you scale an input by a factor of 2x, it is same with the twice of the system. (2x = 2 × x)
This is the first condition that has to be met in order for a system to be linear.
- Additivity
Description of additivity is as follow:
If there is a system f(x) and an input of the system is addition of some constant a and some constant b, an output is equal to the sum of each constant entering the system.
Using the same system in the above example, f(1+2) and f(1) + f(2) are equal.(3 = 1+2)
Thus, linearity is a system in which an input is influenced by the system and an output is affected as much as it is. In other words, the system is “predictable”.
Algebra
Algebra is the part of mathematics in which letters and other general symbols are used to represent numbers and quantities in formulae and equations. (Source)
For instance, x and y are algebra and 3 is a coefficient in y = 3x.
