This is the pt.1 of a series which will guide you through all the linear models of machine learning, from the math theory to writing the algorithm from scratch in Python. You can read the linear models in pt.2!
Here we’ll just make a small recap of high-school math to introduce what is a linear function. Why is it useful? Well, because:
A linear model is nothing more than a linear function whose parameters are estimated from data using a loss function through the gradient descent algorithm.
The juicy linear algebra and calculus will come soon, don’t worry ;)
Now, let’s consider a line: it’s a linear function that maps the value of x to that of y, and it’s usually written as:
Where:
- y is the dependent variable
- x is the independent variable
- m is the slope
- b is the intercept
It is named linear simply because it can be plotted as a straight line. Let’s take a look at some linear functions for some visual insight!
Formally, a linear function is both a polynomial function of degree zero or one and a linear map. But what does it mean?
A polynomial function is just a function with multiple (=poly) terms (=nomia), written in the form:
Let’s take a look at the 2 possibilities:
- 0th degree:
- 1st degree:
The degree of the polynomial function is just given by the highest exponent in the function. Let’s take a look at the graphs below for some visual insight!
From these graphs it’s clear why we only the first-degree polynomial function is the one used, and why it’s named linear! Indeed, a 0th degree function is pretty useless since it maps all independent values to a constant, while a function of 2nd or higher degree is no more linear.
Now, a map… the map is just a function which “maps” the input values to the output. A linear map is just a particular map which preserves some properties of the vector space. For a simple visual intuition, imagine a function that transforms all the points of a grid in a cartesian plane without curving the lines and keeping the origin of the axis in the same place (but don’t stress too much. This isn’t actually important to our discussion. It’s just a bit of context).
Do you have questions? Something is not clear? Did you spot an error? :’) Any suggestion? Just reach me out: https://linktr.ee/voidpunk
You can find the whole code of this series as a Jupyter notebook on my GitHub: https://github.com/voidpunk/ML-Notes :)
That’s all for this very short and easy starter, in the next one we’ll dive into the horrific definitions and math of simple linear regression, multiple linear regression, and multivariate linear regression (jk, it’s actually pretty easy math if you know how to look at it :P). Check it out here: >>>pt.2 <<<
