What is Regression? — Samur.AI
Regression? That’s isn’t a common term!
Here’s the definition from Wikipedia.
In statistical modeling, regression analysis is a set of statistical processes for estimating the relationships among variables. It includes many techniques for modeling and analyzing several variables, when the focus is on the relationship between a dependent variable and one or more independent variables (or ‘predictors’).
Here’s a simple version:
Regression (or regression analysis) is a way of determining the relationship between an output(dependent) variable and one or more input(independent) variable(s).
To get a visual understanding, check out this other article here.
Common regression models include:
- Simple Linear Regression
- Multiple Linear Regression
- Polynomial Regression
- Logistic Regression
Don’t Worry. It’s easier than it sounds.
Simple Linear Regression
As it seems, it is really simple. In this type of regression, we have only one input variable(‘x’) and one output variable(‘y’). Simple! The only constant you need determine is the slope through a process called gradient descent. The result is a perfectly straight line. More on that later.
For reference, the equation below represents the solution to a simple linear regression problem.
y = w*x +b
Multiple Linear Regression
This is just an upgraded version of simple linear regression. Instead of one independent(input) variable, there are multiple independent variables. That’s’ the only difference. In case you are wondering what the math looks like, here it is:
y = w1*x1 +w2*x2 + w3*x3 + ….. + wn*xn + b
Where the x’s are input(independent) variables, w’s are the associated weights, y is the output(dependent) variable and b is the bias.
Couple of things to note here.
First, there are multiple input variables and therefore, multiple weights for each input variable. The weights are usually different.
Second, as before, there is only one bias term. This is because the above equation is for only one row (example) of the dataset. Thus, the bias term contributes to the current result only.
The flexibility with multiple linear regression is that almost any number of variables can be fed to the algorithm resulting in a single output. This too produces a plane in three dimensions.
For simplicity, we only visualize a case in three dimensions as human brains find extremely difficult to visualize higher dimensions.
NB: Dimension is a synonym to the number of variables.
But what if the data is not linear?
This is where Polynomial Regression comes in.
Polynomial Regression
Observe the picture below:
Yes, you guessed it right! It’s not a straight line anymore.
The mathematical equations of polynomial regression are something of the following form:
y = w1*x1² + w2*x2³ + w3*x3⁴ + … + wn*xn + b
Please note that the above equation is just an example. The exponents of the variables are subject to differ depending on the problem at hand.
If you still find it difficult to understand, look at the picture below.
I am pretty sure you will be able to recognize it.
y = x²
You see, the dependent variable ‘y’ maps to ‘x²’. Thus, you have the graph as above.
Similarly, different equations have different graphs for the same.
Another cool picture coming up for higher dimensional datasets.
Cool right?
Next, we come to Logistic Regression. But wait! The topic of Logistic regression requires a separate article of its own because it is a topic of such vast and vital importance. Therefore, it will be covered in the next article.
Summary
In this article, you learned about the different types of regression problems and models and obtained an intuitive idea about how the models differ from each other. Next, we will learn about logistic regression and later, we will learn about how gradient descent works, which is the heart of machine learning and deep learning problems. Until then,
If you like this article, consider buying me a coffee :)
