Linear Regression & Logistic Regression: Basics
Yes, we will be learning these first. Of course, we will be doing that together. Step by step.
But then you might ask me, “Why to write one? We already have thousands of articles”.
True. You do have. But here are the reasons:
You remember something when you share. I care. I share. I gain.
I’m done with my course. I am looking for a job. I learn other stuffs. I binge watch on Netflix and Prime. And I still got time! What else ?— here I am :)
To motivate you. Service to the society. On that note, I believe if I can understand you too can.
My profile would look good.
Enough of the talk. Okay so let’s learn everything from the scratch.
Linear Regression
A machine learning algorithm of course. Simple and widely used. It has in its name: regression.
Regression is nothing but a statistical term that forms a relationship between a dependent variable and a set of independent variables. When the independent variable is one, we call it Linear Regression.
Basically, for the machines, it will allow you to predict the outcome based on the independent variable.
What else?
It’s a supervised machine learning algorithm. Now, what the heck is supervised? Let’s talk about it some other day. For now, you just know that it’s a popular machine learning type that produces a model. This model would then predict the outcome based on the training it received on some labelled inputs. (Did it go over your head? Well, don’t worry! We will cover that in a few days).
What’s with the equation?
Both are same. Basically it says, for every change in x there is a change in y. And that is because y is dependent on x.
This is shown by m or B1 (blue line in the figure). It is also called the slope. Change in ‘x’ results in the change in ‘y’. You must be thinking then what is c or B0. Well, it is the value of ‘y’ when ‘x’ is 0. It is called the y-intercept. When plotted on a graph, if ‘c’ is 0 the line then passes through the origin. Else it would shift accordingly above or below the x-axis (In the above figure, the y-intercept is 1).
When there are multiple independent variables, it is called multiple linear regression.
That was pretty easy, right? Yeah, I know. Let’s move on to the next one:
Logistic Regression
When the dependent variable is categorical, logistic regression is performed. If that dependent variable takes two values, it is called Binomial Logistic Regression.
There are other types too:
Multinomial Logistic Regression — If there are 3 or more nominal categories e.g. Monsoon, Winter, Summer etc.
Ordinal Logistic Regression — If there are 3 or more ordinal categories e.g. Product ratings — 1 to 5.
Okay, we are done. Wait, What?? What about the performance? I have heard something called a cost function? Overfitting? Underfitting? Variance bias? You’ve got to be kidding.
Hold on buddy, that’s too much for one day. I’m not going anywhere. All these will be covered in the coming chapters. Didn’t I say earlier, step by step!
Updates:
Performance of Linear Regression Models : https://medium.com/@uk1992.me/regression-and-performance-metrics-accuracy-precision-rmse-and-what-not-223348cfcafe
