Why “Regression” is present in Logistic Regression?

LINEAR REGRESSION vs LOGISTIC REGRESSION (source)

Most of us know that Logistic Regression is a Classification Algorithm! But why does it have the word ‘regression’ in it?

Let’s start with the basics.

CLASSFICATION

It is a process of putting things/items into a category. When you a fruit bowl. You can separate all the fruits by categorizing them by name, type, or size.

Classification can be binary or multi-class. Binary Classification is where there exist only two categories. Multi-Class is where there exist more than two categories.

BINARY CLASSIFICATION

REGRESSION

Regression helps to find a relationship between one dependent and one or more than one independent variable. In regression, the values are continuous in nature. Examples of continuous values are height, temperature, weight, etc.

INDEPENDENT VARIABLE vs DEPENDENT VARIABLE

The dependent variable is the one that we try to predict through regression. The independent variable(s) are the ones that we use to predict the dependent variable.

USING INDEPENDENT VARIABLE(S) TO PREDICT DEPENDENT VARIABLES

The equation shown in the image above is used to predict the results in Linear Regression. In other words, the dependent variable or target value is predicted using independent variables or given values in Linear Regression.

Now we know, regression uses continuous values (independent or given values) to predict a continuous variable (dependent or target value). But why is the term ‘regression’ used in naming a ‘classification algorithm’ where the predicted result is discrete.

WHY ‘REGRESSION’ IS PRESENT IN THE NAME OF LOGISTIC REGRESSION?

Logistic Regression uses the same formula that is used in Linear Regression. The formula is:-

y = m0x0 + m1x1 + .. + c

The difference is that this same formula is passed through the SIGMOID FUNCTION in Logistic Regression before predicting the final result.

In logistic regression, the sigmoid function provides probabilities [0, 1]. Now, the output (y) is such that,

if y >= 0.5, then y = 1

if y <= 0.5, then y = 0

SIGMOID FUNTION IN LOGISTIC REGRESSION

Logistic Regression uses the same formula as Linear Regression. However, this formula is passed through the SIGMOID FUNCTION in Logistic Regression. As we can see in the above image, the sigmoid function uses the continuous value and predicts the output based on the threshold condition. This is how we get a discrete value in Logistic Regression.

Even though logistic regression uses the same formula as linear regression, it is for the probability of a definite outcome.