What is Regression?
Regression is a statistical process for estimating the relationships between a dependent variable (‘target’) and one or more independent variables (‘feature’ or ‘predictor’). The dependent variable (usually denoted by y) is the output, which is the quantity of interest. Regression assumes that y can be predicted from the independent variable (usually denoted by x), which is the input. The estimation target is a function of the independent variables called the regression function.
[Example of training set: Housing prices]

- x = independent variable / feature
- y = dependent variable / target
Regression analysis is also used to understand which among the independent variables are related to the dependent variable. It helps data analysts to find the best set of variables to be used for building predictive models.
There are various kinds of regression techniques, and these are mostly driven by three metrics: 1) the number of independent variables, 2) type of dependent variables and 3) shape of the regression line. The followings are the most commonly used regressions:
- Linear Regression
- Logistic Regression
- Polynomial Regression
- Stepwise Regression
- Ridge Regression
- Lasso Regression
- ElasticNet Regression
We will discuss details of each regression in future posts.
References
