What is Regression?

Changsung Moon
Aug 9, 2017 · 2 min read

Regression is a statistical process for estimating the relationships between a dependent variable (‘target’) and one or more independent variables (‘feature’ or ‘predictor’). The dependent variable (usually denoted by y) is the output, which is the quantity of interest. Regression assumes that y can be predicted from the independent variable (usually denoted by x), which is the input. The estimation target is a function of the independent variables called the regression function.

[Example of training set: Housing prices]

  • x = independent variable / feature
  • y = dependent variable / target

Regression analysis is also used to understand which among the independent variables are related to the dependent variable. It helps data analysts to find the best set of variables to be used for building predictive models.

There are various kinds of regression techniques, and these are mostly driven by three metrics: 1) the number of independent variables, 2) type of dependent variables and 3) shape of the regression line. The followings are the most commonly used regressions:

  • Linear Regression
  • Logistic Regression
  • Polynomial Regression
  • Stepwise Regression
  • Ridge Regression
  • Lasso Regression
  • ElasticNet Regression

We will discuss details of each regression in future posts.

References

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade