Types of Regression Techniques
In this article I’ll telling you some basic understanding regarding types of Regression Technique. We are focusing only it’s types and it’s formal mathematical equations and how they look…so let’s get started.
A regression problem is when the output variable is a real or continuous value, such as “salary” or “weight”. Many different models can be used, the simplest is the linear regression. It tries to fit data with the best hyperplane which goes through the points.Three major uses for regression analysis are determining the strength of predictors, forecasting an effect, and trend forecasting.
Types of Regression –
- Linear regression
- Logistic regression
- Polynomial regression
- Stepwise regression
- Ridge regression
- Lasso regression
- ElasticNet regression
- Linear regression is used for predictive analysis. Linear regression is a linear approach for modeling the relationship between the criterion or the scalar response and the multiple predictors or explanatory variables. Linear regression focuses on the conditional probability distribution of the response given the values of the predictors. For linear regression, there is a danger of overfitting. The formula for linear regression is: Y = bX + A. In this technique, the dependent variable is continuous, independent variable(s) can be continuous or discrete.
2. Logistic Regression It is widely used for classification problems. Logistic regression doesn’t require linear relationship between dependent and independent variables.We should use logistic regression when the dependent variable is binary (0/ 1, True/ False, Yes/ No) in nature. Here the value of Y ranges from 0 to 1 and it can represented by following equation. If the values of dependent variable is ordinal, then it is called as Ordinal logistic regression . If dependent variable is multi class then it is known as Multinomial Logistic regression. Famous sigmoid function is shown as below:
NOTE : I recommended to you just google out with some authentic material and see what is the difference between logistic regression and SVM.(don’t worry if you not get then i’ll make one more article on it.)
3. Polynomial regression is used for curvilinear data. Polynomial regression is fit with the method of least squares. The goal of regression analysis to model the expected value of a dependent variable y in regards to the independent variable x. The equation for polynomial regression is:
y= a+bx²+…..
In below figure you can see we can’t s fit linear line on such data points, okay.
4. Stepwise regression is used for fitting regression models with predictive models. It is carried out automatically. With each step, the variable is added or subtracted from the set of explanatory variables. This form of regression is used when we deal with multiple independent variables. In this technique, the selection of independent variables is done with the help of an automatic process, which involves no human intervention. The approaches for stepwise regression are forward selection, backward elimination, and bidirectional elimination.
5. Ridge regression is a technique for analyzing multiple regression data. When multicollinearity occurs, least squares estimates are unbiased. A degree of bias is added to the regression estimates, and a result, ridge regression reduces the standard errors. Ridge Regression is a technique used when the data suffers from multicollinearity ( independent variables are highly correlated). Here lamda is the srinkage parameter.
6. Lasso regression is a regression analysis method that performs both variable selection and regularization (technique that is used to avoid overfitting). Lasso regression uses soft thresholding. Lasso regression selects only a subset of the provided covariates for use in the final model. Similar to Ridge Regression.
7. ElasticNet regression is a regularized regression method that linearly combines the penalties of the lasso and ridge methods. ElasticNet regression is used for support vector machines, metric learning, and portfolio optimization.it is hybrid of Lasso and Ridge Regression techniques.
Hope this article will help you out to understand what is the different types of regression is ! To understand how simple linear regression works, just go to the given link below, definitely it’ll help you to understand better with real data.
