Decision Tree Regression and its Mathematical Concept
Hello readers , so here we are to understand how the decision tree algorithm woks in the regression based problems and what mathematical implementation is done in order to do prediction on continuous values . But before going through this blog , refer to the below link to understand what is a decision tree and how it works .
Decision Tree Regression
Decision Tree regression is a concept that implements the Decision Tree Algorithm over continuous data in order to do future predictions . This type of regression analysis is done on the data that is non-linear separable i.e a straight regression line cannot be implemented in order to do predictions .
The internal concept of Decision Tree Regression is exactly similar to the Classification concept i.e the model keeps on grouping the data on the basis of particular features through a hierarchy of a tree structure . Now, since the regression problem doesn’t holds the categorical values , the decision tree works by grouping the data into continuous groups up to a particular limit .
For example , let us take an example of a dataset of the Salary of Employees in the company based on their experience and bonus percentage . Now, let us see how decision tree works in this problem of getting the salaries .
In the above diagram , at the level zero , the feature of division is taken as experience . Since , the feature is a continuous value (experience can be 0,0.2….3.5,3.6….6.1,6.9…) therefore the tree automatically divides the data into three parts or three ranges . At the level 1 , the division feature is Bonus % which is a continuous value too . So the decision tree divides it into 2 ranges and gets trained .
Now , whenever the new data will come up to predict the salary of the employee , the average of all the values in the leaf node is taken as the final output or prediction . This is the working of Decision tree regression .
Mathematical Concept
Now , you might be thinking that how the decision tree knows which feature to use as a division feature at a level . Just like gini impurity or information gain is used in the classification problems , the Decision Tree Regression uses the mathematical concept called Reduction In the Variance .
Reduction in Variance = Variance(T,X) -Variance(T)
Here Variance(T) refers to the variance of the values in parent node and Variance(T,X) refers to the variance of the values in the child nodes .
For a feature to be selected to be the division feature at a particular level , the Reduction in Variance should be highest .
— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —
That is all about this blog. Hope it was an informative one for you !!
