The Only Guide You Need to Understand Regression Trees

A Complete Guide to Decision Trees with a Step-by-Step Implementation from Scratch and Hands-On Example Using Scikit-Learn

Build a tree - Image by the author

Table of Content

1. Introduction
2. Decision trees for regression: the theory behind them
3. From theory to practice — Decision Trees from scratch
4. Hands-On Example — Implementation from scratch vs. Scikit-learn DecisionTree
5. Summary
6. References
7. Appendix / Code

1. Introduction

Decision Trees have been around since the 1960s. Despite being one of the simplest Machine Learning algorithms, they have proven to be highly effective in solving problems. One of their greatest advantages is their ease of interpretation, making them highly accessible to those without a technical background. In many industries, Data Scientists still have to build trust for Machine Learning use cases. Explainable baseline models like Decision Trees can help reduce the skepticism somewhat. If someone wanted to make the effort, they could even trace the branches of the learned tree and try to find patterns they already know about the problem.