Decision Tree Machine Learning
A quick look at the ruby gem “decisiontree” by Ilya Grigorik
Why trees? According to the MIT Machine Learning and Statistics Course, They are . . .
- intuitive, popular in medical applications because they mimic the way a doctor thinks
- model discrete outcomes nicely
- can be very powerful, can be as complex as you need them
What composes a decision tree?
- Root node: this is the starting point of the tree.
- Leaf nodes: these are questions or conditions to split the data.
- Branches: these are the directions that guide our decision.
Below is a tree that helps us make a decision on what to do for dinner.
Now to translate this into code using ruby gem ‘decisiontree’.
(side note) Discrete vs Continuous label types?
Discrete values are Boolean, strings, or a set of numbers that won’t change. Continuous values represent a range, and new values for a label can be introduced during testing.
labels: Defines each column in our training data with a key.
training: An AoA of data to train our data, each with the result included.
dec_tree: This creates a new instance of a decision tree. we list our labels as keys with values of either discrete or continuous.
dec_tree.train : This trains our data.
test: This is a single data set that we want the machine to predict the answer for us.
#predict(test): Returns an output that the machine believes to be true.
How does “decisiontree” work?
This gem implements ID3, which is an algorithm with the purpose of creating decision trees with a data set. After storing the first data set, every attribute of the next data set is used to calculate information gain (or entropy). It continues with each set of data, calculating information gain and separating each them into subsets depending on its result, and refining the tree with leaf nodes and branches. As with most machine learning, the more data that is provided to train our machine, its predictions become more accurate.
Real life examples of decision trees . . .
Resources:
ID3- Iterative Dichotomous 3 is an algorithm invented by Ross Quinlan.
Entropy- How Decision Trees make decisions.
