An introduction to model ensembling

Jovan Sardinha
Jul 12, 2017 · 5 min read

Terminology

Intuition

Scenario 1 —  (all three models are correct) 
= 0.7 * 0.7* 0.7 = 0.3492
Scenario 2 — (two models are correct)
= (0.7*0.7*0.3)+(0.7*0.3*0.7)+(0.3*0.7*0.7) = 0.4409
Scenario 3 — (two models are wrong)
= (0.3*0.3*0.7)+(0.3*0.7*0.3)+(0.7*0.3*0.3) = 0.189
Scenario 4 — (all three models are wrong)
= 0.3*0.3*0.3 = 0.027
1111111100 = 80% accuracy
1111111100 = 80% accuracy
1011111100 = 70% accuracy
The majority vote ensemble produces:
1111111100 = 80% accuracy
1111111100 = 80% accuracy
0111011101 = 70% accuracy
1000101111 = 60% accuracy
The majority vote ensemble produces:
1111111101 = 90% accuracy

Ensembling techniques

Rank ensembling techniques

Stacking techniques

Concluding thoughts


Weights and Biases

Teaching machines to be intelligent

Jovan Sardinha

Written by

Weights and Biases

Teaching machines to be intelligent