Forecasting Stock Prices using XGBoost (Part 5/5)
Using stacking models!
In our previous article, we explored using direct forecasting (also known as day-to-day models). In this article, we will explore stacking, which is an ensemble machine learning algorithm.
The problem statement, feature engineering, training-validation-test split steps remain the same as in previous articles, and will not be repeated here. In this article, we only focus on the below steps which will give different results from before. First, we explain what is stacking.
Stacking
Stacking, or stacked generalization, is an ensemble machine learning algorithm introduced by Wolpert in 1992 (link to paper). The stacking approach makes predictions using a number of base models and then using a different (meta) model to train on these predictions.
In general, the methodology for stacking is as follows:
- Split the train set into two disjoint sets
- Train several base learners on the first set
- Make predictions using the base learners on the second (validation) set
- Use the predictions from above (called meta-features) as the inputs to train a higher level learner (called a meta-model)
