Nowcasting, an Exciting Application of Machine Learning

Original image by Jess Bailey on Unsplash modified by the author

What is nowcasting?

Like forecasting, which is predicting the future, nowcasting is predicting the present, i.e., estimating the current or near-term state using real-time data and predictive analytics.

Where nowcasting is used?

Nowcasting plays an important role in fields where the latest information is indispensable for effective decision-making but may not be immediately accessible. Such fields include finance, meteorology, public health, economics, and others. In finance, for instance, real-time prediction of stock price movements is useful for investors to make informed decisions and various available data sources and advanced analytics can be leveraged for this purpose. Similarly, data from sensory inputs and satellite images can be used for accurate short-term weather prediction in meteorology. However, this article will focus on macroeconomic nowcasting — real-time prediction of key macroeconomic indicators, such as GDP and inflation. The official statistics for such indicators are often only available with a lag.

Macroeconomic nowcasting:

Predicting the short-term dynamics of the economy is a crucial input into various economic agents’ decision-making processes. However, accurately nowcasting key macroeconomic indicators can be challenging for various reasons. For instance, official estimates are released with a substantial delay, and the uncertainty in the data and estimates can lead to multiple revisions, sometimes years after their first release. Additionally, various data series are required for accurately nowcasting of macroeconomic indicators — further complicating the process [1].

Macroeconomic crisis, such as due to the Covid-19 pandemic, makes nowcasting even more challenging due to the unprecedented economic impacts of the crisis, the unconventional policies designed to alleviate those crises, and the unreliability of traditional models due to their dependence on lagged data [2].

Nowcasting of monthly GDP growth in Canada during the onset of the Covid-19 using payments data and ML. The plots show the marginal contribution of predictors, where blue is negative GDP growth rates, and red is positive. Used with permission from: Macroeconomic Predictions using Payments Data and Machine Learning

Machine learning for macroeconomic nowcasting:

The use of machine learning (ML) models for macroeconomic nowcasting is highly advantageous due to their ability to effectively handle a diverse set of real-time data, manage collinearity in predictors, and capture nonlinear interactions between predictors and the target variables. Additionally, ML models prioritize improving prediction accuracy — a crucial aspect of macroeconomic nowcasting applications [2].

Examples of use of ML for macro nowcasting:

Popular approaches for macroeconomic nowcasting are the dynamic factor model (DFM) and Mixed Data Sampling (MIDAS) model; however, the use of ML models has been rising recently. This is due to the need to leverage high-frequency, non-traditional, and complex datasets. A few articles listed below emphasize the usefulness of ML models in extracting economic value from such data. Additionally, these articles highlight that ML models can be used in conjunction with traditional econometric tools like MIDAS models to enhance prediction accuracy and gain deeper insights into macroeconomic trends [3, 4].

a. Kapetanios, G. and Papailias, F. Big data & macroeconomic nowcasting: Methodological review. Technical report. Discussion Papers ESCoE DP-2018–12, 2018. (paper link)

b. Richardson, A., Mulder, T. and Vehbi, T. Nowcasting GDP using machine-learning algorithms: A real-time assessment. International Journal of Forecasting, 2020. (paper link)

c. Chapman, J., and Desai, A. Using payments data to nowcast macroeconomic variables during the onset of COVID-19. Journal of Financial Market Infrastructures, 9(1), 2021. (paper link)

d. Babii, A., Ghysels, E., and Striaukas, J. Machine learning time series regressions with an application to nowcasting. Journal of Business & Economic Statistics, 40, 2022. (paper link)

e. Borup, D., Rapach, D. E., and Schütte, E. Mixed-frequency machine learning: Nowcasting and backcasting weekly initial claims with daily internet search volume data. International Journal of Forecasting, 2022. (link)

Nowcasting of monthly GDP growth in Canada during the onset of the Covid-19 using payments data and ML. The SHAP force plots for Feb (top) and Apr 2020 (bottom), where the red arrows show positive and blue are negative contributions. The f(x) is the model prediction, and the base value is the average of all predictions. Used with permission from: Macroeconomic Predictions using Payments Data and Machine Learning

Challenges to overcome:

However, using ML models also lead to a few challenges that could reduce the effectiveness of these models for policy use. In particular, nowcasting ML model problem of overfitting; that is, due to the flexibility of these models, it is easy to overfit them on in-sample data, which could reduce their out-of-sample performance. In addition, these models are hard to interpret, which could be important to understand their predictions — especially if they are used to support policy decisions.

Image created by the author using Keynote

The researchers are making progress toward overcoming these challenges. For instance, one approach recently developed to mitigate interpretability issues is the Shapley-value-based methodology [5, 6]. However, note that although such methods are based on game theory, they do not provide any optimal statistical criterion. To overcome that, for instance, in a recent paper [7], the authors propose ML-based mixed data sampling and develop the asymptotics in the context of linear regularized regressions. However, much progress needs to be made to use such asymptotic analysis for popular nonlinear ML approaches.

References

[1] Giannone, D., Reichlin, L., & Small, D. (2008). Nowcasting: The real-time informational content of macroeconomic data. Journal of monetary economics, 55.

[2] Chapman, J.T.E.; Desai, A. Macroeconomic Predictions Using Payments Data and Machine Learning. Forecasting 2023, 5, 652–683. https://doi.org/10.3390/forecast5040036

[3] Stock, J. H., & Watson, M. W. (2011). Dynamic factor models.

[4] Ghysels, E., Sinko, A., & Valkanov, R. (2007). MIDAS regressions: Further results and new directions. Econometric reviews, 26.

[5] Lundberg, S. M., & Lee, S. I. (2017). A unified approach to interpreting model predictions. Advances in neural information processing systems, 30.

[6] Buckmann, M., Joseph, A., & Robertson, H. (2021). Opening the black box: Machine learning interpretability and inference tools with an application to economic forecasting. In Data Science for Economics and Finance: Methodologies and Applications.

[7] Babii, A., Ghysels, E., & Striaukas, J. (2022). Machine learning time series regressions with an application to nowcasting. Journal of Business & Economic Statistics, 40.