Member-only story
How Reliable Are Your Time Series Forecasts, Really?
How cross-validation, visualisation, and statistical hypothesis testing combine to reveal the optimal forecasting horizon
Imagine you have a crystal ball — a mysterious family heirloom, handed down through generations. It shows its age, its clarity and lustre long gone, with some chips scattered across the surface.
Despite its hazy provenance, the things you see in it still seem to come true in one way or the other, at least in the short-term. It often shows you events far into the future, but how much can you trust it really?
The crystal ball I’m talking about here is of course our time series models, which we’ve built following the same approach underlying Meta’s Prophet suite. I’ve cheekily referred to my implementation as the False Prophet, but it looks like it’s anything but, producing what look to be fairly accurate forecasts (and I’ve got the cross-validation results to prove it).
Yet, it is only a model, and apart from usually being wrong, models also tend to struggle a bit at the extremes and edges; in this context, the extremities being forecasts far out into time.
In what follows, we’ll be building a time series model to predict UK road traffic accidents…
