Why are economic forecasts invariably wrong?
Does it pay to be normal?
Economists are having a tough time of it at the moment. After witnessing a benign economic environment for years, all of a sudden the harsh winds of the pandemic came along, rendering economic forecasting very difficult to do with any degree of accuracy. That has been followed by a post-pandemic bounce which has shown current economic forecasts as a cross between wishful thinking and pure guesswork. However, the reputation of economic forecasting didn’t need too much adversity to call it into question. Economic forecasters have a reputation for getting things wrong. I wonder why that might be?
We have to start with what an economic forecast is, and what the general public takes it to be. I will refer to the above chart because that is very useful in making sense of what is going on. When an economist makes a forecast, ordinarily there will be a range of possible future outcomes that are dispersed around a given number. In the example above, the mean (or average outcome) is 4.27. It doesn’t matter about the units, but if the reader is happier with a less abstract view, then think of it as meaning that the forecast expects, on average, GDP to grow by 4.27% next year.
Where does this number come from? Normally, we would expect the economist to measure the trend of growth in recent years and, assuming ceteris paribus (that’s economist-speak for nothing really changes), to extrapolate that growth trend for a year ahead. Given our point of origin, the speed at which we arrived here, we can estimate where we will be in a years time. That is the central forecast, which is 4.27% in our example.
There is a problem with this approach. We may not travel at the same rate as in the past in the coming year or we may have been mistaken about our point of origin. Economists try to measure the likelihood of being blown off course by measuring the historic dispersion — a measure of how off course we have been in the past - of previous forecasts. This dispersion is expressed in terms of a thing called a standard deviation — how far we would normally expect to be blown off course.
Once we have a standard deviation, we can estimate the degree of confidence we have in our forecast. This is measured by a thing called the confidence interval. A 95% confidence interval is two standard deviations either side of the mean. We have assumed that the readings are distributed normally. In the example above, the confidence interval is between 2.92% and 5.62%, at a 95% confidence level. This is quite a powerful result because what it means, in this example, is that we can be 95% certain that GDP growth in the coming year will be between 2.92% and 5.62%, hovering around 4.27% as a mid-point.
It is at this point that the story tends to get lost. In order to simplify the story, it is common for it to be reported as a forecast growth rate of 4.27%, or just 4%. The nuance and uncertainty manages to be lost. So, if growth actually comes in at 3% or 5%, the economists are blamed for an inaccurate forecast when, all the time, both possible outcomes are within the expected range and the original forecast was correct. Criticism of economists at this level is unfair. The problem lies with the way in which forecasts are reported and interpreted, not the actual forecasts themselves.
Where economists are open to fair criticism concerns the way in which forecasts are constructed. Earlier, we touched upon two core assumptions in making forecasts — that of events unfolding ceteris paribus, and that of events being distributed on a normal distribution. Both can be subject to challenge.
Ceteris paribus is better known to the world as ‘all things being equal’. And yet, we know that they are not. In futures work, we take it as axiomatic that trends bend and break. Economic forecasts assume that they don’t. As an example of what I mean, consider the current inflation forecasts for the UK. The August Monetary Policy Report stated a central forecast of inflation peaking at just over 13% this year. This was widely reported as 13%. What it actually said was that inflation was forecast, with a 95% degree of confidence, to be between 11% and 15%, with 13% as the mid-range reading.
However, in coming to this conclusion, all sorts of assumptions would have to be made about the future course of prices, even for the last four months of the year. For example, there is a hidden assumption in there about the course of the war in Ukraine and about it’s continued impact on energy and food markets. As we have seen already this year, these assumptions can easily be wrong by a large margin, which is why the August forecast of inflation was adjusted higher than the May one. The August forecast assumes a reasonably benign food harvest this autumn. We are already being told that the impact of drought and heat will have an adverse impact upon future food prices, but the forecasts assume that the market forward pricing is broadly correct. If it changes dramatically overnight because, say, the crop yields are well below expectations, the forecasts will be wrong footed overnight.
We are told that much of the inflation of this year is the result of unprecedented events in the energy and food markets owing to the war in Ukraine and the heat and drought of the summer. The key word here is ‘unprecedented’. If events are unprecedented, they cannot, by definition, be distributed normally. This renders the whole mechanism of estimation and forecasting subject to far wider errors. Instead of thinking of confidence intervals in probabilistic terms, we would be better served by considering them in terms of uncertainty. Instead of being 95% certain, we should be ‘reasonably sure, given that this is an educated guess’. If this doesn’t sound at all certain, it’s because it isn’t. We cannot be at all certain when we are in uncertain territory.
Which brings us on to the point. Economic forecasts are invariably wrong because (a) we have misunderstood the nature of economic forecasting — it is a forecast range rather than a single point, and (b) some environments defy forecasts because the environment is changing too rapidly or the situation is too unique for us to be able to draw upon past experience. So what? Understanding this mismatch within our expectations creates an opportunity to make contrarian investments that can yield huge rewards. The 2015 film ‘The Big Short’ is all about such an opportunity. It’s worth watching.
© Stephen Aguilar-Millan 2022
