Uncertainty in Forecasts, Weather and Other, Part 1: Intro
Over at Meteopress, we investigate, research, develop and implement now-casting and forecasting models that are often on the very bleeding edge of the current state of art, but that only makes a somewhat more down to earth and holistic approach to data and deep learning inference all the more important. Understanding of what to expect and how to interpret the data our tools give us is as important as the predictive power of the tools, and I’ll attempt to give some insight into the pains of today weather science in a series of short and hopefully layman-friendly articles.
At middle school level, a generally excellent physics teacher gave us a rhetorical question that fuels my fascination with science to this day. It was simple, and maybe obvious in its technically correct answer even back then, but still very stimulating.
What’s the difference between 1.0 and 1.000?
Precision.
Mathematically, they’re the same; they hold the same value. Computationally, it starts getting hairy, because the ways we deal with the untameable floating point precision are arbitrary and sometimes counterintuitive, and most computational tools would just disregard the zeroes outright, but in this case, they should be all the more same.
In physics, and general observations of the real world, they’re a radically different statement, though. Every floating point should serve as a warning that after the last digit that follows it, the perhaps seemingly least significant but yet infinitely varied uncertainty skulks. And in our case, the uncertainty hiding behind the first number is two orders of magnitude bigger than in the second case.
Perhaps rooted in the laws of our universe itself on some levels, there’s this nagging, ever present uncertainty hiding in the margins of our measuring tools and their calibrations and biases. This chaos has still been barely reigned in when we try to not only forecast, but even just simply accurately observe, describe and note down the weather we’re already subjected to, and are experiencing.
Ill wind
Even in extensively studied, intensely socially impactful weather events that we’ve observed in 2020 in Czechia as well as in the United States are always to a perhaps surprising degree never quite clear in exact observation of what happened; the paths of destruction and the damage they left ended up serving us as the best ways to estimate the velocity of wind and the paths and time span of the phenomena.
Very few classical point-measurements of the tornadoes are available, because even if there was the very best equipped synoptic station right in the middle of the havoc, its instruments would generally be exposed to elements vastly outside of their operating limits. Pieces of the station itself would probably mostly serve as only more shrapnel to estimate wind speed from the way it embeds itself in whatever it hits.
We do have some nice radar footage of the phenomena, and excellent analysis of the telltale aspects seen on the air mass have already been described on them, but as “sparse” a measurement the precipitation radars are in time stepping, these observations might even go just unnoticed if we didn’t have the destruction trail and massive human suffering to pull our attention to them.
Into the chaos
And so, when even just observing, but ever more so forecasting weather, we do have to embrace chaos. We have to be very clear with our figurative statement of “1” potentially meaning anything between “0.5” and “1.49”, even though in practice of something like predicted temperatures, we don’t quite have or aren’t used to language that would encode our uncertainty and lack of precision in our predictions. Everyone sort of intuits at this point that we aren’t promising a whole degree of Celsius precision in temperature on a day a month from now.
Maybe it should be somehow represented in how we show our data, and indeed, in some cases it is; some weather predictions do present the user with spreads of forecast ensembles (which I’ll talk about more in a following article), but it’s hardly the norm, and it’s questionable how useful this additional information is to any one user.
Even without burdening the end users receiving our nowcasts, forecasts and prognoses, we do need to pay considerable attention to what degree of certainty and uncertainty do the individual tools we use to observe and predict present us with, and this is becoming only more crucial with global meteorological climate sadly becoming more and more unstable and ever less predictable, lately.
Fluffy, fluffy clouds
Another issue is in the question of accuracy and exactness versus presentability, where presentability is this another nasty little beast consisting of human expectations, bias and perception. A number is worthless if it doesn’t give the user, professional or layman, the right idea about what’s going on, or what’s likely going to happen. It is our burden, as the builders of the tools, and the meteorologists using them and issuing outward facing forecasts, alerts and whatnot, to make them naturally comprehensible and intuitive to the end user.
We’ve been facing a long term problem of our precipitation nowcasts “blurring” the observed precipitation fronts, and the issue is; statistically, it’s the correct answer. It’s the predictors way of encoding the uncertainty that’s inevitable given the limited, inaccurate information it has about past and present, as well as the inherent imperfection in the model; giving a blurry, hazy answer is indeed the most honest and, as far as statistical analysis is concerned, correct answer, yet once the stormfronts blur too much, they stop looking like the radar echoes that our meteorologists and users alike are used to observing, which messes with their own entire intuitive, learned way of interpreting these pictures. So we have to figure out how to give answers that are somewhere between statistically correct, but also look relatable to the user. I’ll again, talk about this more, in another article.