How far out can we predict the weather? Chaos sets a ‘natural limit.’
Despite super-powered computers and sophisticated algorithms, we can only predict the weather this far into the future.
Mashable’s series Algorithms explores the mysterious lines of code that increasingly control our lives — and our futures.
Everyone knew an attack was imminent. But exactly when depended on the weather.
In early June of 1944, Nazi meteorologists predicted the weather off the French coast would persistently bring gale force winds and rough seas — thwarting an Allied invasion of Hitler’s “Fortress Europe.” But the Allies, seeking to penetrate the Greater German Reich, had a better prediction. They forecast a narrow window of calm conditions on June 6, using hand-drawn maps and sparse observations of stormy weather in the region. And so it was. General Dwight D. Eisenhower famously ordered the June 6 D-Day invasion to eliminate “Nazi tyranny over the oppressed peoples of Europe.” With horrid sacrifice and bloodshed, the Allies breached Hitler’s regime, in large part thanks to one of the most momentous weather predictions in history.
This early weather prediction, however impressive, was primitive. In the last few decades, vastly improved observations of the air and oceans, which feed into modern computer simulations of how the atmosphere will behave, have revolutionized weather prediction. Why, the National Weather Service forecast, with astonishing accuracy, rainfall totals from Hurricane Florence five days beforehand. Today, we’ll never be surprised by a hurricane. And we’ll always know when a blast of Arctic air will pour down into the U.S., well in advance.
Yet, there’s a limit on how far into the future humanity can ever predict the day-to-day weather: Will a storm probably hit my town? Will it be a terrible day for a barbecue? Will driving be too dangerous? The absolute limit on this type of weather prediction is somewhere between two and three weeks, said Falko Judt, a research meteorologist at the National Center for Atmospheric Research. Even the relentless march of technology — with ever faster, better, smarter computers — cannot push beyond this boundary.
“It’s the natural limit,” said Judt, who in 2018 tested the limits of weather prediction on a supercomputer running sophisticated algorithms to simulate Earth’s atmosphere. “It’s inherent to the atmosphere,” explained Judt. “It’s something that is set by nature itself.”
(Importantly, predictions of big, broad weather patterns, like the expansive heat dome over the Southwest in 2020, aren’t confined by this time boundary. Nor are long-term climate predictions, which use math equations to forecast how rapidly rising greenhouse gas levels will heat the planet. Climate models have proven remarkably accurate: They’ve predicted warming decades into the future.)
Why should there be a strict limit, however, on predicting a day’s weather? How can human progress and understanding — having eradicated smallpox, built supercomputers, and sent humanity to the moon — be limited? It turns out we’re limited by an irrepressible agent. It’s called chaos. Sure, chaos can be tamed to some degree, but not conquered. Thanks to chaos, the future isn’t fully determined.
In our universe, chaos is a powerful property of any system that evolves over time. Chaos emerges in a prediction when a small unknown or error amplifies. It’s somewhat like a spacecraft sent to the moon on a slightly off-course trajectory, eventually sailing past and into the black ether of space. Some systems are inherently more chaotic than others (the moon’s orbit today, thankfully, is pretty stable). But Earth’s atmosphere — incessantly evolving with rising, sinking, and whirling air — is primed for chaos. Chaos thrives there because the atmosphere can’t ever be fully known, to perfection. Why not? On the smallest level, it’s impossible to know exactly where an atom is and how fast it’s traveling, a powerful physical law called the “uncertainty principle.” What’s more, even the slightest, little, undetectable whirl or perturbation in the air can dramatically alter the atmospheric future. All of this means we’ll never have an all-knowing grasp of everything unfolding in the expansive, vacillating sky. There will be gaps and uncertainties. And eventually, they’ll destroy the prediction. “
Literal whirls in the wind as big as your thumbnail can have an effect later on,” said Judt. “Even that little uncertainty will cascade up and render the forecast unpredictable by two to three weeks’ time.”
“It’s mind blowing in a way,” he added.
This is commonly called “the butterfly effect,” in reference to a butterfly flapping its delicate wings and initiating a cascade of atmospheric events. “People think it’s a metaphor,” said Kerry Emanuel, an atmospheric scientist at MIT. “It’s not. It’s real.”
Why, on a warm sunny day (when the atmosphere is inherently more dynamic and unstable than a cold, stable night) you too can wave your arms outside and potentially initiate changes in the air that alter the course of events in the atmosphere. “Everyone is contributing to chaos at different times,” said Emanuel, who separately found the limit of day-to-day weather prediction is around two weeks. (“No one pretends to know the number exactly,” noted Emanuel.)
Scientists’ current ability to better predict day-to-day weather events (which presently reach about a week into the future) can certainly improve. It’s a fascinating science. But chaos always reigns by about two weeks. To demonstrate this curious limit, Judt employed thousands of processors at the Cheyenne supercomputer in Wyoming to create state-of-the-art simulations of Earth’s complex, evolving atmosphere. He ran two 20-day simulations: One of real observations taken from a past October day, and the other an artificial creation of that same day modified with a “teeny, tiny” alteration in the weather. Small-scale events, like thunderstorms, materialized in different places within hours. After six days, big weather patterns some 100 miles to 1,000 miles across (like high and low pressure systems) appeared in different places. By a little over two weeks, the two atmospheres were nothing alike.
In the supercomputer, chaos reared its head, and then propagated.
“It doesn’t matter how much faster or how close to perfect your computers are,” said Adilson Motter, a physicist at Northwestern University who researches chaos. Digital computers won’t ever be able to account for all the details of the actual world, he said. Chaos will erupt with the slightest mismatch or rounding error.
The technological limit is similar to building a spaceship that might (somehow) approach the speed of light (670,616,629 mph), the speed limit any object can travel. “We know we can’t go faster than the speed of light,” said Judt. “Even with futuristic technology, we can’t make spacecraft that can go faster.”
Chaos, then, is a defining, unalterable part of our existence. “The discovery of chaos is considered the third greatest discovery of the 20th century,” said Emanuel, behind Einstein’s General Theory of Relativity and Quantum Theory.
Taming Chaos
The D-Day weather forecast lacked a potent weapon: computers.
Soon after the war ended, in 1950, meteorologists made the first weather prediction on a colossal computer. The machine took up a 1,500-square-foot room. Today, weather forecasts are created with far more advanced computer simulations of the weather. These simulations seek to quell chaos.
“I like to talk about taming the butterfly effect,” said Roberto Buizza, a physicist and a former lead scientist for the European Centre for Medium-Range Weather Forecasts.
Every forecast must tame chaos because our information about the atmosphere is inherently incomplete. “We have fully embraced the concept of our Earth systems being fundamentally chaotic,” said Peter Bauer, the deputy director of the research department in the European Centre for Medium-Range Weather Forecasts. “We will never have the conditions perfectly observed or simulated.”
Still, to make a weather prediction today, meteorologists plug in millions of observations of the atmosphere’s temperature, air pressure, wind, and beyond into computer simulations. In the U.S., over 210 million weather observations from weather stations, radars, satellites, weather balloons, buoys, ships, and elsewhere are funneled into computer models each day. Crucially, for each prediction meteorologists run many simulations (perhaps dozens), but slightly alter the unknown weather conditions, or gaps in observations, during each run. The final forecast is a range of futures, but a pattern emerges where the simulations broadly agree. This range is the prediction.
“It’s the only way to get reasonable predictions,” said Katherine Evans, division director for the Computational Sciences and Engineering Division at Oak Ridge National Laboratory.
A great example is the National Weather Service’s hurricane cone, which shows the path where a hurricane is likely headed. The weather agency never shows a hurricane aimed on one specific trajectory to one specific place. That’s because there’s never one answer. There can’t be. “We will never know the answer with weather, ever,” emphasized Evans. “The atmosphere is just a very complex system.” But this range of likely scenarios paints a good picture of what’s coming, often days in advance. “You’ll be warned,” said Bauer.
Thanks to vastly improved observations of the atmosphere (particularly as satellites came online to observe the swirling atmosphere from thousands of miles above) along with faster computers, the forecasting community has extended weather prediction by about a day per decade for the past 30 years. It’s a feat Bauer called “enormous.”
Yet it’s unknown how far into the future humanity will ever reliably predict the weather — or if we’ll even approach the two-week boundary. We’re hovering around the one week limit now, said MIT’s Emanuel. How much farther can we go?
The theoretical limit can only be achieved with technology that doesn’t exist.
Seeing perhaps another two or three days into the future, or even beyond, will require vastly improved observations of the atmosphere and ramped up computing. This won’t be easy. Already, advancements in improving predictability have nearly stalled, even with supercomputing and advanced satellite observations. “It has slowed significantly over the last five to seven years,” said Judt, noting this hints at how difficult it is to tame chaos. Nearing the two-week boundary, meanwhile, might be theoretically possible, but realistically out of the question. That’s a feat requiring a trillion times more computing power and observations of every millimeter of the atmosphere, Judt explained.
“The theoretical limit can only be achieved with technology that doesn’t exist,” Judt said.
If future computers aren’t given incredibly precise information about what’s transpiring in the world — the winds swirling in the disparate Pacific Ocean, a short-lived downpour in the remote New Mexican desert, the flutters of air made by a pelican swooping over the sea — predictions will inevitably derail as the simulation goes deeper in time. It all goes back to chaos. “As soon as you make a small error somewhere, this error will grow and propagate,” said Buizza.
And the prediction fails.
Freedom
A meteorologist for the U.S. Army Air Corps during World War II, Edward Lorenz, would discover chaos after the war.
Lorenz became a weather-curious scientist at MIT. There, he used an early computer (the size of a dresser) to research weather prediction. In 1961, Lorenz ran a simple weather model, simulating two months of weather into the future. After finishing the simulation, he plugged in the numbers again and restarted the same simulated run of weather. Or so he thought. While the machine crunched equations, Lorenz went out to grab coffee. Upon returning an hour later, he found the computer spitting out vastly different numbers (or weather predictions) than the previous run. How could this be? Lorenz discovered he hadn’t typed in the same exact numbers (or initial state of the weather). He made a one-part-in-a-thousand rounding error. A tiny error. But these small errors changed everything. “They were steadily amplifying until they dominated the solution,” wrote Lorenz.
The serendipitous computer experiment proved the slightest difference in conditions will scale up, eventually producing wildly divergent results. “The understanding of chaos was precipitated by computers,” said Motter, the physicist who researches chaos. Had Sir Isaac Newton had a computer to crunch algorithms of the evolving physical world, he too might have spotted the potency of chaos, Motter said.
Yet chaos, while limiting how far we can peer into the future, is also hugely freeing. Chaos is potent evidence that our lives are not determined nor bound by an inalterable flow of events. In other words, that your fate isn’t predictable.
This emancipating idea, however, wasn’t widely embraced as recently as the 18th century. The hugely influential mathematician and philosopher, René Descartes, formulated ideas leading to the popularization of the Cartesian Universe — a place where everything is determined and follows mechanical laws. Everything, then, was clockwork.
“There was widespread conjecture that we lived in a clockwork universe,” said MIT’s Emanuel. But two discoveries quashed the notion of a predetermined reality, he noted: the marriage of Lorenz’s discovery of chaos with the revelation that the exact location of an atom (“quantum uncertainty”) can never be certain. This showed that everything, and the course of the future, can’t be fully known. “It’s the last nail in the coffin of the clockwork universe,” said Emanuel.
“The death of the Cartesian Universe is enormously liberating,” he added. “Who wants to live in a clockwork universe? There’s no free will. Everything is predetermined.”
The limits on weather prediction, then, on a planet swirling with chaos and uncertainty, is a blessing. At least, it’s much better than the alternative.
“We must then wholeheartedly believe in free will,” wrote Lorenz in 1993, after pondering the implications of chaos for decades. “If free will is a reality, we shall have made the correct choice. If it is not, we shall still not have made an incorrect choice, because we shall not have made any choice at all, not having a free will to do so.”
Originally published at https://mashable.com
