Thirty-six years ago, a fierce storm hit England and France, killing at least 22 people and causing over $35 billion in damages. A few days later, the global stock market tanked, with the Dow Jones dropping 22.6% in one day. Both events were poorly predicted by current experts. But the storm led to the development of a new type of prediction system called ensemble prediction. According to climate physicist Tim Palmer, who was instrumental in developing ensemble prediction, economists could benefit from the development of similar types of systems for predicting intermittent instabilities in the global economic system.
Seemingly unrelated things connecting in unexpected ways is as good a place as any to delve into Palmer’s lengthy and genre-busting scholarship. In a lecture yesterday sponsored by the Columbia Climate School, Palmer asserted that chaos theory is one of the “three greatest theories in 20th century physics,” along with quantum mechanics and the theory of relativity. This is explained in greater depth in his most recent book, The Geometry of Chaos: The Primacy of Doubt, an exploration of the tricky scientific underpinnings of uncertainty and chaos theory, and how seemingly stable systems can act quite predictably—until they suddenly spiral out of control.
Over a century ago, French physicist Henri Poincaré determined that “no system exists” for predicting this unpredictability; Palmer builds on this idea, but adds that chaotic systems are “not necessarily strongly unpredictable all the time” but are frequently confined to periods of “intermittent instability.” In other words, if there’s a ghost in the machine, at least it’s a somewhat predictable ghost. Until it’s not.
Palmer expands on the concept of ensemble prediction—a supercomputer-powered approach that essentially encompasses uncertainty for a range of likeliest outcomes—by embracing the critical role ambiguity plays when trying to predict how massive systems such as the weather, the spread of COVID or the stock market, with many billions of inputs, will behave. If you’ve ever watched a weather forecast with a hurricane barreling toward you and placed bets based on the size of the “cone of uncertainty,” you have Palmer to thank.
The path of a hurricane is one thing; the economic fallout from future wars or the state of Earth’s climate 50 years from now is another. In the future-prognostication department, Palmer also proves no slouch: he was among the researchers who won the 2007 Nobel Prize for contributing to the Intergovernmental Panel on Climate Change reports.
But this begs the question: if chaos is naturally built into the system, how can we predict anything at all? How can we possibly predict climate 100 years from now if we can’t forecast the weather more than a week in advance? Palmer’s answer to that question is, it’s not really the same kind of prediction. He illustrates the concept with a visual of a pendulum flitting around four magnets, favoring none.
If you raise one side of the contraption with a wedge, the magnet will still flit around the poles, but it will eventually favor the magnet that is closest to the ground. “The system keeps going in a chaotic way, but you can clearly see the yellow magnet is becoming more favored,” he says. “And if you think of these magnets as weather types, then it’s showing you that climate change is basically a problem of trying to predict how our emissions affect the changing probability of different weather types.” And it’s only by zooming out and looking at the much larger picture that you can observe such general patterns.
When it comes to the future of our planet, Palmer suggests that we need to embrace this chaos and in order to do so, we need to supercharge the shoestring budgets we currently devote to climate modeling with a well-funded and coordinated multinational effort, much like CERN, the international particle-physics collaborative, does for its field. “I don’t think there’s a shortcut to this,” Palmer says. “I’m a great believer that we need something like CERN to develop ultra-high-resolution computer models, so countries can devote computing and human resources to tackle this problem with the detail and the urgency that it needs.”
