Andrew Blum - The Weather Machine: A Journey Inside the Forecast

Perhaps some day in the dim future it will be possible to advance the computations faster than the weather advances and at a cost less than the saving to mankind due to the information gained. But that is a dream.

Lewis Fry Richardson, 1922

In October 2012, my son was an infant. I had been counting time carefully, weighing the weeks and days. I was also spending a lot of time on Twitter. I would sit holding him in a rocking chair, the world scrolling by beneath my thumb. It was in that pose, on a Sunday afternoon, that I watched as meteorologists went into a tizzy. The latest run of something called the European model had just arrived, and it was sending them into paroxysms. An organized low-pressure system hasnt even formed yet in the Caribbean, so a LOT can happen, wrote Bryan Norcross, one of the worlds most renowned hurricane forecasters, that afternoon. But because the scenario is so dramatic, its going to require our attention. The sky outside was clear, and it would be for a week. But the sky on the screen was filled with a storm that didnt yet exist.

Over the next eight days, Superstorm Sandy dumped flooding rains in the Caribbean, headed north across the warm ocean, soaking up energy, then took an extraordinary left turn toward the East Coast, toward New York City, toward us. We pulled down the shades as far as they would go and filled the bathtub with water. The storm came with fury, making the walls restless and twisting the windows in their frames. The lights flickered, and my screen flashed with strange images: the glass carousel on the Brooklyn waterfront floating in the river like a magical barge, downtown streets turned to canals, lampposts sparkling into fireballs. Not far away, the ocean rose up against the land, rushing through living rooms, flooding power stations and corroding the subways delicate machinery. Neighborhoods along the shore were devastated and Lower Manhattan went dark, a disaster film come to life. At the hospital in which my son had been born, nurses and doctors carried twenty-one infants down unlighted stairways, tangled in battery-powered monitors. Across the region, 147 people died during Sandy, 650,000 homes were damaged or destroyed, and total losses exceeded $50 billion. The city felt fragile. I had the sense that our luck had run out.

New York wasnt the first city to get a storm like this, and it wouldnt be the last. In 2005, Hurricane Katrina startled me not merely with its physical destruction but by the way in which its damage exacerbated inequalities, creating ripple effects throughout society. In 2011, Hurricane Irene gave the Northeast its first experience with what felt like a new kind of storm, whose impacts came less from the wind and more from the rain, which fell for longer periods and in greater quantities than before, raising waters to heights no one could remember. Locally and globally, these storms piled up in a way that made them impossible to ignore. There was scientific debate about the link between these storms and broader climate change, but there was also the bald reality of experience. I had the growing recognition that this was real life now, a new era for earth: record-breaking heat and cold, seasons that stretched in strange directions, weather that in all ways was bigger than before. All as foreseen.

And all as predicted. These storms were different, but so was their anticipation. Something had changed in the weather forecast. The hype was louder and longer, leaving time to pick the bread aisles clean and close schools before the skies had even clouded. Cable television and then social media each created a new and relentless rhythm. But even accounting for that increase in volume, the storms really were bigger, and we certainly knew about them sooner.

The extent of this startled me with Sandy. Norcrosss first warning was different not only in degree but in kind. Its going to require our attention, he wrote a full eight days before, like a forecast for his forecast. His broader concern was, as usual, with the track of the storm and its potential impacts. But his more immediate concern was with the outputs of the computer models. The most accurate computer forecast models are in amazing agreement today, he pointed out on Sunday. Its not often that credible forecast models consistently forecast a historic event, he wrote on Tuesday. By Thursday, Norcross was at Defcon 1 and far from alone there. The strong evidence we have that a significant, maybe historic, storm is going to hit the East Coast is that EVERY reliable computer forecast model now says its going to happen. Norcross and his colleagues had a view into the evolution of the atmosphere at the spatial scale of hemispheres and the temporal scale of days. This was a long way past merely watching Sandys development through the space-based camera of a satellite, extrapolating its next move. It was a simulation of the global atmosphere, capable of running ahead of time. Amid a lifetime of weather, it all added up to an improbable, nearly inconceivable, prognostication. I understood that we use computer simulations for weather forecasting. But when had they gotten so good?

In the weeks after Sandy the weather models had a moment of celebrity. They were not new but they were newly powerful. Meteorologists use the word skill to judge the accuracy of their predictions, and it has a specific definition: the measure of their ability to forecast the weather better than climatology, meaning the historical average for the place and date. If the average high temperature in New York on March 1 is 45 degrees, any forecast has to be right more often than those climatological averages to count as skillful. Generally speaking, with each passing decade meteorologists have been able to make that claim one day farther into the future. That means a six-day forecast today is as good as a five-day forecast was a decade ago; a five-day forecast today is as good as a three-day forecast two decades ago; and, most dramatically, todays six-day forecast is as good as a two-day forecast in the 1970s. All of that improvement is thanks to the weather models. It is often credited to faster supercomputers or better satellites. But I suspected it wasnt as simple as that (as if supercomputers and satellites were ever simple). The models were a black box. How did they work? Why were some reliable (and some not)? Who ran them, and who built them? I wanted a look inside.

In my previous book about the physical infrastructure of the Internetall the data centers and undersea cables and tubes filled with lightI discovered that even the most complex systems are still built by people; they exist in real places and evolve according to some human intention. I learned the most while moving slowly, planting my feet and examining the object in front of me, and talking with the people who built it. I could tell that the source of todays weather forecast was a similar kind of story: complex, ubiquitous and urgent. I knew that if I examined the systems that forecasted the weather with patience and rigorif I stopped looking up at the sky and instead looked down at the machines that watched itI might understand this new way of seeing into the future. I wanted to know how the exceptional forecast for Sandy came to be, and what it might tell me about the exceptional forecasts still to come. But I was also curious about the banal, quotidian weather forecasts I looked at every daylike the ones that said it would rain at four oclock three days from now and often shocked me by being right.

Sandy revealed a paradigm shift in weather forecasting, which now depends less on the day-to-day insights of any human and more on the year-by-year advancements in computer simulations. These prescient weather forecasts were possible not because we had developed a remarkable new skill, but because we had a remarkable new tool. Knowing the weather is one of our oldest desires. After millennia of wishing, we had wired up the earth: with satellites and instrumented balloons; with thermometers, barometers and anemometers; with supercomputers and a purpose-built telecommunications system to tie it all together, in order to see ahead of time.