Perfect Storm: the next-generation weather company

I have a suspicion that there’s a new enterprise weather company to be built. To be completely honest, I don’t know very much about weather. I started reading and asking questions a few months ago. But last time I tried this, I met some great friends and great companies, including Eric at Bayes Impact, Sha and Kalvin at Nava, and Alex. So please, tell me how I’m wrong (or right).

Here’s the idea: there have been three tremendous changes in computational technology over the last decade that have yet to be fully utilized in weather forecasting technology. Incorporating these shifts will result in significantly improved weather predictions, just as they have already begun to individually transform other fields. But here we have, dare I say it, the perfect storm:

1. Cloud computing. Weather predictions today are computed on expensive dedicated mainframes. But this computing paradigm doesn’t match the computational needs of weather prediction. The computational power needed to accurately predict weather is highly dependent on conditions: it’s harder to compute a hurricane than a clear blue sky. It would be extraordinarily expensive to build for peak needs in the mainframe paradigm, but this is a use-case well served by cloud computing.
2. Low cost sensors. The dividends of the smartphone wars that have enabled drones and the Amazon Echo will also enable low cost weather sensors, which will create extraordinarily granular weather data. If we had “perfect information” of weather patterns — we should be able to predict it perfectly. We won’t, but we can get much closer.
3. Machine learning. One of the remarkable observations in Nate Silver’s Signal and Noise (2012) is that weather prediction is significantly improved by having expert humans annotate the computer’s prediction. The reason, dated to the book’s publication five years ago, is that computers aren’t very good at reading images. This has has changed. Anything where a human expert is manually applying a set of visual rules on top of a computing prediction is likely highly inefficient.

But not all weather data is equally valuable. Can you predict air streams to provide better guidance for airline fueling? Can you better target hurricanes evacuations? How much more useful can seasonal weather predictions become for agriculture?

I suspect the best places to begin are outside of the United States, where you’re not competing against a freely distributed government service, with a budget of billions, and hundreds of years of expertise.

Now, tell me how I’m wrong.

If you liked this post, you may like my next one. Sign up here to get the next one by email.