Open Data + Big Data = The Future

In 1986 I was addicted to MTV. Like all of the other kids at school we were glued to MTV to tell us what was cool and to have some understanding of the “Real World”. One day I saw the video for Walk This Way and had no idea who Aerosmith was, but my gut reaction was that it was the best song I had ever heard. Mixing rock and rap was such a brilliant idea in retrospect it is hard to imagine no one had ever real tried it. Give credit to a young Rick Rubin (22 year old hip hop producer at Def Jam) for thinking differently and calling up Tim Collins, Aerosmith’s manager to see if the Aerosmith guys would work for $8K a day to record something with his rising stars, Run-DMC. You don’t know until you ask…incredibly that song only took one day to record and revived and launched Aerosmith to superstar status.

Innovators like Rick Rubin tend to embody different perceptions, work without fear and have the art of persuasion & social intelligence to pull off iconoclastic visions that have the power to change old industries. Some can argue that Jay Z and Linkin Park’s Collision Course in 2004 was better than the Run-DMC & Aerosmith mashup. Maybe, but would it have happened without Rick Rubin setting the stage in decades prior? It is now 2017 and standing on the shoulders of giants is what provides the ballast for innovation and now thinking about science — trips to Mars, DAOs, VR, and autonomous vehicles. Those quantum leaps require significant historical advances and removing data boundaries is critical. Elon Musk has open sourced most of his plans for Mars.

If you believe that data is the new oil, there is a ton of data coming online every day. Digital data from governments (fed, state, local) at a global level is increasingly available and transparent. Open data can inform and drive business decisions, research in the sciences, retail and consumer behavior, and social sentiment as we know is able to move the democratic process. Big data analytics and packages make it easy to consume real time data which can help marketers understand shopping behavior, how many ships are moving liquefied natural gas out of Qatar can move oil prices, and so on.

One of the first companies to monetize open data, Climate Corporation (acquired in 2013 by Monsanto for $930 million) saw an application of weather data to help price weather insurance to farmers. This insurance brokerage business had been previously sold over the counter and was a fairly blunt instrument that can now be finely tuned by forecasting local weather patterns using decades of data from the National Weather Service and other geological surveys. Climate was a great deal for Monsanto to help optimize their own sales force’s throughput on its core seed and chemical businesses.

Sitting at the nexus of data, capital markets, fintech and the buyside & sellside I see a massive sea change and adoption in the industry coming in the form of investing and process flows utilizing open and big data together. With the onset of Web 3.0, the tools and faster computing unlocking business decisions and new investment ideas will only be limited to imagination and a firm’s ability to recruit data talent. There is an increasing demand for gluing data together correctly which is a heavy process requiring data cataloging, cleaning, parsing etc before any insight can be generated. Then the insights come next. It will be great to see more and more of industry “Walking this Way”.