Stop being data driven!

I get it, more data has been created in the last 4 years than in all recorded history before that. I get it, quants are moving off Wall Street and into every other business and finding correlations and insights that had previously been undiscovered. I love these two aspects of big data and I’m in awe of what could be done with a body of data like this. However, I’m under no illusion that any amount of spelunking through this massive data set will yield the kinds of “ah-ha!” moments that drive business transformation.

The current and commonly accepted “data-driven” behavior is misguided. It surfs the waves of Agile development, Minimum Viable Products, and continuous improvement by favoring incremental-ism over real value creation. The current data-gasm in tech should be received as a pedestrian, almost pathetic reminder that you have to form solid business hypothesis and validate outcomes. Beyond that, your data doesn’t tell you shit.

Here’s the common data-driven refrain -

Model A — Data as Input

• “I have data that tells me about my product/business performance.”
• “I mine this data for insights and use these insights to drive improvements.”
• “These improvements are measured for impact, driving continuous improvement and making my data set more robust.”

Bully for you! That model for data-driven continuous improvement will work for about a quarter while someone innovates from behind. Here’s an alternative model for using data in your business -

Model B — Data as Output

• My business has a purpose. I imagine ways this purpose can be fulfilled.
• I will run experiments to support this purpose.
• I will get data that tells me how these experiments performed.

To be clear, data is absolutely critical in this model. Only pursue those activities with positive outcomes and kill those with negative outcomes. You must use data to make this determination. However, this model is Business Driven rather than Data Driven. This model allows for those less tangible but equally impacting factors like passion, ambiguity, and intuition. This model doesn’t use data to reduce risk, it uses data to quantify it.

You want to run your business in Model B? Don’t look to your engineers or run-of-the-mill MBAs to drive your business using Model B. It’s generally not in their nature. These disciplines tend to attract model driven people who focus on execution. These people can be great and they’re certainly necessary — and they excel when playing a specific role. Business leadership requires people skilled at connecting with a broad set of stakeholders, the ability to synthesize points of view, the ability to apply and MODIFY existing models to accommodate new information, and to deal with ambiguity. They display traits of imagination, enthusiasm, and EQ. They take risks but never fail as all outcomes are viewed as just varying degrees of success. The good news is these skills and traits can be cultivated. Given the right culture, business practices, and development activities, people can hone these skills and traits. This can lead to unexpected results — but it relies on the curation of these people in your organization.

How do I know Model B works? Intuition and experience. Where’s the proof? The burden of proof is satisfied by years of business, product, and org impact. It’s not necessary to hire a quant to see it.