Applying product practices for excelling in predictive analytics

Unlocking the potential of predictive analytics through a product development mindset

The past few years have seen data-driven organizations move from the minority to the majority. Data-driven decision-making is everywhere, from politics to advertising to consumer products. But when it comes to predictive analytics, we are still on a brink of a chasm — albeit an exciting one.

Organizations are responding to the need for advanced analytics in a number of ways, from building center-of-excellence data science teams to hiring an analyst and providing on-the-job training.

But we’re seeing that very often, the most critical component gets missed. Between understanding the novel (and sometimes complex) concepts intrinsic to data science, deciding on a team structure with no prevailing one, and managing the often high costs, organizations too often miss the point of advanced analytics: guiding better decision-making that results in better business outcomes.

Too often, the “why” is lost in technical details. In many ways, this is exactly where product development was at 30 years ago. We can leverage all lessons learned to make sure we extract real value and drive results.

In the next few paragraphs, I have extracted the most critical elements in building successful products that can help teams in building effective models.

Make sure the data team knows who they are solving for.

It is so intuitive to anyone building products that, to succeed, they have to focus on addressing the needs of a real user. This is the reason it became a common and necessary practice to start any product or even feature design by defining the target user. Any product manager from B2C to B2B knows they have to identify their users.

This is no different for a predictive model; it can only be effective if addressing the need of at least one business owner within the organization. This business owner has to become a real partner in the process from definition, evaluation, deployment, testing, and refinement. This is the only way to ensure you are solving their real pain and creating models they are excited to put into action.

Define what you are solving for.

There is a reason why Simon Sinek’s legendary book Start With Why became a must-have read for any product manager. If you don’t have a clear and super-focused understanding of your product’s ‘why,’ and precisely what you are solving for, there is very little chance it will bring value to your users. It is no surprise that defining the ‘why’ behind any product became the norm for any product development process.

The same is true with building predictive models. This might sound trivial; however, so many ML-based models don’t have a clear definition of the business objective that is serving the business owner. A product with no clear definition of what it is solving might result in a beautiful and fully functional product that no one uses. In the same way, failing to define the objective of the model might result in building a beautiful ML model that might never see the light of day.

Focus on experiments and a feedback loop.

Delivering successful products is a complex problem and not just a complicated one. Complicated problems are hard to solve, but they are addressable with a set of recipes, meaning they can be solved with a linear approach that includes rules and processes. Complex problems involve too many unknowns and too many interrelated factors to be deduced to rules and procedures.

Since product success relies on so many unknowns, the best practice today is to address it as a complex problem. To deal with that, product delivery includes constant experimentation and learning, each getting you closer to the desired state. The same applies to the process of building effective models. Even if your model is super accurate on past data, the path to effectiveness depends on many unknowns, like the adoption by the business owner, changes in the data, and organizational changes, not to mention small things like COVID-19.

To make sure your model produces business value, you have to measure its performance on live data. Based on that, you must have a way to refine it quickly, test its effectiveness, learn from it, improve it, and so on. This process has to be done in full collaboration with the business owner and within short cycles.

You have to know how the model will be used.

Imagine this scenario: A product team for a commerce site builds the ability to find the perfect price for each product. They have tested how this feature works on a prototype, and it is working correctly. When you ask them how this will be used on the site, they say, “I don’t know…I never really thought about it.”

Sounds ridiculous, right? Well, that is the case for so many teams building predictive models. And this is the perfect path to failure.

Just like product building, you need to start building the model with the end in mind. Will you need to push the data for another system to consume? Will you need to create visualizations that will empower manual action? Will the output be used daily or weekly? And so on. There needs to be a clear path on how these models become effective, i.e., how they bring real value to the business owner.

This is why at Pecan, we make sure any project starts by filling in this statement:

The [Who you are solving for] would like to [What you are solving for] so that we can use the outcome to [How will the model will be used].

By having this definition and starting with the end in mind, we create the kind of iterative, collaborative, and fast-building process that does not focus on just having a great model but that drives results for the business team.

originally posted on LinkedIn