3 Strategies to Deliver Exceptional AI Value
Often AI projects meet model KPIs but not the business KPIs. What’s the secret sauce which will ensure they deliver business value?
This is a second part to the series of AI strategy. For the first part, see: 23 Questions to Ask for Successful Data Science Project.
42
Let’s say that some day in the future, humanity builds the all-powerful AI: The most intelligent of them all and the one that knows everything and can answer anything. All of humanity (or human-robot mutanity) erupts in joy because now we can finally get answers to life’s existential questions that have troubled us since the dawn of evolution. What is life? What is the meaning of existence? Why does the universe exist at all? And so on. And so imagine that they ask this entity …the Deus Ex Machina, ..what is the answer to life? The machine says, give me few million years and I will tell you. After millions of years of computation and memory processing, the machine finally announces that it has found The Answer! So the whole universe waits with bated breath and comes to the console where the answer will be disclosed. And finally the super AI reveals the answer!
“Its 42” !
“42 ???? What is that ?” humanity asks.
“It’s the answer!” says the AI.
“Answer to what?”
“Answer to everything!”
“But that is not very helpful …. To which exact question of life is this answer? Can you be specific so that we get value out of this?”
The AI says, “Well, in that case, you should have asked me the exact question…Now, go back and don't bother me until you find the right question!”
This paraphrased story from The Hitchhikers Guide to Galaxy is the story of most data science projects. Most executives think that one AI project will magically make everything better. The common paradigm is that no matter the business problem, AI is the solution, since AI is a brand-new and cool and all-encompassing entity. While there are lots of things AI can do, there’s one thing it can’t do (at least today), which is to find the right problem.
One model to rule them all !
Can AI solve all my problems? Maybe, but it can’t solve any problem until data scientists know which problems matter most to the business. How do Business KPIs link to Model KPIs? Having the right people in the room to answer that question is the most important part of equation. The common (and almost unbelievable but true) misconception some executives have is that AI is a panacea in itself. For example, a question we sometimes get asked …So we heard about reinforcement learning. Tell us what it can do for us. This is typically a wrong way to look at technology, instead it should be: Here’s the problem we have. Can you tell us how to solve it using latest & greatest methods?
Another common pitfall is that AI is a monolith. This is the idea that one single model is enough to solve a complex problem or many of them. In reality, every AI application is built from various models trained for a very specific tasks and then assembled to work with other models. The business success depends as much on finding that specific tasks as on the models themselves. More models typically means more effort, more money and more time. The question then becomes whether that allocation of resources is proportional to the problem. It’s always better to ask…
Is this the right problem for AI? Have we identified the right problem and is it defined in the right manner, validated by the right people, and are we measuring its success with the right quantity?
3 Pillars to lay a foundation for great AI Value
1. Socratic Questioning
- Socratic questioning is a disciplined method of successively questioning and going deeper. It proceeds by asking why this, why not that?, and what if? The focus is not so much on how to do something but on why we want to do what we want to. In this case, start by asking business stakeholders to describe their problem and then follow up, slowly peeling the layers of the onion. Here is a ready resource for following Socratic questioning in this context: 23 Questions to Ask for successful data science project.
2. Holy Trinity of Business Impact
- The Achilles hill of stakeholders is that having an open-ended problem in AI normally amounts to a money pit where no value comes out of it. The problem has to be SMART. It must be solvable within a time-frame and budget that stakeholders agree to.
- To think strategically about this, create a checklist of three things that together define business impact. If a business problem has a checkbox on all three things, then its a good problem to go after.
- Consider whether the problem has a business impact which is viable in the long term, whether it has eminent value, and whether it is doable within the given time and budget. Writing down ideas and then validating these factors with business stakeholders goes a long way toward ensuring success. The data collected can have a weight-age system to make decision making more transparent and data-driven.
3. Storytelling == AI Value
In the end, any AI project is only as good as a story it can tell. It’s true that every dataset CAN tell a story, but it will NOT until you build your characters at the onset.
- Know the cast of characters ..who is your audience, why do they want the information, and why should this matter? …It should have suspense, some drama and most importantly a point of view. A good data science story is one from the point of view of a narrator. That narrator can be a sales rep, support agent, director of marketing…or multiple characters whose lives will be impacted by this data science project.
- Be sure to link the problem to Business KPIs. Or map the relationship between model KPIs and business KPIs at the defining stage of the project.
- For example, how does 90% accuracy on retention translate to NPS scores? Or how does 70% precision on bad debt reduce the new sales rate? Or how much reduction in staff does a NLU project to classify complaints yield? More often than not, such analyses reveal that the metrics that matter to the business can differ from those that matter to data science. Aligning them needs to happen prior to the start of project — not at the end.
How does this differ from Design Thinking?
Design Thinking originally created for product design solutions is ultimately based on a persona system. With many AI projects, the personas are totally absent or misleading. Be sure to get stakeholders and not data scientists to sign off on which personas matter most.
You can have a machine learning model that maximizes profit but what if the business values growth over profit?
Growth vs. profit is a key business decision and any data science project can be a righteous loser if it doesn't take that fact into account. The personas that are relevant to determine such AI value are Value Creators and Value Assimilators.
Value Creators and Value Assimilators
Consider the AI process as having two actors: Value creators (business stakeholders who are impacted by the model outputs) and Value Assimilators (Business Strategists). For example, with an attrition model the Data Science Team & Sales Representatives might be the value creators while the executive team, planning, sales & marketing might be the value assimilators. Or the Analytics group might be the be value creators while the Finance Team (who put together budgets using the models) might be the value assimilators.
In order to extract maximum AI mileage, these two groups need to be on the same page, on the same paragraph, and even on the same sentence. When the rubber meets the road, we see lot of friction and its important to avoid that friction early on by agreeing to a quantitative definition of AI value.
Strategize to tell your story from the point of view of these two groups and have that cast of characters prepared early on. The road to AI value is not hard, but the best fruit comes when the seeds are planted properly.
This is a second part to the series of AI strategy. For the first part, see 23 Questions to Ask for Successful Data Science Project.
Call to Action
- The notebook and data that form the basis for this blog are available on Github.
- Click to learn more about the IBM Data Science Elite Team.
