Machine Learning: A Guide to Understanding, Adoption & Improvement
Part 4: Engagement
In the last post, Exploration, I shared why connecting a business imperative to Machine Learning (ML) and succinctly defining your objective are such important elements for a successful ML initiative. This post will review how you can validate if Machine Learning will produce the results you expect and want in a real-world setting.
Proof of Concept vs. Proof of Value
There is an important distinction between a Proof of Concept (POC) and a Proof of Value (POV), which often makes the difference between the abandonment or operationalization of a Machine Learning initiative. POCs have a role and can be thought of as a subcomponent to a POV, a POC answers the question from a data science perspective “can we do this?” — can we identify humans and objects in an image or can we predict how many streams a new song will have?
These are important initial questions to ask, but to really understand if there is value creation potential then additional steps must be taken. That’s where the POV comes in. The POV extends the POC into real-world settings to gain empirical evidence that the initiative will produce the expected results. This often entails allowing end-users to interact with the model as part of their daily work, i.e. if the initiative is meant to increase prescription adherence then do we see improvement?
Conducting a Proof of Value (user adoption)
A successful POV has a few key characteristics including user involvement, simplicity and usability.
Coming out of the Exploration phase we have our use-case; one that has a strong connection between the business imperative and Machine Learning, and of course it is measurable. Now we want to make sure the stakeholders (users) participate in the POV, because in most cases this will be their first time interacting with advanced analytics. We can make the introduction easier by following a couple best practices. First, make sure the user has direct experience and familiarity with the data. When they see new insights within the data, it will be much easier for them to connect with what they are seeing. The second-best practice is to keep frequent contact with users during model development to give the data scientists the maximum opportunity to iterate and make small adjustments instead of big course corrections.
Keeping the model simple upfront is also quite imperative for a successful POV. For example, deliver comparisons, contrasts, and new groupings. The ability to show new patterns to users and executives is a great way to build momentum behind the POV. Also, I particularly like the idea that we teach our users with simple trees but implement Machine Learning — it’s the equivalent of showing how a manual screwdriver works then when it comes time to build, we use a power driver.
Finally, during the POV we will want to get real-world feedback and validation, which means our insights need to be useable. It is important to not prescribe to the users what they will receive, meaning if they just want a simple list then provide a list, but if they want a visualization then give them a visualization — whatever will make validating efficient and effective for the user.
Where we can go from here
Once you have validated the strategy via a Proof of Value, you will need to put the model into production. There is no value creation until we can deliver the insights where, when, and how the user needs them. And that is the subject of our next post — Execution.
— — — — — — — — — — — — — — — — — — — — — — — — — — — — — -David is the practice lead for Advanced Analytics and Data Visualization for Slalom Philadelphia. Slalom is a modern consulting firm focused on strategy, technology, and business transformation.We help companies tackle their most ambitious projects and build new capabilities. @slalomphilly
