Driving Business Value with a Data Science Operating Model
Written by Tom Marek, Para Sen
“Nine out of 10 businesses fail to generate meaningful financial returns from AI investment” [1]
Over the last half century there have been several cycles of both optimism and disfavor around the use of artificial intelligence. Those investing in this innovative technology are often brought to abandon it due to the lack of a tangible return on investment (ROI). The latest cycle has been powered by huge advances in cloud and open-source technologies, along with access to large volumes of data. However, even with the right ingredients for success there must be careful consideration of how they all come together to deliver the desired outcomes. We have synthesized our learnings from several engagements into what we call Slalom’s Data Science Operating Model (DSOM).
In this article, we present our thoughts on what a data science operating model is, why it should be a priority, what value you can derive from it, and some of the hurdles you might encounter on your journey.
What is an operating model?
At heart, an operating model is an engine that combines the right people, processes, and tools to help an organization deliver business value in a scalable, consistent, and responsible way. It is the mechanism that provides a systematic and reliable way of delivering on a business’ core objectives. In short, it is the ‘how’ of getting business done.
An operating model can be applied at the level of a single business unit or an entire enterprise, depending on the need and the scale of the requirements. A more cohesive, consolidated approach is always recommended, but if there is potent justification to have an operating model scaled up or down, it should be applied.
Why invest in a DSOM?
“87% of machine learning projects never make it beyond an experimental phase into production” [2]
The most successful technology innovations are those that can leverage cutting-edge tools to make an outstanding business and human impact. A DSOM helps to spark innovation, sustain delivery, and scale capabilities driven by a business outcome.
Creativity, Productivity, and ROI
As companies continue to discover the power of data for their insights, products, and business objectives the need for an operating model becomes increasingly acute.
A DSOM enables the right infrastructure and processes for companies to build and operationalize data science products and use cases effectively with a laser focus on business value and ROI. In contrast, the dearth of a solid operating model means that organizations often have a long pipeline of great ideas and unfinished projects that never reach completion.
Data Democratization
Increasingly, organizations are moving towards the principles of data democratization to enable their people to leverage information and generate both real-time insights and actions and long-term strategy. This necessitates a solid data strategy and platform to power the responsible and effective use of information. A DSOM provides a consolidated foundation that enables not only the development of great data science products, but also the enablement of data literacy which is a core element of data democratization.
AI/ML at Scale
In order to achieve the standards of maturity necessary for AI (artificial intelligence) at scale, organizations must invest in advance in the right mechanisms to enable the experimentation, development and deployment of suites of products and use cases. Getting the right skillsets, infrastructure, processes and operational rhythms is extremely important.
A DSOM is relevant for organizations at every point in the spectrum of analytical maturity, from those that make infrequent but regular forays into advanced analytics, to much more mature organizations that have plenty of machine learning (ML) champions, but still lack a systematic way of choosing and executing the right data science use cases.
A DSOM supports business value generation in the following ways
· Idea generation for pipeline of data science use cases
· Governance processes for people, data, tools, and business value measurement
· Portfolio planning and prioritization of high-value use cases
· Measuring ROI from the very start and throughout the product lifecycle, at both a use case and portfolio level
· Improved use of resources and infrastructure leading to better quality and resilience of products and tools
· Attracting, building, and retaining the right talent, as skillsets and teams are deployed on high-value, impactful projects
· Operational support for rapid and efficient deployment (AI for All/ML Ops)
In short, it supports your desire to utilize data science the right way for maximum business value generation, improving benefits for customers and employees alike.
Core components of a Data Science Operating Model
Strategy: Align data strategy with business strategy
It is important to define a data strategy that works not just for today, but that will guide future changes required within the organizational functional structures, roles, processes, policies, and metrics that will be implemented. This journey will require executive sponsorship and a focus on organizational change management.
Organization and People
The right organizational and functional structures, along with open communication channels should be established to ensure teams are working well together. This allows first movers to take advantage of early organizational investment in an operating model, and fosters innovation and collaboration.
Also, think in terms of a layered decision-making process where discussions and ideation can happen at multiple layers in the organization, from the most strategic executive discussions to the individual business units or teams. This can be done through a combination of Strategic Steering Committees, Centers of Enablement (CoE), Communities of Practice (CoP) and Data Governance Boards. One important outcome is that data science teams have what they need to be successful:
· Unlocking ‘fit for purpose’ data allowing them to focus on the core of their work with lower ‘data wrangling’ overheads
· Better collaboration to ensure a faster flow of finished products actually going live
Processes
In our framework we have defined a staged process allowing the agile ingestion, evolution, delivery, and maintenance of progressively prioritized use cases and projects. In concert with this phased agile process, we apply an ROI discovery and definition model that supports optimal assessment and recovery of investment and benefits.
Creating lightweight but clear processes around use case intake and prioritization allows teams to form better working relationships and become much more productive.
In the Concept stage data science use cases are rapidly assessed by a steering committee. In the Define stage, initial prioritization and epic work breakdown occurs and a subset of use cases receives further investment. In Prioritize some use cases are funded, and others might go into a holding pattern or be retired entirely. Investment and resources deepen through the Build phase, with quality, performance, and maintenance considerations being primary in the latter stages of the process. The entirety of the process is supported by a progressively matured ROI evaluation that forecasts ROI for future comparison to actual ROI.
Metrics
Metrics should be used consistently to help drive the operating model. Some key metrics could be around ROI, length of projects on hold, movement of use cases across stages, human and labor resources utilized, and percentage of projects completed in defined time periods. The important point is to agree on a set of metrics that reveal portfolio health and creativity.
Tools and technology
All tools and technology should be examined and selected based on the value they will drive. Intelligent investment of resources and funds are the best way to facilitate continued interest and investment. Here we talk not just about the technology underlying data science solutions, but also the right program and portfolio management tools and dashboards to deliver and monitor current and forthcoming work. Transparency and comprehensive oversight support informed decision making, faster delivery, and necessary course correction.
For e.g. the dashboard below will allow you to track core metrics such as total cost, total return and return ratios. The dashboard also provides insights into teams that are ready to start moving up the enablement maturity model (e.g. — a See, Do, Teach or Do, Partner, Support model.)
When’s a good time to start?
If you have more than one high-value analytical use case, either in the planning or the delivery phase, you should already have started to consider your roadmap and your ability to deliver on it. What a DSOM offers you is scalability and quality, and as we all know those are things you need to start planning for early in the development lifecycle to get them right.
On the other hand, if you already find yourself with a slew of data science use cases, a DSOM can have a transformative effect on how you are building and deploying AI at scale.
In short, while it’s never too early for any aspiring data-driven organization to begin implementing a DSOM, there is still much productivity to be gained at every stage in the analytical maturity journey.
Common Challenges
The creation of a DSOM is not without hurdles, but they are worth overcoming. Any issues you encounter while setting up a Data Science Operating model are issues you could reasonably expect to face during your data science journey. A DSOM would simply surface these issues in a more systematic and timely manner.
· Justification of investment — An upfront investment of time and effort requires appropriate justification. Here is where you can start to apply principles of ROI to showcase the business value of an operations. Build your bold vision (Modern Culture of Data).
· Data is key — the quality of the data underlying your data science initiatives will play a constraining role in the speed with which you can progress. A good operating model will help to highlight which uses cases are suffering from the lack of quality data. An inadequate data strategy will be one of the first.
· Getting the right talent — many organizations face the prospect of hiring or training for the required skillsets. An important consideration is that AI works best when benefiting from the right collaboration of business, technology, and data science representatives.
The benefit of identifying challenges early on is that priorities can be adjusted to move ahead with quick wins while creating a longer view to tackling the harder issues methodically and effectively.
Conclusion
There is much hyperbole around AI and Machine Learning, which often distracts from both setting and achieving real, attainable goals. A strong operating model is a consistent way of grounding and stabilizing the substantial benefits that can be derived from the power of advanced data science. Start with a consideration of the core components, mechanisms and processes that deliver value and innovation for your business, and make sure this innovation is backed by a solid return on investment. Most of our partners and clients already have some of the constituent elements of an operating model. There is rarely a need to start from scratch, and if there is, there are many ways to move efficiently with committed leadership, a defined vision, and the right resources.
Other Related Articles:
ROI in AI: Measure value to deliver value
MLOps Part 1: Assessing Machine Learning Maturity
MLOps Part 2: Machine Learning Pipeline Automation with AWS
Footnote:
- “Are You Making the Most of Your Relationship with AI?” BCG, October 20, 2020, accessed November 3, 2021, https://www.bcg.com/en-gb/publications/2020/is-your-company-embracing-full-potential-of-artificial-intelligence
- Enrique Dans, “Stop Experimenting With Machine Learning And Start Actually Using It”, Forbes, July 21, 2019, accessed November 3, 2021, https://www.forbes.com/sites/enriquedans/2019/07/21/stop-experimenting-with-machine-learning-and-start-actually-usingit/?sh=252f67ed3365
