ROI for Business AI Remains Elusive
For AI projects to go beyond ‘nice-to-have,’ they must better demonstrate their business value, but how?
By Thomas Davenport
In the summer of 2020 my colleague Laks Srinivasan and I convened a virtual meeting of chief data and analytics officers (CDAO) at the RoAI Institute to discuss the issue of how best to achieve return on AI. We wanted to learn to what degree these senior leaders shared our perspective that AI faces an important economic return issue, and what, if anything, their companies were doing to address it. The industries represented included automobile manufacturing, consumer products, healthcare, insurance, logistics, marketing services, and professional services.
Defining the Problem
Perhaps we are guilty of selecting participants with whom we often agree, but the participants did share our view that AI projects face a critical ROI issue. Craig Brabec, then head of insights and analytics for Ford and now Chief Data Analytics Officer at McDonald’s, commented that, “Let’s face it — we’ve all benefitted from the hype around AI. But now we have to deliver.”
Business stakeholders, he argued, may expect outsized impact from AI given the hype levels. In addition, expectations about the speed of AI development were rarely tempered. Benefits from AI do not accrue as rapidly as many anticipated with dependencies on large volumes of data, process and system integration, and a supportive culture. Dr. James Adams, Senior VP and Chief Medical Officer, Northwestern Medicine, commented: “When it comes to AI in healthcare, the strategy is easy — everyone wants it perfect, free, and now.”
One participant mentioned that organizations often spend too much time and emphasis on AI tools, technologies and models, and not enough time on the measurable, incremental value of AI projects. Consequently, project teams can’t find sufficient funding to be effective.
Several of the executives agreed that anecdotes about value from AI projects only last so long. “If it’s a one-off anecdote, it erodes as a win pretty quickly,” one commented. There is also the problem that “AI is everywhere and nowhere.” A data and analytics leader commented that, “Once an AI project is mature and well understood, it ceases to be acknowledged as AI.” This, of course, makes it difficult to continuously attribute value. The executive suggested perhaps defining AI for the organization relatively conservatively, and to point out long-standing AI capabilities.
None of the discussion participants commented that “return on AI” wasn’t an issue for them. One said, “We’re describing AI as a progression from data to insights to outcomes,” and without outcomes AI simply doesn’t matter. There was widespread agreement with the comment. Of course, some of the issues around return on AI would be faced by any technology, or at least any new technology. But others are specific to AI because of its insight and decision orientation, or because of the extreme levels of hype and fear around the technology.
Mo Chaara, Vice President, Enterprise Analytics & Data and CDO at Philip Morris International, and formerly Senior Director, Advanced Analytics Group |Data Sciences, Machine Learning & AI at UPS, commented: “If we open the door to AI, how far do we go beyond manual and human-driven processes. Are we ready for it?”
Another major challenge when it comes to ROI on AI, according to Vipin Gopal, Chief Data and Analytics Officer, Eli Lilly, is, “What is included in the denominator? Measurement methodology has not kept up with advancement in AI.” Automation projects have become popular as they are easier to measure; there is a barrier to investment in innovation that has significant value creation potential in the long run. Dr. Adams at Northwestern Medicine said, “Though the ultimate value creation from AI is high, immediacy of financial return is a barrier where 40% of the hospitals are running on 1% operating margin.”
Assessing and Improving Return
Participants in the discussion took a variety of approaches to assessing and improving the return on AI in their companies. Maximizing return on AI requires broad awareness of AI across the organization, and therefore, investment in data and AI literacy is important.
Another key focus was to prioritize existing projects based on their degree of value and their orientation (summarized as “hold, fold, or double down” on projects). One executive mentioned that it is easier to measure AI projects involving operational improvements, but it’s also important to retain some that are more forward-looking and complex. Another agreed, “We try to emphasize value creation as well as cost savings, even though the latter are much easier to measure.” In the same vein, a key question at one company is, “How do we leapfrog with this technology?” That organization tries to set up business/technology experiments that allow easy comparison of the new use case to the current state.
The discussion of approaches to achieving return also touched on culture and reengineering business processes for AI. One company represented in the discussion has a strong “design thinking” culture, and that has been helpful in AI projects as well. Design thinking exercises are used to ensure that a business or customer problem is being addressed in the right way, with consideration of multiple alternatives. Several participants also mentioned a data-oriented culture; Gopal at Eli Lilly said that, “A culture of data, new processes, and retraining is a soft requirement for making AI successful.” Another said that, “We think that training, culture change, and reverse mentoring are important to support AI literacy, adoption, and use.”
Of course, adoption and use of AI are prerequisites to economic returns on the technology.
On the technology side, there was also a discussion about the importance of foundational capabilities like data and systems engineering for realizing benefits from AI.
Roles and Responsibilities
It is apparent at many of these organizations that the CFO and the Finance function play a critical role in achieving return on AI. As Craig Brabec put it, “Finance is the only scorekeeper.” One company is working with the Finance organization on developing metrics and methodology to do assessments of value in AI projects; another has informal consultations with the CFO for all major initiatives. Another company representative mentioned the need for educating the CFO and Finance on how to measure return in multiple ways. The executive noted, “Financial value from business outcomes as certified by the CFO is critical to AI adoption in our business.”
But in addition to the CFO and finance function, several companies emphasized the important role that other senior executives can play. “I’ve seen it across the several companies I’ve worked for,” one commented. “Right sponsor and right attention make all the difference.” Another data and analytics leader commented that the “pull” for new AI initiatives can come from leading business owners touting their own successes with AI: “Business leaders can make their outcomes go viral.”
In addition to spreading enthusiasm for AI’s potential, several leaders mentioned the importance of calming fears. Chaara noted, “There is fear out there of what AI can do to jobs and workers.” Those fears, it was argued, can only be resolved by a joint effort of the CEO and the organization’s senior executives, and an internal communications campaign.
It’s clear that the broad and prominent visibility of AI in the world today has created both an opportunity and a problem for organizations hoping to achieve return on the technology.
It’s important to keep expectations under control, to monitor and publicize success and actual results, and to minimize fears of job loss or major changes in skill requirements.
The companies involved in this discussion are aware of these issues and are working to address them.
First appeared on Forbes.com September 24, 2020 here.
Tom Davenport is President’s Distinguished Professor of IT and Management of Babson College, a Digital Fellow at the MIT Initiative on the Digital Economy, and a Senior Advisor to Deloitte’s Analytics and Cognitive practice.
