Weaving AI into Your Organization
Sponsored by Sov.ai
Human-AI Interaction and The Road Ahead
Inspired by the human mind, artificial intelligence (AI) has, over the past decade, entered almost every sphere of the modern business. From our phones to digital assistants, to even how we order coffee offline, AI has helped automate multiple functions that previously required human contact or effort. However, despite the great promise, this has not always met with success.
To that end, in this blog, I would like to discuss some of my experiences with the deployment of AI in business settings. Specifically, I review three experiences of mine that should give pause to businesses as they implement AI support in their interactions and decision-making. First, I speak about my experience with an algorithm that was introduced as a decision aide to guide a manager’s marketing actions. We then discuss the role that goal functions play in reducing friction when the AI supports business decision making. Finally, we explore how an honest AI interaction can lead to better human-ai relationships.
Before I describe these three experiences in more detail, I want to make clear that I firmly believe that AI implementations in business will improve efficiencies and decision making. However, during this process, businesses should not abdicate their responsibilities in ensuring that the AI implementation is done well and monitored continuously.
Data Maketh the AI
Imagine that you manage a leading brand at a well-established Fortune 100 company. You handled the firm’s advertising budget for well over a decade. While it is true that advertising budgets can be more efficient and ROI can always improve, you observed better than industry-average returns for your advertising budget over the years. However, you are now in the age of AI decision aides. Following other firms in your industry, you too decide that your firm could use an AI decision aide to improve the efficacy of your marketing budget. You bring in a somewhat well-known startup that touts its impressive marketing AI-engine. The startup works with your team to identify and define the data in a manner that allows their AI to recommend optimal marketing decisions. After six months of work, the AI finally recommends that you need to reduce your budget for television by 50%, print, and radio advertising by 80%, increase Facebook advertising by 300%, digital marketing by 150%, and store promotions and advertising by 30%. Overall, this would have also led to a significant reduction in the marketing budget. You now have a decision to make, should you trust the AI or your experience? Could you have been so wrong in your allocation decisions over the past decade? Is the budget reduction recommendation in line with your expectations?
The aforementioned problem might seem contrived, even though the numbers are, I dealt with this exact issue in 2018. Who was wrong here? As you might have rightly guessed, the AI was incorrect in this case. But why? The algorithms used in the analysis seemed to be correct on paper. However, as it turned out, the implementation was anything but. For example, to incorporate the firm’s complex web of store relationships, national and international marketing spend, and multimedia allocation of budgets the startup decided to aggregate data over time, across channels, regions, and product types within the brand. While this might appear harmless, improper aggregation of data introduces multiple types of biases into the data — which then alters how one uses the data to make decisions. Furthermore, because the effects of TV and print cannot be directly measured, the model seemed to think that there was a none-to-minimal effect of advertising spend in these media. This was further exacerbated by the fact that the startup lacked domain knowledge, and the firm did not insist on close interaction between its employees and the startup while the algorithm was being developed and implemented.
This example highlights the need for close interaction with any AI implementation from the very beginning. If the employees at the firm were incentivized to work closely with the startup as it implements the decision aide, the AI might have provided more plausible recommendations that would have helped guide decision-making. Additionally, because market structures and conditions change, such interactions and inspections should span beyond the initial stages of the AI implementation. With this example, I do not intend to detract from the innumerable benefits that the firm could have enjoyed with the decision aide if it had been implemented correctly. Rather, I would like to point out that firms should incentivize employees to work closely with any such implementation from the beginning, and on an ongoing basis, to ensure that AI updates its information correctly and provides recommendations that comport with reality and enhance business outcomes.
Know What to Aim for
The astounding success of self-learning AI such as AlphaGo and Watson notwithstanding, AI today still relies primarily on human ingenuity in its design and deployment, and human interaction to generate data necessary for its continual performance. Humans assist AI in its learning and improvement just as much as the AI assists humans in performing tasks or automating decisions. However, speaking to businesses, the boundless optimism in the improved efficiencies delivered through automation and optimization overshadow the often stark reality that AI programs are error-prone, susceptible to human biases, and not very good at performing well on cases different from training. For example, AI has been shown to amplify human biases based on gender and race, and perform poorly on data outside of the training set, even though it was trained on data that were similar (examples in finance, medical imaging, and diagnosis). While advances in research in this field will help solve several of these issues, similar issues and biases will emerge. Therefore, it is imperative for businesses to not blindly base their belief on the results from the algorithm, but learn how the algorithm and the human can work together to achieve improved results and increased efficiencies.
In the aforementioned example, the firm was lucky the AI recommended such extreme shifts in allocations. This gave them pause to think about their decision making and analyze whether the AI’s predictions were worthy of implementation or not. In many cases, the AI could only be slightly wrong, however, over time, blindly following these recommendations without experimenting with other allocations could yield a slippery slope to extreme budget misallocations that will only manifest when it’s too late, due to the delayed effects of advertising.
For example, visualize a business with a sales team that is now enhanced with the introduction of an AI. The algorithm used in this bot provides, based on the characteristics of a transaction, the optimal price the salesperson must sell the product to the customer. However, after implementing the bot, you find that salespeople overrule the AI’s suggested price in favor of a lower (or higher) price about 60% of the time. Should you now think that the salespeople were incorrect in their judgment? This could be true in some instances, but more often than not, it is likely that the salesperson overruling the bot had unique information about the transaction or client (e.g., the salesperson was looking to build a longer-term relationship) that the bot recommending the optimal price was not trained on. Because the data observed lay outside the algorithm’s training data, the prices recommended were no longer optimal.
Additionally, one must also question the idea of optimality in such scenarios. For example, was the bot trained on pricing strategies that improve profit per transaction or profit with respect to a long-term relationship? Was the bot trained on data exclusive to the firm, or using scenarios that might not apply to the client firm? In each case, automation could in some scenarios deliver sub-optimal or completely incorrect outcomes. In addition to this, it is important to know that in many cases, when conditioning for optimality, Goodhart’s law applies. Forcing salespeople to behave optimally via the algorithm could lead to sub-optimal or undesired outcomes. For example, if profitability per transaction is the measure to be optimized, the algorithm and salespeople will act to maximize profit irrespective of the effects on long-term relationships. Similarly, if retention is the goal, the short-term profitability might suffer. To overcome this, it is often recommended that several seemingly unrelated success measures be combined to design optimal goal metrics. Such multi-objective goal functions are more stable and less susceptible to the vicissitudes of a singular ill-specified target objective. Accomplishing this requires not only several iterations, over time and individuals, of human-AI collaboration and a deep understanding of the data generation process, but also a keen knowledge of the multi-objective goal functions and their behavior across multiple scenarios, common and rare.
Build Enduring Relationships
Recent advances in Natural Language Processing (NLP) and machine learning algorithms have led several firms to incorporate AI assistants, or chatbots, into their online and mobile platforms. The chatbots often manage the customer relationship from banking, browsing, shopping, and searching, to other pre- and post-purchase experiences. For example, bots improve call center productivity by answering the simpler questions around product use, returns, refunds, and replacements leaving the more complicated questions for human reps. The bots do so well at managing the experience that customers several times are unaware that they are even interacting with an AI, often assuming that it is a human answering their questions, i.e., until they encounter an error and the interaction stalls or they are forced into pre-programmed questions and answers. Further advances in NLP will no doubt make this even harder to detect and create even more seamless interactions. Given these amazing advances, how should firms use bots in managing the user experience?
A recent study explored this question in the customer service space. They find that chatbot disclosure reduced overall purchases (by on average 80%) — in other words, “undisclosed chatbots are as effective as proficient workers and four times more effective than inexperienced workers in engendering customer purchases. However, a disclosure of chatbot identity before the machine–customer conversation reduces purchase rates by more than 79.7%.” This result seems to indicate that people’s subjective perceptions dominate their objective experience when they know that they are dealing with the chatbot instead of the human option. This result seems quite disconcerting, however, in the same study, they also find that as the customer’s experience with the bot grows, these negative effects diminish over time.
But what if there was no human option available? What if people already knew they were dealing with an AI, and then the firm made algorithmic innovations that made the bot more “relatable”? For example, what if the bot spoke in the customer’s local language? I explored this question with a firm in Asia that was introducing a local language feature into its existing English language AI mobile shopping platform. To determine the impact of localization we first ran an experiment with new users that downloaded either the English language or bi-lingual version of the bot. Users were unaware prior to use which version they had. We find that users who downloaded the bilingual version tended to have a worse experience (about 80% more uninstallations) than those who downloaded the English version — however, those who stayed beyond the first few days with the bilingual version tended to order up to 50% more in the bilingual language version when compared to users with the English only version. For the next study, we widely advertised the bilingual feature and then studied the impact on the usage of existing users. In this case, existing users were those who already experienced the platform in English and then upgraded to the bilingual feature. Here we found that when users were aware of the new feature (positioned as a local product innovation in this case), they were not only more forgiving but were also more willing to try the product. We observed in many cases, existing users ordered about 20% more over the next 2 months when compared to the previous 2 months. Making the bot more relatable (in the local language) also led to an increase in the number of referrals made (almost double).
Given the results of these different studies, what does it tell us? Consumers, especially those who have limited experience with AI, tend to prefer human interaction to interactions with chatbots. However, with increased chatbot experience, the initial subjective assessments are replaced with more objective task success measures. When consumers are already expecting a certain experience with the bot, making the bot more relatable without priming the consumers about the changes, could lead to more negative evaluations of the experience. This could be because the bot’s attempt to relate might seem contrived rather than genuine in the consumer’s mind, hence making the consumer more critical to experiences in the local language. On the other hand, priming consumers to expect this change, and making them partners in the innovation seems to change the consumer’s perception of the innovation to the point that they not only react more positively to the change but also share the experience with their family and friends. In sum, even though consumers will never perceive a bot as a replacement to a human being, to build enduring relationships between the customer and your bot, ease the customer into their association with the bot, and make them partners in the technological innovations introduced in the bot.
The three examples described above are illustrative of common difficulties that firms face with AI implementations. As businesses proceed down this path and anticipate the advantages that AI offers, it would be useful to also note the myriad ways that AI implementations could fail. While some failures might be unavoidable, many could be prevented by a more careful implementation, trusting the AI to do its job, but verifying it at each step of the implementation.
