For AI, Often More Is Better: Don’t Let Decision-Makers (Man or Machine) Do It Alone
Why do some leaders want to play in a one man band? A recent HBR article, reviewed 11 myths about decision-making, most of which were some variant on the theme of “I’ll do it by myself.” Sometimes it’s a straightforward “This is my decision alone; I don’t need to involve others” or an “I trust my gut” or an “I have all the information I need.” Other times it’s the view that “There is only one way to do this.” The result is the same: a failure to get input into a decision.
Yet, there is strong evidence that “phone a friend” or “run it by a colleague” provides the input you need, sometimes to win millions. Getting input into a decision isn’t a new strategy for ensuring you haven’t missed anything.
Sometimes it’s referred to as getting a second opinion. But an “opinion” is not really what it is. As the American comedian, Rodney Dangerfield, joked, “My psychiatrist told me I was crazy and I said I want a second opinion. He said, ‘okay, you’re ugly too.’” Those are opinions. But we’re talking about getting more input into a decision, making sure that we are not seeing only one side of the story.
The vast majority of decision-makers report that they want to be data-driven, and companies are expanding sourcing of external data. They want new data sources to improve their decisions and/or better train their AI models. They want to ensure they see all sides of the story and reduce the risk of potential biases.
The “Friend” You Call Might Not Be Human
“Siri” and “Alexa” are those “friends” a growing number of people call. AI adoption has accelerated on the consumer level, and increasingly so for businesses as well. According to several recent polls, the majority of companies have accelerated their AI adoption. A study by PwC found that 52% of companies have accelerated their AI projects during the pandemic, and 25% now have processes fully enabled by AI with widespread adoption. Of all the respondents in the PwC survey, 86% say that AI will be a “mainstream technology” at their company in 2021.
Another survey highlights similar adoption trends. A Harris Poll with Appen also found that 55% of companies accelerated their AI strategy in 2020 due to Covid, and 67% expected to further accelerate their AI strategy in 2021. And, respondents indicated commensurate budget increases: AI teams with budgets from $500k to $5 million range grew from about one-third in 2020 to 53% in 2021; those with less than $500 million fell. Companies and their decision makers are increasingly turning to their AI assistants to inform decisions.
New Rules On The Playground
Concerns about the effects of AI are loudly voiced by advocates of AI ethics and regulation. Approaches vary, however. And, the patchwork of existing and draft regulations and guidelines reflect this diversity of opinion. According to Algorithm Watch, 173 guidelines and frameworks for ethical AI have been published by governments, companies and academics. But there is one that is poised to break through the noise.
The EU has been the vanguard of AI policy. A proposed regulation was published earlier this year. As Martha Bennett at Forrester Research highlights in a recent report, EU regulation is binding, immediately becoming law in each country as opposed to an EU Directive which requires member level legislation. In the case of AI, the EU wants no room for interpretation at the national level, and no potential for the rules to be diluted or extended. And, as for enforcement, they really mean it. Maximum fines exceed those for GDPR violations, up to 6% of global turnover or a $36M maximum, whichever is higher. GDPR’s maximum penalty is only 4%.
The crux of the regulation is similar to those in the 2019 EU Ethics Guidelines of Trustworthy AI in which prohibits practices that “have a significant potential to manipulate persons … subliminally… in order to materially distort their behaviour… causing harm.” The new regulation establishes four risk categories — unacceptable, high, limited, no risk — and provides a tool to assess AI models. This applies to each model deployed — and requires constant vigilance. With most other regulations calling for studies and falling back on past “fair” practices legislation, like “fair housing” and “fair credit,” the EU regulation is likely to be the one most adopted, as it applies to anyone doing business in Europe. And, just like with GDPR, other governments both national and regional will likely follow suit.
With AI, Often More Is Better
So we know there will be more and more AI. That’s a given. And, that brings more concerns (which proposed regulation will not likely alleviate). These concerns were voiced by two panels on AI and its potential impact on society that I participated in a few weeks ago at GITEX. One panel focused on AI ethics and regulation, the other on recommendation engines and how they could potentially take over the decision-making process. As I argued on the panels, to address these concerns and to make AI effective and ethical, we need more of a few other things:
- More monitoring
- More metrics
- More data
- More literacy
Manage the AI — like any other worker. Imagine that you hire a new employee. You have a job description. You identify requirements based on the key responsibilities and tasks of the role. You establish metrics to measure the success of the new employee in performing that role. You assign a manager to monitor the work and performance of the employee. And, perhaps you have established processes for how to address deviations from that performance, or in the worst case scenario for addressing violations of the company’s code of conduct or ethics. We’ve all heard cases of rogue employees. And, those who’ve managed employees know that even the best employees need guidance and sometimes correction.
This process also applies to the deployment of an AI model. From job description to performance monitoring and evaluation, the model must be managed. The moment you deploy the model it deteriorates or changes behavior depending on the data that flows into it. In the best case scenario, the model will learn and improve in its role. But it might deviate from expectations, and eventually might not be a good fit.
Sometimes we hear that a model can’t be explained. That is also true for a human employee. As with the case of the employee, the model must be continuously monitored and evaluated based on performance. And, deviations must be addressed. Models can be retrained, replaced, or retired. Just as humans don’t make decisions alone, neither do AI “workers.”
Adopt multiple, measurable metrics. The challenge in evaluating an AI model lies in the metrics used to judge behavior or outcomes. We’ve all heard the stories of bias in employment offers and medical treatment. News of these incidents have sparked the growing concern about AI ethics and fairness. But what is fair? If it is fair to me, is it also fair to you? My mother always said, “Life isn’t fair.” But that’s too simple. The problem lies in the definition. And, according to academics, there are 21 different definitions of fair. Then how do we measure our AI?
It’s not enough to simply eliminate demographic information such as race or gender from training data, because in some situations that data is needed to correct for biases. As my former colleague Brandon Purcell summed it up, “Unawareness is not fairness.” Instead researchers are testing new metrics for evaluating the impact on certain subgroups, and digging in deeper to understand differences between the total populations and outcomes. Companies are experimenting with these academic findings to improve the outcomes of their models. For example, Amazon is experimenting with a fairness metric called conditional demographic disparity.
In the meantime, as new methods of understanding the impact of AI models, the best approach is to make a cocktail — a mix of metrics. Evaluate AI models according to multiple measures of fairness. Also explore intersectionality, or combinations of different demographic characteristics. These methods of monitoring AI require more and diverse data.
Embrace data diversity. More data is better, but diverse data is best. In several of the cases mentioned above in HR and healthcare, the data used to train the models embodied existing biases. The data was not representative of the population or target audience. Training AI models requires a massive amount of data. Alibaba’s Ant Group in China uses over 3,000 variables to evaluate loans to small businesses. The process, which has delivered funds to over 16 million businesses, takes three minutes and involves zero human bankers. The default rate so far: about 1 percent.
In a recent healthcare example, more data improved care during the pandemic. In the spring of 2020, the CDO published a model predicting COVID risk. However, the model was trained on Medicare data alone, representing a predominantly elderly population. Several providers retrained the model on Medicaid data, representing poorer patients of all ages. The model was then open sourced and six other providers contributed to model training. The model helped these providers prioritize care and outreach for vulnerable populations, and was recently nominated for a Gartner Eye on Innovation award.
Fortunately, many companies recognize the need for more data. In the findings of the Harris Poll, an overwhelming majority of organizations have partnered with external training data providers to deploy and update AI projects at scale. Those who use an external data provider are more likely to report that data diversity and bias reduction are “extremely important,” compared to those who do not. They recognize the need to address potential biases with more and diverse data. They recognize the importance of more training for the new “worker.”
Data literacy establishes a strong foundation. Training isn’t only for the AI “worker.” Everyone in the organization must be part of a data literacy program. Everyone in the organization plays some role in capturing, protecting or using data today. When employees don’t recognize data or understand the importance of data and the value that it brings to the organization, mistakes can easily be made. Data literacy isn’t about the business analysts or data scientists. They are already experts. Data literacy must start with data awareness, and that applies to cashiers, techniciens, teachers, nurses, administrators, decision makers, and everyone else. When data is captured correctly and protected appropriately, it can be used effectively. And, when the organization achieves a certain level of data literacy, potential issues with data and its application in the workplace are more likely to be surfaced and addressed. A deeper dive into data literacy is a topic for another blog. But for AI, more literacy is better.
And, that means better governance. The bottom line is that AI requires governance, and governance today must include ethical and responsible use of AI. New regulations may establish requirements for AI. But as Martha Bennett, VP and Principal Analyst at Forrester Research, pointed out, it’s not about having a “compliant system.” It’s about establishing the processes to ensure compliance. We talk about “responsible AI’’ or “ethical AI’’ but ultimately it’s the company that must be both ethical and responsible — and accountable.
Forward looking companies will use the regulation to establish these processes, getting their data house in order. Because the ultimate objective is not only to comply with regulation but to comply with the expectation of customers and to deliver value from their AI models — the new AI workers — and the data that powers them.
