AI’s bias problem — and what we need to do to solve it

Photo courtesy: CNET

Today, artificial intelligence (AI) has spread into all corners of life. Take Amazon Alexa for example. We wouldn’t even bat an eye at someone asking Alexa to play a song, hail an Uber, or pull together a grocery list. AI is becoming just as common in business too. Virtually everyone has watched a movie or TV show on Netflix. Those suggested titles? That’s Netflix using AI to review past watching behavior to recommend new content to keep you engaged and subscribed.

AI presents tremendous value for Amazon, Netflix, and practically any modern company, helping to unlock new insights from data and guide major decisions. But even with everything possible with AI, there are a few things to watch out for — high on the list: unintended bias. We need to recognize where potential biases come from to prevent problems from popping up in our AI applications and make sure they deliver the intended results.

The two big reasons for AI bias: data and training

One of the biggest ­causes of AI bias is lack of diversity in data samples. Put simply, an algorithm is only as good as the data you put into it. For example, airlines will routinely run sensor data from aircraft engines through AI algorithms to predict needed maintenance. But if a system is trained with data from flights over the frigid upper half of North America, and then applied to flights over Central and South America, the data will likely fall outside of the model’s parameters and provide the wrong recommendations.

To be honest, it can be hard to get complete, well-rounded datasets to train an AI system, so most just use whatever is easy and readily available. Sometimes, we might not even have the data at all. For example, software to help human resource teams select diverse candidates can have a tough time if it only has data on homogeneous workers to work from.

The other big driver of bias happens when we have rushed or incomplete training algorithms. For instance, a chatbot designed to become more personalized through conversations can pick up bad language, unless we take time to train the algorithm not to. Microsoft learned this the hard way with its Twitter bot “Tay” that started posting some very politically incorrect tweets. Concerns around proper algorithm training are why it’s hard to justify AI use cases in areas as crucial as criminal justice.

Why do some rush training? You could say it’s due to the popularity of agile programming, which promotes short iterative development. On top of that, excitement around getting AI going leads to premature applications. Many don’t spend the time and effort in planning and design work, particularly when it comes to current AI limitations in processing things like common sense, fairness, and impartiality. This is where people with domain knowledge will be key. Domain experts can help think through potential biases, train the models accordingly, and govern over the machines to see that they don’t fall out of line.

Diversity in data and teams can help

The best way to prevent AI bias is to use comprehensive datasets that account for all possible use cases. If there is imbalanced data, we can look to external sources to fill in the gaps and give the system a more complete picture. In a nutshell, the more comprehensive your data, the more accurate the AI will be.

Diversity in the teams working with AI can also address training bias. When we only have a select few working on a system, it becomes skewed to the thinking of a small group of individuals. By bringing in a team with different skills and approaches, we can have a more holistic, ethical design and come up with new angles.

Take for instance one wealth management firm that had multiple teams train an algorithm for driving higher trading income. The obvious approach was to look at data on 30–35 year old, single male day traders, the expected sweet spot. One of the teams — with members including customer experience and domain experts — not only addressed the original objective, but also identified another opportunity in the 50–55 year old, single women demographic — a highly investing segment that was previously untapped. Such diverse teams discover questions we wouldn’t know to ask.

AI can also minimize bias

For all that has been said about AI bias, we are actually using AI to address unconscious biases in human decisions and actions. Going back to the recruiting example, job descriptions might have unconscious gender biases that can deter men or women from applying for a position. An AI program can be trained to review descriptions and flag and replace words that can be construed as hypermasculine or feminine, such as taking “war room” and making it more neutral with “nerve center.”

There are a few keys things to keep in mind when addressing unintended bias in AI. There needs to be comprehensive data coverage to make sure we can train the machine to work with all scenarios. Diverse teams with domain knowledge can help, bringing in different angles and ideas. But we can’t rush it. We need the right design, planning, and governance in place to make sure the technology delivers on its promise.