This post was initially published here.
There’s a common personality trait between many people who work in data science, and that’s that we love having meaningful impact in our roles. Many of us wouldn’t have chosen this career if our work wasn’t both valuable and important to our companies direction. We need to accept that this also means we should be uncomfortable in our work and put ourselves in uncomfortable positions.
“Ethical design is not superficial”
— Laura Kalbag
In many ways, the way we’ve developed software in the technology industry has been to ease the pain points for our customers, but maybe we’ve been optimizing for the wrong thing. We should ensure humans have ultimate control of automated decision systems especially with these final decisions impact a persons life quality of life health or reputation. AI systems should be one input combined with a human’s domain knowledge to make a final decision. Put that responsibility back on our customers even though they may rather defer to a computer generated decision.
In situations where computers need to make significant decisions they should always be explainable, able to be overridden, and structured so the people impacted can appeal these decisions and institutes can reverse them. Without creating these safeguards we run into the issues of algorithmic bias. Believe it or not, this is the way we deal with AI currently in many industries such as housing, healthcare, and human resources. There’s no way to create effective human in the loop systems without explicitly defining the responsibilities of humans and responsibilities of AI.
There’s much care and consideration involved in creating these systems because we need to identify the full range of harmful uses as well as beneficial uses. There are always blind spots in unintended consequences and we need to have frameworks that allow us to mitigate the harm caused by these. An aspect of machine learning that we hardly ever discuss or prepare to deal with is that people previously discriminated against are still discriminated against in our systems. So in every case where there is bias, like redlining in housing, or anti-black and sexist hiring practices, our systems will only amplify that unless we work to specifically prevent it.
We have to deal with the bias of humans who are working in the loop with AI systems who reinforce their bias through how they accept or reject automated decisions. Typically this is a question left unanswered by most data science teams. To be able to mitigate the amount of harm caused by our AI systems we need to outline several factors. Who can report the AI system as not working properly? Who do these reports go to? Is there a way to turn off or suspend use of a system? Who gets notified when there is bias perpetuated by the system? Ultimately, what are the consequences of these systems functioning in ways we didn’t expect?
Going hand-in-hand with protecting a users livelihood, we need to be respectful of personally identifiable information and have strict outlines for how that information is used. We need to define who has visibility into the systems that show PII. We have to value ethics, equity, and accessibility at the forefront of our work or risk spending our life’s work on subjects wholeheartedly deemed biased, like facial recognition. Systems used to make decisions about people using their historical data need to provide an understandable level of security. We can build user trust by valuing transparency. Instead of having these shadow systems working while users are unaware, it’s better to explicitly state in entity as an AI system and what a user’s recourse is if they stumble upon a bug.
Lastly, in order to incentivize engineers and data scientist working towards ethical goals, we should reward team members for finding ethical issues. This can be with financial incentives as a percentage of the money saved by finding the issue or company — wide recognition for prioritizing people to profits. We need to change our incentive structure to reach any level of ethical Machine Learning. As long as companies incentivize speed to market, AI as their competitive advantage, and profitability over users, we can’t reach true ethical or responsible AI Solutions.