Worried about biased AI? Worry about human bias first.
How human bias is reflected in artificial intelligent applications
It is quite likely that your most recent job application has been analyzed by a machine and in case you got lucky, it passed it over to the recruiter to hire you.
However, for those cases where you didn’t get lucky, it would at least be good to know that you were not filtered out for the wrong reasons, like for your gender, hair color, or political preference.
In the AI field, many techniques have been developed to minimize bias, but when the data for hiring-machines stems from the last 20 male-only executives with certain preferences, then even the best AI techniques are most likely unable to help [examples: automated hiring discrimination, AI unlearns human].
The big change, opportunity, and dilemma is the transforming way of developing software. Instead of defining the logic by telling the machine what characteristics a job-applicant should have, thousands and thousands of applicant profiles from the past are taken and matched with the final decision made by recruiters. These information pairs are passed to the machine, which learns the applicant-characteristics and matches is it to results of the application.
This procedure has led to multiple successes, as machines are able to identify tiny pieces of connected information that are too small for humans to realize but can have a significant impact on the final decision. Nevertheless, when humans’ decisions are used to train the system, how much better or unbiased can the AI be?
Since the data used to train a system dictates its future decision outcomes it raises the importance to identify and exclude human bias first, before training a soon to be intelligent system after.
Therefore, each recruiter’s decisions can be analyzed. Correlating their decision outcomes with all available characteristics is one technique. If the correlation with ‘gender’, ‘race’, or ‘political preference’ is higher than normal (can be defined by a threshold), then the recruiter can be assumed to be biased. As a result, his decision data should be excluded from the data set for the AI.
Nevertheless, no subjective decisions can be perfect. This means that slight tendencies always exist. But, bias is averaged in either direction with a large enough data set of many recruiters' decisions.
Finally, another method is adding non-subjective, quantifiable measures to the data. For example, the performance of candidates after they were hired.
Other Forms of Human Bias
Assuming a police station observed a high number of crimes in a particular neighborhood then future officers are more likely to be sent to these areas. Since more surveillance is conducted in these areas now, further crimes are found as result — and so on.
AI is trained by past observations and follows the existing observed trends. Thereby, observed clusters become large while other crime hot spots remain undetected [examples for bias: implicit bias in the police department, Police Patrol Bias].
Another area for human bias is the difference in occurrence. Imagine a healthcare AI is used to detect different stages of Diabetes wounds. However, the hospitals developing the system are in majority surrounded by neighborhoods with white-skin colored people.
After deploying the system in hospitals situated in neighborhoods with a majority of black-skin colored people, the AI suddenly starts to predict the wrong treatment with severe consequences as it is not trained to different skin-colors [examples for bias: AI bias in healthcare, white people favoring AI algorithm].
Methods to Prevent Human Bias
#1: Subjective Human Bias in Past Decision
Following the previous HR example, each recruiter’s decisions can be analyzed by correlating their decision outcomes to identify biased behavior and filter out extreme cases.
Enlarging the dataset with more decisions from different people can be effective, as it tends to average naturally inherited bias. Alternatively, non-subjective data can be added to the pool, too, such as post-hiring performance.
#2: Human Bias in Past Events
As already illustrated with the police institution, future crime is likely to be detected where past crime occurred already, as more officers are dispatched to such areas. Officer deployment systems that are trained on such data directly inherit geographic tendencies.
To detect geographical tendencies, going back to the beginning of time may help. At the start, patrols were likely to be conducted at random until the first cases are found. Only afterward, clusters start to form over time. Thereby, a fresh perspective on the crime landscape can be acquired.
Similarly, intelligent systems can overcome geographical human bias by integrating randomness or structured selection of unknown areas.
However, clustered data is a good way for AI to learn the behavior or movement of crime. When surveillance is increased significantly, crime areas tend to shift. Therefore, AI can be leveraged to adopt such movement patterns by measuring which patrols were more or less effective after detecting an initial crime hotspot.
#3: Human Bias in Occurrence
Gathering the right mix of data is a vital aspect of training an AI. This is especially true when slight changes in prediction performance can have severe consequences, like predicting stages of cancer or diabetes.
Therefore, when more data is available, it is wise to make sure that the dataset contains the target group in all its relevant diversity (e.g. genders, skin-types, ages). However, when data is not available, augmentation strategies provide an alternative, which have received high research attention in past years. Furthermore, test data should be used to validate that most of the different kinds of patients are classified equally well.
Conclusion
Mitigating AI bias plays a large role in ensuring a human-centric deployment of AI systems. Besides various technical approaches to mitigate AI bias, human bias should be given attention first.
Therefore, before the next system is trained, the data needs to be carefully analyzed. Including its origin, collection strategy, completeness and diversity of target group.
For questions on AI or AI bias, in particular, feel free to reach out anytime: david.berend@mine-it.eu or visit safe-intelligence for more.
