Ethical biases in model building and
implementation.
Bias during the design and the implementation of a model should be avoided at all costs. The data scientist working on those should feel utterly concerned since they make this technology possible in the first place and avoid a certain dilution of responsibilities.
Here are three typical biases that you can be aware of while developing your model with insights on how to avoid them.
Evaluation bias occurs when the data used to train the model is not representative of the target population.
Being mindful of the context in which the model will be used at the design stage would help to prevent this kind of bias.
The poor performance of a commercial facial detection algorithm regarding the detection of black women is one example of this bias. The training data set was probably the cause of this representation bias and a benchmark data set, with real-life cases example, should have avoided any evaluation bias. This should be taken into account from the beginning of the design.
This kind of bias also occurs when a performance metric hides important features of the underlying data. In this case, the data represent the reality but the metric responsible for the way this model is trained will mask this subtlety.
A comprehensive and granular metric analysis can mitigate this bias.
Aggregation bias occurs when a single model is used to predict an outcome for a given population having different subgroups with heterogeneous outcomes.
Having one model for each subgroup could solve this kind of bias, the idea being, again, to be mindful.
In the medical field, we know that according to different ethnicities, diabetes will have consequences that are totally different and the treatment applied should be adjusted. Failing to take into account different ethnicities, in this case, could lead to inaccurate outcomes predictions.
Deployment bias occurs when a model is used with a different purpose to which it was created.
This bias occurrence depends greatly on the context the model will be used (socio-technical system moderated by institutional structures and human decision-makers). The context should be kept in mind from the beginning.
One interesting example is a model developed originally to assess the probability to commit a new crime. Instead of being used only for this purpose, this model was used as a proxy to make decisions on the prisoner's sentence duration.
In conclusion, people creating those models should be the first concerned by those topics, and having educated examples in mind will go a long way. Those examples closer to the design process should help to realize more ethical and robust solutions.
PS:
In the resources part below is an interesting article [1] to understand more about what kind of ethical challenges we are currently and will face more and more in the future.
Resources
Most of the article is based on this document:
Harini Suresh, John V. Guttag — A Framework for Understanding Unintended Consequences of Machine Learning — https://arxiv.org/pdf/1901.10002.pdf
Interesting examples for healthcare and concise explanation can be found here (Stanford):
https://www.coursera.org/learn/evaluations-ai-applications-healthcare
[1] https://www.fastcompany.com/90608471/timnit-gebru-google-ai-ethics-equitable-tech-movement
