New Way to Measure Crowdsourcing Bias in Machine Learning

An overview of how to use counterfactual fairness to quantify the social bias of crowd workers

Photo by Edwin Andrade on Unsplash

Crowdsourcing is widely used in machine learning as an efficient form of annotating datasets. Platforms like Amazon Mechanical Turk allow researchers to collect data or outsource the task of labelling training…