You Can’t Quantify Everyone’s Performance
How the attempted measurement of performance with lawless screening algorithms cause harm to reality
Every application system works by assessing applicants, and trying to distinguish which individual performs better than others. School or job applications and performance reviews demonstrate how dangerous this approach can be when wildly unregulated algorithms are applied.
When one applies for a job today, the first few review steps are completely human-less. Resumes get filtered through an algorithm, searching for all the right key words before any human eye meets an applicant’s name. It seems like algorithms speed up this process and make finding those perfect applicants more efficient- but this new gamefied process is rigged and lawless. Cathy O’Neil explains the danger of such algorithms, created to filter people out and measure the skills of individuals. This type of algorithm is just one example of many methods in which big data is used to marginalize populations and quantify people’s activity.
Cathy O’Neil’ details a case of a school teacher falling victim to faulty algorithms that fail to accurately detect the effectiveness of teachers performance in Weapons of Math Destruction.
In this case, Sarah Wysocki, a very well-liked middle school math teacher had fallen victim to a complex algorithm put in place to weed out low performing teachers. The IMPACT test was given to teachers in order to measure their scores as teachers. Wysocki had extremely positive reviews among her students and principal, but half of her lower performing scores in the IMPACT test outweighed her approval reviews, resulting in her termination from the school.
O’Neil unpacks where this algorithm fell short in accurately summing up the performance of this teacher. She explains that students have a multitude of exterior influences that affect how much they’re learning. Does that mean the teacher is at fault for ineffectively teaching her students? The issue with this algorithm and IMPACT test, is that the reality that the algorithm has set up is based on assumptions and data points without enough evaluation. When algorithms are not evaluated and compared to unbiased reality, it becomes an extremely threatening data monster.
These algorithms must be tested and balanced for equity and accuracy, but regulation is extremely complex. Algorithm-based screening for performance measurement results in inaccurate ratings that focus on numbers rather than real big-picture experience. Reviewers must be able to understand what the algorithm states, and be able to integrate this information within the broader context of an individual.
Sources
O’Neil, Cathy. Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy. Penguin Books, 2017.
