A Data Science Superhero Fighting the Creepy Algorithms

In this episode of the Masters of Data podcast, I speak with Cathy O’Neil for a fascinating discussion about algorithms, the incredible value they have in today’s world, and the major challenges they can present. It is undeniable that more and more of our world today is being evaluated, analyzed, and driven by algorithms. You need to look no further than your car insurance, your kids’ education, or your mortgage to see the results of algorithms processing and delivering verdicts on you and your ability to do something you want to do. Cathy O’Neil, a data scientist, mathematician, activist, and author of the New York Times best-selling book Weapons of Math Destruction has spent years talking about the dangers of, as she calls them, “creepy algorithms”, and our need to take the effects they may cause seriously — both to ourselves, and those less advantaged than ourselves. And with an increasing number of today’s worldwide decisions being dictated by algorithms, this is a timely discussion on an important issue.

To start, Cathy gives us an interesting look into her background and how she found her way into the world of data science and mathematics. As she notes, growing up with both parents as mathematicians she fell in love with the discipline of mathematics at a young age and loved, “the purity and elegance of proof”. And as she got older she began to see that the world around her was messy in many ways, but math wasn’t messy, rather it was predictable and able to be reproduced. She began to appreciate that math is unique in that the answers are always concrete and irrefutable, not subjective or pliable. This early love and appreciation for the world of mathematics fueled her through undergrad studies at Berkeley and doctoral studies at Harvard. Then, after a short time in academia, she eventually found herself in a finance role at a hedge fund, right in the heart of the credit crisis. While this experience was challenging, Cathy identifies that it was eye-opening and gave her a unique perspective early on about algorithms, data analytics, how they are often used, and their role in the global landscape.

While this event was no doubt frustrating, having this experience early in her career helped Cathy see the reality of the world around her. As she explains, she was able to see the variations of modeling taking place in finance and how denials of risk is not isolated from outcomes that impact global economies and people’s livelihoods. It also gave her a heightened awareness that models are built with biases and with a denial that the data doesn’t lie because people inherently believe that the math is irrefutable. The grim reality she began to see, was that mathematic models were not only inaccurate or incorrect, but they were even being built at times to generate dishonest outcomes which were having practical implications in people’s lives. As she and I discuss, this was so impactful to her that she began a blog called Mathbabe with the purpose of highlighting things like creepy algorithms, aimed at bringing public awareness to the fact that data analytics are being used in frightening ways to calculate approvals for loans, job security even prison sentences-often times unbeknownst to the world.

This discovery has ignited a passion for Cathy who sees the real, practical harm these practices can bring if they are left unchecked. As she explains to me, one benefit from experience during the credit crisis was that it brought a social awareness to the real problems behind the issue. Awareness is powerful and can result in tangible changes being made, which is why she has rallied to bring the appropriate awareness to these issues surrounding improper uses of algorithms and dedicated herself to addressing these dangerous trends. The idea that people are being unknowing victims of bad algorithms and feeling a moral obligation to discuss them was the catalyst for Cathy to write Weapons of Math Destruction, in hopes of bringing the needed awareness that will lead to real change.

As we discuss though, the solutions do not simply get implemented through heightening global awareness on the issues. While this is the first step, it is not an inherent solution. We have to recognize that algorithms aren’t going anywhere, we just have to find the right solutions to use them well. How do we do that? This is a key part of the discussion. As Cathy sees it, data scientists need to begin taking their ethical obligations seriously, and a Hippocratic oath is a step in that direction. The challenge though is that none of them currently ask the question, “are you denying people their constitutional, legal, or human rights?”, something that everyone in data science needs to be asking. Additionally, though we need the laws already in place to be enforced because laws aren’t being enforced: anti-discrimination laws for hiring, for housing, for credit or for insurance. To see real change take place algorithms that are being used will need to have automatically installed monitors that will keep track of what they’re doing to make sure they’re in compliance with the laws that are supposed to protect citizens. Until that happens she fears we will continue to be subject to creepy and at times, abusive uses of algorithms in everyday life.

Outbound Links & Resources Mentioned

This episode of the Masters of Data podcast

Learn more about Cathy:

https://weaponsofmathdestructionbook.com/contact/

Learn more about her book Weapons of Math Destruction:

https://weaponsofmathdestructionbook.com/

Purchase her book Weapons of Math Destruction:

https://www.amazon.com/Weapons-Math-Destruction-Increases-Inequality/dp/0553418815?ie=UTF8&*Version*=1&*entries*=0

Learn more about her Mathbabe blog:

https://mathbabe.org/

Follow on Cathy on Twitter @mathbabedotorg