Driven by the data monster
I was on a business analytics course recently. Wanted to participate because I am skeptical of the data hype and suspicious of the perceived neutrality of numbers. Coincidentally, I started listening to Cathy O’Neil’s Weapons of Math Destruction after the course which made me believe that my suspicion is more grounded than I thought.
Sidenote: If you don’t want to buy the book, you can start by reading this article or her blog.
The course highlighted some important aspects:
- Analytics is a combination of math and soft skills. And the way it uses statistics is not really statistics.
- The analyst needs something to frame the data or there is too much of it. Framing is a decision.
- The analyst decides on a method, interprets the results of the method usage and shares recommendations. So many decisions being made.
- The definition of success is often a number.
- I suck at Excel.
The fact that data analysis is analysis and not an objective science (like nothing never is) should be clear to everyone, but what may be less clear is that being driven by numbers makes us focus on externalities rather than the real content or value of our proposition. For example, in our previous project where we had goals such as “improving the retention by 5%”, our product development became a numbers game rather than an effort to meet user or business needs. It could help if definition of success was a real value rather than a number but I am no longer sure.
But product development is peanuts compared to what Cathy O’Neil highlighted:
- Mathematical models are simplifications. Simplifications built by humans based on the given task and their values.
- Many models lack feedback. The errors are not corrected, which means that the models end up creating their own reality and use it to justify their results. (She calls these WMDs, weapons of math destruction.)
- Many models are opaque for those rated by the model. With machine-learning, models become opaque for all humans.
- The results will change behaviours. (Yes, numbers game. And playing the numbers game makes us lose sight of what is meaningful.)
- The WMDs tend to punish the poor. And because of point 2, the injustice is more difficult to uncover than before.
Will open data and transparent models save us? Maybe. There is already a Robin Hood of data analytics who looks at the open data in New York and finds injustice.
It was so nice to change from simplified existence to the messiness of human with these two books by Malcom Gladwell. How on one hand we subconsciously process amazing amounts of data to come to quick conclusions which we cannot explain, and how on the other hand our intuitions can be so off. And how difficult it is to understand what makes a community tick. Because there are so many things that control us and are beyond our control.
Not having one explanation makes me so much more comfortable than pretending there is one.
Footnote: Here you can check how prejudiced you are.
