Big Data: In Climbing the Mountains of Truth, We Still Search for Valleys
If you haven’t read Cathy O’Neil’s book Weapons of Math Destruction, you ought to. In it, she applies critical insight to the application of algorithms and Big Data. Her main objection is that we often don’t adequately discuss, or acknowledge, the limitations of algorithms in our daily lives, that when taken too far they can do more harm than good.
Part of this harm can be derived from the fact that algorithms, by being based upon mathematics, are thought of as inherently objective. However, as they are created by human beings they cannot be free of faulty logic and implicit bias. They affect whether we get a job, what we will pay for goods and services, whether we are candidates for medical procedures, and a host of other decisions that directly affect our lives.
Of course, mathematical modelers don’t create algorithms with the intent of doing harm any more than someone building a model airplane intends to do anything else but see it fly. But it is easier to see the results of plane building than formulae composition. This intangibility is made worse because often algorithms are not available for public scrutiny and even if they were there right in front us, we might not deem ourselves qualified to analyze them in the first place. And this is where professions, in general, can knowingly, or unknowingly, do harm and perhaps why George Bernard Shaw once opined that professions are the conspiracy against the laity. Indeed, we should question experts who have a learned advantage and who speak with authority, and objectivity, in the name of science. Otherwise, in the race towards more and more specialization in knowledge based economies, subject matter expertise is the big bang that drowns-out the whispers of doubt at the edges of the universe.
At the center, it is difficult to see planets from black holes. While seeking the illumination of knowledge, we often unknowingly find darkness, sometimes due to cognitive biases, which can be defined as systematic patterns of deviation from rational judgment, whereby inferences about other people and situations may be applied illogically. The ingroup bias informs us that we tend to perceive those in our group more favorably than those outside of it. Of more concern, is that in any group, we tend not only to think more favorably of those within it, but also the information and beliefs held therein. A group with a hammer may only see nails just as a group carrying Big Data may only see algorithms. All of us have different cognitive profiles, but holistic and integrative are not adjectives frequently associated with the cognitive profiles of data scientists. Sometimes the ability to find the needle in the haystack, stops one seeing the haystack.
In her book, Cathy O’Neil shares an experience about a large American media firm that used algorithms to predict employee performance based, primarily, upon a data set structured around employee longevity and the number of promotions. As this organization had a history of not promoting women, the algorithm (per the number of promotions) predominantly chose men as being more likely to succeed. This is one example of many.
Of course, data scientists are aware of the delicate nature of algorithm alchemy. Though it is crucial to understand, and inquire, into the inclusivity of data set creation. O’Neil mentions the necessity for algorithm audits, whereby third parties look at the chosen data sets used to construct algorithms. Indeed, many of us leap on Big Data bandwagons and kick the wheels to make them go faster, with hammers and nails at the ready. When we aim this bandwagon at people analytics, we are playing with people’s lives. Never forget it.
While data science has transformed how we collect and use information, we should question the efficacy of applications with a potentially negative social impact. Historically and comparatively speaking, human progress might seem to be ever reaching skyward, but if subconsciously, or worse consciously, there is an underlying perspective that subject matter experts and professions create the bang, and laity the whispers, perhaps amongst all of our achievements on the mountain of truth, we still subconsciously search for valleys.
