An all-round algorithm with many loopholes
We often assume technology or machines are neutral, but they are not. Recently, I watched“Coded bias”, which is a documentary on Netflix in 2010 that investigates bias in algorithms. Joy Buolamwini, M.I.T. Media Lab researcher, is voicing to find and improve loopholes in algorithms, starting with the discovery of very low facial recognition rates for black women, that is, racist biases. From racist facial recognition and predictive policing systems to scoring software that determines who has access to housing, loans, public support, etc., it addresses the myriad ways in which algorithmic biases have penetrated every aspect of our lives.
Throughout watching, I found out how widely I was influencing or controlling our daily lives. I was afraid of how much potential influence it could have on me without me being aware of it. An algorithm is now perceived as enterprising and objective. I believe this is why we should be wary of them. AI is based on human-entered data and the data reflects our history. Unlike changes in human perception over time, the past remains within the data, and within the algorithm.
Already, countries and companies around the world are trying to dominate individuals through this algorithm. In particular, AI is being developed for political and commercial purposes in China and the United States. It is necessary to be wary of many ripple effects and regulate and monitor algorithms because important tasks are performed based on blind faith in algorithms at the national level and responsibility is dispersed and victims’ damage increases. We have to think about who the algorithm’s automation and efficiency benefits.
If AI is wrong, whose fault is it, does AI itself know that its decision is wrong? How is an artificial intelligence created and used? In the end, AI was created by humans and is located in systems and systems with a deep history of discrimination such as criminal justice systems, housing, workplaces, and our financial system. And prejudice is often reflected in the results that artificial intelligence must predict. History is the winner’s record, and data reflect the human history of prejudice and discrimination. Algorithms that have learned this mechanically are bound to be biased, and impeccable objectivity is fiction.
On what criteria is Artificial Intelligence created and used now, and what bias can it cause? Let’s take a look at the next.
Criteria for Distributing and Using Algorithms
Center for Critical Race + Digital Studies about a machine learning algorithm, highlights that the algorithm might be optimized according to “the purpose”. It should be noted that this purpose is set by a person. This article explains that “Just like any recipe can be optimized for different goals — an entree can be optimized to be healthy, flavorful, authentic to a particular cuisine, or inexpensive — so too can predictive algorithms.” Many machine learning tasks are optimized by default for predictive accuracy. So they “test” the model on prediction problems for which we already know the answer to see how frequently it gets it right. Before the model can perform well on these tests, it may need to be fine-tuned. Once it achieves an acceptable score (what constitutes “acceptable” varies greatly between projects), it is deployed and used on data “in the wild.”
Center for Critical Race + Digital Studies defines three types of bias: Pre-existing, technical, and emergent. These are refined recently, which include:
- Historical bias
When there is a misalignment between the world as it is and the values or objectives to be encoded and propagated in a model, historical bias occurs. In other words, historical bias occurs when the data used to train an AI system no longer accurately reflects current reality. For example, the financial inequality faced by women is one of the issues affected by bias. According to Australian Human Rights Commission, the gender wage gap has narrowed since 2014 but the AI system is making decisions based on previous outdated income data from 2014. The more out of data, data is used the more women the AI system rejects as inappropriate, which can be seen as illegal discrimination under the Gender Discrimination Act.
2. Representation bias
Representation bias occurs when you define and sample a population on which to train a model, which is similar to sampling bias. This bias derives from uneven data collection. Take the same face recognition system in the sampling bias as an example. If the collected data consists mostly of photographs of white men, random sampling will not help avoid bias because it is already present in the data.
3. Measurement bias
Measurement bias occurs when features and labels are selected and measured. Variables are frequently proxies for the desired quantities rather than a pure measurement of a construct. The set of features and labels chosen may exclude important factors or introduce group or input-dependent noise, resulting in differential performance.
4. Aggregation bias
Aggregation bias arises during model construction when distinct populations are inappropriately combined. Diabetes and diabetes-related complications are more common in Hispanics than in non-Hispanic whites. When developing AI for diabetes diagnosis or monitoring, it is critical to make the system sensitive to ethnic differences, either by including ethnicity as a feature in the data or by building separate models for different ethnic groups.
5. Evaluation bias
During model iteration and evaluation, evaluation bias occurs. It can occur when the testing or external benchmark populations do not represent all segments of the user population equally. Evaluation bias can also occur when performance metrics are used that are inappropriate for the way the model will be used.
6. Deployment bias
It occurs after model deployment when a system is used or interpreted in inappropriate ways. The criminal justice system employs tools to predict the likelihood of a convicted criminal relapsing into criminal behavior. The predictions are not intended to assist judges in determining appropriate punishments at the time of sentencing.
How These Biases Affect Marginalized Groups
According to American Civil Liberties Union, bias is in the data used to train the AI — data that is often discriminatory or unrepresentative for people of color, women, or other marginalized groups — and can rear its head throughout the AI’s design, development, implementation, and use.
- Punishment and Policing
The use of AI and algorithm-based tools has increased for individual policing and punishment. A well-known example is the use of facial recognition and predictive policing techniques by law enforcement. Predictive police technology uses historical and real-time data to predict when and where crimes are most likely to occur, or who is most likely to participate in criminal activities or become victims. Various studies have shown that the data supplied to predictive policing programs are historically biased and perpetuate existing over-policing for marginalized groups. Historically biased police data supplied to predictive policing programs are perpetuating and reinforcing over-policing in areas inhabited by marginalized groups.
The Dutch System Risk Indication (Dutch SyRI) system, which determines whether individuals are likely to commit public interest fraud, is another example of “crime pre-detection.” The use of the system, which was mainly developed in low-income and high-immigration areas, has been criticized by the court for violating human rights.
2. Essential Services and Support
Automated systems are also increasingly being used to determine whether a person is eligible for essential services and support. This occurs in the private sector through workplace surveillance, credit checks, private security firms, and mobile apps.
For example, in Finland, the National Non-Discrimination and Equality Tribunal scored applicants based on factors like their place of residence, gender, age, and native language. Access to financial services, such as banking and lending, can clearly be a determining factor in an individual’s ability to pursue economic and social well-being. Credit will also assist marginalized communities in exercising their economic, social, and cultural rights. Other rights, such as non-discrimination, association, assembly, and expression, may be jeopardized by automation that polices, discriminates, and excludes.
AI technology is also used in the context of social security provisions, such as benefits or shelter, and influences the extent to which individuals can rely on these services. By excluding or shutting out those who are already in precarious situations, automated decisions can push them even further into precarity. On the one hand, a system failure or error can lead to serious human rights violations, including violations of the right to life.
3. Movement and Border Control
Automated systems are also increasingly used in immigration settings that impede the rights of refugees, immigrants, and stateless people. The border is made by integrated artificial intelligence using biometric databases such as retina and fingerprint scans, aerial video surveillance drones, etc. They rely entirely on technology in collecting data for these critical decisions. Humans tend to lack control over this technology in these decisions create real difficulties for people under “border surveillance” and often life-and-death despair. For an instance, this causes refugees to burn their fingerprints to avoid tracking the European Union or their country of origin.
These show that the bias of the AI and the scale of those affected by it can be fatal. Data forming the algorithm reflects the human history of prejudice and discrimination and we should be aware of that flawless objectivity is fiction.
