Dark Side of AI — Lack of transparency widens inequality and trust in Systems
As more and more of our world gets connected and automated, we are entering a new era where the government or private entities access to most services/opportunities is pre-determined by deploying Computer /AI algorithms.
The issue with such systems are they are deployed without our consultation or knowledge, nor are these systems accountable to any standards or regulations. These algorithms have typically been trained on biased data collected without explicit consent. Usually, it impacts the already marginalized people in society, such as indigenous people or people of colour.
Typically these algorithms are deployed assuming that computers and AI would remove discriminatory practices, but in reality, they are automating inequality and racism.
Current regulations
The only comprehensive effort to regulate AI and automated decision-making systems in Canada is the Government of Canada’s Directive on Automated Decision-making (“the Canada ADM Directive”). Many other governments, including the Government of Ontario, have begun considering AI and ADM regulations but have not yet passed or implemented comprehensive or dedicated rules.
The Government of Canada has introduced Bill C-11, An Act to enact the Consumer Privacy Protection Act and the Personal Information and Data Protection Tribunal Act, which could impact privacy protections and AI systems. Similarly, PL 64 in Québec on data protection includes new provisions for automated decision-making. Finally, the Government of Ontario has embarked on a major initiative to develop a “Trustworthy AI” framework in Ontario.
Currently, the Canadian Governments directive is very limited in scope and does not apply to private companies nor many sectors of the government, such as law enforcement, etc.
Why is it important?
Lack of transparency in AI can cause various types of individual harm, such as financial loss, loss of employment opportunity or access to credit, loss of freedom, and collective social damages.
AI used to determine hiring decisions has been shown to amplify existing gender discrimination or racial lines. Law enforcement agencies are rapidly adopting predictive policing and risk assessment technologies that reinforce patterns of unjust racial discrimination in the criminal justice system.
AI systems shape the information we see on social media feeds and can perpetuate disinformation when they are optimized to prioritize attention-grabbing content. This can adversely affect political discourse and significantly impact civil society and democracy as it becomes increasingly difficult to agree on the basic set of facts. In the US, a widely-used healthcare algorithm falsely concludes that black patients are healthier than equally sick white patients.
As AI becomes increasingly omnipresent in our society, it becomes increasingly essential that AI deployed in the public spaces for services should be free of bias to increase trust in the system and decrease inequality.
What can be done to reduce bias and increase transparency in AI?
- Companies must publicly disclose to the end-user when they are interacting with algorithms/AI for Automated decision-making.
- Government and Private entities must collect biometric data such as face images, bio-metrics or voice data with explicit consent to train the algorithms.
- Establish an independent National AI Oversight body along the lines of The National Institute of Standards and Technology (NIST) in the US, which establishes and enforces regulations and conducts regular AI Audits or Algorithmic Impact Assessments (AIA) to ensure that algorithms are not discriminatory. AIAs should be made publicly accessible by the companies.
- Algorithmic impact assessments (AIAs) are a tool for assessing possible societal impacts of an algorithmic system before the system is in use. They have been proposed by researchers, policymakers and developers as a way to create greater accountability for the design and deployment of AI systems,which can in turn build public trust in the use of these systems, mitigate their potential to cause harm to people and groups, and maximise their potential for benefit.
- The oversight body should ensure the data that these algorithms are being developed reflect the country’s diverse population makeup and ensure a high accuracy rate among all the various population groups.
- The companies should also disclose the data nutrition label, so it is easier for the public to understand the labels and types of data. Disclosure of data nutrition labels should be made mandatory, along with AIAs. The Dataset Nutrition Label is a standard quality framework for assessing dataset quality. The Dataset Nutrition Label standard is a project of the Data Nutrition Project, a non-profit group that creates tools and practices around dataset quality that encourage responsible model development.
- The oversight body should create an awareness campaign among the public on how to read the data nutrition label and AIAs, so people understand what it means.
- The oversight body should also establish a certification process for non-critical AI based on the white paper published by the Responsible AI Institute (RAII), based on the OECD’s AI principles. The results of the certification process should be publicly accessible to ensure transparency.
- Canada should follow the EU framework on the AI Act (EU-AI Act) to define high-risk AI and establish regulations governing them for public use. These systems can cause both individual and societal harm affecting democracy and society.
- In the EU Framework, High-Risk AI is defined as systems that use sensitive information such as biometrics, facial recognition, etc., in critical sectors such as Education, Health, law enforcement, judiciary, financial industry, immigration, etc.
- Following the EU risk framework of the EU-AI Act, before these high-risk systems become available for consumer use, they are subject to strict reviews known in the AIA, which determine whether the system meets all requirements in the AI Act risk framework. The developer or provider of the algorithms will conduct these assessments, report their results to independent oversight authorities, and make them publicly accessible.
- Following the EU, by banning Real-time remote biometric identification in publicly accessible spaces for law enforcement purposes.
- Following EU law on the Ban on Social scoring leads to unjustified and disproportionate detrimental treatment.
