Pulling Back the Curtain on the ‘Black Box’: How the Digital Services Act Will Legislate Algorithmic Auditing
The promises and limitations of government oversight over algorithmic systems
By Ilse Heine, Policy Fellow, Global Network Initiative
An algorithm is like a recipe, in that it is a set of steps for a computer program to accomplish a goal or solve a problem. Algorithms appear as lines of code, but they are increasingly central in our lives — particularly through their role in online platforms. Facebook’s algorithm determines what content is displayed on a user’s news feed, it learns about the user’s preferences, and is built to maximize, among other things, user engagement. At a time of global uncertainty, it is increasingly clear that recommendation algorithms can also magnify disinformation, hate speech, and violent extremism online. The pervasive role of algorithms has drawn public scrutiny and spurred governments worldwide into action, including the European Union.
Automated Content Moderation
To address the negative consequences of algorithmic amplification, platforms are also turning to automation and machine learning to scale their content moderation practices. An algorithm can be described as a set of pre-set rules that are executed by a trigger, and it should not be conflated with machine learning (ML). ML is a particular subset of artificial intelligence (AI) that allows systems to learn from data automatically without being explicitly programmed. Social media platforms are using ML in various ways, including to assist with the human moderation process by prioritizing content for further review, based on its level of harmfulness. In addition to its benefits for scale and cost, AI-enabled content moderation decreases the human moderators’ exposure to certain types of content that can cause psychological distress.
While AI offers advantages, it also poses risks to the rights to freedom of expression and access to information. Specifically, AI does not have the necessary context to discern certain types of speech, which can result in illegitimate removal. Human rights groups have criticized platforms for removing videos and images critical for documenting war crimes, and they have identified other examples of excessive removals in the context of alleged online extremism. Another study found that AI models for processing hate speech were 1.5 times more likely to flag tweets as offensive or hateful when written by African Americans. Since online behaviors and the context around speech are constantly evolving, it is challenging even for the most resourceful companies to find comprehensive data sets to train ML models, or for ML systems to learn these nuances quickly, thereby increasing the chances for misidentification and over removal. Despite these risks posed to fundamental rights, legislators and civil society alike have limited knowledge about the inner workings of these algorithmic models.
The European Response
The EU has already taken steps to break the opacity around algorithms. For instance, the ‘right to explanation’ is enshrined in the EU’s comprehensive data privacy legislation, the GDPR — users are allowed to request the data behind the algorithmic decisions made about them. Similarly, in the platform to business (P2B) regulation, online platforms are obligated to be transparent about the general parameters that determine online ranking.
The European Commission is also engaging with various stakeholders to conduct an in-depth analysis of algorithmic transparency. The EU Parliamentary Research (EPRS) service published a 100+ page study on a governance model for algorithmic transparency and accountability. Among the report’s recommendations is the establishment of a regulatory body tasked with administering algorithmic impact assessments and investigating algorithms where there is a suspicion of infringement on human rights.
The Digital Services Act (DSA) will likely build on this foundation, but the proposals thus far lack specificity. The draft “own initiative reports” by the rapporteurs from three key parliamentary committees support general algorithmic transparency and risk assessment mechanisms. Echoing the EPRS study, two of the committees (LIBE and JURI) want to establish an independent regulatory body to oversee these tasks. Where an AI decision causes harm, the committees also call for proportionate penalties, liability, and redress mechanisms. However, the reports could provide more details on the possible regulatory elements and structures responsible for their implementation.
Opportunities and Challenges
While the EU is allocating resources to study the nuances of algorithmic transparency and accountability, it is still unclear how the DSA will integrate this evidence. During the consultation process, which ends in September, stakeholders should prioritize building awareness of the opportunities and challenges related to algorithmic auditing.
For one, traditional legal redress will fall short of the Commission’s goals. The LIBE Committee report states the need to “analyse whether [platforms] or the algorithms they use amplify illegal content.” However, it is often impossible to clearly identify this causality. An algorithm operates like a ‘black box,’ particularly the more complicated it is; even the developers themselves cannot always explain how an algorithm reaches a certain decision. Technical experts note that algorithms are not just implementing goals, but shaping them. Moreover, most platforms’ content moderation processes are currently a fusion of human intervention and automation. Given these technical realities, attribution of harm and legal liability would likely be difficult to achieve. Associate professor in law and A.I. ethics at Oxford University, Sandra Wachter specifically analyzes how the EU’s legal protections against discrimination cannot “be easily translated to algorithms as discriminators.”
There are also questions about the right role for and realistic capacity of any independent agency charged with auditing algorithms. While an independent agency traditionally offers increased oversight and reassurance to the public, there is no guarantee that a government agency will have the expertise, resources, and agility to regularly administer and assess algorithmic impact assessments. AI and machine learning are rapidly evolving technologies that impose pressures on large bureaucracies to adapt quickly.
The proposals also state that an EU-wide agency should have the power to penalize platforms if they fail to comply with the proper regulation, but there is no clarity yet on the specific requirements related to algorithmic transparency. This is not for the lack of tools and methods at policymakers’ disposal. For instance, platforms could be obligated to disclose their training data — the initial set of data used to train a machine learning model to predict the intended outcomes. This practice is widely cited, as models can amplify biases inherent in the training data. Ranking Digital Rights recommends the disclosure of accuracy rates for human and automated detection and removal and information about the types of output models generate. The model itself can also be reviewed, including the assumptions and limitations underlying the software modeling tool and “white box testing” to analyze the source code. However, others argue that a review of the inputs, outputs, or code is not enough and that regulators should account for related human behavior as well.
Once more, EU legislators are demanding more than just transparency, but also explainability. While these two terms are often used interchangeably, the latter distinctly refers to making algorithms interpretable to end-users. Explainable AI can help to hone trust, but it also presents new technical challenges and is still in its early research phase. Overall, while the research community is starting to agree on general, overarching AI principles (i.e. transparency, accountability, explainability), there is no consensus on the right methods to achieve them. Research to define and test various options is ongoing.
Looking Ahead
Whatever approach is sought, given the potential impacts this could have on freedom of expression, it must meet the strict necessity test, that is, states seeking to restrict expression must articulate the threat imposed by a specific type or piece of speech as well as the “direct and immediate” connection between the expression and the threat. Penalties should also be proportionate and non-discriminatory. The DSA must also spell out how requirements may vary for different digital services, products, or platforms, including the criteria and processes for audits and identifying companies for audit. Vague rules risk setting unachievable standards and imposing undue burdens. In addition, any requirements and subsequent penalty for non-compliance should be both flexible, to account for technical complexity, and clearly defined, for legal clarity. As noted in GNI’s recent private consultation on the DSA — “achieving the right balance of prescriptiveness and flexibility will help ensure respect for fundamental rights…”
The considerations outlined only touch the surface of the technical, social, and legal challenges around algorithmic transparency and accountability. During the consultation process, relevant stakeholders should continue to engage on how best to achieve algorithmic transparency and to what end. An effective solution will only arise from an interdisciplinary process, involving computer scientists, ethicists, social scientists, and policymakers alike.
The EU’s steps toward better algorithmic transparency and accountability are laudable. Several advocacy groups and scholars are demanding action just as forcefully. Likewise, companies may be hesitant to reveal their ‘secret sauce,’ and are concerned about subjects gaming their systems. However, they also do not want the sole responsibility of determining what are essentially political questions around algorithmic ‘fairness’ and ‘bias’ and should facilitate creative ways to inform others about their approaches. That said, the Commission should not pursue algorithmic transparency in the DSA for its own sake, but rather, prioritize the main objectives — to build a healthy and democratic online ecosystem, open avenues for interdisciplinary input and research, address information/market asymmetries, and protect fundamental rights. Algorithms have significant consequences for these goals, but they are just one component of a larger communications nexus.
