What do we mean when we talk about transparency in content moderation?
Facebook, Google, Twitter, Reddit and other major social media companies are working to provide more information about how they apply their rules when they remove content that we share online. Without good information, it’s easy to assume these massive content moderation systems are biased. But what sort of information do we really need, and how can it help hold giant technology companies to account?
The OnlineCensorship.org project collects stories about people who have had their social media accounts suspended or their posts removed. A collaboration between the Electronic Frontier Foundation and Visualizing Impact, it tries to hold tech companies accountable for the decisions that they make.
In a new journal article as part of a project funded by the Australian Research Council and the Internet Policy Observatory, we analyzed 380 survey responses from OnlineCensorship.org to better understand what information people want and need from social media platforms.
One of the major problems with content moderation has long been that the notices that platforms provide are not specific enough. In our data, we saw more than a quarter of users express confusion about which specific post or act triggered a moderation action or account suspension. The lack of specific information makes it very difficult for users to understand the rules and learn from the experience, or to understand whether their content is moderated by the platform or removed by another user. Platforms have become better at providing notice recently, but users are still fearful of ‘shadowbanning’, where content is silently removed or made less visible.
We also found evidence of a systemic failure on the part of platforms to provide good reasons to explain the decisions they reach.
Only half of our participants expressed confidence that they understood the platform’s decision to remove their content or suspend their account.
In the absence of good information, we saw many users develop and use vernacular explanations about why their content was removed, frequently blaming biased moderators and undue external influence.
Social media companies are now in a crisis of their own making. Because they failed for so long to explain their processes and decisions, they have allowed mistrust to grow and thrive. On all sides of the political spectrum, we saw people conclude that their posts must have been censored because their opinions were not popular or politically correct. We saw these justifications from conservatives, Donald Trump supporters, “alt-right”figures, Bernie Sanders supporters, anti-vaccination campaigners, vegans, and more.
Other users blamed the content moderators themselves — the thousands of invisible workers who have to decide whether millions of social media posts each day contravene the complex rules of digital platforms. Because these processes operate in secret, we suspect that users may more readily infer bias or attribute a lack of contextual knowledge and cultural sensitivity to human moderation teams of major platforms. So, for example, the perception that social media companies are run by young people led a conservative poster to complain that those who “run afoul” of the “social justice warrior” orthodoxy were being unfairly targeted: “Facebook has hired many millennials who’ve been educated in systems that emphasize identity politics and thus, personal offense is indistinguishable to them from actual offense”. This is a claim that social media platforms have worked hard to refute, but the perception of human bias still seems to stick.
We also found that users are not often told how their content was flagged for review, and were sometimes left to guess whether they have been the subject of a complaint by an acquaintance, a government agency, or an opaque algorithm designed to identify potentially problematic content. Users in our dataset expressed distrust of other users, worrying that they were being targeted in coordinated flagging campaigns, resulting in effective bias in moderation decisions. This distrust is worsened by a perception that flagging systems can be gamed, particularly when decisions are made with the assistance of algorithms that automate part of the process.
Meaningful transparency
The lack of clear information available to users breeds folk theories and concerns about conspiracies and systemic bias in content moderation processes.
Building on this research, we helped to develop The Santa Clara Principles on Transparency and Accountability in Content Moderation. The Santa Clara Principles set out a targeted priority list of the most important things that platforms can do, right now, to improve how people understand their content moderation systems.
It’s now been a year since the release of the Principles, and we’re pleased to note that the recommendations about improving the notices sent to users have been taken up by many of the major platforms.
Unfortunately, there is still a long way to go. While notice to individual users has been improved, moderation systems are still plagued by a general lack of transparency. Major platforms all provide regular transparency reports, but these are presented at a high level of abstraction — total numbers for a country or type of rule that was broken.
In our Article, we propose a more targeted concept of transparency that focuses on the information that is required to better understand systems of content moderation in a way that renders them more accountable. Too often, claims about transparency are made in a vague way — and the transparency we get in return is just as vague. We suggest that there is a pressing need for more specificity in identifying what information should be provided and to whom.
For individuals who are directly subject to decisions about the enforcement of rules of social spaces, good notice is a critical prerequisite to understanding why a decision has been reached. But it is not enough. Fostering meaningful transparency means, at least in part, providing more detailed and individualized explanations of content moderation systems as a whole.
The concerns we saw most frequently expressed by users are around bias and undue influence. These concerns can’t be investigated through high level aggregate statistics.
If users (and government regulators) are to develop greater confidence in the operations of content moderation systems, much more granular information will be required. Understanding bias in moderation decisions and the algorithms that support them requires careful attention to the inputs and outputs of these systems and their differential social impact. Analysis of this type will require large-scale access to data on individual moderation decisions as well as deep qualitative analyses of the automated and human processes that platforms deploy internally.
This is the type of transparency that platforms have been reluctant to provide, but without it, we doubt that people will be able to trust the integrity of content moderation systems.
Read the full paper, published in the International Journal of Communication:
Suzor, Nicolas P., West, Sarah Myers, Quodling, Andrew and York, Jillian. (2019) “What Do We Mean When We Talk About Transparency? Toward Meaningful Transparency in Commercial Content Moderation.” International Journal of Communication, 13, pp. 1526–1543.
