Personalizing Content Moderation on Social Media Sites
This blog post summarizes a paper on introducing and analyzing the concept and design of personal moderation. This paper will be presented at the 26th ACM Conference On Computer-Supported Cooperative Work and Social Computing (CSCW) in Minneapolis, MN.
Social media sites empower users by allowing them to create and share the content of their choice. However, the rules around what content is acceptable on a platform versus not continue to be narrowly shaped by the cultural norms of Silicon Valley, where most big platforms hosting user-generated content are located. Given the normative differences across cultures and communities, a one-size-fits-all solution to shaping online content precludes serving the disparate needs of millions of end users.
Recognizing the inevitable conflicts regarding platform-wide content moderation, we call for an alternative approach called personal moderation that we define as follows:
Personal moderation is a form of moderation in which users can configure or customize some aspects of their moderation preferences on social media.
We identify two categories of personal moderation:
- Personal content moderation
- Personal account moderation
Personal content moderation tools let users make moderation decisions on each post based on its content alone and regardless of its source. Examples of such tools include toggles, sliders, or scales for ‘toxicity,’ ‘sensitivity,’ or other attributes, as well as word filter tools for filtering out user-specified phrases (see Fig. 2). These differ from more common personal account moderation tools, such as being able to block or mute undesirable accounts individually or in bulk. I have studied the operations and consequences of personal account moderation tools in my prior research (e.g., [1, 2]). This post focuses on my team’s research on personal content moderation.
These tools are ‘personal’ in that every user can configure them differently, and a user’s configuration applies only to their own account. In addition, they are content-based in that they help users configure moderation choices based on the characteristics of the content they encounter on the platform. Table 1 highlights how personal account and content moderation differ from other common modes of moderation.
In the context of internet history, personal content moderation tools, with all the promises and perils they entail, have found their moment. Critics and scholars are increasingly calling for mechanisms that move moderation decision-making away from centralized platforms and toward individual users to give them greater control over what they do not want to see on social media. Popular platforms like Facebook, Instagram, TikTok, and Twitter now offer word filter tools and sensitivity settings for users to configure over their news feeds and over ‘Explore,’ ‘For You,’ and ‘Search’ products.
In this paper, we sought to understand what considerations come into play when users have the ability to decide on personal moderation settings. We also wanted to understand users’ challenges when interacting with and configuring personal content moderation tools as realized in different commonly deployed designs, including toggles, word filters, and sliders. Finally, we were curious about users’ perceptions of the labor involved in configuring these tools.
To gather deeper reflections and nuanced rationales for users’ preferences, we conducted semi-structured interviews with 24 social media users. We developed a series of probes to prompt interviewees to consider different potential tool designs and elicit more informed opinions. We built a web application that simulated a social media feed and implemented four types of controls based on personal moderation tools that have been deployed or proposed in the past: a word filter, a toxicity toggle, an intensity slider, and a proportion slider (Fig. 3). In the interviews, we asked participants to interact with all four control interfaces and inspect the resulting changes in the simulated feed while speaking aloud about their preferences.
We found that while encountering offensive content on social media is a common experience, some prefer just to ignore such content, while others configure personal moderation settings, and still others get frustrated enough to quit social media altogether. Many interviewees were resistant to setting restrictive filters due to their fear of missing out on relevant posts and their desire to hear others out, even if the content might be offensive. Our analysis also raises critical areas for improvement in the current designs of personal moderation tools from the perspective of end users: increasing clarity in the definitions of various interface elements, incorporating environmental/cultural context, offering appropriate levels of granularity, and a wider leveraging of example content as a means to provide transparency and enable greater control. Our findings also highlight users’ understanding of the cognitive labor involved in personal moderation and, related to it, their perspectives on how platforms and lawmakers share some responsibility.
Building upon our findings, we highlight the importance of addressing online harms while attending to users’ desire not to overlook relevant content and how more context-aware personal content moderation tools can contribute to this goal. We emphasize the value of clarifying the meaning of crucial interface elements and how doing so may necessitate an overhaul of current tools. We argue that designs that let users configure moderation exceptions for specified user groups or cultural contexts would increase the controllability of these tools. We recommend that configuring these settings be iterative for users, and incorporating user preferences inferred from other interactions, such as reporting and seeking user feedback, could further improve their utility.
Finally, we note that offering these tools does not exempt platforms from ensuring the efficacy of their baseline moderation. However, these tools can let users customize their social media experience without infringing on free speech concerns.
For more details about our methods, findings, and design implications, please check out our full paper that will be published in Proceedings of the ACM on Human-Computer Interaction (CSCW) 2023. For questions and comments about the work, please drop an email to Shagun Jhaver at sj917 [at] rutgers [dot] edu. Citation:
Shagun Jhaver, Alice Qian Zhang, Quanze Chen, Nikhila Natarajan, Ruotong Wang, and Amy Zhang. 2023. Personalizing Content Moderation on Social Media: User Perspectives on Moderation Choices, Interface Design, and Labor. In Proc. ACM Hum.-Comput. Interact. 7, CSCW2, Article 289 (October 2023), 33 pages. https://doi.acm.org?doi=3610080
