Less is Not More: Improving Findability and Actionability of Privacy Controls for Online Behavioral Advertising
This post summarizes a research paper that was accepted to the ACM CHI Conference on Human Factors in Computing Systems, which is a premier international conference of Human-Computer Interaction. This work was a result of a wonderful collaboration with Ruiyi Wang, Weikun Lyu, Nick Cook, Dr. Hana Habib, Dr. Lorrie Cranor, Dr. Nikola Banovic, and Dr. Florian Schaub.
Table of Contents
Motivation and Research Questions
TL;DR: A very quick glimpse of takeaways 👀
Design Variables
Formative Study: Interviews/User Studies
Main study: Experiment on Facebook
Results
Takeaways
Personal Notes of the Lead Author
Motivation and Research Questions
Privacy settings provided by tech companies are hard to find. How can we design ad settings so that they are more findable? And once designed and deployed, how do they impact user behavior and sentiment towards the settings and company?
Major tech companies that rely on ads for business, such as Meta and Google, argue that users have control over their data via ad privacy settings. Unfortunately, research shows that is not the case. For example, Habib et al. (2022)’s study showed that Facebook users are not aware of Facebook’s existing ad settings or/and have a hard time finding them.
The problem is, companies have little incentives to explore alternative interface designs for ad settings (at least, as of now). So in this work, our team, which consists of mainly academic researchers, asked: (1) How can we design ad settings so that they are more findable on social platforms? (2) And once designed and deployed, what would their impact be on users’ behavior and sentiment towards the settings and platform? (2) is important, because if findings show findable ad settings do impact users’ behavior and sentiment, they can potentially serve as a foundation for regulation or companies’ decisions.
To tackle these research questions, we conducted a formative study to explore and finalize ad control designs, built a Chrome extension to augment the designs on Facebook, and conducted an experiment to study them in a realistic setting (i.e., in the context of participants’ own Facebook accounts).
TL;DR: A very quick glimpse of takeaways 👀
Our study results show it is feasible to design more findable ad controls, which means platform companies can do better when it comes to making ad settings findable and more broadly, usable. We provide concrete design ideas on how to develop more findable ad settings.
Design Variables
To dive into the specifics, in our study, we focused on two design variables.
Entry point. The first one is the entry point, which means the initial interface a user would click on to find a path that leads to the correct ad setting. We were interested in the entry points in the main feed where users spend most of their time, such as the the top menu bar, the left menu bar, the top of the feed, and within ads.
Level of actionability. We were also interested in the level of actionability, which means how actionable are the provided options in the ad control interface. Platforms typically provide ad controls with low actionability, because they provide links to the general ad settings page, where the user has to spend time searching the page to find the right setting that they want to use. In contrast, we came up with designs with high actionability, which means the interfaces directly surface links to specific functionalities (examples below).
Formative Study: Interviews/User Studies
We first conducted interviews with 20 participants to explore and finalize designs regarding entry points and level of actionability. We explored and selected designs by showing them to participants via mock-ups and interactive prototypes. Below, we describe the final designs we chose based on the formative study. To read more about other designs that we considered, please check out the paper!
1. Entry Point Location
We narrowed down the location of ad controls’ entry points to within ads and top of the feed, mainly due to their high findability compared to other designs.
Ad menu. This design consists of an ad button with a blue background and a gear icon (figures below). When a user clicks on the button, the contextual menu appears, with Facebook’s original options displayed under the header “For this ad” and our augmented options underneath “For all ads.” Almost all participants in the formative study liked having the ad control’s entry point within ads.
Feed dashboard. This design is a “privacy dashboard” at the top of the feed. Participants’ reaction to the dashboard in the feed was mixed compared to the ad menu. But we decided to include it as Facebook has historically surfaced privacy-related reminders on the main feed. (Because some pilot study participants gave feedback that it took too much space in the feed, we made it collapsible via a button at the top right corner.)
2. Actionability
We were interested in finding out whether users were open to high actionability interfaces because companies typically provide ad settings with low actionability, and users could have become used to it. However, our participants in the formative study preferred or were open to high actionability interfaces, which suggested that these designs were worth testing in our main study.
Low actionability. For interfaces with low actionability, we included two links: one to the Ad Settings tab of the Ad Preferences page where many ad privacy settings are located, and another link to the general settings page (figures above). We also considered Facebook’s original interface, which served as the control condition, as having low actionability.
High actionability. For interfaces with a high level of actionability, we included entry points to Facebook’s existing settings that are a mix of data privacy settings and ad content curation settings, which we considered the most important based on prior research (figures below).
Main study: Experiment on Facebook
We built a Chrome extension that augments the four types of designs on Facebook, so that we can measure our designs’ impact on user behavior and sentiment in a realistic setting. Then, we conducted an experiment on Facebook with 110 participants who installed the extension to experience one of our conditions. For the control condition, the Chrome extension did not augment any designs but only collected users’ behavioral data on Facebook.
Using the Chrome extension, we gathered participants’ behavioral data of clicks and page history only while they were browsing Facebook. Participants also answered survey questions so that we can measure their perceived usability of existing Facebook ad settings, perceived usability of augmented ad controls, and perception of Meta.
There were five conditions in total: control condition (Facebook, which is the baseline), ad menu with low actionability, ad menu with high actionability, feed dashboard with low actionability, feed dashboard with high actionability.
During the experiment, we asked participants to complete three tasks (shown in a randomized order), which involved finding an ad setting after reading scenario-based prompts.
Task 1: Find advertisers that used tailored audience list
We asked participants to find one advertiser that targeted to them using audience lists. We included this task because audience lists are an advertising technique that not many people are aware of, and are considered very invasive once users learn about them.
The right setting to find was the “Audience-based advertising’’ popup (in the Ad Settings page). We considered this task as the most difficult because it had the fewest entry points compared to the prior two.
Task 2: Manage ad topics
We asked participants to find a way to see fewer ads about a certain topic. We chose this task because prior work has shown users express a strong need for having more control over what kind of ads they see. The respective setting for this task is the Ad Topics page.
Task 3: Stop personalized ads based on online activity on other websites and apps
This task asked participants to stop Facebook from targeting them with personalized ads based on online activity on other websites and apps other than Facebook’s products (i.e., websites/apps other than Facebook and Instagram). We chose this task because prior research has shown privacy-concerned users mostly want ways to prevent data sharing and tracking.
The right settings were either Off-Facebook activity or the “Data about you from partners popup’’ in Ad Settings (recently renamed to “Activity information from ad partners” ).
Results
1. Findability of Facebook’s Ad Setting Functionalities
Ad controls within ads and at the top of feed both increased the findability of ad settings for the most difficult task of finding Audience-based advertising, with top of feed being more effective. High actionability increased the findability rate for all tasks. At most, one design (feed dashboard with high actionability) increased the findability rate by 41% for the task of finding Audience-based advertising.
2. Participants’ perception of Facebook
Our tests could not find any significant differences when comparing treatment conditions to the control condition. Regardless, it is worth noting that overall participants’ trust in Facebook was low across all groups.
3. Participants’ perceived usability of Facebook’s existing ad settings
Our designs positively impacted participants’ perceived usability and sentiment towards the settings interfaces across all tasks (although the questions to which ratings showed difference slightly varied). The below graphs show participants who eventually ended up at the same settings offered by Facebook, and our conditions changed how they got there. This demonstrates how findability and actionability have a substantial impact on how settings are perceived, an important metric to consider from the perspective of user experience.
4. Participants’ sentiment towards and perceived usability of the new ad controls
Another interesting finding was that participants perceived ad controls with high actionability and low actionability as usable, with high ratings for both. Our tests could not find any significant difference between ratings for high actionability and low actionability. However, participants did significantly prefer the ad menu more than the feed dashboard, although the usability ratings were overall high for both.
Takeaways
- Our findings show that we can design more findable ad controls, which means companies, perhaps unsurprisingly, can do better to make ad settings more usable. Many regulations across countries say that companies should provide ad controls to users, but they do not concretely specify how they should be designed. We believe that regulators should provide research-informed requirements to companies on how to design ad controls, based on studies like this one.
- The ad menu increased findability of ad settings and users also preferred it more than the feed dashboard, which makes it a promising design companies can adopt. While the dashboard is more findable, it should be designed in a less intrusive way, as it seems like that was the reason why participants preferred it less.
- Participants were open to using more actionable interfaces. We caution against designs that hide important functionalities under the guise of being minimalist.
Personal Notes of the Lead Author
- One aspect of this paper that I really like, is that I think it nicely shows it is important to consider how to design interfaces, and not just algorithms.
- Building a Chrome extension was very useful for this study, as it let us study user behavior in a realistic setting. It also let us collect log data of users’ clicks (while they were browsing on Facebook). But for future studies, I am also interested in building mobile apps, because many people use phones to access social media.
- In order to really understand how to design ad settings that are usable but also align with companies’ ad-based business models, we need long-term field studies and experiments. Collaborations between academics and companies would be a superb way to achieve this.
