User Study of Tracking Protection in Firefox Nightly
A recently landed feature in the experimental Firefox Nightly build integrates proactive tracking protection for Firefox Nightly users coupled with Do Not Track (DNT). With tracking protection active, ads and tracking technology are blocked before they reach the user’s browser. Subsequently, tracking ads and tracking pixels are not loaded by the browser and are not visible to the user. Further, by not loading the ads, tracking protection speeds up page load times.
In January 2015, the Firefox user research team conducted a three-day diary study with twelve demographically-diverse Firefox users in the US to understand their experience with Firefox Nightly’s implementation of Polaris tracking protection. For the diary study, participants were asked to download and install Firefox Nightly. Participants were instructed to turn on Tracking Protection in Firefox Nightly use it as their primary browser for the duration of the study. Participants responded to cues and direct questions while keeping notes of their daily online activities. After the diary study was completed, we conducted hour-long follow-up interviews with four of the participants via Vidyo.
We wanted to answer the following questions:
- What does “tracking protection” mean to participants? How do they define it? How do they believe it works?
- How does the implementation of tracking protection in Firefox Nightly match participants’ mental models of what tracker blocking means?
- Does tracking protection disrupt participants’ current experience of browsing the web?
Findings and Observations
Definitions of Tracking and Tracking Protection
Most participants understood that tracking involved a third party using a tool or technology to identify unique browser users and to collect data about those unique users’ activity online. Participants provided different descriptions of how they believe tracking works and what information being tracked. Most participants used more general vocabulary to describe the more tangible aspects of tracking, such as tracking technologies track “my likes based on my website visits” (P15) or “what I might be interested in.” (P10) A small number of participants described the technical details of how online tracking works with a high level of accuracy.
Who Do Participants Believe is Tracking Them?
Overall, the majority of participants were able to describe at least one or many employers of tracking technology. While most participants did not describe the ecosystems of tracking and the relationships among companies involved in tracking, they did mention the piece of tracking technology that is most visible to them: targeted ads.
We also asked all the participants: “who do you believe is tracking you online?” Most participants mentioned Google, Facebook, or “marketing companies.” Two participants said the US government was engaged in tracking online activities
Benefits about Tracking Protection
Feeling more secure and private. The most commonly cited benefit among participants was the feeling of greater security and greater privacy. Although some participants who said this did not articulate specifically how tracking protection worked or made them feel more private and secure. Some participants clearly believe the tracking protection feature’s labeling and/or its visible effects (e.g., blocking the display of some ads).
Fewer ads, less clutter. Among the participants who noted the absence of targeted ads with tracking protection on generally viewed the absence of targeted ads as positive benefit. Most of these participants viewed ads as “clutter” (P2) on a site or as a “distraction” (P4) from their intended browsing activities. A few of these participants said that they have sometimes found targeted ads helpful, but also said they did not miss the ads when they were absent.
Faster page loading. Some participants believed tracking protection speeded up their browsing experience by blocking tracking technologies that took extra time to load on a page. This extra speed was seen as a benefit.
In our follow-up interviews, two participants believed that tracking protection slowed down their browsing because “the browser has to do extra work.” (P2). The other two participants described how tracking protection improved the speed of page loading.
Brings attention to the issue of tracking ads. A few participants noted how their relationship to and understanding of tracking changed over the course of the study. As P2 wrote, “It was very thought provoking because even though I did not use the internet much on my phone or computer I started to think about what a difference tracking can make…”
Relevant for mobile. Further, when asked about tracking protection on mobile, many participants said they hadn’t considered online tracking on mobile before. As P10 wrote, “None of my other devices give me the option of turning tracking on and off. And Firefox Nightly has definitely changed my perception of tracking because being that it gives me the option to turn it on and off it also makes me curious what the other browsers are using…It’s led me to various questions.”
Frustrations with Tracking Protection
Confusion when targeted ads disappeared. Some participants found the blank areas of missing content where targeted ads were blocked as visually jarring and confusing. As P8 said, “I did notice unusual behavior with page loading or missing content. My reaction to the missing content were usually were Ads would appear…I eventually closed the page because it was odd.” Eventually, participants who mentioned the blank content grew to like having targeted ads blocked.
Options/Preferences Labels unclear. Some participants did not understand the functionality of the opt-in selections in the Options/Preferences menu. The two options sound very similar: “Do Not Track” is a technical framework to send a signal to ad servers requesting to opt out of tracking and “Prevent sites from tracking me” refers to the Firefox Nightly tracking protection feature. For these users, these two selections are a distinction without a difference.
Limited discoverability of interface and alerts. Participants were only alerted to the presence of the shield icon in day two of the study. Only one participant (P15) said he noticed the icon before it was pointed out in the instructions. Once they discovered it, participants did express an accurate understanding of the shield’s functionality to turn tracking protection on and off for the site.
Perceived video playback issues. Video playback, page loading issues, and crashing were issues participants attributed to tracking protection. In our subsequent tests, we demonstrated it is unlikely that tracking protection was the actual cause of instability or video playback issues. Regardless, likely because their attention was focused on the tracking protection feature as the subject of the study, tracking protection was perceived by some participants as the cause of these problems.
Additional Observations
A feature for more than a privacy-centric niche. We sometimes think that privacy protection tools are valuable to only a small portion of privacy-centric users (3–8% depending on the study) or general users at specific moments (for example, using the private browsing window). It is noteworthy how many of the participants found value in tracking protection.
Most participants in this study were not already focused on tracking and privacy-related issues. Tracking protection in Firefox Nightly exposed them to a tool that offered them a good deal more privacy than they were accustomed. Some participants responded positively and wanted to continue to have tracking protection as an option. A few other participants said they would like to use tracking protection after they perceived it becoming more stable. As P2 said, “I do plan to use tracking protection beyond this study because I liked the way things loaded better and I feel safer for some reason. I will not seek out additional tools because I don’t think I use the internet enough that I would need something like that.”
Invisible and visible. Participants focused almost exclusively on the visible manifestations of privacy such as targeted ads. During the follow-up interviews, we asked participants about the aspects of trackers that they could not see. P4 and P12 described ad networks and Google but did not fully convey the complex web of relationships among advertisers and tracking technology. Instead, participants only described the targeted ads they saw or did not see with tracking protection active. This more limited understanding makes a good deal of sense given the abstract nature of privacy concerns and the behind-the-scenes nature of tracking tools.
In order to improve the user experience of the tracking protection feature, we recommended at the time a few changes to the design:
- Better labeling in the interface, using more common language to distinguish between “Do Not Track,” “Tracking,” and “Privacy.”
- Show the invisible aspects of tracking to users to demonstrate visually (via a counter and a list of blocked trackers) to users what is being block on the sites they visit. Making the invisible elements more visible would allow us to show more fully the impact of tracking protection. As some participants indicated in their diary study, these details are valuable to them and enlightening about tracking in general.
- Many participants did not mind targeted ads disappearing, but some did find the missing ads affected the design of sites they visited. Whether users support ads or not, many sites are designed with space for ads. In order to retain the intended design of a site, we recommended filing in these empty spaces with wireframes of the ads being removed. Wireframes would also benefit new tracking protection users who were uncomfortable with sites appearing empty. [This proposal may be technically challenging to implement with accuracy.]
- Tracking and privacy are complex and sometimes abstract issues. Introducing the tool with on-boarding and a visual tutorial is essential for adoption and education.
Originally published at blog.mozilla.org.
