User Study: New Firefox Privacy Permissions Panel

The Firefox User Research team will now be posting results from more of our smaller research studies. Our aim is to share our results and provide some insight into our research and product development process.

Imagine you just came home with a framed picture. You need to find a nearby hardware store so you can buy nails to hang your picture up on the wall. You open your web browser and navigate to a well-known hardware chain’s web site to find a nearby location. When you find the page with store locations, the hardware chains’ web site asks you to share your current location. Maybe you’re in a hurry…sharing your location in this case means you don’t have to type in your current address.

A well-known American hardware store chain’s locations around Portland, Oregon

With Firefox, we believe that personal information should never automatically be handed over to a website without the user’s permission. After all, a web browser should be more than an application that blindly accesses web sites. Firefox mediates between you and those web sites to protect your privacy and security. When we talk about this role at Mozilla, we talk about how Firefox is the user’s agent on the web. Among our goals for Firefox is the mission to protect our users from privacy and security risks and to help them make more informed decisions.

A Better Privacy Permissions Panel

From previous internal research, the Firefox product team was aware that Firefox’s current privacy panel UI is ambiguous to users. For a geolocation request in Firefox’s current implementation, users are presented with a drop-down box displaying a catalog of options:

Current Firefox (FF50) Privacy Permissions Panel

• Share location with the current site (just this once)
• To always share their location with this specific site,
• Defer sharing their location
• Never share their location

Further confounding the decision process for users is the “X” widget in the upper corner of the panel that dismisses the dialogue. What does dismissing the dialogue mean? How do users interpret the “X”? Do they believe that “X” means “don’t allow” location sharing ? Or do users believe they are deferring the choice and simply dismissing the dialogue to move on with their current task?

The Firefox privacy team wanted to improve the labeling and also the interactions in order to help Firefox users make a more informed decision. The team iterated on several designs that attempted to resolve these questions. The goals of the redesign were:

• Communicate more clearly to users the choice they being asked to make
• Capture the expression of user intent with more certainty
• Present actions with more understandable outcomes

Version of the new Firefox Privacy Permissions Panel for Geolocation. This version has no “X” widget.

The team settled on a simpler design with better labeling and fewer interactions. However, questions remained about the possible usefulness of the “X” for users. What if users did not want to make a choice at that moment? What if they just wanted to make the dialogue go away and move on with their browsing? Would users be annoyed at being forced to make a choice? The team was unsure.

Two Studies

To evaluate the new design, the Firefox UX research team conducted two unmoderated, remote studies with 25 participants in the US, Canada, and the UK who selected Firefox as their primary web browser. Sessions were run between September 23, 2016 and November 1, 2016. For these initial studies, we focused on Firefox Desktop, not Firefox for Android or iOS. After downloading a special build of Firefox Nightly with the new privacy panel designs, participants followed a talk-aloud protocol to complete a series of tasks that triggered different privacy permissions panels. Participants’ test sessions were recorded and reviewed.

In each study, we divided participants into two groups. One group of participants used a special build of Firefox Nightly in which the new privacy permissions panel had the “X” widget; the other group had a version without the “X” widget.

Version of the new Privacy Permissions Panel with an “X” widget.

Study One

In the first study, participants were told to use Firefox Nightly to complete three primary tasks:

1. Navigate to Starbucks.com and find a store nearest them (triggering panel requesting geolocation).
2. Revisit the Starbucks.com site to observe participants’ reactions to being prompted again or not (depending on their selection of “Remember my decision”).
3. Navigate to talky.io and follow the instructions to start a web-based video chat with an unspecified friend.

Here’s what we learned:

• All participants appeared to understand the basic request presented to them in the permissions panel: sharing their current location with a site and allowing access to their computer’s webcam and microphone for a web chat site.
• “Allow” means allow the site access to the requested resource or information. “Don’t allow” means deny access.
• None of the participants appeared to be visibly or audibly frustrated or annoyed by being forced to make a choice between “Allow…” and “Don’t Allow.”
• Among the group of participants with the option to dismiss the permissions panel using the “X” only one did. When asked about her the meaning of clicking the dismissal widget, she stated that she believed clicking “X” was the same as “no.”
• Only one participant selected the “Remember this decision” checkbox. It’s unclear if participants saw the box or not. However, several participants expected the browser to not prompt them again. As Participant WxP3 said on being shown the privacy panel on the second visit to Starbucks.com, “I did not expect to see [the privacy panel again] again. I expected it to remember my location. I guess it doesn’t.”

Study Two

We wanted to test the new geolocation privacy panel, but with an additional layer of complexity. Unlike the Starbucks geolocation prompt which has an obvious transactional purpose (it’s on the Find a Store page only), how do participants react to the geolocation prompt when the purpose isn’t obvious?

Again, participants were divided into two groups with a special version of Firefox Nightly: one group with the “X” widget and one without the “X” widget.

In the second study, participants were told to use Firefox Nightly to complete three primary tasks:

1. Navigate to T-mobile.com. A panel is triggered on loading the site, but on the T-mobile.com page, the transactional meaning is less clear.
2. Navigate to Starbucks.com and find a store nearest them (triggering panel requesting geolocation).
3. We asked participants to locate their current settings for their privacy permissions.

Here’s what we learned:

• We observed similar results with regard to participants’ understanding of the permissions panel: all participants understood the general request.
• More participants in the “X” group selected “X” this time and all but one of them had the same interpretation of “X”: “No, I don’t want [the site] to access my location.” (Participant WxP7)
• One participant clicked “X” and interpreted “X” as both minimize and as “No.” Participant WxP3 said, “I closed it down…I really just minimized it. There was a pop up that was in my way and it wanted to share my location and I either said “no” or just minimized it. I just kind of got it out of my way.” For this participant, making the dialogue go away and not wanting to share their location are the same thing.
• The context of the request for geolocation is important in users’ decisions. Several participants who selected “Don’t Allow” for T-Mobile said they did not know why T-Mobile was asking for their geolocation. As Participants WOxP6 said, “I felt like T-mobile didn’t need to know my location because I was just looking for a phone.”
• Very few participants were able to relocate the status of their decision about sharing geolocation in Firefox. (TIP: It’s currently located in the address bar when you visit the specific domain.)

Observations

In spite of making a quick decision, participants do understand in general terms what is being asked of them. A simple “Allow” or “Don’t Allow” interface appears to be adequate for almost all of our participants to make a clear decision.

Further, the “X” widget is redundant because for all participants it carried the meaning of “Don’t Allow sharing.” In expressing her desire to make the dialogue go away and say “no,” one participant provided an additional layer of meaning. Based on this observation and the lack of frustration expressed by participants, we feel confident that the “X” dismissal widget can be removed because fewer possible interactions make for an easier user experience.

In the decision process, the context about the relevance of geolocation provided by a website appears to be very important. Based on participants’ answers to follow-up questions, it’s clear why Starbucks is asking them for their location. Participants may not want to share their location with Starbucks for privacy reasons (in fact a small number of participants said regardless of value they never shared their geolocation for this reason), but the request is clear to them. By contrast, it’s not clear from the testimony of participants why T-Mobile is asking for their geolocation data.

Technologies like geolocation in the browser can be potentially useful to both sites and users. But these tools can be used as gimmicks or in a way that does not provide visible user value. Web sites and developers need to situate why they are asking for users’ location in a meaningful context. When websites aren’t providing context for requesting this information, they are adding an extraneous task for users to perform that increases frustration and potentially eroding trust in the web site’s purpose and intent.

For Further Study

Which UI elements do participants see? Why did so few participants notice or select the “Remember my decision” checkbox? How can we improve users’ understanding of how to make persistent decisions?

Since most participants could not locate with the permissions management interface, how can we improve it? Many participants chose the Options (on Windows) or Preferences (on OS X) to locate permissions management.

How can we better support users like the small number of participants in our two studies who never wish to share their location?

Revisit privacy permissions on Firefox for Android and iOS. Is the experience or choice different on a mobile browser? How can we improve the experience there?