Firefox Test Pilot
Published in

Firefox Test Pilot

Gaining Insights Early: Concept Evaluation for Firefox Send

Earlier this week, the Firefox Send experiment launched in Test Pilot. The experiment allows people to transfer files in a simple and secure way using Firefox.

The idea for Send stemmed from what we’ve learned from past research about what people do online. For instance, from the multi-phase study on workflows conducted by the Firefox User Research team last year, we know that transferring files — images, text documents, videos — to oneself or other people is an atomic step within many common workflows. Once the Test Pilot team has an idea for an experiment, one of the first steps in our process is to evaluate the viability of that idea or concept.

Concept evaluation vs. usability testing

During the early idea stage of an experiment, our user research efforts focus on trying to understand the problem space the experiment is intended to address. We do this by seeking answers to the following types of questions:

  • Does the experiment idea address an existing need participants have?

The findings from this early research help determine the kinds and amount of effort that our product, UX, visual design, content strategy, and engineering team members will invest in the experiment moving forward. At this stage, we are less concerned with how usable an early design of an experiment is because we know that as we grow our understanding of existing needs, behaviors, mental models, and attitudes, the design could change significantly.

Research design for Firefox Send

To evaluate the idea for Firefox Send, we conducted remote interviews with individuals who reported having transferred a file online in the last week. Five participants were recruited to represent a mix of age, gender, ethnicity, U.S. location, level of educational attainment, household income, and primary desktop browser. Participants were asked to complete a series of think-aloud tasks using a desktop prototype of Send and were also asked about the ways they currently transfer files online. Each interview lasted approximately 45 minutes, and in addition to the researcher, other members of the Test Pilot team joined the interviews as notetakers and observers.

One of the screens from the prototype used in the Firefox Send concept evaluation

What We Learned

  • Participants use a variety of methods to transfer files online. Email was the most common method reported. Methods like email that involve uploading locally-stored files in order to share were perceived as more secure — because of greater perceived control over local files — than sharing that commenced from files already in the cloud.

What We Needed to Do After Research

The Send concept evaluation study produced a list of 15 detailed recommendations for the Test Pilot team. To summarize, we needed to take take three actions:

  1. Make the Send functionality more discernible (including accessible) in the browser

What’s next

Now that Firefox Send has launched, we will monitor metrics and conduct additional qualitative research to understand the usability of the experiment for people transferring files as well as people receiving files. Give Send a try.

The report

The full report contains detailed findings and recommendations

View the full report from this study. The Test Pilot team is working hard to make all of our user research reports public in the remainder of this year. As we do this, links to other study-related documents may break. If you have any questions about details related to Test Pilot user research, please contact Sharon Bautista at:



Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store