Gaining Insights Early: Concept Evaluation for Firefox Send

Earlier this week, the Firefox Send experiment launched in Test Pilot. The experiment allows people to transfer files in a simple and secure way using Firefox.

The idea for Send stemmed from what we’ve learned from past research about what people do online. For instance, from the multi-phase study on workflows conducted by the Firefox User Research team last year, we know that transferring files — images, text documents, videos — to oneself or other people is an atomic step within many common workflows. Once the Test Pilot team has an idea for an experiment, one of the first steps in our process is to evaluate the viability of that idea or concept.

Concept evaluation vs. usability testing

During the early idea stage of an experiment, our user research efforts focus on trying to understand the problem space the experiment is intended to address. We do this by seeking answers to the following types of questions:

• Does the experiment idea address an existing need participants have?
• Is the experiment intelligible by participants who might use it?
• Which, if any, mental models do participants most closely associate with the experiment idea?
• Is the experiment idea comparable to any tools participants are already using? If so, how is the experiment idea unique?
• What expectations do participants have of the experiment?
• Do participants have any concerns about the experiment?

The findings from this early research help determine the kinds and amount of effort that our product, UX, visual design, content strategy, and engineering team members will invest in the experiment moving forward. At this stage, we are less concerned with how usable an early design of an experiment is because we know that as we grow our understanding of existing needs, behaviors, mental models, and attitudes, the design could change significantly.

Research design for Firefox Send

To evaluate the idea for Firefox Send, we conducted remote interviews with individuals who reported having transferred a file online in the last week. Five participants were recruited to represent a mix of age, gender, ethnicity, U.S. location, level of educational attainment, household income, and primary desktop browser. Participants were asked to complete a series of think-aloud tasks using a desktop prototype of Send and were also asked about the ways they currently transfer files online. Each interview lasted approximately 45 minutes, and in addition to the researcher, other members of the Test Pilot team joined the interviews as notetakers and observers.

One of the screens from the prototype used in the Firefox Send concept evaluation

What We Learned

• Participants use a variety of methods to transfer files online. Email was the most common method reported. Methods like email that involve uploading locally-stored files in order to share were perceived as more secure — because of greater perceived control over local files — than sharing that commenced from files already in the cloud.
• Participants could not tell what the Send feature did based on the early UI for the browser toolbar.
• Participants expected to be able to email the share link from the Send UI, which would require integration with email services.
• Participants were unclear whether people receiving files transferred via Send had to use Firefox to access the files.
• Participants expected file view and download settings to be more flexible than suggested by the prototype UI. For example, one participant noted that the default one download limit would be problematic for her because she sometimes needs to download the same file on her work computer and then on her computer at home.
• No participants expressed preference for cloud versus peer-to-peer file sharing.

What We Needed to Do After Research

The Send concept evaluation study produced a list of 15 detailed recommendations for the Test Pilot team. To summarize, we needed to take take three actions:

1. Make the Send functionality more discernible (including accessible) in the browser
2. Make the secure nature of transferring files via Send apparent
3. Give people more control over how long and/or how many times shared files can be viewed

What’s next

Now that Firefox Send has launched, we will monitor metrics and conduct additional qualitative research to understand the usability of the experiment for people transferring files as well as people receiving files. Give Send a try.

The report

The full report contains detailed findings and recommendations

View the full report from this study. The Test Pilot team is working hard to make all of our user research reports public in the remainder of this year. As we do this, links to other study-related documents may break. If you have any questions about details related to Test Pilot user research, please contact Sharon Bautista at: sharon@mozilla.com