Firefox Test Pilot
Published in

Firefox Test Pilot

Screenshots is Shipping in Firefox 56

Let’s start with the big news: we just shipped Firefox Screenshots with Firefox 56. Screenshots began life as the Test Pilot experiment Page Shot, and this marks the first time an experiment from Test Pilot has graduated into Firefox.

This is a big deal for Firefox users and a big deal for the Test Pilot team who have worked on this project for the last eighteen months or more, but most of all this is a huge moment for all of our brave Test Pilot participants. Your feedback, usage and support have changed Firefox for millions and millions of people. We couldn’t be more proud or more grateful.

What was Page Shot?

The page shot selection interface.

Page Shot was a smart screenshotting experiment. Screenshots are pretty much ubiquitous these days, and even though there are a lot of options in the market on desktop, we knew from user research that we had an opportunity to innovate. Page Shot brought a few differentiating features into the mix:

  1. Smart selection: Page Shot was built for the web and let users intelligently select individual elements on a web page for shooting. Want to capture a specific image? Just click it and Page Shot will do the rest.
  2. Smart search: We built Page Shot to not only capture screenshots, but to capture lots of metadata as well. Page Shot extracted text and other attributes from underlying pages (within the shot area) and stored it along with each screenshot. In practice, this meant that shots of text were fully searchable.
  3. Full page shooting: Page Shot could take shots of full websites (up to 5000px), and not just visible page content.
  4. Expiring shots: Page Shot let users save shots to the web, but these saves expired after two weeks. We let users modify expiration dates on shots if they liked so they disappeared more quickly or stuck around longer.

Why Ship in Firefox?

In terms of sheer user-adoption, Page Shot has been our most popular Test Pilot experiment to date. Over the duration of its active run in Test Pilot, Page Shot had nearly twice as many daily enrollees as any of our other experiments. The difference in adoption made us pay special attention to Page Shot from the start.

Page Shot had nearly double the daily active enrollees as the next experiment. Also interesting to note is how many of our participants disappear around the New Year holiday.

Just as importantly, we saw sustained usage throughout the life of the experiment. Test Pilot doesn’t put a ton of focus on marketing since we believe our audience is sizable and diverse enough to give us the kinds of feedback we need to judge success. We tend to look at retention (do people keep using an experiment) more than growth (are more people coming to an experiment over time) as a key metric, and Page Shot retained users extremely well. People took shots and continued taking them.

Given a stable user base, we saw strong overall retention for screenshots.

We learned that Page Shot had significant network effects beyond Firefox. Because shots were sharable, we saw lots of people viewing shots across lots of different contexts. While Page Shot was a desktop-only experiment, we saw lots of shot views on mobile devices.

Sessions other than those initiated by taking a shot by device over the last six months of the experiment.

Further, while Page Shot was only available on Firefox, we saw significant traffic from other browsers like Chrome, Internet Explorer and Safari. The graph below shows that while Firefox normally accounted for the majority of Page Shot sessions, there were specific instances where other browsers spiked dramatically. We hypothesize that these spikes occurred around specific screenshots that went viral.

Viral events led to big spikes in sessions on other browsers.

While we don’t have any direct evidence that Page Shot led to growth for the Firefox product, we nevertheless really like the idea that Page Shot gave people a way to express their use of Firefox in public through social sharing.

Ultimately, the decision to move Page Shot to Firefox was made because of the combination of all of the above factors. Positive signals around adoption, retention and network effects all played a part in the decision. We also just thought Page Shot was really cool.

How is Firefox Screenshots Different from Page Shot?

Since last spring, our team has focused on the (herculean) task of transforming Page Shot into Firefox Screenshots. There has been a massive amount of behind-the-scenes engineering work from the Test Pilot, Firefox and Add-ons teams to make this possible.* On the product side, instead of adding new features before launch, we stripped out features with a focus on simplifying user experience, improving user interface polish and ensuring Screenshots works across all locales.

So What’s Changed?

First things first, we changed the icon.

The UI in Screenshots is significantly more polished than it was for Page Shot. We touched up colors, animations, transitions, layouts and icons across the application. We also added a beautiful landing page and a new onboarding flow to introduce users to the experience.

Crucial beziers!

Users in Test Pilot loved Page Shot’s smart selection tool, so we drastically improved the overall polish of that feature. We also added googly eyes.

The Screenshots selection interface.

At the same time, we decided to temporarily remove full page and visible shooting features from Screenshots. Not only were these features not performant enough to make the final cut, but we did not include the option to directly download full/visible page captures without uploading to the Page Shot servers. In order to respect the privacy of our users, we backed out these features while a new user experience could be evaluated. As the graph below shows, people were definitely using these features in Page Shot, but not as much as saving and downloading cropped shots.

The yellow and green lines represent full and visible page saves.

Additionally, we added a cloud icon next to the save button on the shot selection screen. Again, we did this in order to promote transparency about saving to the web and better signal that Screenshots is backed by a service.

The Save icon before and after

We also removed smart search from Screenshots. While we still like the feature, we wanted to take time to reason about the privacy implications of capturing page metadata. Page Shot showed us that our audience didn’t search much anyway, which made the decision easier. Compared to the more than a quarter-million shots taken in the last six months of the experiment, there were only 1100 total interactions with the search bar.

Nobody searched so we killed the feature for launch.

What’s Next

While we’re launching Screenshots today in Firefox 56, our other area of focus for the last six months has been on Firefox Quantum. Quantum is a brand new Firefox coming this November that drastically improves the performance and style of the browser. As we’ve built Screenshots for the 56 release, we’ve also had to track and implement many interface changes so that we fit seamlessly into the next generation of Firefox.

When Quantum launches this November, you’ll notice that Screenshots fits beautifully into the new interface. Quantum’s design language is all about decluttering your browser, and Screenshots will take its place alongside bookmarks, Pocket saves, synced tabs and other things you collect and share while browsing with Firefox.

Don’t worry, you’ll be able to pin Screenshots to the search bar with a right click.

For Quantum, we’re also bringing back full and visible-page shooting. Thanks to great work by Neha Khanna over the summer, we have a far more flexible design than before. Instead of saving shots to the web by default, Screenshots will preview the shot and prompt users to save to the web or download directly. We’ve also resolved some performance issues by using JPEGs instead of PNGs for very large images.

Beyond Quantum, we’ll continue to add more features to Screenshots. On our latest development builds we have two more features under active development:

  • Accounts integration so that users can connect Screenshots to their Firefox Accounts and access them on all of their devices.
  • Annotation tools for doodling on, meme-ifying, and otherwise marking-up screenshots.

We’ve already started rolling out Screenshots to some users in Firefox 55, and we’ve seen downloaded shots edge past saves to the web. We’re eager to conduct further user research around the order and design of the download and save buttons and hope to add more options like saving images directly to clipboard.

Screenshots users in Firefox 55 split evenly between uploads and downloads, but recently downloads have pushed ahead.

As we officially launch Firefox Screenshots, we’ll continue to actively solicit feedback from our new users to figure how to keep improving. We chose to keep the Beta label on Firefox Screenshots to signify that this is the beginning of something and not the end.

We’re excited to get to work!

*The saga of the technical transformation from Page Shot to Firefox Screenshots is another really excellent story. We’ll tell that one here as well.



Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store