Graduation Report: Activity Stream
Activity Stream launched as one of the first Test Pilot experiments. Our goal with Activity Stream from the beginning has been to create new ways for Firefox users to interact with and benefit from their history and bookmarks. Web browsers have historically kept this valuable information tucked away, limiting its usefulness. We were inspired by some smart features in Firefox, like the Awesome Bar, and wanted to bring that kind of intelligence to the rest of the browser.
We believed that if people could easily get back to the pages they had recently viewed and saved, they would be happier and more productive. We wanted to help people rediscover where they had been and help them decide where to go next.
Here’s what we learned
Our first attempt at this included two new features in Firefox: a new New Tab page and a Library view to see all of your bookmarks and history ordered from newest to oldest.
While we we were equally excited about the possibilities of both of these features, we found very quickly that people spent much more time interacting with New Tab. We decided that splitting our efforts on these two features wasn’t the best way to positively impact most people in Firefox and made the decision to retire the Library.
The good news is that this gave us more time to focus on New Tab. The first version included 4 major sections: Search, Top Sites, Spotlight (later renamed Highlights), and Top Activity. Each of these sections changed and morphed as we collected feedback through surveys, A/B tests, and user interviews.
Search
Up first was Search, which might be the most obvious. Or maybe it wasn’t. When we asked people what this search box did, many answered that it would search their history and bookmarks. That’s a pretty good guess considering the other items that are on the page. The problem is that it actually searches the web using your default search engine (Google or Bing for example). Because of this feedback, we changed the label of the search box to say “Search the Web”. This seemed to clear things up for most people.
One of the most surprising things that we learned while running this experiment is that around thirty percent of New Tab interactions were with that search box. You might wonder why that’s so surprising, but if you look at Firefox closely, you’ll notice that there are actually two other search boxes above this one: the Awesome Bar in the top left and the Search box in the top right. We believe that the New Tab search box is this popular because it’s in the content of the page and reminds people of the familiar search box from their favorite search engine.
Top Sites
After the search box, we have the ever popular Top Sites which are, well, your top… sites. To be more specific the sites (or pages) that show up here are ones that you have visited both frequently and fairly recently. This is the same technology that powers the Awesome Bar in Firefox, and it’s called frecency. Basically it’s good at guessing which sites you might want to visit based on your recent browsing. We made some minor changes to the algorithm that powers Top Sites but the bigger changes that we made were visual.
The Top Sites tiles in previous versions of Firefox used large screenshots of the sites you visited. We wanted something that was both more compact and easier to recognize, given the other items that were on the page, and decided to use icons to represent each site.
This seemed like a pretty obvious solution that would mirror the app launchers that people were familiar with on both their phones and laptops. The problem was that it wasn’t all that obvious in the end. Many sites had poor quality icons that were very small. This made it difficult for people to recognize their favorite sites.
We addressed this by creating our own collection of high-quality icons. Unfortunately for our icon artists, there are an endless number of sites on the web and therefore too many icons for us to hand curate. The other problem with icons is that they’re great for home pages but not so good for specific sections or pages on a site. So you might see a Reddit or CNN icon that looks like the home page when it was actually a specific page on the site.
This made it difficult to guess where an icon might take you. In the end, we settled on the best of both worlds. For home pages with a nice icon, we give you that in all its glory. For sections of a site or where a large icon isn’t available, we combine the small icon with a screenshot to give you some extra hints about which page you’ll land on.
Highlights… or was it Spotlight?
Next up on New Tab was the ever changing Spotlight section. The name Spotlight didn’t last for too long thanks to another feature with that same name in a certain popular (mac)OS. We settled on the name Highlights as a replacement even though to this day we worry that it isn’t quite right. We’ve debated the name several times since but always end up back at Highlights. The original idea for this section is that it would be the “highlights” of your recent activity in the browser that you would see in the more expansive Library view.
We actually spent a lot of time iterating on this section. Our goal was to provide a similar feature to Top Sites but in reverse. Rather than showing you the things you visited most, we wanted to show you the things you had just discovered and might want to get back to again. Ideally these would be things you might have bookmarked had you thought of it.
We ended up with a fairly sophisticated system where Firefox would assign each of your recently visited pages and bookmarks a score, and it would show you the items with the highest score each time you opened a new tab. We gave bookmarks more points since you had told us they were important and that way they would hang around and be available to you for a little bit longer.
For many of us on the team, this was a really great feature that we loved using. Unfortunately, when interviewing users, especially those using New Tab for the first time, they found Highlights to be confusing. They didn’t understand why items weren’t in chronological order (thanks to the scoring system) and when the section was empty, they didn’t know what to expect.
We made a number of changes to address these concerns. We went back to a simpler version of Highlights that is mostly chronological with bookmarks showing up first. We also added little ? bubbles to explain the different sections and give users quick access to customization. Finally, we added message boxes to explain the sections when they were empty.
Top Activity
Last (and maybe least) we had Top Activity at the bottom of the page. Somewhat like Highlights, Top Activity was meant to be some of the most interesting things from your recent history. In reality, it was just the first few items from the Library view.
This actually turned out to be a more effective feature than we had anticipated. We had a lot of positive feedback about easy access to the most recently visited pages. We did soon realize though that Top Activity and Highlights were remarkably similar features and decided to combine them. Through a few different iterations we ended up with the 9 cards you are familiar with in Highlights today.
Customization
Something that became clear through much of our testing is that people wanted to customize their New Tab. We found ourselves wanting the same thing in different ways. Some people wanted two rows of Top Sites. Others wanted to remove the search box and still others wanted to choose between just history or bookmarks in Highlights. So we added a whole slew of customization options to a nice side panel where it’s easy to see what your New Tab will look like as you make changes.
Recommended by Pocket
So those are all the sections right? Well almost! Last year we tested some content recommendations with our good friends at Pocket. We had some mixed results back then and some technical challenges that kept us from doing additional tests. Since that time though, Mozilla acquired Pocket, and we’re now part of the same company! This made it even easier to run experiments together and so we did. Recently we shipped the latest version of this feature called Recommended by Pocket, which helps you find the most interesting articles from around the web.
We rolled this out as a test so that not everyone received this feature to begin with. We compared how much people used New Tab with and without this feature, and we were excited to find that people used New Tab more when this was enabled.
These results gave us the confidence to ship Pocket recommendations in a number of key countries including the United States, Canada, and Germany.
Conclusion
All of these lessons and iterations came together into the really great New Tab experience that we have today:
Many of the details are different but most of the big ideas are very much the same. We have stayed focused on helping people connect to the places they’ve been and hope to help them find where they might want to go next. We could not have done any of this without the amazing help, feedback, and patience of you, our loyal test pilots. Thank you so much for joining us on this journey!
Here’s what happens next
The exciting news is that we shipped this feature as part of Firefox Quantum! We continue to learn and iterate and look forward to making these features even better for all of our Firefox users.
Thank you again for your help, and we encourage you to participate in helping other Test Pilot experiments learn and grow the same way that we’ve done with this one.
