Four key takeaways from our election night live notifications experiment
Start early, recruit clearly, get everyone in a room, and stress test
After writing up how we planned, designed, and helped build the Guardian’s live updating notification for the US presidential election with the UK apps team, we thought there was still space to define some takeaways for publishers looking to try live data-driven notifications.
Election night was our first attempt to launch this (or any) new notification format for iOS users, and it was the first time that an experimental format we validated through smaller tests was integrated into the Guardian apps.
Some of the takeaways are specific to our process, but we hope what we’ve pulled out here will be helpful to anyone looking to try a similar format.
1. Live-updating alerts make sense to readers and drive app traffic.
Live updating alerts provide value at a glance. The Friday following the election we sent a survey to our notification subscribers asking for feedback on the experience. About 83% of the more than 6,000+ users who responded told us that the alert clearly and effectively communicated the important results in one place, letting them know what the news was, at that moment.
The live-updating alert format was easy to use and understand. The majority of users expanded the alerts, proving that it was easy to do so. Even more told us that they understood how the live-updating alert worked, even though a persistent alert that replaced itself with new data was an entirely new format for our audience.
The live-updating alert generated significant app traffic during the event. These alerts reached an audience of more than 230,000 Guardian app users, and drove 800,000 page views to the Guardian’s election night live blog and pages containing a full breakdown of results. Traffic from the alert accounted for about 6% of overall page views to the live blog on election night. A majority of alert subscribers (74%) contributed to that traffic by engaging with the alerts — tapping through on the alert to open the Guardian app.
2. In-app sign up is a successful recruiting tool for new formats, but the call out and description needs to be very clear and simple.
Sign-up should be easy and options should be tailored to your anticipated audience’s interests. The language and graphics on our sign-up page were clear, quick and simple, and we customized the opt-in/opt-out function based on a user’s location. Specifically, where possible, we chose to default US users to being opted in to the alert, with the option to opt-out; for users outside the US we defaulted them to being opted out, with the option to opt-in. We decided to differentiate the default behavior based on location since we thought users in other countries might be less interested in live US election results than those in the US.
As much as possible, explain exactly what you are offering at the point of sign-up. More on that call to action: We went back and forth amongst ourselves and with other editors in the Guardian US newsroom about what the sign-up screen should say and how to describe this brand new format. Were they auto-updating alerts? Live alerts? What would pique a user’s interest, but also help them understand the offering? In the final wording we attempted to strike a balance.
Happily the attention to detail paid off: 62% told us in survey feedback that it was clear the alert would update, with an additional 23% reporting it was somewhat clear.
Since a notification is small, first determine the hierarchy of information that should appear, and be disciplined about the character limit. We recommend forming a multidisciplinary team early on, including representatives from editorial, product, development and design, to determine the key pieces of information you’re trying to deliver, and how to best display them in the alert. Tackle questions like: If someone doesn’t expand the notification, what is the most important information for them to still see? In the expanded view, will all the information fit on a smaller phone screen? Asking these and other questions will help you ensure that a majority of your users have a great, and complete, experience with the alert.
3. Successful collaboration on new formats requires extra planning, communication, and possibly, new tools.
Start planning work well in advance, and seek out newsroom input on every piece of the experiment, from sign-up page to follow-up survey. While many newsrooms are starting to collaborate more often with product teams, developers and design groups within their organizations, the process still isn’t seamless. For example, in this project, we worked out nearly all of the details for in-app sign-ups directly with the UK apps team, without first realizing that the newsroom would want to weigh in on that copy, the visuals and how the alert was offered to app users. Because we didn’t consider the sign-up strategy as one of the editorial considerations for the alert, we weren’t as inclusive as we should have been with that decision making. This meant that, just days before the alert launched, we had to either jam in or completely roadblock a few small but significant changes to the sign-up process.
When experimenting with apps, discuss development deadlines up front. For many newsrooms, changes to web features can be made just before or even during a live event. When building or integrating with a native app however, teams need more lead time to make changes, since making changes to apps sets off a much bigger chain of events. With apps, changes must be coded, tested on multiple devices, submitted to an app store, approved and then made available to users for download.
It would have been wise of us to discuss the differences between web and app development with stakeholders in advance, so it was clearer why changes could and couldn’t be made in the days leading up to the election. It also would have helped to signal that the app approval processes differ between iOS and Android, and that most elements of the alert needed to be hard coded into the apps well in advance before they could be submitted to the App Store or Google Play.
For this alert, the sign-up screen and the integration of the AP data feed needed to be finalized before the app updates could be submitted for Google and Apple’s deadlines, facts that we didn’t make clear enough to the editors working with us on the project. The upshot: make sure to kick off the collaboration by identifying all of the decisions that will need to be made, by whom, and by when.
Consider building an editorial tool that helps facilitate the creation of content, and also a deeper understanding of the format. The content management tool we built for this alert allowed our team, and in theory the news editors we work with, to independently edit the alert title, body copy and graphics, and was created with simplicity and flexibility in mind. When working with a new format, having a tool with an easy-to-use interface can help all involved visualize the output.
App notifications need time to be tested on phones of varying screen sizes and operating system versions, in order to find and fix issues. Staying within the character limit in a notification is a continual challenge, as is determining the number of lines you can display in a notification before some information gets cut off. Make sure to test your content on a wide variety of screens and across platforms to make sure the most important information always appears. We made some last-minute adjustments when we realized that phones with a slightly larger than factory-set font size would cutting off the popular vote percentage. We shortened “Popular” to “Pop.” and managed to restore the full content on those phones, but looking at a device with those settings in advance would have helped.
4. In order to ensure a positive user experience, great visual and experience design are just as critical as quality content, and require as much precision and flexibility as other elements of coverage.
If you want to use a graphic in a notification, ask yourself first: “What additional useful information does it convey?” Notifications have limited space, so anything you decide to include needs to have a clear purpose. We decided to include a data visualization of the great 8-bit candidate avatars, designed by the Guardian US Interactives team, on top of a bar chart showing the number of electoral votes each candidate won relative to the 270 vote threshold.
We included the graphic not only because it was eye catching, but also because it was a more visual way to represent the electoral vote data, especially the ratio being apportioned in real time. If you decide a visualization is an important element of your notification, make sure to label what it is and what it represents.
Action buttons should only offer the most important and interesting functionalities. Since interacting with a notification beyond just tapping on it to go to an article is a very new concept, it’s important to make sure that the interactions you make available in the action buttons are clearly labeled and are useful or interesting. Again, just like with the content within the notification, you want to talk with your team about the hierarchy of the interactions, and show them in order of importance.
Because one of our goals was to gauge the audience’s appetite for information beyond that contained in the notification, the first two action buttons gave access to the live election night blog and also the full election results pages. The third button was a mechanism to opt-out of the alert, which was important to surface and make easy for the user to do, but was less important than giving easy access to more content. The action buttons are a powerful way to add value to a notification, so make sure you appropriately dedicate them to functions that help you learn about what your audience wants from new alert formats.
Notifications are usually subscribed to and sent through a service outside your CMS, so stress test that service in advance, especially for a spike in alert subscriptions. On election night some people who signed up for the alert indeed never received it because our push service was overloaded with subscription requests and couldn’t register them all. We recommend stress testing the push service in advance. Additionally we will be looking at ways to send a message to users confirming that they are signed up.
As always, if you have thoughts or questions about the alert, or even how we’ve structured these takeaways, reach out to us here in the comments, on Twitter, or by e-mail at innovationlab@theguardian.com.
The Guardian Mobile Innovation Lab operates with the generous support of the John S. and James L. Knight Foundation.
