Using Web Notifications for News
The Guardian Mobile Lab’s first self-built experiment with sequential and interactive notifications on Chrome.
Read this, and then sign up for our next experiment! It’s intended for Android users on Chrome, with a similar offering to come for iOS soon.
We’re building! Now that the Guardian Mobile Innovation Lab team has grown and our first developer, Alastair Coote, has been on the job a few weeks, we’re excited to share insights from our first experiment with a house-built tool!
We’re starting to look at notifications, one of our five areas of focus, and on Friday, May 6, we ran an experiment using a web notifications system that Alastair built. Our experiment was to run an explainer-type set of notifications that coincided with the release of April’s US jobs report.
This first experiment was intentionally small, sent internally to a handful of subscribers within the Guardian newsroom. We wanted to test the workflow and process (and you know, the tool itself) before taking it public. For that reason, we don’t have a large set of data to share. We’re planning more experiments for larger events, and a larger audience.
With that caveat, we learned a few things from the process. So, here’s what we did.
Wait, what are web notifications?
Essentially, web notifications make it possible to send a push notification (the type of home screen or banner alert we expect to receive from apps) using the Chrome browser rather than an app. Web notifications for Google Chrome first became available in April 2015, but this April they were updated to make sending data and context through them much easier, and therefore of more use to news organizations.
Web notifications become important when Chrome as a mobile browser is brought into consideration. With them, any user who uses the Chrome browser on an Android phone or desktop — unfortunately iOS devices are currently unreachable — can receive notifications directly to their home-screen or lock screen, even if they haven’t installed your organization’s app.
We started experimenting with them for a few reasons.
Web notifications have the power to reach a less loyal audience. News organizations’ push notifications, which have the proven power to drive traffic to a story, are currently constrained by the limit of who has downloaded the app and subscribed to notifications. With web notifications, the user still needs to opt in to the notifications, but there are no further steps needed, like downloading, or even navigating the app store. Put simply, they have an easier onboarding procedure. A less loyal reader, or one who likes your content but maybe isn’t interested enough to download the app, or is constrained by their own data or storage limitations, can now receive notifications with a smaller initial investment.
Notifications still appear on a user’s home screen or lock screen. Like push notifications, web notifications show up on the user’s home screen or lock screen. This lowers the the barrier to entry to receiving the information news organizations are sending, and hopefully, entices users to read more about what the notification is alerting them to.
You can format notifications in new ways. In looking at what news organizations are doing with notifications right now, we found that most of the experimentation happening today is largely in tone. With web notifications, it’s possible to also experiment with form, too: for example, by sending sequential notifications, notifications with action buttons inviting the user to interact directly with content, and notifications that self-replace with updated information (without bothering you with thousands of pings from new notifications).
No major newsroom is testing them (publicly). Web notifications in their current form are pretty new. Some news organizations are experimenting with progressive web apps, but we haven’t seen any web notifications in public yet (though please let us know if you are testing them or planning to do so!) so we’re excited to share what we learn.
Web notifications are fast to set up. Once Alastair set up the underlying infrastructure for our experiments (we started with nothing and now we have a website, a server, and this whole notification system) he was able to create the types of notifications we were looking to use without having to create an app to package them. This may change as we hope to reach iPhones (web notifications are not currently supported by iOS).
For this notification system, Alastair used AWS SNS, a message broadcasting/subscription service provided by Amazon, and AWS Lambda, an on-demand computing service that runs code in response to events and automatically manages the underlying computing resources. He chose to use Lambda because, particularly as we experiment, Lambda will grow with us or shrink depending on how many users we attract. Lambda runs on demand, so it’s cheaper. Instead of having a server running 24/7, the code executes when we send a message then immediately shuts down again. Additionally, we can run it multiple times simultaneously, so when we are sending out notifications we can spin up several Lambda instances side by side. Without that, we would have to send them one by one, which produces an overall slower experience.
With that set up, Alastair wrote the code itself, which is available on our GitHub page. The code listens for messages broadcast through SNS, and rebroadcasts to our web notification subscribers. He added endpoints to add/remove subscriptions via API requests, and finally created our public-facing page to allow users to sign up for notifications.
The setup:
This is the first experiment the Guardian Mobile Lab has run that was based on our own tools, rather than harnessing the power of an off-platform tool to serve a journalistic function (for example our previous experiments on WhatsApp or Periscope). It came together as the result of an idea from the newsroom, specifically our business desk.
The first Friday of every month is jobs report day. For business reporter Jana Kasperkevic that means she comes in early and drinks a lot of coffee in preparation for the report to be released at precisely 8.30am. The report is prepared monthly by the US Bureau of Labor Statistics. It tracks how many jobs have been created or lost in a given month, as well as unemployment statistics, and details which industries are growing or stagnating. Because of those last details in particular, the jobs report is an incredibly important piece of data as it is an indicator of the economic health of the United States.
From earlier conversations about the lab’s purpose and projected areas of study, Jana knew that we were looking for a story on which to test out our notifications ideas, and offered the jobs report as a possibility. We agreed that this would be a good story for a few reasons. First, the jobs report comes out monthly, which means we have a scheduled and replicable experience on which to test. Second, while the jobs report is routinely reported in a quite matter-of-fact way, there are plenty of more granular insights that might be of more use to a particular user, but that are not the focus of the main app notification.
Once we settled on this as our story, we then thought about what the best type of notifications for this scenario might be. There is a predictable flow of information in a jobs report, but also, so many numbers that it’s hard to parse out quickly what is important or why.
We decided on an interactive strategy that we thought would help people get to the heart of what is most meaningful in the jobs report: is this good news, or is this bad news?
Our experiment was to send a series of notifications immediately following the publication of the Guardian’s article about the jobs report. We imagined this as a type of explainer: a notification giving a data point and then briefly explaining, through the categorization of good news or bad news, why this was important. We wanted to test if notifications like this, with action buttons that allowed the user to chart their own path in response to a series of facts, were helpful to readers, if they were enjoyable, and if we could build them into the existing workflow.
After sending an initial notification with the headline of the jobs report, we followed this with the provocation, “Now do you want the good news or the bad news first?” We offered users the opportunity to interact with two action buttons: good news with a thumbs up emoji, or bad news with a thumbs down emoji.
Users were then taken through the series of six notifications in order of which action they chose. After all the good news notifications were seen, only bad news was offered, and the other way around. Upon getting to the end of the notification series, the user was prompted to read the full article, or take a survey on the experience.
With the back-end pieces of the puzzle, what we hoped to learn, and the basic premise in hand, we then came up with the timing and copy with Jana and Dominic Rushe, her editor.
The next steps were to plan and run this experiment:
- We talked with Jana and Dom, to get an idea what her morning looked like and figured out when in the course of her work we could send the notifications.
- We created a Google Form with a survey to send to members of the newsroom after the experiment had taken place.
- We sent the sign-up page to the Guardian US newsroom.
- We did a last minute run around the office to recruit and put the sign-up page on the newsroom slack channel to promote it and get a few more sign-ups.
The execution:
This experiment was largely about figuring out the workflow of sending web notifications. For this, we worked with Jana to see how she prepared for jobs report day. She is able to largely pre-write her story the day before, leaving room for actual numbers. She also pre-wrote eight notifications she thought might be relevant based on what she was hearing ahead of the jobs report. The day before the report came out a new study was released reporting a dramatic slowdown in jobs. We tailored our notifications to this assumption, and had them edited and copyedited.
On Friday, Jana was at her computer waiting for the 8.30am release. We were there, too, as she learned that the jobs number was actually far lower than anticipated: only 160,000 jobs created in April (as compared to 250,000+ job the month previous). Jana did some quick work to tweak her article, and handed it over to her editors so it could go up as soon as possible. The Guardian article, US economy adds just 160,000 jobs in April — further sign of a slowdown, was published at 8.38am.
After publishing the first article, Jana immediately started to add color and more detail for a mid-morning refresh, which went live around 9.40am. We had discussed before that the time in between the two versions would be when Jana would be able to tweak or altogether rewrite the notifications we had previously prepared. In the end, we decided to not send two of the prepared notifications that were no longer relevant, and rewrote or added the correct numbers to the other six.
Once we settled on the correct notifications and had all the numbers, Alastair deployed the notifications at 9.01am. We saw that 25 members of the newsroom interacted with the notifications, and of those who began the interaction, one third went through the entire series. Five members of the newsroom completed surveys afterwards.
What worked:
The notifications were sent! While this was a relatively small experiment, it was the first test of our building powers. The notifications went out as planned. We were delighted.
On the morning of the jobs report, we had four people working on the experiment. Jana was running editorial lead, writing the article and tweaking the notifications, Madeline merging Jana’s work and the correct numbers into the mobile lab spreadsheet and handing off the finalized notifications to Alastair, Sarah was monitoring the experiment and finalizing the final notification with Alastair, who was was running technical lead and deploying the notifications. We may be able to run this with a lighter workforce in the future, but for a first technical experiment, this number of people proved appropriate.
We successfully integrated into the newsroom workflow. By planning ahead with Dom and Jana, we were able to figure out when the best time to send the notifications would be, in a way that didn’t interrupt or hinder the publication of the initial article or the notification sent to the Guardian’s US app audience, which were of primary importance.
By sitting near Jana as she was tweaking her report, we were able to get numbers from her, check in with her about her timing, and then push all the notifications without overly disturbing her or creating additional work.
Our users enjoyed the experience. Our final notification was a message announcing the end of the notifications series and directing users to read the article or take a survey about what they thought. While the survey size was very small, users told us that they enjoyed the notifications, and most importantly, would be willing to use them again.
The same basic system can be used for different types of alerts with no app update required. In addition to the sequential notifications experiment we ran on Friday, the same fundamental system could be used for a different type of notification interaction. For example: a poll where those notified feed answers back to us (useful to gauge opinion), or a notification that updates without a second notification appearing (could be an interesting way to detail primary results over time). As we continue to experiment, we are looking to build up a base of users. Once opted in, these users will never need to worry about updating an app to be able to take part new experiments, we’ll just push them the latest.
What could have worked better:
Many of the things that could have gone better on this experiment stemmed from this being our first time out of the gate, and also the fact that, unfortunately, many users saw the notifications on desktop and not on mobile, as intended. In the future, with a larger experimental group, we will be able to have more informative data from which to draw conclusions.
Need more users on mobile. The Guardian is an iPhone-issue newsroom, and so, unfortunately, in this first test very few of our users experienced the notifications on mobile and instead interacted with them on desktop. In the future, as we open up our experiments to the public, we will be looking for a way to include iPhone users, but for now, the notifications only work on Chrome on desktop or Android. Users unfortunately can’t receive these notifications through the Chrome app on iPhones.
Timing was a little slow. Twenty minutes went by in between when the initial article went up and our notifications were sent. Though not long in the scheme of an entire day or a private experiment, it’s a lifetime in a breaking news cycle. Were we to try sequential notifications again, we could probably cut about 10 minutes off this time simply by better understanding when each person is busy and can’t step in, and by knowing when to say something if the timing is too slow.
Notifications weren’t sticky enough. Several would-be users reported missing the notifications on desktop because they disappeared too quickly. Additionally, on Android, as the notifications appeared alongside the user’s other personal notifications, ours did not remain tethered to the top of the screen. As the mobile lab, we don’t want to invest too much time in solving a desktop-first problem, but it is a good reminder to make sure the notifications have some degree of permanence on all platforms so that users who do want to see them are able to.
Need more data to confirm that users understand the interaction. Because many of our users were on desktop, we didn’t walk away with as definitive a sense that our users understood the action as we would have liked. On Android, you need to “pull down” or “pinch” the notifications to surface the Good News and Bad News action buttons. After a demonstration of this experiment at last week’s Google I/O, we also heard anecdotally that some users believed the action buttons were actually prompting them to “rate” the alert as either “good news” or “bad news”, rather than offering a choice of more good news or bad news.
Lessons learned:
Buy-in from the newsroom continues to be paramount to the success of these experiments. This continues to be the most important lesson from our experiments thus far. There’s a lot of interest in the newsroom here at the Guardian US about what we’re doing and news on mobile in general. We have found that the best ideas have come from sitting down with reporters and editors and talking about what challenges they face and ideas they have for ways to make coverage more compelling, or, in the case of this jobs report, ways to make information granular and mobile-friendly, but delivered in an interesting way.
Newsroom innovation doesn’t happen without collaboration from all stakeholders. The lab can do its best work only by taking into account the existing practices of the reporters and editors and seeing how we can tweak these to create a mobile-first experience.
Interactive and sequential notifications would be difficult to deliver in a moment of real breaking news. There is a lot that has to come together in order to send a series of interactive notifications. Not only does the editorial content need to be there, to run a similar experiment with interactive notifications, you need to have a flow worked out. During breaking news, with no advance warning, even pulling off the content side of sequential notifications would be difficult, because the reporter needs to have a predetermined flow of information.
There’s more to learn about tone and timing. From talking with users we also learned that there were many differing opinions about the tone of the notifications and how many we sent. Also, because so many of our users were not on mobile, there is a lot still left to be learned about how to deliver this sort of explainer-type information on mobile and then best way, contextually, to deliver according to a user’s preference.
We’re still at the beginning of our notifications experiments, and have more to learn, but we’ll be working in the open and you are welcome to join. Our next experiment will be on June 7, where we hope to test out self-updating and interactive notifications around that day’s presidential primaries. The intended experience is for Chrome on Android devices. Simple sign-up is available on our website.
Let us know if you have additional ideas and observations in the comments or email us: innovationlab@theguardian.com. We look forward to hearing your thoughts.
The Guardian Mobile Innovation Lab operates with the generous support of the John S. and James L. Knight Foundation.
