NYC Media Lab Notes
The State of the Art for Notifications
June 17th, 2015
Last week, NYC Media Lab convened a workshop on notifications, drawing insights from experts in user experience and product design for two panel discussions. The first focused on the future of notifications and tried to put them in context of changes in digital media such as the introduction of wearables and the Internet of Things- you can read about it here. The second panel looked at the state of the art and considered best practices for designing notifications from practitioners. Here are notes from the second of the two sessions.
The state of the art for notifications
This panel considered what we know from experiments in designing notifications for digital media products. From apps like Yo! through to content products such as weather, education and news, this panel shared best practices and tried to get at what we should regard as the ‘state of the art’ in the realm of notifications.
Event-based alerts perform much better than generic notifications
First up on the panel was Matt Hartman, Director of Seed Investments at betaworks. Matt works with multiple companies at betaworks that are taking a novel approach to notifications, and he shared some top level learnings from their experiences. For instance, trigger-based, or event driven notifications drove a 12% open rate, whereas non trigger based notifications only drove a 2% open rate.
“We have an app called Digg Deeper,” said Hartman, “which will let you give all your top news stories, and alert you to certain things that are trending.” These event triggers drive more than five times more engagement than other types of notifications.
Consumers are still learning how to interact with notifications just as device capabilities are evolving
Hartman also noted that “there’s a little bit of consumer learning to be done right now. Yo is one of the companies that I think is doing really interesting zone of notification layer. You can slide a notification aside and a lot of times, you can take actions within them. More and more of the interaction can happen on a notifications layer. For those of you that have tried the Apple watch, they’re doing a really good job of giving you some contextual responses. If you text someone, ‘hey, do you want coffee or tea?’, when you get the alert on the watch the first two replies are ‘coffee’ and ‘tea’. It’s doing some reading of what you’re doing. By the way, that’s inconsistent with how Apple allows you to do notifications on the phone. You can’t send a payload that says, ‘Here’s your choices; coffee or tea.’ You have to pre-program when you initially ship the app, here are all the choices people are ever going to be able to make.” Apple is only starting to give more options to developers, and consumers are only starting to understand what is possible.
Notifications are sacred and must be highly prioritized by content providers
Denis Hurley, Director of Future Technologies at Pearson, first learned this lesson in an experiment he ran with students from NYU. “We gave them the Samsung Gear Smartwatch and they were given three notifications a day, asked two different questions, one of the following six; how tired, or they’re hungry, stressed, angry, lonely or sad are you? At the same time, they were wearing a Fitbit, so we were aggregating their actions. We also asked them to send an email to one designated email inbox about how they thought throughout the day.” But even in this limited experiment, the students found the notifications intrusive. Given the option, “they would have stopped the process. The notifications were really intrusive on a day-to-day basis. The bottom line is that notifications are sacred and really high priority means that it’s something that’s actionable, and we should really enable prioritization.”
Not all notifications are created equal
Kaytee Nesmith, Senior Product Designer at NPR has focused on how users experience notifications in her customer-centric design process for the app NPR One. “The interesting thing that I found after talking to several people,” she said, “was that there seem to be this sort of consensus that there’s very distinct categories of notifications. There are the ones that we’re all familiar with that are like really intrusive, that interrupt you during your day and that might not be relevant information to you at all. Those are the ones that tend to get shut off permanently or at least, some level of modification happens on those, and it’s permanent.”
“There were two areas of notifications that people tended to define: the necessary and the tedious notifications, usually from messaging apps, or Twitter.” But, a third kind of notification is emerging that Nesmith regards as the goal. “Notifications that are contextually aware and are respectful of your time, that contain something that’s really important to you or really interesting to you. These will be based off of data.”
Let’s get real: notifications are mostly annoying
Noah Chestnut, Product Lead for BuzzFeed’s brand new News App, put it plainly. “Notifications are annoying. Everyone is really optimistic that ‘we’re going to crack it. It’s going to get better.’”
According to Urban Airship, Chestnut noted, in 2013 50% of app users would opt in to push notifications. “In 2014,” said Chestnut “for a social app, it’s about four out of then. For a gaming app, it’s about two out of then. For a finance app where like it’s your bank account, it’s four out of then. People are just not even saying, ‘I’ll give you a chance. I’ll let you win me over.’ They’re saying, ‘No.’
The answer, to Chestnut, is two fold. “When I think about notifications, two things come to mind. One is that we have to deal with all of the baggage of all of the other apps and products we had no hand in building. In addition to the Apple, and Google, and Microsoft not helping us play better together. Then secondly, we have to be smarter about what the app actually does when we make our pitch to say, ‘hey, do you want notifications?’” Chestnut believes the app should begin to tailor its notification behavior based on use, continuing to learn more about the user, rather than asking for a blanket opt-in at the time of download.
“Notifications, if you think about where a lot of people have first encountered them, it comes from SMS. It comes from text. It doesn’t come from the New York Times or Facebook. That type of relationship is still really valuable, and it’s a lot more valuable than anything you can do on your homepage, or even the home screen of your app.”
Maybe the metrics are all wrong
Chestnut also noted that the way media companies measure the success of notifications is suspect. “Click through rate is a really weird metric for a lot of notifications. When I get a headline from the Breaking News App, or from Circa and I read it, I don’t need to read the article. I read the notification, I’m good to go, especially on a watch. Yet, the only metrics we have are, ‘Did we send it?’ Did somebody open it? Did somebody uninstalled the app?’
And, media companies need to consider this problem collectively. “We are all friendly, but it’s not like we’re sharing our data on these things for very valid reasons,” said Chestnut. “Because of that, we don’t know when someone didn’t like a notification they got from X app that might have actually tapped on something from Y app. Apple or Google don’t have an incentive to give us that data either. It’s the type of thing where we are all competing, yet we’re all being judged at the same level, and we all have repercussions on our peers in this way.”
More should be done to consider the psychological ramifications of notifications
David Carroll, a faculty member in the Parsons Design & Technology MFA program, asked the panel about the psychological and cognitive impact of notifications. He noted studies on the effect of digital media on neurochemistry and the possibility of dependencies, and asked the panel about behavior segmentation related to the neuropsychology of different use digital media use cases.
Matt Hartman took the question on. “We have an app called Home Screen, which you should all download. It takes a screenshot of your home screen, and then it will do image recognition on all of the apps that you have and tell you which are popular and suggest other apps for you to try.” Hartman suggested in future the application may look for evidence of notifications badges on apps, to assess how many people are leaving them unopened or what the average number of oustanding notifications are for certain types of applications.
Noah Chestnut hopes BuzzFeed is not causing anxious behavior. “When Buzzfeed News is telling you, ‘X, Y, or Z happened in the world,’ and you missed it by an hour, I really pray we’re not stressing someone out. I also hope that our best users aren’t the folks who are freaking out that they have haven’t read every single headline that day. I think in the long run, that’s not good for any of us, including the people who are using our app.”
Timing is everything
Dheerja Khaur, Head of Product at The Skimm asked the panel about the importance of timing. “If I think about my routine, there are times during the day where I don’t want to see anything and there are times during the day when I love a notification like ‘here’s your read of the day’. But when I get out of the meeting and I have 15 notifications, I hate that.”
Nesmith responded that she has “been doing a lot of looking at timing. We have sort of bulk data, but we know people tend to listen more in the morning, on their commute.” So the question becomes, “how can we use that data in order to give you a notification that’s going to bring you in and get you in that routine?” The goal for Nesmith is personalization, but right now she is focused on issues like managing notification delivery across time zones.
On the power of notifications to change behavior
Two questions from the audience wondered whether we are seeing significant evidence of notifications changing behaviors.
Matt Hartman gave an example from his experience with Yo!, which built an API so developers could use the Yo! platform to send notifications. A group of developers in Israel came up with a notification application to warn Israelis about rockets launched into Israel.
Denis Hurley believes people notifications that modify behavior in some circumstances, as born out by his experiment with the Samsung watch. If the mood tracking queries discovered the student was feeling sad, the student wanted wanted to knkow “Well, what do I do with the information? They wanted feedback to say, ‘Go outside, get some vitamin D. It’s a beautiful day right now.’”
Ultimately, the panelists agreed that there is much yet to be learned about how to deliver notifications in the best way, and that the conversation will continue to evolve as the platforms and devices evolve, as well. For some time the state of the art will remain a moving target.
Justin Hendrix is Executive Director of NYC Media Lab. Reach him at justin [dot] hendrix [at] nycmedialab [dot] org or follow him on Twitter @justinhendrix.
