Adventures in Android:
User Notifications
An Introduction
Let me tell you a story real quick. At Spire, we continuously build and support a cutting-edge platform, and our product team is a tiny group of just 4 developers and a designer. Together, we‘ve built some sweet little native apps on iOS, Android, and the web as well as a robust internal API. With such a small team, we don’t have much time for big support projects. However, when Urban Airship notified us that they were changing their business model and shutting off our push notifications unless we signed an expensive paid contract, we started doing a little research.
Over the course of a week, Rob and I put in some late nights and rapidly built out a lightweight cross-platform push notification server in Node. Since we figured our situation with Urban Airship was probably commonplace, we decided that we’d open-source our project in hopes that it might help out others in need. That project became yodel. Now, I’m primarily an Android engineer and I love exploring new Google tech, so I was excited to try out some of the “user notification” functionality we all heard about way back at I/O 2013. User notification capability had been locked down to specific developers for a good while, but luckily it had recently been made available to all developers. Due to previous limitations, the popular node-gcm project on which we built our Android notifications still only supported the classic messaging structure, so I embarked on yet another sub-project to fork node-gcm and build in support for the latest features of Google Cloud Messaging.
So what are User Notifications?
To make sure we’re all on the same page, let’s quickly review how classic push notifications work on Android. You have 3 main players: 1) an Android device running your app, 2) your notification server, and 3) Google’s GCM service. The process for receiving a push notification for your app on a specific device is as follows:
- Your app registers its host Android device with the GCM service.
- Your app receives a registration ID from GCM after successful registration.
- Your app sends over the device’s registration ID to your notification server, where it is stored for future use.
- Your notification server sends a notification message to GCM, specifying the device registration ID (and potentially other registration IDs) as the recipient.
- The GCM service takes that message data and sends a push notification to the device that is associated with the registration ID.
- On the device, Android routes the push notification data to your app, and your app decides how to display or otherwise handle it.
Registration IDs identify devices running your app, but there isn’t any other relationship beyond that. No implicit connection with a specific user account on your app and no association with any other devices.
Enter User Notifications. The user notification system brings an additional layer of connectivity to the classic system, adding the ability to create a one-to-many relationship between a user and a collection of devices. Rather than sending a notification to a user by specifying a list of devices that you locally manage on your own server, you provide a single token called a notification key. The notification key is a long string generated by the GCM service upon request. To have a notification key generated for a specific user, your must take the list of the device registration IDs associated with that user, provide a locally-unique string ID for your user, and send this data via HTTP POST request to a specified GCM endpoint. The response will then include your user’s brand new notification key. In the future, when the user adds or removes devices, you can simply send the changes to GCM via the same endpoint. I’ll get into the specifics further in this post, but the idea is that GCM manages your user’s list of devices, not you.
In addition to basic GCM user-device management, the user notification system includes capability for upstream messaging. This means that with the right configuration, you can send GCM-powered messages from a device back to the server as well as to other devices. At its core, GCM’s upstream messaging system runs as an XMPP server. This means that for your server to receive upstream messages from devices, you will need to set up a local XMPP client to authenticate and connect with GCM. However, If you simply want to send messages from a device to other devices registered under a notification key, the Android GCM library makes it very simple to simply send a message straight to those devices, bypassing your own server entirely.
Server Side: Managing Notification Keys
On the server side, I’m going to assume that most people will set up some sort of subscribe/unsubscribe API route that allows clients (like your Android app) to request or opt out of notifications on the device. Yodel only accepts commands through a shared Redis queue, so I built a quick and dirty HTTP layer for testing if you want to see a basic example of a notification server API in action. [Note: I’m not going to get into the XMPP client details in this post, but keep an eye out for future entries.]
When a user subscribes to notifications for the first time (step 3 in the list above), the server receives the device’s registration ID. Since no notification key yet exists for this user, we’ll need to request one via POST request to GCM’s user notification HTTP endpoint (see note). This POST request is in JSON format and requires an operation type, a notification key name (typically your app user’s ID), and a list of initial device registration IDs. On all notification key operation requests, we must include the following HTTP headers for proper parsing and authentication:
Content-Type: "application/json"
project_id: "<PROJECT NUMBER>"
Authorization: "key=<SERVER API KEY>"<PROJECT NUMBER> is your Google API project number (see note) and <SERVER API KEY> is your Google API Server API Key.
For now, we need to perform a “create” operation. The HTTP request body should take the following form in JSON:
request: {
"operation": "create",
"notification_key_name": "<yourUserIdentifier>",
"registration_ids": ["SFKGKjflskfjQF", "AdfEfdfDf234", etc...]
}Upon a successful POST request, you should receive a basic JSON response containing the notification key:
{ "notification_key": "yourSuperLongNotificationKey" }Once you have this notification key, you need to hold onto it and hold onto it dearly. Ideally, you should store the notification key (and notification key name, if it differs from your user ID) along with your other user data. There’s a big gotcha I discovered while building my server (and a big oversight, in my opinion): If you lose a notification key, you can’t get it back. To make it worse, You can’t regenerate a new notification key with a previously-used notification key name. If you try, you will get a 400 response that says “notification_key already exists.” That’s it. Your old notification key is lost to the ages, and your notification key name is unusable. To prevent your keys from being irreversibly bungled, either set up a data structure that reliably prevents the loss of the notification key, or set up a system where you can easily generate a new notification key name (not your user ID) and request a new notification key if needed. Once you have a reliable system set up, this isn’t such a big deal. However, it can be a real bitch when you’re just trying to do some early testing without any solid data persistence.
In the event that the same user installs your app on a new device and subscribes for notifications, you will need to associate that new device’s registration ID with the user’s existing notification key. This is done by performing an “add” operation using the aforementioned HTTP endpoint:
request: {
"operation": "add",
"notification_key_name": "<userIdentifier>",
"notification_key": "<existingNotificationKey>",
"registration_ids": ["<deviceRegistrationId>"]
}Upon a successful POST request, this will add the provided registration ID to the list of device registration IDs associated with the user’s notification key. (As a side note, it’s possible to make this request without specifying the notification key name. However, providing the key name prevents potential mismatches and is recommended by Google.)
Along similar lines, if a user unsubscribes from notifications for a particular device, you will need to perform a “remove” operation. Except for the operation name, the request structure is exactly the same as the “add” operation:
request: {
"operation": "remove",
"notification_key_name": "<userIdentifier>",
"notification_key": "<existingNotificationKey>",
"registration_ids": ["<deviceRegistrationId>"]
}When an operation is successful (whether it be create, add, or remove), the response will always simply contain a single notification key.
A final note on managing notification key data: there is no “delete” operation for notification keys. Therefore, you cannot explicitly delete a notification key. However, a notification key will be automatically deleted if every registration ID has been removed. The real difficulty is determining whether a “remove” operation has removed the last registration ID. In all cases, the response to a successful “remove” simply returns the affected notification key, even if the notification key is automatically deleted in the same instant. This is problematic, because until you request another “add” or “remove” operation, you don’t have a clue that the notification key is missing. My personal solution was to retry an “add” operation by automatically switching it to a “create” operation whenever I received a response with 400 status code and “notification_key not found” error message. This is kind of dirty (referencing specific error messages can be dangerous, since they could change), but it’s currently the most efficient way to ensure the notification key is either updated or recreated in the same call. Of course, if the notification key is recreated rather than updated, I must then store the new notification key in place of the old deleted key.
Server Side: Sending User Notifications
I won’t spend an enormous amount of time on the basics of sending user notifications. The process is nearly identical to sending a classic push notification but with a couple variations and notable quirks. For the most part, the official documentation on sending notifications is quite solid, and if you want to see our real-life example, you can peruse sender_base.js within yodel-gcm.
As with managing notification keys, GCM provides an HTTP endpoint (see note) for sending push notifications. Usage is very similar to what we discussed before: build a JSON request body with the proper parameters and send it as a POST request. You will need to include the proper headers for authentication and parsing like so:
Content-Type: "application/json"
Authorization: "key=<SERVER API KEY>"And a JSON body for the “send” request might look something like this:
{
"data": {
"message": "You have received a notification!",
"randomKey": "randomValue"
},
"to": "<notification key>"
}This is practically identical to the process for sending a classic push notification; however, rather than providing a “registration_ids” attribute, you must include a “to” attribute, assigning the recipient’s notification key as the value. Note that the GCM documentation mentions that you must provide a notification key via the “notification_key” attribute; however, the “notification_key” attribute isn’t recognized by the GCM endpoint despite what the docs say, and you will continue to receive the confusing error “Missing ‘registration_ids’ field.” This is a significant defect on the part of the official docs, and it took a helpful StackOverflow answer to point me in the right direction.
Once the data is successfully posted to the GCM “send” endpoint, you should receive a response with a 200 status code and a little extra info:
{
"success": 2,
"failure": 0
}This response describes the number of devices to which the push notification was successfully sent (as well as the number of failures). In the event of a partially-successful request, you might get a response like this:
{
"success":1,
"failure":2,
"failed_registration_ids":[
"regId1",
"regId2"
]
}In this example, you see that the push notification was successfully sent to 1 device, but 2 devices failed to receive the message data. In a typical setup for sending classic push notifications, you would then retry the request, sending the notification to the list of device registration IDs that failed (hopefully with a retry limit and exponential back-off). This is where things get a little hazy for user notifications. The GCM documentation simply says to retry if you encounter a partially-failed request. However, if you retry the last “send” request again, you will inadvertently send the 1 successful device a duplicate notification (since you are specifying a notification key that is tied to both successful and failed devices). Unfortunately, there doesn’t seem to be any special feature for retrying a “send” request for a user notification. To properly retry for failed devices, you will need to create a classic “send” request, including the “registration_ids” attribute and setting it equal to the array of failed registration IDs returned in the last response. This isn’t terrible, but it does add a bit more server-side registration ID handling that I hoped user notifications were going to remove.
Client Side: The Basics + Notable Quirks
Nothing really changes for the client when receiving user notifications, and that’s a good thing. It allows you to upgrade your server to support user notifications without breaking existing clients in the wild. (If you need a refresher on implementing a GCM client, you can always check out the official docs.) The interesting work on the client consists of sending upstream messages. As I noted earlier, I haven’t implemented an XMPP client on my server, so I can’t write much about receiving upstream messages on the backend. However, I did a little experimentation with device-to-device upstream messaging, and it’s pretty fascinating.
To get started in device-to-device messaging, you need to pass the notification key to the client. Ideally, you could return the notification key in your server’s response to a “subscribe” request and store it locally on the device. Once you have the notification key, you can use a permutation of the following code to send a message to all devices associated with that key:
final notificationKey = getNotificationKey(); // Not shown
final GoogleCloudMessaging gcm =
GoogleCloudMessaging.getInstance(CONTEXT);// gcm.send(...) will not (and should not) run on the main thread.
new Thread(new Runnable() {
@ Override
public void run() {
try {
Bundle data = new Bundle();
// The string "user_command" is arbitrary
data.putString(“user_command”, “dismiss_all”);
gcm.send(notificationKey, “user_command”, data); } catch (IOException e) {
e.printStackTrace();
}
}
}).start();
Within your GCM Intent Service that listens for incoming push notifications, you can then add something like this:
...
if(GoogleCloudMessaging.MESSAGE_TYPE_MESSAGE.equals(messageType)) {
// Check for a custom command
String command = extras.getString(“user_command”); if (command != null && command.equals(“dismiss_all”)) {
// Dismiss all the app's notifications!
getNotificationManager().cancelAll();
}
} else {
// Handle normal notification messages
}
}
...
By passing in this “user_command” extra with gcm.send()’s data bundle, I can essentially send any command (as well as any other custom data) to all other devices that belong to a particular user. My app on those devices can then decide how to handle those commands. In this particular example, I set the command to “dismiss_all” on the sender side. On the receiving side, I dismiss all notifications if that command is received. This is just one scenario; the possibilities are practically endless!
For a more comprehensive example of using device-to-device notifications on Android, you can check out my basic demo app on GitHub.
The Myth of Auto-Synced Notifications
During Google I/O 2013, we saw presenters tout an amazing new feature that originally piqued my interest in user notifications: auto-synced notification dismissal across all a user’s devices. Swipe it away on one device, and it disappeared on all other devices as well. Within the official documentation for user notifications, synchronized notification dismissal is listed in the first bullet point:
“If a message has been handled on one device, the GCM message on the other devices are dismissed. For example, if a user has handled a calendar notification on one device, the notification will go away on the user’s other devices.”
This led me and others to naively believe that if we simply started sending notifications using notification keys, the GCM service would automatically send a dismissal prompt across all a user’s devices. That is not the case. Auto-synchronized notification dismissal is not a built-in feature of user notifications at the time of writing. Luckily, it’s not the end of the world. Using device-to-device messaging capabilities, you can easily detect a local notification dismissal on your app and send your own custom-crafted dismissal messages to a user’s other devices. On the receiving end, you can then listen for your own dismissal messages and use the included data to trigger a dismissal of a specific notification. It’s not as effortless as automatic notification dismissal, but it’s not that hard either.
Conclusion
While user notifications and the accompanying upstream messaging capabilities are quite fascinating, the current version is not without its share of pitfalls. However, developers can still take advantage of the unique device-to-device and device-to-server messaging to build some amazingly synchronized and connected platforms.
As I mentioned throughout this post, you can learn more by browsing our various open-source projects aimed at making GCM and cross-platform push notifications a simple addition to your current project:
yodel — A lightweight Android and iOS push notification server written in Node.js and powered by a shared Redis command queue.
yodel-gcm — Our fork of the popular node-gcm library for easily sending GCM push notifications.
yodel-demo-http-server—A quick and dirty HTTP layer for demoing a Yodel server on the same machine.
yodel-demo-android—A demo Android app that connects to the yodel demo HTTP server.
