Actions on Google — Starting a conversation via Push Notifications

Colin Wren
Jun 12, 2020 · 6 min read
Photo by 수안 최 on Unsplash

In my last post I wrote on adding a conversational interface to an app by using the Firestore document for the user as an integration point as part of a proof of concept (PoC) for an app I’ve got in the works.

The integration with the app wasn’t the end goal of the PoC however, the main objective was to see if I could build something that periodically engages the user in conversation and in doing so allows the user to add data without having to remember to fire up the app to add it.

Actions on Google allows for this via the means of a push notification which when clicked will start the conversation at the desired intent so the user can answer a few questions, after which the conversation is ended and the user’s record is updated.

As with the previous post I’ve created a demo project that can be used as a basis for a web app and Action to explore how the integration works:

User Verification

In order to send the user push notifications we first need to make sure they are signed up to use our app which is a requirement for using the app linking functionality of Actions on Google.

App linking allows the Action to receive information about the user from Google and also gives access to storage on the user object that can be used by the Action to keep track of state, which in our case will be if we’ve asked the user if they want to receive notifications already.

This user state can only be accessed by users who are logged in to the device they are using to interact with the Action, a check for this can be carried again conv.user.verification which will return VERIFIED if the user is able to move forward.

If the user is not verified then we need to close the conversation with them as subsequent code will fail to run if they do not have that verified status, we can prompt this via the SignIn method from the actions-on-google library.

In order to redirect the user to sign in we first need to create a new Intent which listens for the actions_intent_SIGN_IN event and is fulfilled by our Firebase fulfilment function.

Once the user is logged in we can access their information such as their name to personalise the experience

Prompt the user to allow notifications

Once we know that the user is logged in we can prompt them to enable push notifications. This is as simple as returning a response and two suggestions; one for if they want to allow notifications and one for if they don’t.

Regardless of the way the user answers we will set a PUSH_NOTIFICATION_ASKED flag in the user.storage so that when the user interacts with the Action again we don’t re-prompt them.

Once you’ve got permissions to send the user push notifications and you go through the flow a second time the push token will be undefined so it’s important that you don’t re-attempt this call otherwise you may encounter an error.

Intents

The suggestion response will be backed by an intent, with the intent asking the user for permissions to send push notifications and then redirecting them to another intent that will collect the user’s push token and store it.

First, we need to create the Intent for starting the permission flow, this needs a new Intent in Dialogflow with the following setup:

  • Namesetup_push
  • Events SETUP_PUSH
  • Training phrasesSend me doggo reminders
  • Actionsetup.push
  • Fulfilment — Enable webhook call for this intent enabled

This will listen to the Send me doggo reminders suggestion and run the fulfilment function which in turn will trigger the permissions flow.

Second, we need to create the Intent for receiving the permissions once the user has permitted the action to send push notifications, this Intent needs this setup:

  • Namefinish_push_setup
  • Eventsactions_intent_PERMISSION
  • Training phrasesfinish_push_setup
  • Actionfinish.push.setup
  • Fulfilment — Enable webhook call for this intent enabled

Fulfilment

In the fulfilment code we’ll add the following functions to the intents:

setup_push
Will create the call to UpdatePermissions which will take the Add Doggo intent as a command, this will be the intent that gets fired off when the user receives the notification

finish_push_setup
With the user agreeing to push notifications our finish_push_setup intent will get fired with the following arguments, which we’ll use to get the push token:

  • PERMISSION — Did the user permit push?
  • UPDATES_USER_ID — The user’s push token

We want to store that UPDATES_USER_ID in our database as we will need to supply it as part of the push notification payload.

Once the user is signed in we can store the PUSH_NOTIFICATION_ASKED value in the user storage allowing us only prompt the user once

Actions on Google Integration setup

The last thing to do is to set the Intents up in Actions on Google, this will tell Actions on Google that these Intents exist and that our Add Doggo Intent wants to send notifications.

In Dialogflow, open the Actions on Google Integration Settings (under Integrations, click the Integration Settings button in the Actions on Google box) and add the Add Doggo Intent to the Implicit Invocation list.

In the same dialog press the Manage Assistant App button to open up the Actions Console, here under the Develop -> Actions menu item we need to edit the Add Doggo entry to allow push notifications.

The settings for this are under the User Engagement section and we need to enable the Would you like to send push notifications? option and supply a name for this engagement.

Once this is saved our fulfilment functions will have the correct permissions to set up the push notification permission.

Sending a push notification

Now we’ve got the user’s push token we can start sending them notifications, this is done via an authenticated API call with a payload containing the user’s push token and the intent we want the notification to trigger and the title for the notification.

In order to use this API though we first need to enable it, this can be done at https://console.developers.google.com/apis/api/actions.googleapis.com , you just need to press the Use API button to set it up.

Authenticating with Google

In order to send a push notification we need to get a JWT from Google that we can use to authenticate with their server, this JWT needs to be generated with a service account that has the ‘Project Owner’ role, more details can be found on Google’s Push Notification documentation.

You can use the googleapis library in Node to get a JWT to use to authenticate with Google’s services using the following command in the folder where you push Service Account lives:

This will print a JWT can use to send push notifications

Once we have the JWT we then use it to send the payload to https://actions.googleapis.com/v2/conversations:send which will then send the push notification to the user, this takes the shape of:

replace TOKEN with the UPDATES_USER_ID value we got earlier

On receiving the notification the user’s device will start that intent when the user responds to it, this means that if we created an intent that asked the user a set of questions to update their records we’d be able to prompt them periodically and engage them in a conversation that results in their records being up-to-date with minimal effort on their part.

Summary

By sending push notifications to the user we can engage them in conversations to update their data, allowing them keep things up to date without having to boot up your app to do it

Being able to start a conversation with the user makes your Action more valuable if there’s a need to add data over time as you bring the conversation to them instead of hoping they remember to add that data.

This type of conversation however does need to be designed to make data entry as quick as possible, using suggestions and cards to allow the user to jump straight to the point so that they don’t drop off halfway due the notification arriving at a bad time and the interaction taking too long to complete.

Get the code

I’ve pulled together a demo project that contains the Action and test code for push notifications, this is also available on the actions-on-google-push-notifications branch of the demo project I created as part of my blog — Adding a conversational interface to your app with Dialogflow.

Get smarter at building your thing. Join The Startup’s +792K followers.

Sign up for Top 10 Stories

By The Startup

Get smarter at building your thing. Subscribe to receive The Startup's top 10 most read stories — delivered straight into your inbox, once a week. Take a look.

By signing up, you will create a Medium account if you don’t already have one. Review our Privacy Policy for more information about our privacy practices.

Check your inbox
Medium sent you an email at to complete your subscription.

Colin Wren

Written by

The Startup
Colin Wren

Written by

Technical Lead at BJSS. Interested in Automated Testing, Dev practises, Metal, Chiptune. All views my own.

The Startup

Get smarter at building your thing. Follow to join The Startup’s +8 million monthly readers & +792K followers.

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store