Google Assistant meets Ktor

Implementing an Assistant webhook endpoint with Ktor

For the longest time I had wanted to experiment with backend services, but coming from the mobile world of Android it always felt a bit daunting to dig into all the backend services companies like Amazon, Google and Microsoft provide and knowing which ones I really needed.

SpaceX Falcon 9 rocket launch. Image credit: Javier Mendoza / AP

Then I ran into Ktor, and it ticked a couple of important boxes that made the transition from Android smooth.

• Written in Kotlin
• Super simple setup and configuration
• Supports building with Gradle

It seemed to be the bridge to the backend world I was looking for.

The first baby steps

All I needed now was an interesting use case for a hobby project. At around the same time we got a Google Home, and at work we were building a voice controlled robot powered by Dialogflow. My son was also in his first year of primary school, and I wondered what he was eating at school. Instead of asking him I naturally checked if there was possibly a REST API I could query.

And of course there was! I had my use case — When I ask Google Home what food is being served at school, it should speak the menu of the day. To do that, I had to connect the Google Assistant/Dialogflow to a webhook that would get the lunch information and serve it back.

My intention was that the webhook endpoint would be a Ktor app hosted on Heroku. Perhaps the stars were aligned, because Google had also recently released an excellent Kotlin library for Actions on Google that made parsing of requests coming from Dialogflow and building responses a breeze.

Fast forward a couple of late evenings and I had a working Google Assistant agent connected to a webhook endpoint. By using Dialogflow entities for schools, the agent was able to identify the name of the school and extract it as a parameter. Together with the date, these two parameters would be sent to the backend where they would be extracted and transformed to be used as parameters to the API of the school lunch provider.

What struck me the most was how quick and easy it was to do all this with Ktor.

To boldly go…

In this section we’ll go through the steps needed to make a webhook backed agent. We’ll create an agent in Dialogflow and write a Ktor app that handles requests from Dialogflow and responds back with a text that the Google Assistant can speak out loud.

Since the school lunch API is a bit limited — it’s in Swedish only and restricted by an API key — we will instead query an open 3rd party SpaceX API to get details about their latest launch.

Build better voice apps. Get more articles & interviews from voice technology experts at voicetechpodcast.com

This is what the end result will look like in the Assistant.

Creating the agent

Before you start, you will need a place to host the Ktor app. I’ve used Heroku but you could just as well use Google App Engine.

Create a new agent in Dialogflow by signing in at https://console.dialogflow.com/. When the setup wizard asks if you want to create a GCP project, accept that as the Stackdriver logs may be useful for debugging later on.

• In the Intents menu, create an intent called Latest launch
  Under Training phrases add Tell me about the latest launch
  Under Fulfillment enable the Enable webhook call for this intent checkbox
• Save the intent and go to the Fulfillment menu on the left
  Enable the webhook and enter the URL to your endpoint
• Enable the Google Assistant API at https://console.cloud.google.com/apis/api/embeddedassistant.googleapis.com/. You’ll need this to generate the debug logs.

That’s it! If you want, you could add a few more training phrases to let the agent pick up on variations of the request.

Creating the Ktor app

Use the IntelliJ IDEA plugin to create a new Ktor project. When prompted with which features to install, check Features->DefaultHeaders and Content Negotiation->GSON.

Depending on the hosting you’re going to use, you’ll need to add some additional files and modifications to the build.gradle file. Fortunately, Ktor has instructions for how to set it up on each hosting platform.

The full sample is available at https://github.com/fejd/ktor-spacexlaunch but let’s have a look at some bits of it here.

To do the REST API calls we’ll use Retrofit and to parse requests and compose responses we’ll use the Actions on Google library.

In the Application.kt file, we’ll add handling of the POST request from Dialogflow. SPACEX_WEBHOOK_ENDPOINT = “/spacex-webhook”

SpaceXActionsApp is a DialogflowApp and the @ForIntent annotation parameter should match the name of the intent in the agent. it.details is the human readable text that will be spoken by the assistant.

The SpaceXRepository class sets up the API class SpaceXApi using Retrofit.

The LatestLaunch data class is generated from JSON using the JSON to Kotlin Class IntelliJ IDEA plugin. Once installed, the SpaceX API data classes can be generated by running the action Kotlin data class File from JSON (found under File -> New). When prompted, we’ll paste the JSON we get from executing:

curl -s https://api.spacexdata.com/v3/launches/latest | jq

Press Generate and the data classes will be created.

All we need to do now is push the app to the host and test the webhook in the Dialogflow console.

That’s all there is to it! With a tiny amount of Kotlin code, we’ve built a backend service using Ktor. It fetches data from an open API and then communicates the SpaceX launch details back to the Google Assistant.

This post just scratches the surface of what Ktor and Google Assistant can do together. For more inspiration on how these two technologies can be used, and be combined, check out the Actions on Google samples and the Ktor samples.

Thanks to Oleksii Masnyi and Martin Begleiter for reviewing the contents of this post!

This post has been cross posted on the Jayway blog.