Dialogflow Webhook, Golang and Protobuf
By Paul Lhussiez (Lab Backend Developper)
In the previous article about Dialogflow we created a bot using a webhook written in Go. At the time of writing, there was no Dialogflow SDK for Go, and I had to create a library to bootstrap the structures of a DialogFlow webhook call (which you can find here). Now with the v2 version of the Dialogflow API, there's a Go SDK which allows you to control Dialogflow's behavior. The v2 is now enabled by default.
But that’s not the main point of interest of this release. When they released parthe SDK, they also released the Go code that was generated from the protocol buffer definition, and that’s what we’re going to use to properly handle an incoming webhook request.
Dialogflow Webhook
Dialogflow is great to create conversational bots, but sometimes you want to do more than just answer with predefined text. You might want to send an email, query an API and format its response to answer your user, or even store information in a database. That’s the goal of Fulfillment in Dialogflow. When you enable the fulfillment on a specific intent, you tell Dialogflow to send a query with a payload to your backend, as shown in the previous article and the schema below:
Protocol Buffer (aka. Protobuf)
Protocol buffers are Google’s language-neutral, platform-neutral, extensible mechanism for serializing structured data — think XML, but smaller, faster, and simpler. You define how you want your data to be structured once, then you can use special generated source code to easily write and read your structured data to and from a variety of data streams and using a variety of languages.
Basically protobuf allows you to define how your data is structured once. Then you can generate the source code that will allow you to easily read and write in that format with the language you like the most or need to use.
Handling Webhook
We’re going to start with a simple program that uses gin as the HTTP router, and logrus as the logger. So write that down in a main.go file.
This program only starts the router and listens on the 8080 port, or exits with a proper error message if it fails to do so. It also accepts posts on the /webhook endpoint but doesn't do anything with it.
Two options here. We’ll first see how to use the new go module approach if you’re already using go1.11. If you’re not using modules or go1.11 yet, we’ll also see how to use dep to manage our dependencies.
Using go modules
The behavior can change if you’re inside your GOPATH or outside. For this example we'll just create a new directory anywhere outside our GOPATH and put the main.go file we created in the previous section inside it.
Here we initialized our package and gave it a name. It created a mod.go file which just contains the package name and nothing else. Things get interesting when we run the go build command though:
And voilà. Not only did we installed the proper versions of the two libraries we’re using (namely gin and logrus) but also their transient dependencies.
Using dep
We’ll first init dep by running dep init.
Then we can add the following package:
Handler
Now we’re going to import the generated code from protobuf:
We now have access to all the structures and especially the WebhookRequest one, which will allow us to properly unmarshal a webhook request. But now we have a problem. We have the Go code that corresponds to the protocol buffer, but Dialogflow will send JSON requests to our endpoint. Luckily, the protobuf team thought of that for us and created a package named jsonpb.
Package jsonpb provides marshaling and unmarshalling between protocol buffers and JSON. It follows the specification at https://developers.google.com/protocol-buffers/docs/proto3#json. This package produces a different output than the standard “encoding/json” package, which does not operate correctly on protocol buffers.
We can now modify our handler:
If you used dep to handle your dependencies, run another dep ensure, if you used the go modules approach, nothing to do but running go build.
We can now use the wr struct to retrieve the data Dialogflow sent us. For example if you want to retrieve the output contexts:
Conclusion
You can find the complete code here.
