Wit.ai Explained — Part 2— Building a bot with Ruby on Rails
In the first part of this series, we went through some of the concepts that Wit.ai introduced in their (relatively) new Bot Engine.
In this part, we are going to build a Rails application using the Ruby SDK for Wit.ai to illustrate how you can use Wit’s new powers in your code to build a chatbot.
To be able to keep up with this tutorial, you’ll need to have an intermediate grasp of Ruby on Rails and it’s paradigms.
The complete code for this example is available here on Github.
Overview: What are we building?
We’ll be building a simple Rails API that will serve as the back-end of our chatbot. It will have only two endpoints:
- POST /start – to initiate a conversation with our bot. It shall accept no parameters.
- POST /message – to send a message to our bot. It takes only one parameter; message (a string).
To send messages to the client application (Angular app, Android app etc.) that would be consuming your API, we’ll use PubNub realtime messaging. You’re going to need either a service like PubNub or WebSockets to send messages to your client application.
We’re doing that because we don’t want to respond to our User’s messages in the Request-Response cycle of an HTTP request. This might hold up the conversation unnecessarily and it would mean that we can’t send messages to our users unless they send one first.
Finally our chatbot is going to use the Wit Application and Story that I talked about in the last post. If you haven’t gone through the tutorial to recreate that application in your own Wit Console, you can find it here. You can also download a copy of the application that you can import via the Wit Console.
It’s absolutely necessary to get that done before proceeding with this example.
The Wit Ruby SDK
Before we start writing code, I think it’s useful to go over some of the capabilities of Wit’s Ruby Library.
It has a Wit class constructor that takes a Hash as an argument. This Hash can have the following as it’s elements:
require 'wit'
actions = {
send: -> (request, response) {
puts("sending... #{response['text']}")
},
my_action: -> (request) {
return request['context']
},
}
client = Wit.new(access_token: access_token, actions: actions)1. access_token
This is your server side token you can obtain from your Wit Console. This parameter is required.
2. actions
This is a Hash too and it’s where all the fun happens. Remember when I explained actions in the first part? The actions you define while creating a story need their corresponding function/method in your code. The actions hash takes your action names as keys and their implementations as lambda functions.
Asides from your own defined actions, there’s one other very useful action you can add to your actions Hash. It’s called “send”, and it’s called upon when Wit is sending a message to your user. It is via this method that you can do stuff like save all Wit’s responses to the database or send them to your client application.
The Wit Methods
The Wit class has three methods that you can use to interact with Wit from within your code.
- message: This is the simplest one. All it does is extract the meaning from a string. It returns a Hash of entities and their accompanying data.
response = client.message('what is the weather in London?')- converse: This does quite a bit more. It returns what your bot should do next. This could be to respond to the user with a message, to perform an action or to wait for further requests. To get the whole sequence of actions, one would need to call converse repeatedly until the type is “stop”.
response = client.converse('user-session-id', 'what is the weather in London?', {})- run_actions: This is the method we’re most interested in. It’s a higher lever method to converse. What it does is that it runs actions (duh). As opposed to .converse, in which you have to call successively, this just takes the lambda functions you passed into the Wit constructor and calls them when it should. You can also pass in the maximum number of actions to execute (defaults to 5). This makes our life much easier as we’ll see soon.
client.run_actions('user-session-id', 'what is the weather in London?')You can read more about these methods in the official documentation.
Let’s Build
So let’s create a new API only rails application.
rails new chatbot --apiPrelim Setup
The first thing I usually do is set up my Gemfile. Though it’s impossible to predict beforehand all the gems one would need, there are a few I pretty much always use.
source 'https://rubygems.org'
gem 'rails', '~> 5.0.0', '>= 5.0.0.1'
gem 'puma', '~> 3.0'gem 'responders', '~> 2.0' #for respond_to and respond_with
gem 'rails_param' #to validate/coerce request parameters
gem 'rack-cors', :require => 'rack/cors' #to set up cors
gem 'pubnub' #pubnub ruby library
gem 'wit' #wit ruby sdk
group :development, :test do
gem 'spring'
gem 'sqlite3'
gem 'listen'
end
group :production do
# production gems for heroku deployment
gem 'rails_12factor'
gem 'pg'
end
The comments beside some of the gems aim to describe what they do. However, I recommend that you go to the github page of each gem for a more in-depth understanding of their functionality.
The next thing to do would be to prep our ApplicationController.
class ApplicationController < ActionController::API
include ActionController::ImplicitRender
respond_to :json
rescue_from RailsParam::Param::InvalidParameterError do |exception|
render json: {errors: exception}, status: 422
end
def default_serializer_options
{root: false}
end
endThe above code does a couple of things:
- including ActionController::ImplicitRender handles implicit rendering for a controller action that does not explicitly respond with render, respond_to, redirect, or head.
- rescue_from RailsParam::Param::InvalidParameterError catches the exceptions from the rails_param gem and renders the JSON response.
- overriding default_serializer_options removes all root nodes from our JSON responses
Now, we create an extensions folder within our /app folder and tell Rails to autoload everything in it by adding the following lines of code to the /config/application.rb
config.autoload_paths += %W(#{config.root}/app/extensions)
config.autoload_paths += Dir["#{config.root}/app/extensions/**/"]The /extensions folder is for custom Ruby classes and modules that don’t fit into any of the standard Rails folders. It is in this folder we shall keep ruby files to manage the code that will interact with the Wit and PubNub API’s.
Lastly, we setup CORS. To do that, you need to uncomment the code in the /config/initializers/cors.rb file.
It is important to restrict the domains/origins that can consume your API in production (except you’re building a public facing API). You can read more on how to do that here.
The default configuration that we just uncommented lets every and any one access our API.
Singleton Primer
From going through the wit-ruby documentation, we can easily see that we will need an instance of the Wit class in our controller(s). However, it doesn’t make sense to create a new instance of the class every time someone makes a request that corresponds to a controller action.
One way to do this would be to create an initializer that instantiates the Wit class and stores it in a global variable. Generally, global variables are bad, so we’re gonna use the Singleton Pattern to get this done.
Singletons are simply classes that can have only one instance. Ruby comes with a Singleton module that one can include in your class to make it a Singleton. If a class Klass includes Singleton, Klass.instance will always return the same object and Klass.new would return a NoMethodError.
I also think having Singletons in our /extensions folder makes everything neater.
WitExtension Singleton
Create a new wit_extension.rb file in /extensions. What we need to do now, is create a WitExtension Singleton class and in it’s initializer, we set up a Wit client, it’s access_token and actions.
require 'wit'
require 'singleton'
class WitExtension
include Singleton
def initializer
access_token = "7DO5OGFBNMKCLW57NIIO5I7CS27RAJCU"
actions = {
send: -> (request, response) {
# do something when bot sends message
},
findRestaurants: -> (request) {
#
}
}
@client = Wit.new(access_token: access_token, actions: actions)
end
def client
return @client
end def set_conversation(conversation)
@conversation = conversation
end
end
Thanks to the above code, we can call WitExtension.instance.client anywhere in our Rails application and it would return the same instance of WitExtension and hence, the same Wit client object.
Also, we added a set_conversation method so that we can pass a reference to the conversation to WitExtension. Why we’re doing this would become apparent later.
Note that you shouldn’t actually have your access_token or any other token like it just lying around in code waiting to be put in version control. You should use the secrets.yml for development and environment variables in production.
Let us add more code to our actions Hash to get it to update the context with the right keys at the right time.
Here’s the WitExtension class again, with more stuff added in.
require 'wit'
require 'singleton'
class WitExtension
include Singleton
def initialize
access_token = "7DO5OGFBNMKCLW57NIIO5I7CS27RAJCU"
actions = {
send: -> (request, response) {
},
findTheatre: -> (request) {
context = request["context"]
entities = request["entities"]
showTime = first_entity_value(entities, "datetime") || context["showTime"]
movie = first_entity_value(entities, "movie") || context["movue"]
if showTime
context["showTime"] = showTime
context.delete("missingTime")
else
context["missingTime"] = true
end
if movie
context["movie"] = movie
end
if showTime && movie
theatre = search_theatres(showTime, movie)
context["theatre"] = theatre
new_context = {}
else
new_context = context
end
@conversation.update(context: new_context)
return context
}
}
@client = Wit.new(access_token: access_token, actions: actions)
end
def client
return @client
end
def set_conversation(conversation)
@conversation = conversation
end
private
def first_entity_value(entities, entity)
return nil unless entities.has_key? entity
val = entities[entity][0]['value']
return nil if val.nil?
return val.is_a?(Hash) ? val['value'] : val
end
def search_theatres(showTime, movie)
# perform search query magic
puts "Searching for Theatre..."
return "Random Theatre"
end
endThe code above in the findTheatre action extracts the entities from Wit’s request parameter and updates context’s keys and values based on whatever it requires to operate.
It is from the returned context Hash that Wit decides what to do based on the presence and/or absence of any of the keys.
So when we pass missingTime, the bot simply asks for the time because that’s what we told it to do.
PubnubExtension Singleton
PubNub is a pubsub realtime messaging API/service and I think they’re pretty awesome. You can read more about their product and services here.
We’ll be using the pubnub ruby library to harness some of their powers.
In our case, we’re going to use PubNub to send messages to our client application and hence, like with the Wit class, we’ll need a singular, globally available instance of the Pubnub class.
require 'pubnub'
require 'singleton'
class PubnubExtension
include Singleton
def initialize
@client = Pubnub.new(
subscribe_key: 'sub-c-f7a78ae4-829f-11e6-974e-0619f8945a4f',
publish_key: 'pub-c-1c361ca2-73cf-4ea3-bdab-eefaa0d3187d',
:logger => Rails.logger,
error_callback: lambda do |msg|
puts "Error callback says: #{msg.inspect}"
end,
connect_callback: lambda do |msg|
puts "CONNECTED: #{msg.inspect}"
end
)
end
def client
return @client
end
endThe code above should be in /extensions/pubnub_extension.rb and it does pretty much what one would expect it to;
- declare and initialize the client variable so that it shall exist by the time PubnubExtension.instance is called and
- have a getter for said client.
Do not forget that it’s bad practice to leave your keys in code/version control. You should use environment variables and/or secrets.yml
Models
Now that all of that is done, we need to create our models. We’re gonna only have two, relatively simple models; Message and Conversation
rails g model Message body:string conversation_id:integer kind:stringAnd for Conversation
rails g model Conversation uid:string context:textWe can now run our migrations.
rails db:migrateWhat we’re gonna do next is set up associations, validations and to randomly generate a uid when a conversation is created.
In /app/models/message.rb
class Message < ApplicationRecord
belongs_to :conversation
validates_inclusion_of :kind, :in => ["outgoing", "incoming"], allow_nil: false
endIn /app/models/conversation.rb
class Conversation < ApplicationRecord
serialize :context, Hash
before_create :set_uid
has_many :messages
private
def set_uid
return if uid.present?
self.uid = generate_uid
end
def generate_uid
SecureRandom.uuid.gsub(/\-/, '')
end
endChatController
We’re going to have only one controller called ChatController and like I said earlier, two actions/endpoints.
In /app/controllers/chat_controller.rb
class ChatController < ApplicationController
def start
@conversation = Conversation.create
WitExtension.instance.set_conversation(@conversation)
PubnubExtension.instance.client.channel_registration(action: :add, group: 'main', channel: @conversation.uid)
render json: @conversation
end
def message
param! :message, String, required: true
param! :uid, String, required: true
@conversation = Conversation.find_by_uid(params[:uid])
WitExtension.instance.set_conversation(@conversation)
@message = @conversation.messages.create(
body: message,
kind: "incoming"
)
WitExtension.instance.client.run_actions(@conversation.uid, params[:message], @conversation.context)
puts "SENDING TO WIT #{params[:message]}"
end
endThe start method/action creates a conversation and registers that conversation as a channel on PubNub.
The message method/action receives a message from our client application, creates an incoming message and saves it to the database then sends the message to Wit.ai using the run_actions() method. The conversation context that has been persisted in the WitExtension is what shall be passed into run_actions() so that it can continue where it left off.
We can clean up the code in the controller to make it more verbose and clearer about what it’s doing at every point.
class ChatController < ApplicationController
def start
create_conversation
add_channel_to_main_group
render json: @conversation
end
def message
validate_message_params
set_conversation
create_incoming_message(params[:message])
WitExtension.instance.client.run_actions(@conversation.uid, params[:message], @conversation.context)
puts "SENDING TO WIT #{params[:message]}"
end
private
def validate_message_params
param! :message, String, required: true
param! :uid, String, required: true
end
def create_conversation
@conversation = Conversation.create
WitExtension.instance.set_conversation(@conversation)
end
def add_channel_to_main_group
PubnubExtension.instance.client.channel_registration(action: :add, group: 'main', channel: @conversation.uid)
end
def create_incoming_message(message)
create_message("incoming", message)
end
def create_message kind, message
@message = @conversation.messages.create(
body: message,
kind: kind
)
end
def set_conversation
@conversation = Conversation.find_by_uid(params[:uid])
WitExtension.instance.set_conversation(@conversation)
end
endNotice how we use WitExtension.instance.set_conversation to pass our conversation to the WitExtension when we create a conversation and when we’re retrieving a message’s conversation.
What about sending messages to the user?
Remember the send lambda function we created in our WitExtension?
We can get the response that Wit intends to send to our user from the arguments of that function and use PubNub to send it to our client application.
send: -> (request, response) {
PubnubExtension.instance.client.publish(message: response['text'], channel: @conversation.uid)
@conversation.messages.create(body: response['text'], kind: "outgoing")
puts("sending... #{response['text']}")
},Also, we created and persisted a new outgoing message so that our database can have record of the entire conversation between Wit and our users.
Routes
The only thing left to do now is to set up our routes and that’s a very simple thing to do.
In /config/routes.rb
Rails.application.routes.draw do
post '/start', to: 'chat#start'
post '/message', to: 'chat#message'
endDeployment & Testing
This can be easily hosted on Heroku by following their guide and you can test the API by using Postman.
That’s all for today folks. Remember, the complete code for this example is available here on Github.
What’s Next?
- In the third part of this series, we shall do the exact same thing we did here but with Sails.js and the official Wit Node library.
- In the fourth part, we’re doing the exact same thing AGAIN, but this time with the HTTP API. I feel like it makes sense to show how something like this can be used in a native Android or iOS app. Anyway, I’ll choose one mobile operating system depending on how I feel at the time.
