Wit.ai Explained — Part 3— Building a bot with Sails.js
In the first part of this series, we went through some of the concepts that Wit.ai introduced in their (relatively) new Bot Engine.
In the second part, we built a Rails API application using the Ruby SDK for Wit.ai to see how to wield Wit.
In this part, we’re going to build the same thing but with the Sails.js framework. It’s pretty much the same tutorial, but everything pertaining to Ruby/Rails has been changed to use JavaScript/Sails.
To be able to keep up with this tutorial, you’ll need to have an intermediate grasp of Sails.js and it’s paradigms. To learn more about Sails.js, you can check out my other blog; hellosails.com.
The complete code for this example is available here on Github.
Overview: What are we building?
We’ll be building a simple Sails API that will serve as the back-end of our chatbot. It will have only two endpoints:
- POST /start — to initiate a conversation with our bot. It shall accept no parameters.
- POST /message — to send a message to our bot. It takes only one parameter; message (a string).
To send messages to the client application (Angular app, Android app etc.) that would be consuming your API, we’ll use PubNub realtime messaging. You’re going to need either a service like PubNub or WebSockets to send messages to your client application.
We’re doing that because we don’t want to respond to our User’s messages in the Request-Response cycle of an HTTP request. This might hold up the conversation unnecessarily and it would mean that we can’t send messages to our users unless they send one first.
Finally our chatbot is going to use the Wit Application and Story that I talked about in the first post. If you haven’t gone through the tutorial to recreate that application in your own Wit Console, you can find it here. You can also download a copy of the application that you can import via the Wit Console.
It’s absolutely necessary to get that done before proceeding with this example.
The Wit Node.js SDK
Before we start writing code, I think it’s useful to go over some of the capabilities of Wit’s Node.js Library.
It has a Wit class constructor that takes in the following parameters:
const {Wit, log} = require('node-wit');
const client = new Wit({
accessToken: MY_TOKEN,
actions: {
send(request, response) {
return new Promise(function(resolve, reject) {
console.log(JSON.stringify(response));
return resolve();
});
},
myAction({sessionId, context, text, entities}) {
...
return Promise.resolve(context);
}
}
});1. accessToken
This is your server side token you can obtain from your Wit Console. This parameter is required.
2. actions
This is the object where all the fun happens. Remember when I explained actions in the first part? The actions you define while creating a story need their corresponding function/method in your code. The actions parameter takes your action names as keys and their implementations as functions.
Asides from your own defined actions, there’s one other very useful action you can add to your actions object. It’s called “send”, and it’s called upon when Wit is sending a message to your user. It is via this function that you can do stuff like save all Wit’s responses to the database or send them to your client application.
The Wit Methods
The Wit class has three methods that you can use to interact with Wit from within your code.
- message: This is the simplest one. All it does is extract the meaning from a string. It returns all the entities and their accompanying data.
const client = new Wit({accessToken: 'MY_TOKEN'});
client.message('what is the weather in London?', {})
.then((data) => {
console.log('Yay, got Wit.ai response: ' + JSON.stringify(data));
})
.catch(console.error);- converse: This does quite a bit more. It returns what your bot should do next. This could be to respond to the user with a message, to perform an action or to wait for further requests. To get the whole sequence of actions, one would need to call converse repeatedly until the type is “stop”.
client.converse('my-user-session-42', 'what is the weather in London?', {})
.then((data) => {
console.log('Yay, got Wit.ai response: ' + JSON.stringify(data));
})
.catch(console.error);- runActions: This is the method we’re most interested in. It’s a higher lever method to converse. What it does is that it runs actions (duh). As opposed to .converse, in which you have to call successively, this just takes the functions you passed into the Wit constructor and calls them when it should. You can also pass in the maximum number of actions to execute (defaults to 5). This makes our life much easier as we’ll see soon.
const sessionId = 'my-user-session-42';
client.runActions(sessionId, 'what is the weather in London?', {})
.then((context) => {
console.log('Waiting for next user messages');
})
.catch((e) => {
console.log('Oops! Got an error: ' + e);
});You can read more about these methods in the official documentation.
Let’s Build
So let’s create a new API only sails application. Make sure your Node version is 6.x.x, the Wit Node SDK won’t work except it is. To update node using npm, follow this tutorial.
sails new chatbot-sails --no-frontendService Primer
From going through the documentation, we can easily see that we will need the Wit client and it’s methods in our controller(s).
Services in Sails.js are pretty easy to understand. They’re simply JavaScript objects with functions that can be used everywhere (controllers, policies … etc). You can read more about services in this hellosails services tutorial.
Create a new WitService.js file in /api/services. Then npm install --save node-wit.
/**
* WitService.js
*/
const Wit = require('node-wit').Wit
const token = "7DO5OGFBNMKCLW57NIIO5I7CS27RAJCU"
let conversation = {}
const actions = {
send(request, response) {
// do something when bot sends message
},
findTheatre(request) {
// perform action here and update context
}
}const client = new Wit({accessToken: token, actions})
module.exports = {
client: new Wit({accessToken: token, actions}),
setConversation(conv){
conversation = conv
}
}
Thanks to the above code, we can call WitService.client anywhere in our Sails application and it would return the Wit client to be used to send messages or run actions.
Also, we added a setConversation() method so that we can pass a reference to the conversation to WitService. Why we’re doing this would become apparent later.
Note that you shouldn’t actually have your accessToken or any other token like it just lying around in code waiting to be put in version control. You should use environment variables in production.
Let us add more code to our actions to get it to update the context with the right keys at the right time.
Here’s the WitService again, with more stuff added in.
/**
* WitService.js
*/
const Wit = require('node-wit').Wit
const token = "7DO5OGFBNMKCLW57NIIO5I7CS27RAJCU"
let conversation = {}
const actions = {
send(request, response) {
...
},
findTheatre({context, entities}) {
const showTime = firstEntityValue(entities, "datetime") || context.showTime
const movie = firstEntityValue(entities, "movie") || context.movie
if (showTime != null) {
context.showTime = showTime
delete context.missingTime
} else {
context.missingTime = true
}
if (movie != null){
context.movie = movie
}
if (movie && showTime){
const theatre = searchTheatres(showTime, movie)
context.theatre = theatre
var newContext = {}
}else{
var newContext = context
}
return Conversation.update({uid: conversation.uid}, {context: newContext})
.then((updated) => {
return Promise.resolve(context)
})
}
}
const firstEntityValue = (entities, entity) => {
const val = entities && entities[entity] &&
Array.isArray(entities[entity]) &&
entities[entity].length > 0 &&
entities[entity][0].value
;
if (!val) {
return null;
}
return typeof val === 'object' ? val.value : val;
};
const searchTheatres = (showTime, movie) => {
// perform query magic
console.log("Searching for Theatre")
return "Random Theatre"
}
const client = new Wit({accessToken: token, actions})
module.exports = {
client: client,
setConversation(conv){
conversation = conv
}
}The code above in the findTheatre action extracts the entities from Wit’s request parameter and updates context’s keys and values based on whatever it requires to operate.
So when we pass missingTime, the bot simply asks for the time because that’s what we told it to do.
At the end of the action, the conversation’s context is updated with the new context so that it can be used to call runActions so that context will be remembered and the bot can continue from where it left off.
You’ll notice that when the correct, complete parameters have been gotten from user input, the conversation’s context is updated to an empty object. This is to reset the “state” of the entire conversation so that the bot knows to start all over from the beginning after successfully completing a story.
PubnubService
PubNub is a pubsub realtime messaging API/service and I think they’re pretty awesome. You can read more about their product and services here.
We’ll be using the pubnub node library to harness some of their powers.
In our case, we’re going to use PubNub to send messages to our client application and hence, like with the WitService, we’ll need to be able to use PubNub in our controllers.
/**
* PubnubService.js
*/
const Pubnub = require('pubnub')
const publishKey = "pub-c-1c361ca2-73cf-4ea3-bdab-eefaa0d3187d"
const subscribeKey = "sub-c-f7a78ae4-829f-11e6-974e-0619f8945a4f"
const client = new Pubnub({
publishKey: publishKey,
subscribeKey: subscribeKey
})
module.exports = {
client: client
}The code above should be in /api/services/PubnubService.js.
Do not forget that it’s bad practice to leave your keys in code/version control. You should use environment variables.
Models
Now that all of that is done, we need to create our models. We’re gonna only have two, relatively simple models; Message and Conversation
sails generate model Message body:string kind:stringAnd for Conversation
sails generate model Conversation uid:string context:jsonWhat we’re gonna do next is set up associations, validations and to randomly generate a uid when a conversation is created.
In /api/models/Message.js
/**
* Message.js
*
*/
module.exports = {
attributes: {
body: {
type: 'string',
required: true
},
kind: {
type: 'string',
required: true,
in: ["outgoing", "incoming"]
},
conversation: {
model: "conversation"
}
}
};In /api/models/Conversation.js
/**
* Conversation.js
*
*/
const uuid = require('uuid');
module.exports = {
attributes: {
uid: {
type: 'string',
required: true,
unique: true,
defaultsTo() {
return uuid.v4();
}
},
messages: {
collection: "message",
via: "conversation"
},
context: {
type: 'json',
defaultsTo: {}
}
}
};To generate the v4 uuid, you need to npm install uuid.
The context attribute in our conversations model, to be used in runActions().
ChatController
We’re going to have only one controller called ChatController and like I said earlier, two actions/endpoints.
In /api/controllers/ChatController.js
/**
* ChatController
*
*/
const _ = require('lodash');
module.exports = {
_config: {
rest: false,
shortcuts: false,
actions: false
},
start(request, response){
Conversation.create()
.then((conversation) => {
WitService.setConversation(conversation)
response.status(201).json(conversation)
})
},
message(request, response){
const data = _.pick(request.body, ["uid", "message"])
Conversation.findOne({uid: data.uid})
.then((conversation) => {
WitService.setConversation(conversation)
createIncomingMessage(data.message, conversation).then((message) => {
WitService.client.runActions(conversation.uid, message.body, conversation.context)
console.log(`SENDING TO WIT: ${message.body}`)
response.status(204)
})
})
}
};
const createIncomingMessage = (body, conversation) => {
return Message.create({
conversation: conversation,
body: body,
kind: "incoming"
})
}The start method/action creates a conversation.
The message method/action receives a message from our client application, creates an incoming message and saves it to the database then sends the message to Wit.ai using the runActions() method. The conversation context that has been persisted in the WitService is what shall be passed into runActions() so that it can continue where it left off.
Notice how we use WitService.client.setConversation() to pass our conversation to the WitService when we create a conversation and when we’re retrieving a message’s conversation.
Also, you have to npm install lodash for the controller to work.
What about sending messages to the user?
Remember the send function we created in our WitService?
We can get the response that Wit intends to send to our user from the arguments of that function and use PubNub to send it to our client application.
send(request, response) {
PubnubService.client.publish({
message: response.text,
channel: conversation.uid,
},
function (status, response) {
// handle status, response
})
Message.create({body:response.text, conversation:conversation, kind:"outgoing"})
console.log(`sending... ${response.text}`)
},Also, we created and persisted a new outgoing message so that our database can have record of the entire conversation between Wit and our users.
Routes
The only thing left to do now is to set up our routes and that’s a very simple thing to do.
In /config/routes.js
module.exports.routes = {
'post /start': 'ChatController.start',
'post /message': 'ChatController.message'
};Deployment & Testing
You can can sails lift test the API locally by using Postman.
That’s all for today folks. Remember, the complete code for this example is available here on Github.
What’s Next?
In the fourth part, we’re doing the exact same thing AGAIN, but this time with the HTTP API. I feel like it makes sense to show how something like this can be used in a native Android or iOS app. Anyway, I’ll choose one mobile operating system depending on how I feel at the time.
