How to deploy a Facebook bot

Probably during this months you’ve heard about Facebook opening a beta stage on the development of bots for the Messenger platform.

Messenger

Several bots are currently available, probably you’ve chatted with Poncho, the weather forecaster, or with CNN bot for flashing news.

Maybe you don’t still now how to build and deploy your own one. First of all you need to know that you are gonna interface with Facebook for connecting your bot to one of your pages, you will not use Facebook to create a new bot. Each bot must be run on a owned server, thus whenever a message will arrive to your Facebook page, it will be forwarded to the server which will decide if answer or not. If your bot will have something to say it will answer to Facebook which will answer to the user. Basically your bot will send a structure similar to this to Facebook:

  {
    "recipient":{
        "id":"USER_ID"
    }, 
    "message":{
        "text":"hello, world!"
    }
  }

Here is a part of the JSON object you will receive when Facebook forwards to you a message:

    {
      "id":"PAGE_ID",
      "time":1460245674269,
      "messaging":[
        {
            "sender":{
            "id":"USER_ID"
          },
          "recipient":{
            "id":"PAGE_ID"
          },
          "timestamp":1460245672080,
          "message":{
            "mid":"mid.1460245671959:dad2ec9421b03d6f78",
            "seq":216,
            "text":"hello"
          }
        }
      ]
    }

You can handle all these objects and reply to Facebook with every language you want, you can even start an all-day-long session and reply manually with curl to each message you will receive, but this is something we don’t want. For this tutorial I have chosen Node.js for setting up a server that will handle all messages, it is fast, easy and has a huge list of available packages.

Pre-coding part

Before setting up the server we will need to create an app for our bot on Facebook. Head on Facebook and create an App:

https://developers.facebook.com/quickstarts/?platform=web

Choose a suitable name, remember that you will have only few opportunities to change the app name. When your App ID is created you can skip the Quick Start tutorial and land on your App dashboard. Add a product on the right side column and choose “Messenger”. Now you have to setup you server webhook address. However, I doubt you own a server right now (at the end of this tutorial you will). As for now you will use your Notebook as a server, this is practical because you will change the server code often and the changes will be reflected immediately without uploading stuff on a production server.

Another thing to mention is that Facebook only accept secure connections (hopefully) between him and our server, preventing messages sniffing and MITM attacks. So you are going to buy an SSL certificate from Digicert, just kidding! You will use a powerful tool called Ngrok the will tunnel our connection between Facebook and us to himself. We will talk insecurely to Ngrok servers and those ones will talk to Facebook in a secure way. Browse the official Ngrok website and download the tool:

ngrok - download

No installation or PATH modification in needed, just unzip it and launch it. A console will appear, type:

ngrok http 8080

Ngrok will establish a connection to our port 8080 to their servers. Copy the https address on the console, click “setup webook” on Facebook and paste it. Choose a key-phrase (whatever you want) that Facebook servers and our one will use and tick all four options. If you “Verify and Save” you will receive a 404 error since our server is not up. Let's turn it up!

Coding

We will use Node.js, let’s download and install it.

Download | Node.js

To understand if node was successfully installed open a console and type “node -v”, it should return you your node version. Well done! If you remember, before, I said that Node is popular because it has a lot of packages! In fact you will use this feature so many times, remember: fire was already discovered, no need to re-invent it.

Create a new folder for your project and browse it, now you have to initialize your project by typing “npm init”, fill all the gaps and then create a new index.js file. Open it and paste this code:

const http = require('http')
const Bot = require('messenger-bot')

var bot = new Bot({
  token: 'PAGE_TOKEN',
  verify: 'VERIFY_TOKEN'
})

bot.on('error', (err) => {
  console.log(err.message)
})

bot.on('message', (payload, reply) => {
  var text = payload.message.text + "!"

  bot.getProfile(payload.sender.id, (err, profile) => {
    if (err) throw err

    reply({ text }, (err) => {
      if (err) throw err

      console.log(`Echoed back to ${profile.first_name}               ${profile.last_name}: ${text}`)
    })
  })
})

http.createServer(bot.middleware()).listen(8080)
console.log('Echo bot server running at port 8080.')

Replace PAGE_TOKEN with your page token, you can generate it in your App dashboard under Messenger section(you will have to choose a page, the one on which your bot will be installed), and the VERIFY_TOKEN with the key-phrase you have submitted in the webhook form.

This script is your entry point. It is taken from Remixz who has done a beautiful Node package, the one we will use:

remixz/messenger-bot

You must save the code, install Remixz package and run the server by entering these commands in the console while in you project folder:

npm install messenger-bot --save
node index.js

Your server is now probably up! Listening on port 8080. The last step for connecting our bot to Facebook is to “Verify and Save” your webhook now that the server is up. If Facebook did not saved your parameters, just re-paste your Ngrok secure tunnel address, your key-phrase and tick all 4 message options. Now “Select a page to subscribe your webhook to the page events” which is your bot page. Facebook will now handle your webhook.

Test it

Start a new conversation with your page and you will see the bot in action, it will answer you back what you type to him with an additional exclamation mark!!

Now lets structure your code. The bot will catch all messages with this function:

bot.on('message', (payload, reply) => {
  bot.getProfile(payload.sender.id, (err, profile) => {
    //reply...
  })
})

In the payload there is a JSON structure similar to this one:

        {
          "sender":{
            "id":"USER_ID"
          },
          "recipient":{
            "id":"PAGE_ID"
          },
          "timestamp":1460245672080,
          "message":{
            "mid":"mid.1460245671959:dad2ec9421b03d6f78",
            "seq":216,
            "text":"hello"
          }
        }

You can check if the message contains some keyword and perform accordingly some actions.

if(payload.message.text.toLowerCase().contains(“help”)) {
    getHelp(reply, profile)
}else ...
  
function getHelp(reply, profile) { 
    replyToUser(reply, profile, `Hi ${profile.first_name}, ... `)
}
function replyToUser(reply, profile, answer){
    var text = answer.slice(0, 300)
    if(!!answer) {
      reply({ text }, (err) => { //need to pass exact name text
        try{
          console.log(`New Reply`)
        }catch(e){console.log(e)}
      }) 
    }
}

Remember that you have a text limit size of 320 chars.

Answer with useful messages to clients

When you create a bot, the first thing you want to do is serving users with some useful data. For example Poncho, the bot presented by Facebook during the FBF8, will tell you weather forecast for the city in which you live. Well, Poncho needs to fetch this data somewhere, probably it uses some sort of local database. Well, if you have a database ready you can fetch data from there. If you do not have any sort of data available the best thing to do is to use someone else’s data! You will need to query an API server. Here is the documentation for calling Yahoo Weather APIs.

Weather - Yahoo Developer Network

Or here you can find the immense Musixmatch db, which can give you information about singers, albums and even lyrics.

Home

You can easily query those API servers Request-promise, a great Node package.

Well, all these sites have an API calls limit, maybe you can ask for data only 2000 times per day. However, our bot must answer at anytime of the day. To outrun this limitation you can cache your APIs calls. Memory-cache is a great package that does this for you. You can store a key-value object when you query for the first time an API endpoint and then use the local copy for the next requests. You can even set an expiration time for the cached items, if you query some weather forecast better to cache for a small period, if you query movies information you can cache the results even for days.

Here there is an example of query+cache call to a server.

var cache = require('memory-cache')
var rp = require('request-promise')
var cachedResult = cache.get(item)
if(cachedResult !== null){
  processData(cachedResult)
}else{
  var options = {
     uri: 'http://iloveapis.co/api/v2/item/'+ item,
     json: true, // parse the json in an object
     timeout: 10000 // 10 secs to wait for reply
  }
  rp(options)
    .then(function (response) {
       processData(response)  
       cache.put(item, response, 30000) //caching
    })
    .catch(function (err) {
        console.log(err)
    });
}

A couple of things to point out

Whenever you restart your Notebook or simply Ngrok for creating a tunnel between your Notebook and Facebook’s servers, Ngrok will give you a new address. Just browse the Facebook Developers Console and change the address in the webhooks section of your App.

Sessions

Poncho remembers where you live. Your bot can remember anything a user says. You can even store in a database each message the bot will receive. But this requires a lot of space and a powerful database, one could be Mongo. You can instead use sessions in your main Node script. You can create an Object session = {} and add a key for each user that messages the bot.

function findOrCreateSession (fbid){
    var sessionId;
    // Let's see if we already have a session for the user fbid
    Object.keys(sessions).forEach(k => {
      if (sessions[k].fbid === fbid) {
         // Yep, got it!
         sessionId = k;
      }
    });
    if (!sessionId) {
      // No session found for user fbid, let's create a new one
      sessionId = new Date().toISOString();
      sessions[sessionId] = {fbid: fbid, context: {}};
    }
    return sessionId;
}  
const sessionId = findOrCreateSession(payload.sender.id)
var session = sessions[sessionId]
session.context.yourfield = 10

So now you can remember what a user said and formulate a conversation.

• Which movies do you prefer?
• Dramas -> session.context.favourite = “drama” //store
• So check Philadelphia with T.Hanks
• Tnx, give me another one -> he likes drama, he said //retrieve
• Check Kramer vs. Kramer

Deploying on a real server

Well, now your bot lives on your pc. If you shut it down the bot isn’t going to answer users anymore. You need a free server, most of the people out there will tell you: go with Heroku for your Node app. No! Heroku is a great service, but for the free plan your app has to sleep (bot down) for at least 6 hours each day. We want a free and continous service.

Here it comes Openshift!

OpenShift: PaaS by Red Hat, Built on Docker and Kubernetes

Developed by Red Hat it gives you a server with SSH connection and a list of pre-installed softwares. Sign up and when you are done browse to your Openshift console. Add an application, scroll down and choose Node.js (Latest), so your server will auto update Node for you. Thats a big feature.

To connect to our server we must install RHC Openshift package. It is a little bit tricky. You need to install Ruby (if you are on Windows, use the 2.1.8 version), then open a console and type:

gem install rhc
rhc setup
// fill all the infos required 

Gem is the package manager for Ruby software, similar to what npm does for Node.

When you completed the setup, rhc have installed on your pc a SSH key, so your pc will be trusted when you talk to your server. Now you need to upload your bot on it. You need to have Git installed in order to perform this operation.

Git - Downloads

When it is installed you must clone the server folder into a local copy that will reside on your Notebook. You must find the address of the repository(the folder) on your Openshift console. In the picture is underlined in green color.

git clone yourbigid@app-server.rhcloud.com

Now you have a local copy of what is running on your server, it is just a welcome page. You can now copy all your bot files in the folder you have just cloned.

To update the server files with the new one you have to run three console commands

git add .
git commit -m "description of the changes"
git push

Git will start to upload your files, then it will build your project and start the server.

A couple of thing to point out

Your bot when deployed on openshift must be serverd on a certain ip address and on a certain port. So in your bot code just add this few changes

var server_port = process.env.OPENSHIFT_NODEJS_PORT || 8080
var server_ip_address = process.env.OPENSHIFT_NODEJS_IP || ‘127.0.0.1’
http.createServer(bot.middleware()).listen(server_port, server_ip_address)

Handle exceptions

To avoid unexpected crashes of your bot surround all your functions with try blocks.

try{ ... }catch(e){console.log(e)}

Great your server is up! If you browse it you must see a message similar to this one: Error, wrong validation token, since in our request there isn’t the keyphrase that we have set up previously.

Head on Facebook Developer Console, select your App and change the webhook address, you don’t need anymore to put your Ngrok address, just put your server address.

Connect to the server and check logs

It is useful to login your server and check for bot errors. With RHC it is so simple.

rhc tail -f app-root/logs/nodejs.log -o “-n 50” -a appname

It will show the last 50 line of console.log() of your application. It is even live, so you’ll see all new log lines without re-run the command.

If something went wrong, and you want to restart your app just type:

rhc restart-app appname

You can even restart the app at anytime without your Notebook by logging in Openshift website (the site is mobile friendly).

Get Facebook approval

Probably you have noticed that you are the only one who can talk to your bot on Facebook. Well this is a security reason provided by Facebook to avoid spamming of bots. You have now to get Facebook approval.

You must request to Facebook a permission called pages_messaging. Click App review on the right column while in the Developer Console, make your App public and request the permission.

Now you have to edit the notes, confirming that your app will not spam any user and you must upload a video of how a user can talk to your Bot. You have to show all its behaviours. This is the one I have submitted.

Now you must provide a picture for your App and a Privacy Policy URL. Click on the Settings section on the right column and upload an Image, then generate your Privacy Policy here:

PrivacyPolicies.com: Privacy Policy Generator

Fill all the infos, then copy the policy and paste it on a store-paste website, like Betterbin or Ghostbin. Copy-paste the generated URL to Facebook Privacy Policy URL form.

Now you have to add a platform in your App settings. Choose Page-tab and link your Privacy Policy URL in the Secure Page Tab URL. However, if you have a real website to link you should link it.

Click on the right column App Review and start a submission. Facebook team will take one week to review your submission, after that you can spread the word and tell your friends about your Messenger bot.

Tips

You can recieve a beautiful Messenger URL by going on your Facebook page settings (not Facebook App settings) Settings -> Page Info -> Facebook web address, here type your Page name. You will recieve two addresses

facebook.com/pagename -> links your page
m.me/pagename -> starts a conversation with your bot

Here is mine: m.me/pokemonbot

If you read till here probably you set up succesfully your Bot, click the Recommend button and comment with your Messenger URL.