Create a Slack Live Blog using Node.js, Express, RethinkDB, Socket.io, & Handlebars

Delivering content broadcasted by Slack in real-time

GIF taken from working example

Why?

I’m a developer at a digital design and technology studio called PHDL. One of our awesome clients is an experiential marketing firm based in San Francisco called Manifold. Like many companies, Slack has become instrumental to Manifold’s daily workflow, communication and culture; so much that they came to us with an idea: is it possible to allow anyone on their team to publish content from a Slack channel onto a section of their website in real time? We had previously built a carousel that updated for them in the “contact us” section of their website, but to update it required admin access and WordPress familiarity — any funny inspirations to put on the page usually died on the vine if the site admin wasn’t around, and so the idea was born. We would use Slack to allow anyone at Manifold to publish content to this page on their WordPress site.

After doing some initial research, I didn’t like similar implementations that already existed. I found them to be inflexible, and so I set out to create an easy and flexible solution that can fit into any system or framework. If you can open a socket connection and receive data via an API, this setup should work for you. Resources for: Slack, Node.js, Express, RethinkDB, Socket.io, Handlebars

The Flow

The Flow Expanded

  • User sends message to specific Slack channel (e.g. #blog)
  • Slack bot watches the #blog channel and receives the message
  • Slack bot creates a RethinkDB entry based on message data
  • Node server is connected to RethinkDB with a real-time change feed active (watches for changes, emits event and data)
  • RethinkDB notifies Node server that data has changed and sends the data
  • Node server emits ‘new post’ event with new data over Socket.io
  • Node server serves Front End at “/”
  • Client has an active Socket.io connection waiting for the Node server to emit events with data
  • Client receives socket events and data, then updates the DOM accordingly

Three Distinct Parts

As you can see above, the application is split into three main parts:

1. Slack Bot

The Slack bot is a Node.js app that listens to a specific Slack channel (e.g #blog) and saves all data to a database using RethinkDB. It can be switched on and off by typing !on and !off commands in the chat.

2. Node Server

The Node server is an Express server that is connected to RethinkDB with a real-time change feed. When data is changed in the database, RethinkDB sends the data to the Node server through the change feed. When the Node server receives this data, it emits a socket event that the Client is connected to and waiting for. For added flexibility, the Node server also has an API endpoint to GET post data. This allows you to receive the posts without the need to connect to the database directly. This is useful for Front-End applications that cannot call a database.

3. Client

The Client is the end user. In this case, the Client is a website being served by Express at http://localhost:5000. When a user loads the website, a new Socket.io connection is established between the Client and the Node Server. Events and data are passed from the Node Server to the client in real-time. Past posts (prior to the user’s Socket.io connection) are rendered on load using Handlebars.

Now that we know a bit about the process, we can start coding.

Setup

Start off by cloning my sample repository using git:

git clone https://github.com/caseymorrisus/Slack-Live-Blog.git

You should see a file named server.js which contains the following:

Don’t worry about all of the dependencies for now, we will go over each part as we implement it.


Express Config

We will start off by adding the following Express configuration to the end of server.js:

Explanation

First, we tell Express to use Handlebars as its template engine.

We then set up the middleware and tell Express to serve static content from publicDir which was set to /public

After setting up the middleware, we can start the server.


Socket.io Server Config

Configuring the server to use Socket.io is fairly simple. All we are doing is opening a socket connection and logging to the server when a user connects and disconnects. This connection will be used later to emit events to the client.

Define Routes

Next to add into server.js are the two routes that will be served. The first route can be viewed at http://localhost:5000. It contains the following:

Route: “/”

Explanation

I won’t go into too much detail for the sake of brevity, however, I’ll continue to explain the most crucial pieces.

Below is how we set up a new route within Express. The following responds to GET requests at the route of “/”.

Connect to RethinkDB with the host “localhost” and the port “28015”. If you are hosting your database on a different server, replace localhost with the IP of that server.

Get the five most recent entries in the “posts” table.

Finally, we render the “home” view (home.handlebars) when the route is connected to. Send JSON object with 5 most recent entries to be rendered.


Route: “/posts”

Finally, we create a route which responds with a JSON object at http://localhost:5000/posts. This object contains the twenty-five most recent entries in the “posts” table within RethinkDB. While this functionality is not used for this specific implementation, it is a nice feature to have. Having this API allows the posts to be pulled safely from the database. This is useful for Front-End applications that don’t have the ability to connect to a database. Anything that can call an API now has access to your data!


RethinkDB Real-Time Change Feed

Finally, the last thing we add to server.js is the code needed to set up a real-time change feed for RethinkDB:

To explain further, the code below is for the change feed. We tell RethinkDB that we want to watch for changes in the ‘posts’ table. Whenever an entry is changed, it is sent to the Node server via the change feed. When the Node server receives changed data from RethinkDB, it sends that data to the Front-End (client) through an open socket connection and event ‘new post’.

The View

On the Front-End, we will be using Handlebars as our template engine. The data we are sending from Express is handled within views/home.handlebars. This file is inserted into views/layouts/main.handlebars where it says {{{body}}}. Anything you want to know about Handlebars can be found within their documentation.

Client Socket.io Configuration

Open public/js/production.js and you will find the following:

The majority of this file deals with DOM manipulations and animations within jQuery. That is not the focus of the article so I will continue to the important parts.

In the entire file, these two lines are the most important:

First, we open a socket connection with our Node server. Second, we wait for the Node server to emit a ‘new post’ event. Third, the Node server emits an event when the database has changed. A JSON object is passed along to the client through the ‘new post’ event. How you handle this data is up to you. In this case, I’m simply appending them to the DOM and keeping a maximum of five posts at any one time.


I thought you said something about a bot?

Start off by opening bot.coffee, as you can probably guess, we’ll be using CoffeeScript from now on. Once you open up the file, you should see a list of dependencies.

Each code snippet from here on should be appended to bot.coffee. Next, we configure Slack. You will need to visit the URL in the snippet to generate a bot and Slack Token.

Slack Bot Configuration

Next, we setup a simple object to store the bot’s options. For now, we are only using one option but this can be extended later.

Now, we will create a few helper functions to be used at a later time. Everything below is explained within the comments. If you want to use a different channel name, change ‘blog’ to the name of your desired channel.

Due to the significance of indentation within CoffeeScript, I’ve included the rest of the file in a single gist. Again, everything is explained in the code’s comments.

If you’d like to see the entire bot.coffee file you can find it here.

Starting The Server

We now have everything we need. You can start the Node server and bot in one command by entering npm start in the command line at document root. The page served at http://localhost.com will update in realtime when you post to Slack.

Wrapping Up

As you can see, it’s fairly easy to get something simple set up. The bot is looking a bit bare, but you can always add more features as they are needed. Some ideas for features and commands include:

  • Admin permissions (only admins can use the bot)
  • Update posts using the bot
  • Delete posts using the bot
  • Command to list most recent posts in Slack

Be warned, the code in this article was written as a prototype and proof of concept. It has since been refined, expanded upon, and changed for a production environment. If you use this in production, it is at your own risk. You can find examples in each resources documentation explaining what to do when deploying to production.