4 Things I Learned While Building a Slackbot With AWS Lambda

By eleith

tl;dr: Lambda promises to simplify how we build services in the future. While building a slackbot internally for Coursera, I learned a few lessons about Lambda worth sharing.

When I first heard about AWS Lambda, I couldn’t wait to get my hands on it. One year later I made the decision to stop stalling and use it to power my first Slack integration.

At Coursera, we use Slack so heavily that merely switching tabs out of Slack can be costly to our productivity (assuming you define our time in Slack as productive…).

We have created a number of Slack apps to help in this endeavor. We have /boba to schedule boba runs and /lunch to find out what’s for lunch (because who wants to walk to the fridge to read the menu?).

Along those lines, I created /buggy.

Because our team not only builds Coursera but we use it every day, we find a number of bugs ourselves. We have a central #bugs channel and each team has their own channel where numerous other bugs gets reported.

However, when someone reports a bug, we have an awkward dance of who or whom actually should file the Jira ticket. /buggy aims to sidestep this awkward dance and bias us towards action.

With /buggy we now interact with our bug collector (Jira) more efficiently (labeling, assigning, previewing, searching and more) without ever leaving Slack.

The ever helpful Bryan suggested I use Lambda for the project. This sent me down a surprisingly ziggy path where I came to learn that Lambda is the definitely the future … just not for tomorrow.

Here are four things I’ve learned in the process. I’ll definitely use Lambda again, but for now, I plan to move /buggy to an Express service running on an EC2 micro.

AWS Lamba Scripts Are Easy to Write

To build a Lambda script, you only need to create a function that takes in a request object and calls a method on the context argument when you are done. Here is an example:

var AWS = require('aws-sdk') exports.handler = function(event, context) { context.done('helloooo “the future”!'); }

You can write your Lambda scripts in Nodejs, Python or Java.

However, be aware of a few non trivial caveats:

LAMBDA USES NODE v0.10

At Coursera we write all our frontend (and Nodejs stuff) in ES6 (ES2015). Yet, I quickly learned that Lambda only runs Node v0.10, a year old version of Node that lacks a number of fixes and ES6 support.

Just for reference, Python is pegged at 2.7 and Java is pegged at 8.

Writing without ES6 was not a showstopper but I did run into other walls.

The Node Jira library [https://github.com/floralvikings/jira-connector] supports oauth authentication with our Jira instance using a common Node oauth library. Unfortunately, there seems to be a bug with the oauth library when trying to sign oauth requests that seem to work with Node v0.11 and up, but fails on the Node version Lambda uses.

At this point, I had a few choices: write my own oauth library, don’t use oauth, or re-write my application in Python.

I made the wrong choice.

Lamba Python Scripts Requires Non-Trivial Packaging

The Python Jira library has a number of dependencies that needs manual installation. One of them, to my dismay, is the Python crypto library, which doesn’t seem to be your ordinary Python library.

With Lambda, if you need to use Python libraries you need to package them all together and upload them in a zip file. with Node, all the libraries are in a node_modules and thus this packaging is straightforward.

With Python you have a bit more work because of the need to use virtual_env. Pycrypto seemed to further complicate these steps and I decided to switch gears and make life more simpler and dump oauth all together and go back to the simplicity of Nodejs packaging.

Lambda Requires Learning Two Other AWS Services

You can test the output of your Lambda function directly within the aws console and if you write your own script to bundle and upload your bundle directly to AWS, the development cycle gets pretty fast. However, this testing is limited to a pre-defined input that you specify within the console.

To debug external calls to your function, you have to use yet another AWS service: Cloudwatch Logs. This one is pretty easy, since the AWS console has built a good UI for this, if not slightly frustrating for how many clicks and refreshes you end up doing.

Yet, before you can proceed in testing, you’ll soon realize that your Lambda function is unreachable from the outside world. To fix this, you’ll need to learn a third AWS service: API Gateway.

With the API Gateway, you can open external access, once you figure out how to use the semi-complicated interface. This service seems pretty powerful, allowing you to manipulate and transform all incoming requests and outgoing responses, in addition to setting authentication schemes in front of your endpoints.

In the end, you can learn these services quickly, it just requires a lot of poking, documentation reading and maybe getting lucky on stack overflow. However, it does feel a bit heavy when you go into the project originally thinking it would be super simple.

Trying to not return any message to Slack became frustrating though, because the default API Gateway always transforms my response into a null which would then print out a ‘null’ in Slack.

Luckily I found help on medium on how to mangle up the response just enough through the API Gateway to prevent unwanted text responses in Slack.

Slack requires a response within 3000ms. however, my apI calls to Jira were taking more time, so I switched to using Slack’s delayed response messaging, but Slack still requires a response from the original request within 3000ms.

I attempted to kick off an apI request to Jira and then return a response to Slack. However, once you send off a response, your script dies immediately.

Lambda actually complained that other processes were still running (my request to Jira) and this clued me into the problem.

I dove a bit to better understand how Lambda actually runs a modified version of Node to support this behavior. This makes sense, because they need to spin up Node quickly (so you don’t get charged for bootup times) and they need to shut down Node as quickly as possible (again, so you don’t get charged for cleanup times). With this information, it makes sense why they have to peg down a version of Node (though I still wish they would upgrade to Node v4.2 ASAP).

I still haven’t solved this problem and my Jira API responses frequently take over 3000 ms. However, Jon had the brilliant idea of spliting the script into two Lambda functions and in the first one, kicking off the second Lambda response, thus allowing me to return a response quickly, while the other Lambda function makes the Jira call and sends a delayed response to Slack.

In the end, I really enjoyed playing around with Lambda.

I am very excited by the whole notion of simplifying what it takes to build an application down to the core of what I love to do (just writing code) and abstracting away what is farther removed from what I’m building (preparing and maintaining hardware, services and systems).

Lambda is definitely the future.

But for tomorrow, since we are going to be writing more bots for Coursera, I think we’ll move on to hosting a simple EC2 machine and wire each bot to an Express router. For now, I am looking forward to writing ES6 and using oauth without hassle on the next version of buggy!f

Originally published at building.coursera.org on February 17, 2016.