Learn AWS API Gateway with the Slack Police

There’s a second edition of this story that solves the same problem in a significantly different way. In the context of learning API Gateway, you should read and understand both approaches.

For an even more in depth look at API Gateway, please see this article written by Alex DeBrie from April 2019.

I bet you’re familiar with those times where someone asks a question in #general and it leads down a rabbit hole of lunch decisions or questions about subdomains. You’re sitting there thinking, “isn’t there a #nerds channel for this kind of stuff?” The answer, of course, is yes, but you don’t want to be the one to put your boss on blast.

It’d be nice if there were some sort of…Slack Police. You could just call the Slack Police and they’d get the riffraff under control.

Not everyone likes cheese on their burgers.

There are many ways to make this happen. Here’s one, which may be educational if you’re interested in AWS’ API Gateway or Lambda functions.

Slack Integrations

The Slack Police is really just a slash command that any user can trigger. The command takes an optional bit of input, and then sends a message back to the channel where the command was triggered.

The first thing to do is setup a new slash command integration for your Slack team.

Set the Command to whatever you’d like, and just put a placeholder in the URL for now. Everything else you can leave at the default, though you may want to specify the Autocomplete help text or Descriptive Label. Also make a note of the Token, since we’ll need it a little later.

Slash commands work by sending a request (in this case a POST request) to a given URL. If the URL returns any sort of response, that response body will get sent back to the user who triggered the command as a Slack message. The message is sent only to the user who triggered the command, and it comes from slackbot, not from the Slack Police.

We want a message to get sent to the entire channel, so that’s actually not going to work for us. In fact, we don’t want the user who triggers the command to get any sort of message from slackbot (just the message from Slack Police, like everyone else), so we’ll have to make sure that the URL being hit doesn’t send back any sort of response body when the command is triggered. (Spoiler alert: it would be nice to send back an error message to the triggering user is the command fails, so in some cases we will actually have the slash command URL respond with some data.)

NOTE: Since this originally written, slash commands have been updated with additional functionality for handling the response from the URL. It is now the case that a JSON payload can be returned, which can indicate if the response should be treated as “in channel” or “ephemeral”. An “ephemeral” response works like a standard response, and is only visible to the user who triggered the command. “In channel” responses are visible to the entire channel, which is what we’re trying to achieve, but the command itself (and the user that triggered it) is also visible to the entire channel. Since Slack Police is meant to be anonymous, that isn’t particularly useful in this case, and the steps below for using an Incoming Webhook to get messages back into Slack are still necessary. But in cases where anonymity isn’t required, “in channel” slash command responses make it even easier build slash commands with Lambda.

The easiest way to get a message back into the channel is with an Incoming Webhook. The Webhook gives us a URL that some JSON can get sent to, and it will get posted to a channel as a message. If we have the server that’s receiving the request from the slash command send the Webhook, we get the effect we’re looking for.

In most cases the response will be blank, but if the command or webhook fail we can send back an error message that only the user who triggered the command will see.

Once again, start by creating a new integration for the Incoming Webhook. What you choose for Post to Channel doesn’t really matter. We want the message to get sent back to whichever channel the command came from, and that will get set when the Incoming Webhook gets sent. Setting it to @slackbot is a good failsafe. The Descriptive Label, Customize Name, and Customize Icon can be whatever you’d like. You’ll need the Webhook URL in a few minutes, so keep it handy.

That’s nearly all the work that needs to be done in Slack. The rest of the job is creating an endpoint somewhere that the slash command can hit, and that can use the Webhook to send messages back. Let’s use API Gateway and Lambda to build that out.

API Gateway

API Gateway lets you define HTTP endpoints that are backed by various services. Since the slash command is configured to POST requests to its URL, we will need a POST method in the API Gateway.

Nested Resources can help organize your API methods

Start by creating a new API if needed, and then a new Method. There’s no Lambda function to connect the method to yet. For now select Mock Integration, just so we can do some initial setup.

You’ll see a flow diagram that represents this API endpoint. In this case, the Client is the Slack, the data coming from Slack (through Method Request and Integration Request) to the Endpoint is the POST request that the slash command makes, and the data coming from the Endpoint (through Integration Response and Method Response) is the response that generally will be blank being sent back to Slack. The Incoming Webhook doesn’t appear on the diagram, because it will be handled entirely within the Lambda function.

There are a few reasons that this looks overly complicated, when really all we want is to funnel an HTTP request to a Lambda function.

For one thing, Lambda is always going to expect to receive data from an API Gateway as JSON, and there are cases where the data being sent to the API isn’t JSON. That’s the case with slash commands coming from Slack. Slash commands make POST requests with an application/x-www-form-urlencoded Content-Type, so the body of the request looks something like:

token=gIkuvaNzQIHg97ATvDxqgjtO&command=/weather&text=lorem

…which is decidedly not JSON. The Request steps of the API Gateway flow let us do things like reformat request data (headers, query parameters, body, etc) into something that will work with Lambda. API Gateway even allows for different formatting based on the content-type of the request.

Another thing that API Gateway’s complexity affords us is the ability to construct responses determined by the result of the Lambda function. Lambda itself is not aware of things like HTTP headers or status codes. API Gateway lets us specify exactly what kind of response the method returns based on what the Lambda function returned. We’ll see that come into play in just a minute.

The Method Request configuration is where authorization for the method can be controlled. It is also where you would list any HTTP Request Headers or URL Query String Parameters that needed caching or that would get used for transforming the data being sent to Lambda. Our slash command API method doesn’t need any authorization, has no query string parameters, and is indifferent to the request headers, so these settings can be left at their defaults.

The Integration Request step is more important to our API. This is the step that ties the actual request to the backing service. In our case, we know that we have a URL encoded request body, and Lambda is expecting JSON. API Gateway’s Mapping Templates provide a way to make that conversion happen.

In the Mapping Templates section of the Integration Request, we want to add a template for the Content-Type of the request we are expecting. In this case it’s application/x-www-form-urlencoded. Once the content-type is added to the list, there’s an option for Input Passthrough, which send the request body along to Lambda unchanged (and thus would have to be JSON already), or for Mapping template.

It’s worth noting that the request body is always available in the Integration Request, without any sort of configuration in the Method Request (unlike headers or query parameters). The default case API Gateway is design to handle is a request body that is JSON, so without any configuration it could get passed through to the Lambda function. Even if the request body is not JSON, the value remains available within the Integration Request.

Selecting Mapping template allows us to build up the JSON that will get passed to Lambda, using data from the request and what was configured in the Method Request step. API Gateway uses Velocity Template Language and JSONPath to construct JSON that conforms to your needs.

In the case of this slash command, we could try to use VLT and JSONPath to rip apart the URL encoded request body and rebuild it as a JSON object, but since Lambda functions are based on Node.js, and Node provides built in functionality for parsing URL encoded query strings, that would be a lot of unnecessary work. Instead, we can simply wrap the request body as-is in a JSON object, so that we can get the value to Lambda for parsing.

{
  "postBody" : $input.json("$")
}

The mapping template for this request ends up looking like a very simple JSON object, with a single key-value pair. The key is arbitrary, and the value just needs to be the request body as the slash command sent it. API Gateway provides the $input variable (as well as $context and $util) for working with properties of the request. The json() function evaluates a JSONPath expression, and returns the results as a JSON string. The $ (root object) expression returns the entire request body, which is what we need in this case.

Now when a POST request comes in to this API endpoint, the method will map the still-URL-encoded form data request body into JSON that looks something like:

{
  "postBody": "token=gIkuvaNzQIHg97ATvDxqgjtO&command=/weather..."
}

That is what will get passed to the Lambda function, giving us access to all the data the slash command sent when the user triggered it.

Note: Slack slash commands do allow you to switch to GET requests. While this would avoid the need to wrap the body in a JSON and then parse it in Lambda, it means that instead each query parameter needs to be mapped explicitly in the mapping template as separate key-value pairs, which ends up being more work. It is a viable option, though.

The API Gateway method configuration is also used to construct the response. We know that if any response body gets sent back to the slash command, it will be shown to the user as a message from slackbot, and generally we don’t want that. But we do want to notify the user if there was a problem while the Lambda function was running, like if the Webhook fails.

Remember, all that gets returned to API Gateway from the Lambda function after it’s run is a single value. There are no headers or status codes, so API Gateway provides ways of inspecting the return value, and conditionally building an HTTP response with the headers, status code, and body that we want.

The primary way of responding from the method conditionally is using the Lambda Error Regex property of integration responses. Looking at the Integration Response configuration, you’ll see the default integration response, which has an HTTP status code of 200 and no regex.

This is a good fit for times when the Lambda function succeeds, and we want the slash command response to be blank so that the user does not see a slash command error.

Within the default integration response, there is a default Mapping Template as well. The Content-Type is application/json, which is fine for this case, even though we know it won’t be returning any JSON (or anything). The slash command has no content-type preference, and also does not expect any specific headers, so it’s not necessary to define any Header Mappings.

The problem with the default Mapping Template is that it’s configured as Output passthrough. Even if the Lambda function returns no data when it’s successful (which will be the case), this template configuration will still cause the slash command to display a message to the user: null.

In order to prevent this, we’ll define a mapping template rather than use passthrough. The response still needs to be empty, though, so the template is essentially doing nothing. It just needs to be something that API Gateway will process and returns nothing.

Putting a bit of VTL into the mapping template which just sets a variable is enough to get what we want. With this template in place, when the method returns the 200 status integration response, Slack will behave properly (no null).

There’s still the matter of the error message, though. When there’s a problem in the Lambda function, it will return an error to API Gateway. Normally, when the function succeeds, it returns nothing. That allows us to define a very simple Lambda Error Regex to catch errors occurring in the Lambda.

.+

A regex that matches any string is enough to identify when the function returns an error. The regex can’t be added to the default status 200 integration response (or it would not match successful empty return values), so we’ll need to make a new integration response.

If you try to create a new integration response you’ll find that there are no available Method response status options. Response statuses only become available in the Integration Response step if they have a corresponding Response defined in the Method Response step.

So jump over to Method Response configuration, and add a Response for HTTP status code 400.

Once that’s done and you return to the Integration Response configuration, you’ll see that you’re able to add a new integration response for method response status 400.

Set the regex for this response, and now any time the Lambda function returns a value (which for us will only be if there’s an error), the method will return 400 rather than 200. If a mapping template isn’t added to the 400 response, the default passthrough behavior will send the entire stringified JSON error back to the user. Using a very simple mapping template, though, would allow for the error message that gets sent back to be a bit more user friendly.

The following VTL mapping template will grab just the errorMessage from the returned error along with a friendly message, and wrap them up as a JSON Slack message payload. (The full error with the stack trace will still get logged in CloudWatch to help with debugging.)

#set($inputRoot = $input.path('$'))
{
    "text": "Sorry, but there was a problem running your command: $inputRoot.errorMessage"
}

Note: The name and icon that show up in Slack along with the error message come from the slash command integration, not from the Incoming Webhook. It’s up to you if you want them to match the successful command responses, or appear to come from slackbot, or something else. Also be aware that Slack treats any non-200 response from a slash command as an error. If you do want to take full advantage of JSON payloads with “ephemeral” or “in channel” messages, they would need to come from a status 200 method response.

That mostly wraps up what needs to be done in API Gateway. Once the Lambda function is set up, the Integration Request will need to be switched from Mock Integration to the function.

AWS Lambda

The actual program behind the Slack Police is the Lambda function that takes the input from the slash command and sends the Incoming Webhook to get a message back into Slack.

Building Lambda functions is fairly straightforward, and there are many good resources.

You can find my version of the function in this gist.

Remember to add the slash command token and Webhook URL from the Slack integrations you setup above.

There are a few interesting bits to look at.

var params = querystring.parse(event.postBody);

Here is where the original URL encoded body from the POST request the slash command makes gets parsed into an object that we can use throughout the rest of the function. postBody is available on the event object because of the mapping template that was defined in the Integration Request.

if (params.token !== slashCommandToken) { 
  console.log('Invalid Slash Command token.'); 
  context.fail(new Error('Invalid Slash Command token.')); 
  return; 
}

The slash command sends the Slack integration token, so we can confirm that any request being made is from a trusted source (i.e. Slack). If the token were ever compromised, you would rotate it in the Slack integration settings and update the value in the Lambda function.

Notice that if the token is invalid, context.fail() is called with an Error. This (read: anything) is what the Integration Response regex is looking for when determining if the response is a 200 or 400.

if (params.text && params.text.indexOf(' ') != -1) { 
  webhookText = params.text; 
} else { 
  var intro = "Okay folks, let’s move this conversation"; 
  var to = (params.text ? (‘#’ + params.text).replace(/##/, '#') : 'elsewhere'); 
  var webhookText = [intro, ' ', to, '.'].join(''); 
}

This determines the message text sent with the Webhook. If the text value of the slash command includes any spaces, the value itself is used as the message. Otherwise it’s assumed that the value is a channel name, and an intimidating message is sent directing people to that channel (or “elsewhere” if no channel was provided).

var payloadForParams = function (params) {
    return {
        channel: ['#', params.channel_name].join(''),
        text: textForParams(params),
        link_names: 1,
        mrkdwn: true

Setting the channel on the webhook payload overrides the configured Post to Channel integration setting, which is why every Slack Police message isn’t sent to @slackbot, even though that is what was configured. The link_names option determines if channel or user names are parsed and clickable when displayed in Slack.

if (response.statusCode < 400) {
    context.succeed();
} else {
    context.fail(new Error(msg));

If the Webhook fails, context.fail() is again called with an Error. If it succeeds, context.succeed() is called without a return value, which prevents the status 400 regex from matching on successes.

Save the Lambda function code, and we’re almost done. Set the Integration Request to that Lambda function, deploy the API Gateway, and, lastly, set the URL on your slash command integration settings in Slack (from way up at the top) to the URL of the deployed API Gateway method.

You should now be able to issue a slash command that sends a request to an API Gateway method, which wraps the request body in JSON using a mapping template, and hands the JSON off to a Lambda function that parses the data so it can send a Webhook back to Slack, and also returns some value to API Gateway which determines if the value is an error or not, and responds to the slash command request appropriately, so that either the user is notified of the error, or the channel where the command was triggered posts a message from the Slack Police.

Two things. I realize now this probably should have been a video. Also, I find the documentation for API Gateway to be slightly non-obvious. The first few times I looked at it, it wasn’t always clear how different parts of method configurations worked together, or what default values or behaviors were. Hopefully this overwrought example makes it a little more obvious how different aspects of API Gateway would be used in a real-world application. Something like the CORS example shows some other parts of system, and I never even mentioned models, so this was by no means comprehensive.