GitHub Board Slack Notifications

Part 2/3: “How I learned to spell Lambda”

Before continuing see Part 1 and the TL;DR there!

Recap

In the last part we conclude the research part of the initiative and made a decision: We’re building a lambda function. This function receives a GitHub webhook and forwards it to Slack.

My story

I’ve never created, deployed or maintained a lambda function on AWS (or anywhere else) ever. The following is my process and learnings.

🕵🏻‍♂️ First let’s do some research…

Found this bad boy: https://github.com/kentaro-m/lambda-pr-notify-bot

Looking pretty darn close to what I want

Here is their diagram of how it works.

Basically exactly what i need; minus the number three (3. Get Pull Request) arrow.

And it’s written in JS! 🎉

Alright! Let’s setup for AWS Lambda!

AWS Lambda

Incidentally we use AWS at IDAGIO but I didn’t want to start bothering our DevOps team with my learning experience so I opted to set it up for myself and then ask for help when we’ll do the actual deployment.

You need a phone number but then on the (never ending) Free Tier you 1M free requests per month and 400,000 GB-seconds of compute time per month.

The (Original) Game Plan

So now I’m staring at the AWS Lambda console trying to figure out what to do. Not sure how to execute it, but a plan I had. 🤔

1. get the code that extracts the info from the github event payload working in the AWS Lambda “IDE” (Cloud 9) using their test button
2. get the endpoint exposed publicly
3. train a Github Org webhook on it
4. check the AWS endpoint logs to see if it worked
5. target the slack POST endpoint
6. customize the events i’m responding to and the response.

⚠️ When I tried to use the code from the lambda-pr-notify-bot repo I had found I realized that I can’t `import`and i should be old school `require`ing. Otherwise i would need to setup babel in the AWS Console. See here

I can test inside the IDE but I need a sample of what the GitHub webhook sends. Found a fixture for my test: https://raw.githubusercontent.com/kentaro-m/lambda-pr-notify-bot/master/test/fixtures/mention_review.json

^ This is good sign!

I managed to run my script using their test function but to get it exposed I would need to understand another AWS Service: API Gateway. It has a testing utility but it requires some deeper knowledge of AWS Security and at this point I was losing steam…

Is it security? Where da logs?

I’m losing patience and I’m super confused

Let’s see if anyone has done this before… ⏱

YES! Found this amazing tutorial using serverless 🎉

What is serverless? Serverless is amazing (esp for this kind of thing)

Develop, deploy, troubleshoot and secure your serverless applications with radically less overhead and cost by using the Serverless Framework. The Serverless Framework consists of an open source CLI and a hosted dashboard. Together, they provide you with full serverless application lifecycle management.

Serverless, my hero! 🦸🏻‍♂️

If you follow the instructions on the tutorial everything will work great, the APIs are a little old but it will get you up and running. In no time I had a GitHub message in my slack!

Needs some polish and my team-mates are amused but this is serious progress.

One caveat on my setup was that i needed to setup a separate profile for the AWS login i was using for myself and my work one. For this you will need to use the AWS_PROFILE env variable, like so.

$ AWS_PROFILE=pers sls deploy

Since i had a working base, it was time to add a little bit of flare to it.

☀️ Also I could see logs, including responses and console.log statements by running

$ AWS_PROFILE=pers sls logs -f stargazer

As I was feeling confident I made a new plan

1. secure the endpoint
2. find a way to test locally with different GitHub event payloads
3. customize the Slack message to fit my needs
4. productionize the bot

Secure the endpoint 🔒

The tutorial also guided me through setting up the org-level web-hook. It however doesn’t include any security aspect.

The webhook configuration gives you a way to setup a SECRET

From the docs

Setting a webhook secret allows you to ensure that POST requests sent to the payload URL are from GitHub. When you set a secret, you’ll receive the X-Hub-Signature and X-Hub-Signature-256 headers in the webhook POST request. For more information on how to use a secret with a signature header to secure your webhook payloads, see “Securing your webhooks.

For this I brought in octokit and webhooks.

npm install --save @octokit/webhooks

And boom! Endpoint secured!

Test locally 🧪

Serverless comes in with a handy tool called invoke and it’s friend local . With these you can actually invoke your endpoint with a .json file and see how it behaves. Super handy!

You can see the payloads in github under your webhook but they will be different when coming into your event handler. The event handler actually contains both the body of the event and the headers.

$ sls invoke local -f validate -p ./fixtures/move.json

I could run my bot against these fixtures and see the results in slack

Customize the Slack messages 👩‍🎨

At this point I started thinking what I wanted my messages to look like. And what information I would like to present to my users.

However I quickly realized I needed to make more requests to GitHub to extract information about the ticket being moved and the column it was being moved from. I needed arrow number 3 after all.

Undefined is not your friend

So I brought in more of octokit into my project

npm install --save @octokit/request

In this case you will require a GitHub Token for your bot to talk to the API. I’ve created a personal Token and added it into the env vars in serverless.yml

# serverless.yml
environment:
  - GITHUB_TOKEN: “<YOUR GITHUB TOKEN>”

⚠️ The project board are still an experimental API so configuring your defaults like above will not be enough to work with GitHub Project Boards.

You need to enable experimental API use:

Now I can get information about issues (like title and url). A few examples:

I also figured out how to differentiate between notes and issues using content_url param on the payload.

Notice one is called “Note” and one “Issue”

Time to learn more about what Slack notifications are made of! 📚

Turns out Slack has quite a cool and complex system for making notifications, called Block Kit, which includes buttons and time pickers and some layout options. They also have an online builder. Super nifty!

I went with a minimalistic approach of using a markdown text followed by a context section (with small avatars) and a actions section with two buttons: `See Issue` (or Note) and `See Board`.

Productionize and make public 💫

Leftovers🥢

One thing in my TODO bucket still is a way of specifying a project to channel relationship for the bot.

Git Repo

Move it into a git repo, make sure no secrets are leaked. For this I opted to add the serverless.yml file into .gitignore and include a sample one with the repo. Update the readme, make sure to cleanup fixtures as they might leak information.

Productionize

Move over to IDAGIO Infra. Start collecting feedback!

End of Part 2

Continued in Part 3: “So close, yet so far”

Lessons from Part 2:

1. AWS Lambda has a free tier that should be suitable for small applications
2. Serverless exists and it’s awesome
3. When working with GitHub webhooks you will probably eventually need to get data from GitHub and you will need a token for that
4. BlockKit and most of Slacks documentation is great
5. Sometimes you think it’s a small leftover but when you look closer… Part 3

Thanks for reading! 🙇‍♂️