Serverless: Creating a Webhook Consumer in 5 minutes

Interacting with third-party APIs can present challenges when you want to keep your internal data up to date with changes. Many providers offer list endpoints that you can call periodically and update data. However these scheduled tasks can take a long time with a large amount of data. You have to write code to compare changes, and most providers have rate-limiting which makes the task challenging.

Webhooks solve this problem by pushing data to your app when events happen. Recently I was testing and debugging our Ambassador API webhooks (using services like requestb.in and hookbin.com) to create temporary endpoints that accept POST data and display the request, response body and headers. This worked great!

However, Ambassador’s webhooks have 🔒 security features that require more advanced testing. All I wanted to do was set up a simple API endpoint to test encryption. I didn’t want to introduce throwaway code or complexity to our app’s url scheme, nor did I want to go through the process of shipping code into our development or production app. This seemed like a great opportunity to go “serverless” and create a separate microservice to consume the data coming from the webhook.

All I wanted to do was set up a simple API endpoint

Creating an API Endpoint in 5 minutes

I was able to set up an Amazon API endpoint and a lambda function in less than 5 minutes. I immediately added code and quickly deployed changes to speed up the testing and debugging process. Awesome!

What you’ll need:

• node package manager (npm) https://nodejs.org/en/download/
• AWS Account https://aws.amazon.com/ and your access keys

Steps:

1. Install serverless CLI
2. Add a few environment variables
3. Create template from node.js sample project
4. Add the encryption code
5. Deploy!

Let’s break it down:

The Technical Details

Using the serverless CLI, I created a template, made some code changes, and deployed to AWS.

1. Install serverless CLI:

npm install serverless -g

2. Add environment variables:

export AWS_ACCESS_KEY_ID=<your-key-here>
export AWS_SECRET_ACCESS_KEY=<your-secret-key-here>

3. Create template from node.js sample project

serverless install -u https://github.com/serverless/examples/tree/master/aws-node-simple-http-endpoint -n test-webhook-endpoint

“-u” is the template, “-n” is the project name / directory where you want to install.

This will create the necessary directory and configuration files, as well as the function handler.js. This is the real time saver. Now all I need to do is add any custom logic I might need.

4. Add the encryption code

Note: The token used for the webhook is stored in serverless.yml file. This will add an environment variable to reference. In a production environment, it will need to be secure. For this example I used “SAMPLE_WEBHOOK_TOKEN”.

service: test-webhook-endpoint
frameworkVersion: ">=1.1.0 <2.0.0"
provider:
  name: aws
  runtime: nodejs4.3
functions:
  webhookConsumer:
    handler: handler.endpoint
    environment:
      WEBHOOK_TOKEN: "SAMPLE_WEBHOOK_TOKEN"
    events:
      - http:
          path: webhook
          method: post

The environment variable is referenced in the code with process.env.WEBHOOK_TOKEN. Below is the completed code which recreates the hmac security header for verification using the token and request body.

'use strict';
var crypto = require('crypto');
module.exports.endpoint = (event, context, callback) => {
var securityHeader = event.headers['X-MBSY-HMAC-SHA256'];
  const hmac = crypto.createHmac('sha256', process.env.WEBHOOK_TOKEN); 
  hmac.update(event.body);
  const validationToken = hmac.digest('base64');
if (validationToken == securityHeader){
    // perform any custom logic with event.body here
    // for sake of this post, we'll just log the body.
    console.log(event.body);
    const response = {
      statusCode: 200
    };
    callback(null, response);
  }
  else{
    const response = {
      statusCode: 401,
      body: JSON.stringify({
        error: "Unauthorized request."
      }),
    };
    callback(null, response);
  }
};

5. Deploy!

serverless deploy

Testing

Now I can test by POSTing to the endpoint using curl with the test data below. Replace the url with your AWS endpoint.

curl -X POST \
  https://XXXXXXX.execute-api.us-east-1.amazonaws.com/dev/webhook \
  -H 'cache-control: no-cache' \
  -H 'content-type: application/json' \
  -H 'postman-token: baa4b8da-8ba2-1482-db46-cf3fdec65632' \
  -H 'X-MBSY-HMAC-SHA256: BlcnaPSd2rzLlNKG1qCK2QQt9Vp8PB91uGnWIfjHYjI=' \
  -d '{"created_at": "2017-07-16T21:45:17.458000", "event_data": {"campaign": {"description": "Invite your friends to join us.", "id": 12345, "name": "Test Click Campaign"}, "click": {"created_at": "2017-07-16T21:45:17.458000", "ip": "127.0.0.1", "referring_url": null, "social_name": null}, "contact": {"email": "test@example.com", "first_name": "Test", "id": 1234567, "internal_id": null, "last_name": "Contact"}, "url": {"short_code": "dcBb"}}, "event_id": 12345, "event_type": "click-tracked", "webhook_version": "0.2"}' \
  -w "%{http_code}\n"

Now I can check the output of our code by using Amazon CloudWatch (already set up for me 🙌). After making the request, the webhook consumer endpoint returned a 200 request and logged the data. 💥

Summary

Using webhooks is an efficient way to get data from a third-party service as it is updated. This will save you the trouble of writing a scheduler to hit the API and processing changes over time. Shortly after an event happens, your service will get a request and can process the relevant data immediately.

Serverless is a surprisingly simple way to manage your lambda functions and takes out the tedious task of configuring AWS settings and permissions. Lambda with API Gateway also allows you to scale up easily if needed, so you could use this in a production environment.

Code: https://github.com/forgingdestiny/node-webhook-consumer-serverless
Read more about serverless: https://serverless.com/
Read more about the Ambassador API: http://docs.getambassador.com/

P.S. Join us at the Embassy, we’re hiring!