Error Handling in AWS Lambda and API Gateway

The Lambda and API Gateway offerings from AWS have provided a powerful new mechanism for quickly developing REST APIs without the overhead of creating infrastructure and boilerplate code to spin up web servers. Once you realize SAM is a mess and move to Serverless things really start to fly. Inevitably you will reach the point where this proof of concept needs to start to become more real. That means adding functionality such as authentication and error handling/reporting. After scouring the web and finding the results unsatisfying, I decided to publish my experience here in the hope that others may benefit.

Goal: Provide appropriate status codes and messaging to callers of the REST API such as the Angular UI while providing minimal intrusion of HTTP status codes into the Lambda code itself.

Round 1: Default Serverless Error Handling with Lambda Integration

Serverless does a great job at creating reasonable defaults and removing boilerplate so that the developer (or DevOps engineer) can focus on “more important things”. It only makes sense that the first thing to try would be what Serverless provides out of the box.

Using Lambda Integration, we created a simple endpoint in our serverless.yml:

createDeal:
  handler: create-deal/index.handler
  events:
    - http:
        path: create-deal
        integration: lambda
        method: post
        cors: true

When we deploy our stack to AWS, we see the following in API Gateway:

Serverless has created a number of default regular expressions that map to error codes. This is great so long as the errors that are thrown from our code include the appropriate status code in the error text string. Let’s look at an example:

exports.handler = async (event) => {
    if (!event.dealName) {
        throw new Error('[400] Missing required property');
    }

    return createDeal(event); // returns created deal object
};

Easy, right?

Maybe not. What happens if we get an error back from createDeal() that does not conform to this standard? We can’t rely on all called methods and libraries to know that error strings need to contain an HTTP status code. Sure, we can perform a try/catch, but that still places the logic of determining the HTTP status code in our endpoint code. Additionally, any consumer of the response will need to either use or parse out the status code from the error message. Lastly, if an error is thrown without one of the mapped status codes then the API Gateway will return a 504 Bad Gateway response, which is of little value to the caller. We can do better.

Round 2: Lambda Proxy Integration

A common recommendation when creating new Lambdas is to use the “proxy integration”. Serverless-stack has a great writeup on how to do this with Serverless so I will try not to be repetitive. In essence, the developer is responsible for setting the HTTP status code when creating a response and responsible for catching any errors in the code so that an appropriate response can be created. Let’s take a look at the examples based on the serverless-stack writeup:

createDeal:
  handler: create-deal/index.handler
  events:
    - http:
        path: create-deal
        method: post
        cors: true

Note that instead of using the lambda integration we are using the default (proxy) integration.

exports.handler = async (event) => {
    const dealInfo = JSON.parse(event.body);
    if (!dealInfo.dealName) {
        return {
            statusCode: 400,
            headers: {
                'Content-Type': 'application/json', 
                'Access-Control-Allow-Origin': '*'
            },
            body: {
                message: 'Missing required property'
            }
        }
    }
    const newDeal = await createDeal(dealInfo);
    return {
        statusCode: 200,
        headers: {
           'Content-Type': 'application/json', 
           'Access-Control-Allow-Origin': '*'
        },
        body: JSON.stringify(newDeal)
    }
};

This approach has its pros and cons. Like the previous solution the developer is responsible for catching all errors and returning the appropriate status code and message. Also like the previous solution we will get a 504 Bad Gateway if the response is not in the correct format such as when an uncaught error is thrown.

The serverless-stack solution includes a utility function to simplify the response return buildResponse(statusCode, body) . This simplifies some of the common code around the response such as header information, but does not address the concerns of uncaught errors and status code mappings in the lambda function.

Round 3: Middleware with Middy

A popular approach to abstract a lot of the boilerplate is to use a middleware solution such as middy. Middy wraps the lambda call and allows the developer to attach any number of useful pre and post-execution code that can be used to modify the request or response. This does not address the issue of declaring HTTP status codes in the lambda function, but it does provide some powerful capabilities for integrating with other common libraries like http-errors. Unfortunately the net gain for this use case is no better than the more lightweight solutions above. What we end up with is a similar result with more dependencies and additional code:

createDeal:
  handler: create-deal/index.handler
  events:
    - http:
        path: create-deal
        method: post
        cors: true

(using the same proxy integration)

const middy = require('middy');
const {httpErrorHandler, cors} = require('middy/middlewares');
const createError = require('http-errors');
const {autoProxyResponse} = require('middy-autoproxyresponse');
// wrap lambda handler in middleware
const handler = middy(async (event) => {
    const dealInfo = JSON.parse(event.body);
    if (!dealInfo.dealName) {
        throw new createError.BadRequest({message: 'Missing required property'});
    }
    return createDeal(dealInfo);
});
// declare middleware to use
handler
    .use(autoProxyResponse())
    .use(httpErrorHandler())
    .use(cors());
// export the handler
module.exports = {handler};

The CORS middleware is a welcome addition as it allows us to skip manual entry of the ‘Access-Control-Allow-Origin’ header, but we need to add the third-party middleware auto-proxy-response in order to wrap the returned value in a valid lambda response format (with status code 200) or manually create the response object ourselves.

Round 4 (Final): Mapping error codes in API Gateway

After much deliberation over whether to use the proxy response (as seems to be recommended by both Serverless and AWS), I came to a conclusion that I’ve seen repeated across the web: it makes short-term development quicker but does not provide the separation I’d expect of the HTTP response from the lambda code. This separation can be used to simplify the developers’ lives making it harder to shoot themselves in the foot by failing to catch an error while also reducing boilerplate to make the code more readable as a unit of work. What this does add is extra Serverless configuration in the form of response templates and error code regex matching. Let’s touch on each of these points.

Lambda code:

exports.handler = async (event) => {
    if (!event.dealName) {
        throw new Error('Missing required property');
    }

    return createDeal(event);
};

We return to a simple lambda code in a world where HTTP status codes are blissfully unknown. If createDeal() throws an error that is not caught and rethrown we will not break the gateway handling.

serverless.yml:

createDeal:
  handler: create-deal/index.handler
  events:
    - http:
        path: create-deal
        method: post
        cors: true
  response:
    headers:
      Content-Type: "'application/json'"
    statusCodes:
      200:
        pattern: ''
      400:
        pattern: '^Missing.*'
        template: ${file(resources/error-response-template.yml)}
      500:
        pattern: '^(?!Missing).*'
        template: ${file(resources/500-response-template.yml)}

error-response-template.yml:

'{
  "message": $input.json("$.errorMessage")
}'

500-response-template.yml:

'{
  "message": "Internal Server Error"
}'

All of the HTTP status code handling has been abstracted into the API Gateway configuration. API Gateway will apply the regex pattern to the errorMessage field from the response to determine which status code and response template to use (see documentation). In this case I have configured 3 possible status codes:

• 200 — if there is no error message
• 400 — if the error message starts with the string “Missing”
• 500 — if the error message starts with any other string (catch-all)

These will work in addition to the existing 401 and 403 status codes that the gateway will return when authentication or authorization fails using my authorizer (different topic).

I’ve additionally created standard response templates that can be reused across endpoints, thus reducing copy and paste and standardizing on a return format.

All in all I believe this provides the simplest developer experience while maximizing consistent behavior and response format. The developer needn’t worry about HTTP status codes while writing lambda functions, and the possible status codes that could be returned are documented in a single location rather than throughout the code. There is some cost and complexity around performing regex matching on the error message, but for this use case that is an acceptable tradeoff.

I hope you’ve found this explanation helpful!