Spot-on error management with AWS Lambda, API Gateway, CloudWatch and Serverless

Thomas Aribart
May 18 · 6 min read
Computer connected to server (serverless) with bug breaking the connection.
Computer connected to server (serverless) with bug breaking the connection.

Strange as it is, I believe having bugs is healthy. Because too often, an app that has no bug is an app that has no monitoring system set up to detect them in the first place.

I’ve recently discovered the serverless ecosystem. To be more precise, I worked on a project that used the Serverless framework to deploy NodeJS Lambda functions, connected to, among others, API Gateway and CloudWatch.

After two months of development, we realized that we needed an efficient way to throw and detect Lambda execution errors. We also realized that it was not that trivial: It took us a few weeks to get a reliable system in place.

This article shares some of the learnings we made on the way. In particular, how to:

  • 🏓 Throw and harness errors throughout your Serverless app
  • 🚦 Return relevant HTTP responses, with customizable status codes and payloads
  • 🎯 Monitor your execution errors in CloudWatch
  • 🔧 All that with minimum boilerplate and maximum reusability

Sounds interesting? Let’s dive in!

#1 Throw and harness errors

Here, we will add two properties to it:

  • statusCode: A related HTTP status code that can be parsed by API Gateway if needed.
  • key: An string identifier that adds context to the status code and can be used for front-end translation purposes.

Now, we could harness those errors by wrapping our Lambda functions with a try/catch, but let’s use middy.js instead: If you’re not familiar with it (and you definitely should!), Middy uses the middleware design pattern to allow a better segregation of business logic from configuration such as JSON validation, logging, or in this case, errors handling.

Neat! So what does our handleError middleware looks like? In the case of a synchronous lambda, it depends on how it is integrated to API Gateway.

#2 Proxy integration

Lambda proxy integration
Lambda proxy integration
You can select Proxy integration in the API Gateway console

In a proxy integration, API Gateway passes the raw request as-is (headers, query string parameters, URL path variables, payload, authorization context etc…) to the integrated Lambda function, and expects a similarly integrated response from its return results. Meaning that any status code will be correctly interpreted by API Gateway to build the request response.

A successful proxy-integrated lambda…
Image for post
Image for post
…and its results in Postman 📮

With that in mind, we can make full use of our custom errors with a proxy HTTP response building middleware:

Image for post
Image for post
It worked 💪

However, in this configuration, CloudWatch considers that the Lambda function was executed successfully, and the thrown error is not captured.

Image for post
Image for post
🤔 Where’s the error ?

Alternatively, if we throw the error in our middleware (or, equivalently, pass the error either as the first argument to the callback), the error is captured, but API Gateway does not interpret it and always returns a 502.

Image for post
Image for post
There it is! 🙌
Image for post
Image for post
But we lost our payload 😭

The only way to have custom errors both captured by CloudWatch and interpreted correctly by API Gateway is to move from proxy to lambda integration.

We can do that by adding integration: lambda to our Lambda functions configuration. This had small implications on the shape of our Lambda functions events and responses, but nothing unmanageable*: You will find a detailed implementation on the GitHub repository that I made along with this article.

*Largely thanks to the default integration mapping templates set up by Serverless

#3 Lambda integration

The good news is that Serverless sets up default method and integration responses that do most of the job, so no additional boilerplate is needed! Check the Serverless documentation for more details. Long story short: Throwing any string containing [400] will actually trigger a 400.

Image for post
Image for post
Serverless default integration responses

We can use this to our advantage by reworking our formatting middleware:

And, sure enough, there it is! The best of both worlds:

Image for post
Image for post
Image for post
Image for post
Awesome 💪

Note that this only worked for the status codes set up by default by Serverless. So we should add a check in our custom error constructor:

So meta 🤯

#4 Handling non-custom errors

We can improve this by detecting and parsing incoming errors, and using 500s only as a last resort. For instance, Axios errors have a handy isAxiosError property:

We can create such parsers for each type of error we can encounter, and give them to a parseError function:

Finally, we can use this function in our formatErrorForApiGateway middleware:

Image for post
Image for post

And voilà 👍 Thanks to Serverless and middy.js, we set up a neat error management across our synchronous lambdas and API Gateway with minimum boilerplate!

#5 Using mapping templates

Image for post
Image for post
Look at that body…

The error key is already troublesome to parse, any customization would be hard to implement (what if you want to pass a payload ?), and forget about using any other status code than the eight ones provided by Serverless!

But fear not, there is a way ! Called mapping templates.

Mapping templates are “recipes” that API Gateway use to reshape responses after standard mappings have been applied, giving control on the responses bodies, headers and status codes. Different templates can be triggered depending on the content of the Lambda function return values, through matching patterns.

Let’s use them to replace the body of our error response body by the value in the error message:

Now, let’s shape the error body the way we want:

And voilà! Nice and clean:

Image for post
Image for post

Last but not least, we can extract the response configuration in a separate file to avoid having to apply the same mapping templates to each Lambda function:

Conclusion

  • Activating proxy integration for your Lambda functions requires less configuration, but hides execution errors from CloudWatch
  • To solve this issue, Lambda integration with default Serverless integration responses is a good way to go
  • Mapping templates will give you maximum freedom on the shape of your HTTP responses, at the cost of some additional boilerplate

Feel free to check out the GitHub repository that I made along with this article: You will find detailed implementations of each solution, as well as correct Typescript types and a bonus sendMessage middleware which sends an EventBridge event when an error occurs 👍 It’s ready to deploy if you want to try it for yourself!

Thanks for reading 🙏 I hope that you learned something from this article and that it will save you some time on your next Serverless project!

Serverless Transformation

Tools, techniques, and case studies of using serverless to…

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store