Building a testable/continuously deployed API using Amazon API Gateway
There are a ton of advantages of using AWS to develop your API: a serverless infrastructure, scalability, rapid prototyping. But you can quite quickly get stuck in an untestable and hard to deploy mess.
You’ll find many good articles out there on how to get started with making a simple API — this isn’t one of them. Instead we’ll talk through how we overcame some of the challenges we faced when working on an API for the data warehouse at www.thetab.com
Moving Amazon API Gateway development away from the AWS UI
The ability to create APIs using “a fully managed service that makes it easy to create, publish, maintain, monitor, and secure APIs at any scale” is a magical thing. However, developing them through the UI has serious problems — how do you deploy changes, keep a full history and test your API? There is the concept of ‘stages’ in API Gateway, but it’s very limited.
And then there is simple but annoying stuff like this:
Swagger allows us to produce a YAML definition file for our API, that can then be edited, stored in git, and deployed using our normal deployment process. It will even produce the API documentation. It’s fiddly as f***, but it’s worth the pain.
You can export a swagger + API Gateway Extensions file from AWS. In our experience, this file was 95% correct but wouldn’t import using the API Gateway Import API but with a little massaging it worked (hint: cacheKeyParameters should be method.request.path.universityId or method.request.querystring.filterBy). Once we had this working using our deployment services (snap-ci.com) we then stopped making changes using the AWS UI.
Fully automated documentation
This is so simple, it feels like cheating. We copy the swagger file to a S3 bucket that contains swagger UI.
Although the API is not public facing the documentation allows developers to know what different methods do without having to ask each other. And if we’re honest, not having to speak to each other is the holy grail for nerds. 🙊
Unit testing the backend
The backend for the API is implemented in node.js and run using AWS Lambda. The unit tests were fiddly to get working but using aws-lambda-mock-context and sinon stubs we were able to test the backend.
A safety check using smoke tests
Despite the process above, it’s still possible to screw something up, so we implemented a runscope test suite that will get/post each API endpoint and test the response value. This gets called for both test and production.
Should anything be wrong, we’ll get a Slack notification.
Takeouts
Amazon API Gateway is a game changing service. At a tiny cost, we can build an API that can handle massive demand, has no service maintenance to worry about and is easy to get started with. There are some pain points around the development process though but hopefully some of the pointers in this post will save you some of the time we spent overcoming them.
Do give me your feedback or suggestions.
