API-Gateway ❤️ CloudWatch
Not a match made in heaven, but a working relationship
API Gateway — The workhorse 🐴
Ever since API-Gateway was released by AWS back in 2015 people got surprised by their AWS bill: It was way more expensive to go server-less than expected.
Mostly this can be rooted to one point: API-Gateway is not intended as proxy or Load Balancer: If none of its features are used, it is a very expensive Load Balancer — with less features.
The API-Gateway shines when it comes to more advanced features; and one of those is to call other AWS APIs without the use of a Lambda or an SDK in between:
With API-Gateway one can build custom REST API via
- Its resources, methods, and its models to provide a stable API
- Combining the request & response mapping the Apache Velocity templates to expose just the right amount of an AWS API
- While providing security via IAM and such
- And use caching & API-keys
without the hassle of scaling backend logic just to expose the actual functionality via another AWS service
CloudWatch-The old accountant 👴
While CloudWatch is decent at what it does (well it’s not DataDog and not Splunk) it’s also old: It was first released in 2009 and the API definitely feels that way.
While the API was overhauled in 2010 it still is from a time when XML was considered a good data format & Internet Explorer 8 was released 😭. And it shows:
The CloudWatch API is built around Query parameters and the data format is XML (or JSON if kindly asked).
The goal here is to expose a custom metric from CloudWatch via the API-Gateway without any Lambda & SDK. There are three main hurdles to overcome:
- API-Gateway is really built around JSON & REST APIs— Query Parameters will need special handling
- CloudWatch expects timestamps in ISO8601 but API-Gateway’s Velocity does not provide access to DateTool
- CloudWatch has a very unique way to express lists. Once this is clear things become way smoother
Another ‘hurdle’ are the CloudWatch error messages if something errors out: These error messages are less helpful to say the least.
It’s all about the headers
The first solution to the puzzle is to sent the right headers to the CloudWatch API from the API-Gateway. As the API-Gateway is built around JSON so we have disguise this origin:
The API-Gateway integration must sent the following static headers:
Accept = "application/json"
Content-Type = "application/x-www-form-urlencoded"
This allow the CloudWatch API to start accepting requests from the API-Gateway and respond with JSON payloads
URL query parameters for the win
As the CloudWatch API uses query parameters for each and everything, here is the request mapping (yes for
application/json— or whatever our API-Gateway should expose to the client).
This is the most basic API request:
and there are some things to notice:
- There cannot be any newlines in the mapping
- The CloudWatch API state that it respects the
Operation) but this always end in an error. Therefore the API-Gateway action integration is just set
For a realistic request and to fulfill the stated requirements from the beginning the array notation from the CloudWatch API must be used.
It follows the notation
$NAME.member.1 for a string array. For complex objects it is
$NAME.member.1.$PROPERTY — while the numeric counter must start by 1!
So putting it all together adding parameters from the API-Gateway resource path:
Many things are happening here, so let’s break it down:
- The version is set to
2010-08-01to use the most ‘recent’ API schema
- The metric to retrieve is in the namespace
customNamespace/custom— and URL encoded
Statisticsmust always provided in array format. Only
Maximumis selected but the array notation must be used nevertheless.
metricNameis a path parameter derived from the API-Gateway resource. It is available via the
- Same goes for the
endTime(here a query parameter) from the caller. The format must be in ISO8601 AND URL encoded by the caller already (which is already the case as it is a query parameter for the API-Gateway itself)!
- Period is set to static
- Finally a
Dimensionsfor the custom metric is selected. To do so, the dimension (
userIdhere) must be selected.
The filter value is provided via
Dimensions.member.1.Value: We use the caller’s Cognito identity. Here we let the API-Gateway perform the URL encoding.