API Gateway + Lambda Compression

Configuring gzipped JSON responses with API Gateway and Lambda

In order to return gzipped application/json from API Gateway, there are some quirks/limitations to consider (described at the end of this post). This is because there’s no easy way to enable automatic compression of responses in API Gateway. However, there is binary data support which means we can configure a lambda to return compressed responses.

I. Setup Your Lambda

Your lambda will be responsible for zipping the responses. Here is a simple Python 3.6 lambda. The important bit here is the output format and the headers. This is because this lambda will be configured as a proxy integration.

• You must specify 'isBase64Encoded': True when returning gzipped responses and False when returning uncompressed data.
• To enable CORS, you must specify the Access-Control-Allow-Origin in the response.
• When you set up your lambda integration request in API Gateway, make sure you check the “Use Lambda Proxy integration” option.

Python 3.6 Runtime

II. Configure Binary Media Types

Enable binary support for your API by adding the media type to be treated as binary.

This is done by adding application/json as a binary media type in the “Binary Support” tab.

Caution: this can break existing OPTIONS methods on your resources. This will be fixed in the next step.

III. Configure OPTIONS Method

By adding application/json to be handled as a binary media type, this may cause some OPTIONS methods to fail with an “Internal server error”. This is most likely caused by API Gateway handling of empty responses and converting to binary.

To fix this, we’ll have to modify the content handling for any failing OPTIONS methods. Unfortunately, there is currently no way to configure content handling for MOCK integrations through the web Console. You can use the following CLI command:

aws apigateway update-integration \
    --rest-api-id <api-id> \
    --resource-id <resource-id> \
    --http-method OPTIONS \
    --patch-operations op='replace',path='/contentHandling',value='CONVERT_TO_TEXT'

The CONVERT_TO_TEXT content handling value forces API Gateway to treat it as text. More details can be found in the content type conversions docs.

IV. Deploy!

Once you’ve deployed your API, you can hit your resource and see your JSON response!

curl https://<id>.execute-api.<region>.amazonaws.com/<stage>/<res> \
    --compress \
    -H 'Accept: application/json' \
    -H 'Accept-Encoding: gzip'

Note that you must specify the Accept header. Without this header, API Gateway will not convert the lambda response into binary, causing the request to fail.

Some Things to Consider

• You must use Proxy Integration in API Gateway for your lambda.
• Using Proxy Integration means that you cannot leverage body mapping templates or transform responses.
• Lambda must handle returning uncompressed and compressed responses depending on the request accept encoding.
• You need to ensure that your existing resources and methods do not have any conflicts with application/json being set as a binary media type. The OPTIONS method is just one example to be aware of.

This is due to the fact that as of October 2017, there is no way to easily configure API Gateway to compress uncompressed data passing through API Gateway. This means that your lambdas must zip and base64 their responses. BobK@AWS writes:

What is not currently supported is:

1. Compressing uncompressed data passing through API Gateway (e.g. compressing data before sending to Lambda or your HTTP endpoint).
2. Using mapping templates with compressed data (decompressing/modifying/recompressing).