Shenanigans of Serverless
Serverless computing with its massive scalability and low cost is the next big thing for the cloud. The devil however lies in the details, as I discovered while working on some of projects. This article focuses on Lambda, the flagship serverless offering by AWS. Lambda has intricacies and hard limits which can throw up real challenges unless some clever design principles are adopted.
1. 30 second API timeout
A popular use case of Lambda is for building APIs. Here Lambda is paired either with API gateway for REST or AppSync for GraphQL. While Lambda can have a maximum timeout of 15 minutes, both of these impose hard limits of 29 and 30 seconds respectively. Hard limits cannot be increased, period. Exceed the timeout period and your call fails with an error response.
To overcome this limitation one should allocate more memory to a Lambda or find a workaround. We could ditch the RESTful approach and use websockets provided by API gateway, but this could be an overkill for your use case. Alternately we can opt for an asynchronous pattern.
Solution- Asynchronous Lambda invocation
Here API gateway immediately returns a 200 response and asynchronously invokes the Lambda function.
This approach suits an invoke and forget approach. However a lot of complexity arises if the client seeks a definite response from Lambda:
- A second API call is needed to fetch the results, this time a synchronous one. This is done by skipping the ‘InvocationType’ header (explained ahead) and adding appropriate logic to your Lambda. Alternately a second Lambda could be used.
- A polling mechanism may be needed if execution time can’t be known in advance. This is because a second call faces the 29 second limitation.
- A persistent data store like DynamoDB will be needed because Lambda is stateless. You have no guarantee whether the same Lambda instance into which you passed data shall be invoked by your second API call. Further, Lambda instances are short lived and will take away your data on shutdown.
- A call tracking mechanism so that results are passed to the right user and to the right call. The results must also be delivered to the user in the right sequence.
A disadvantage of the above approach is that results can only be received sequentially. To overcome this, you may make the first call synchronous and pass a unique identifier to the user. The user polls for each result with help of this unique identifier.
Implementation
- In Integration Request
Under HTTP headers add the following
Name: X-Amz-Invocation-Type
Mapped from: method.request.header.InvocationType
Here the user must pass ‘InvocationType: Event’ as header for asynchronous invocation, otherwise this will by default will lead to synchronous invocation.
To make the endpoint asynchronous only set mapping as ‘Event’(in quotes) and skip the next step.
2. In Method Request
Add ‘InvocationType’ under HTTP Request headers. Basically user enters the ‘InvocationType’ header which is mapped to ‘X-Amz-Invocation-Type’. This tells API gateway whether the all is synchronous or asynchronous.
2. 10 MB data limit
Again, this is a hard limit imposed by API gateway. For files below 10MB we can enable binary inputs from settings.
Beyond 10MB we need to use a service like S3 for uploads. But this means we need more than one API calls. The approach depends on the service you want to develop:
- Your own website or backend: Use AWS SDK for uploading files to S3 and then call your APIs.
- API as a service: Here we need 3 API calls in total. Since we do not want end users to know our back end secrets, we make use of S3 signed URLs for upload.
Brief working
- Create an S3 bucket and set its CORS policy to allow POST or PUT requests.
<?xml version="1.0" encoding="UTF-8"?>
<CORSConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
<CORSRule>
<AllowedOrigin>*</AllowedOrigin>
<AllowedMethod>PUT</AllowedMethod>
<AllowedMethod>POST</AllowedMethod>
<AllowedHeader>*</AllowedHeader>
</CORSRule>
</CORSConfiguration>2. Create IAM role with S3 write permissions. Generate the access key and key secret. Basically we create a signed URL using this IAM role to give temporary S3 write access to the client..
3. Use the Keys in your Lamba function to generate signed URL
4. Return pre-signed URL to user on first API call. User uses it for uploading files.
5. User makes second API call. Perform processing on data and return response.
For more details please read Sam Quinn’s article on S3 uploads.
Returning large files
The 10MB limit also applies while sending data out of API gateway. Again we will use S3.
- Lambda uploads file to S3 and gets the item URL.
- Lamba returns the S3 URL as response.
- The trick here is to redirect your client to the S3 URL. For this we send the URL along with a 302 status code (aka redirect).
We need to take some security measures here. You may not want unauthorised users to access the file in case they get hold of the S3 item URL. This again adds a lot of complexity.
3. Lambda vs Lambda-proxy integration
Again this is related to API gateway.
- Lambda integration: API Gateway performs request and response transformations on data entering or leaving API gateway.
- Lambda proxy integration: API Gateway does not take part in transformations.
For a detailed comparison please read Lakshman Diwaakar’s article here.
Simple enough? Now lets look at the shenanigans:
- The new HTTP API option in API Gateway: At Re:Invent 2019 AWS launched a new variant of API gateway with a funny sounding name, HTTP API. Basically its a stripped down version of the existing offering, very similar to Lambda-proxy integration. It is simpler to use and comes at a lower price. However it lacks core features of the former like SDK generation, API key generation, usage plans and Cognito/IAM authorization.
Also the nomenclature could’ve been better, because both variants are REST APIs over HTTP protocol.
2. Monolithic vs multiple lambda functions: Lambda proxy integration opens up a new design pattern where all calls are routed into a single, monolithic lambda function. A single Lambda handling multiple requests will theoretically stay warmer and reduce cold starts. This pattern is simpler to implement, for example one could fit a full fledged Flask or Express server inside a single lambda.
The trade offs for this approach also have to be considered. Serverless Hero Yan Cui makes a strong case for using single-purpose Lambda functions. Putting all your code together will affect capability discovery, debugging and scaling of your Lambda functions. Further if your Lambda takes high volumes of incoming calls, the cold start benefit will start to diminish. This is because of the way Lambda works, i.e. a new instance is spawned every time when existing instances are busy and a new call is received. Ultimately its a trade off between simplicity and performance.
3. Using Lambda-proxy integration? Be prepared to write more code. The asynchronous invocation feature I wrote about earlier will have to be developed by yourself.
4. Using Lambda integration? The next heading is for you.
4. Data input formats
This is related to how API Gateway accepts data and passes it to Lambda. API gateway has first class support for query strings, path parameters, headers and JSON data. However other MIME types like ‘x-www-form-urlencoded’ and ‘multipart/form-data’ are second class citizens.
a. application/json
Here JSON data is passed through the request body. API gateway performs some transformations on the data and passes it to Lambda in JSON format.
{ Name : 'John Smith', Age: 23}If we select Lambda integration, we perform these transformations in API gateway itself. This is done with help of the Velocity Template Language(VTL).
According to Wikipedia
Apache Velocity is a Java-based template engine that provides a template language to reference objects defined in Java code. It aims to ensure clean separation between the presentation tier and business tiers in a Web application (the model–view–controller design pattern).
Lambda integration may be unsuitable if this is not your primary use case. Proxy integration may be a better bet for users unfamiliar with Java based templating.
In Lambda proxy you perform transformations and templating in Lambda itself. Here API gateway transforms the request body, headers, path and query parameters, metadata etc into a JSON object and passes it to Lambda.
b. application/x-www-form-urlencoded
Here the body of HTTP message sent is essentially a giant query string. name/value pairs are separated by the ampersand (&), and names are separated from values by the equals symbol (=).
'Name=John&Age=23&special%20characters=%24%23%40!'Here the Non-alphanumeric characters are replaced by `%HH’, the two hexadecimal digits representing the ASCII code of the character.
Here’s where the fun begins. API gateway can accept input in any form, Lambda accepts only JSON format. However API gateway provides no simple way to convert ‘x-www-form-urlencoded’ into JSON.
Conversion of ‘x-www-form-urlencoded’ string to JSON can be done in two ways:
- Lambda-proxy integration: Conversion performed by Lambda. This means you can use the programming language of your choice.
- Lambda integration: API gateway performs conversion with help of velocity templates.
We’ll see both the methods in action here. Let the encoded input body be as follows:
key1=first&key1=second&key2=third- Lambda-proxy integration: Here I created a simple Lambda function which parses x-www-form-urlencoded using a core NodeJS module.
2. Lambda integration: A VTL template from stack overflow for x-www-form-urlencoded seems to be popular among serverless community. Basically it converts your string into JSON and passes the value to Lambda.
Go to integration request section in API Gateway and add the above mapping template. Be sure that content type is set as application/x-www-form-urlencoded, not application/json.
Now lets look at the results
Here we had passed two values for key1, however we get back only one. The template keeps the latest value for each key and discards others (‘first’ discarded). Besides the template is hacky at best. Surely there must be a way to fit multiple values in an array using VTL, but at the time of writing I could not find such code in the public domain. If you are able to sneak it out of Pentagon, do let us know in the comments.
Here’s what an AWS official wrote on the Amazon Web Services Forum:
there is no direct support for parsing “application/x-www-form-urlencoded” parameters from the body. If you want to parse parameters from your POST body, you would need to parse them using a mapping template.
An alternative would be to send the entire payload to your lambda function and parse the key value pairs from within your lambda function.
How about not using ‘x-www-form-urlencoded’ at all? The issue is that HTML cannot produce JSON on its own. HTML forms can produce only ‘x-www-form-urlencoded’ , ‘multipart/form-data’ or ‘text/plain’. We need to use Javascript to produce JSON, which may force changes on your client side.
While it may tempting to go with the API Gateway’s default JSON option for building your API, this may force you to change your web front end design. Here’s a detailed comparison of which format you prefer and when. It’s worth noting that Stripe and Twilio, two popular API providers stick to ‘x-www-form-urlencoded’ for requests.
We can have a compromise here. The request body can be directly passed into Lambda. The VTL template can take care of query and path parameters. But if that’s not the case we should stick to Lambda proxy or HTTP API.
Binary payload with application/x-www-form-urlencoded
A major advantage of x-www-form-urlencoded is the ability to send binary data. That is to say, it allows files to be encoded sent through an HTTP requests. Binary data is not supported by JSON. We can send binary data less than 10MB in size to Lambda using API gateway.
- Go to settings and find binary media types section
2. Add the MIME type you wish API Gateway to support. To allow all formats, enter the following:
*/*
While sending binary data remember to add ‘Content-Type’ header containing the MIME type you wish to support.
Lambda proxy method
It is important to take note of ‘isBase64Encoded’ property. This should be set as true if we have enabled binary media support, even if the response body contains textual data. Otherwise we get back a binary encoded string for both binary and text data. ‘isBase64Encoded’ by default is set to false. Note that ‘isBase64Encoded’ is not a header but a flag for API Gateway.
Rather than set it manually, we can directly take its value from the API Gateway event.
Remember to add ‘Content-Type’ header containing the MIME type you wish to support.
Lambda integration method
This one is again is nuanced. There are two ways to do it:
- Add the exact MIME type in supported which you want to support. ‘*/*’ which we used earler will not work. Why? Another shenanigan. The issue with this approach is that we need to know what types will be supported in advance.
2. The description for binary Media support is self explanatory.
API Gateway will look at the Content-Type and Accept HTTP headers to decide how to handle the body.
Earlier the Content-Type header was enough. Now we need Accept header as well. Both hold the same value, that is name of the MIME type. This in our case is image/png.
3. Our mapping sends the request body directly into Lambda.
4. Lambda function simply returns back the data it received without any change.
5. Make the request
But what when setting file headers is not possible? This can happen when the POST message contains data of a different format or when we make a GET request. The proxy integration discussed earlier will work properly, but Lambda integration method will need modifications. We need to add the MIME type headers manually using API gateway.
- Add the exact MIME type as done earlier
2. In method response add a new response header ‘Content-Type’.
3. In integration response map ‘Content-Type’ header to the correct MIME format, which in this case is ‘image/png’ (with quotes). Set ‘content handling’ to ‘Convert to binary’.
Here we hardcoded ‘image/png’ into the header mappings. Alternately we could make Lambda pass the required header map it as follows:
The Lambda function in this example returns an image stored in an S3 bucket. Lambda converts the image into a Base64 encoded string. API gateway converts this into Binary format and adds the correct header.
Result
x-www-form-urlencoded drawbacks
- It can transmit either text or binary data, but not both. Only a single binary file can be sent.
- Every non-alphanumeric character is replaced by 3 characters (‘%HH’), making this encoding inefficient for large files.
This format is good for textual data and small files. There’s a second MIME format supported by HTML forms to address the issues facing ‘x-www-form-urlencoded’ format.
c. multipart/form-data
‘multipart/form-data’ is the format of choice for transmitting large files. It allows both text and multiple files to be sent together. This is achieved by separating each item using a string boundary.
Each (key, value) pair is encoded as follows
--<<boundary_value>>
Content-Disposition: form-data; name="<<field_name>>"<<field_value>>
So for two values we get
----------------------------024892011799986865440046\r\nContent-Disposition: form-data; name=\"key1\"\r\n\r\nvalue1\r\n----------------------------024892011799986865440046\r\nContent-Disposition: form-data; name=\"key2\"\r\n\r\nvalue2\r\n----------------------------024892011799986865440046--\r\nThe boundary value is automatically generated but we can also manually assign it using the ‘Content-Type’ header:
Content-Type: multipart/form-data; boundary=--------------------------632614546983522473131640API gateway does not support ‘multipart/form-data’. But we can pass the request body to Lambda and parse it from there. However this too has caveats, and I found that all these solutions only support Proxy integration.
- Lack of good standalone libraries to parse ‘multipart/form-data’. The most popular one for NodeJs is ‘aws-lambda-multipart-parser’ but it faces many issues and is not actively maintained. I couldn’t find such a parser for Python.
2. High level frameworks like Flask or Express could be used. But they require Proxy integration and can be an overkill for a small function.
3. Use a Lambda middleware framework like Middy.js. This approach too requires proxy integration and can bloat your function. I couldn’t find an equivalent library for Python.
It seems that NodeJS has a better module ecosystem for Lambda compared to Python and other languages.
d. Other exotic content types
From these examples 3 patterns to handle MIME types have emerged:
- Lambda integration: Use VTL templates in API gateway to transform the input into a suitable JSON format.
- Lambda proxy integration: Perform transformation in Lambda using libraries supported by your selected runtime.
- Compromise: Use VTL template to send request body directly to Lambda. VTL could still be used to transform path parameters, query parameters and headers.
Conclusion
From the various nuances discussed here, it is clear that Serverless computing is not simple as shown to be. Issues which may be trivial for a regular server may require heavy lifting in serverless.
Part 1 was about challenges arising from hard limits imposed by API gateway upon Lambda. Part 2 will focus on the specifics of Lambda itself. Stay tuned and do not forget to follow!
Want to be friends? Or looking to hire a developer? Catch up with the writer on LinkedIn.
