You Serve Lambdas Too Cold. Warm It Up!

“Colorful lines of code on a MacBook screen” by Caspar Rubin on Unsplash

One of the most common issues people face when using AWS Lambda is the cold start. It doesn’t even matter if it’s java or nodejs. Trust me, a real-life app will have a noticeable bootstrap time: you can easily reach 1-second cold start even with nodejs.

What It Means for the End User

Let’s benchmark an AWS Lambda written in java exposed via AWS API Gateway. Check out this article for details on how you may create such AWS Lambda.

We will use siege to simulate a single user making a bunch of requests.

->siege -c1 -d10 https://5smlp6f63h.execute-api.eu-west-1.amazonaws.com/api/ping
** SIEGE 4.0.4
** Preparing 1 concurrent users for battle.
The server is now under siege...
HTTP/1.1 200     1.00 secs:       2 bytes ==> GET  /api/ping
HTTP/1.1 200     0.03 secs:       2 bytes ==> GET  /api/ping
HTTP/1.1 200     0.03 secs:       2 bytes ==> GET  /api/ping
HTTP/1.1 200     0.04 secs:       2 bytes ==> GET  /api/ping
HTTP/1.1 200     0.03 secs:       2 bytes ==> GET  /api/ping

There is nothing surprising. The first request is slow, the subsequent are fine. The user can handle a single lagging request to your backend, right?

WRONG.

If you open any website, the requests to the backend will probably look like the following:

A user initiates more than 1 concurrent request. 1 user ≠ 1 concurrent request.

Thus, 1 user may easily mean 5 concurrent requests (we will use the magic number 5 just an example throughout the article):

->siege -c5 -d10 https://5smlp6f63h.execute-api.eu-west-1.amazonaws.com/api/ping
** SIEGE 4.0.4
** Preparing 5 concurrent users for battle.
The server is now under siege...
HTTP/1.1 200     0.84 secs:       2 bytes ==> GET  /api/ping
HTTP/1.1 200     0.88 secs:       2 bytes ==> GET  /api/ping
HTTP/1.1 200     0.91 secs:       2 bytes ==> GET  /api/ping
HTTP/1.1 200     0.94 secs:       2 bytes ==> GET  /api/ping
HTTP/1.1 200     1.00 secs:       2 bytes ==> GET  /api/ping
HTTP/1.1 200     0.03 secs:       2 bytes ==> GET  /api/ping
HTTP/1.1 200     0.03 secs:       2 bytes ==> GET  /api/ping
HTTP/1.1 200     0.03 secs:       2 bytes ==> GET  /api/ping
HTTP/1.1 200     0.03 secs:       2 bytes ==> GET  /api/ping
HTTP/1.1 200     0.02 secs:       2 bytes ==> GET  /api/ping

This means some of users of the API above will see 1 second delay when AWS Lambda is not cached (it’s cached for about 5 minutes after invocation).

If your AWS Lambda is a backend process which may tolerate such lags, you’re lucky. However, in order to show smooth experience for end users you have to warm up the lambdas!

Warming Up the Lambdas. Attempt #1

Since AWS Lambda is cached for 5 minutes, the most straightforward solution that you may think of is to trigger the required lambdas every 5 minutes (or maybe 4 minutes, to be sure).

This won’t work. Because AWS will cache just 1 lambda instance. A user with 5 concurrent requests will now look something like:

->siege -c5 -d10 https://5smlp6f63h.execute-api.eu-west-1.amazonaws.com/api/ping
** SIEGE 4.0.4
** Preparing 5 concurrent users for battle.
The server is now under siege...
HTTP/1.1 200     0.03 secs:       2 bytes ==> GET  /api/ping
HTTP/1.1 200     0.88 secs:       2 bytes ==> GET  /api/ping
HTTP/1.1 200     0.91 secs:       2 bytes ==> GET  /api/ping
HTTP/1.1 200     0.94 secs:       2 bytes ==> GET  /api/ping
HTTP/1.1 200     1.00 secs:       2 bytes ==> GET  /api/ping
HTTP/1.1 200     0.03 secs:       2 bytes ==> GET  /api/ping
HTTP/1.1 200     0.03 secs:       2 bytes ==> GET  /api/ping
HTTP/1.1 200     0.03 secs:       2 bytes ==> GET  /api/ping
HTTP/1.1 200     0.03 secs:       2 bytes ==> GET  /api/ping
HTTP/1.1 200     0.02 secs:       2 bytes ==> GET  /api/ping

This means that only 1 out of 5 concurrent requests is handled by the cached lambda. It’s better, but the experience will still be sluggish.

Warming Up the Lambdas. Attempt #2

Now, since the previous attempt improved things. We will try to create 5 such triggers.

This will not work as well. Effectively, the result will be almost the same as in the previous attempt, because the triggers are NOT fired simultaneously. CloudWatch does not allow to configure this behaviour.

Warming Up the Lambdas. Attempt #3

At this point, you want to switch back to AWS EC2, but don’t give up, I’ve got a simple solution for you!

In order to warm up our target lambda, we will need another lambda. That’s right, we will create a “warming” lambda, whose responsibility is to ping the target lambda with the required concurrency level. This way we will force AWS to cache as much lambda instances as needed.

The warming lambda code is very simple. Here we are just creating 5 concurrent requests:

var AWS = require('aws-sdk');
var lmb = new AWS.Lambda({ region: 'eu-west-1' });
exports.handler = function(event, context) {
    Promise.all([...Array(5).keys()]
        .map(() => lmb.invoke({ FunctionName: '...' }).promise()))
        .then((data) => context.succeed(data))
        .catch((err) => context.fail(err));
}

And this is what we’ll see when the user performs the 5 concurrent requests now:

->siege -c5 -d10 https://5smlp6f63h.execute-api.eu-west-1.amazonaws.com/api/ping
** SIEGE 4.0.4
** Preparing 5 concurrent users for battle.
The server is now under siege...
HTTP/1.1 200     0.03 secs:       2 bytes ==> GET  /api/ping
HTTP/1.1 200     0.03 secs:       2 bytes ==> GET  /api/ping
HTTP/1.1 200     0.03 secs:       2 bytes ==> GET  /api/ping
HTTP/1.1 200     0.03 secs:       2 bytes ==> GET  /api/ping
HTTP/1.1 200     0.03 secs:       2 bytes ==> GET  /api/ping

This is much better! However, keep in mind, that 6 concurrent requests will not be handled that well: 1 of the 6 requests will be slow.

Thus, you can configure the concurrency level by yourself by changing the number 5 to something else.

The Example Sources

You can use for your tests the simple java example here. Note, that the repo has its own CloudFormation template which does not contain the warming feature. To use the warming feature, you may use the following CloudFormation template:

https://console.aws.amazon.com/cloudformation/home?#/stacks/new?templateURL=https://s3-eu-west-1.amazonaws.com/bleshik/aws-lambda-servlet-example-warming-CloudFormation.json

What’s Next?

We’ve already discussed how to turn your plain old java servlet application serverless. Next, I’ll show how to deploy your JAX-RS application on AWS Lambda, which is quite similar to deploying the plain old servlet, but still worth mentioning.

Comments, likes, and shares are highly appreciated. Cheers! ❤️