AWS Lambda — Best Practices

We at KredX have been using AWS Lambda for a while now, influenced by the fact that Lambda is scalable yet simple and cost effective. A developer could just focus converting the requirement to code, and easily get the code up and running with Lambda. However, there are a few design factors to be considered while coding a Lambda function considering Lambda container life cycle.

Dependency Initialisation

Any expensive initialisation logic you add to your Lambda function can easily increase execution duration and hence eat into your monthly billing budget. Time taken to instantiate your lambda function is invariably a part of paid time when it comes to billing. Leveraging the fact that Lambda containers are reused across requests depending on request frequency and container instance availability, you can design your code to perform initialisation only once per container instance creation. It means that initialisation is done only on ‘cold’ starts and not when the container is ‘hot’.

An easier example with Java is that the Lambda handler implementation class can initialise the dependencies at the implementation class level, either as a part of the constructor or as a static block level. Remember that the handler implementation class is instantiated only once per container instance. One needs to understand that this approach makes the lambda function stateful and not stateless, which is fine considering how expensive your initialisation logic is.

Lambda Timeout

A Lambda function needs to be designed knowing that AWS allows maximum 15 minute execution duration at a time. Execution of the function is terminated by AWS once the runtime duration breaches this threshold and AWS retries executing the function again. It only means that you need to make sure that this retry by AWS does not break your business logic.

Latency

We, the engineering team at KredX, have moved all our worker jobs from EC2 to Lambda achieving throughput and cost optimisation. Lambda functions are cheap when latency is in milliseconds, or say even seconds. They can get pretty expensive when the jobs frequently start running for minutes and if you are not able to bring down runtime execution duration, consider sticking with EC2 or ECS as the latter options will be relatively less expensive. Do leverage the pricing calculators provided by AWS to make objective decisions here.

Allocating Resources

Lambda is cost effective in terms of expense. That does not mean that you could simply get away by allocating minimum memory. AWS allocates underlying hardware based on the memory you allocate to your function. Opting for higher memory can sometime improve latency, so you need to try around a few options.

AWS does not allow configuring Lambda runtime resources at granular levels. One of the Lambda functions written in Java at KredX failed due to insufficient Metaspace and we had to allocate increased memory. An ideal solution would have been configuring the JVM to use higher Metaspace size, but AWS Lambda simply does not allow tuning JVM.

Throughput

There is a default cap of 1000 concurrent executions of Lambda functions at an AWS account level. You may not like the conservative approach, but it is enforced for a good reason. One does not want a bug in the developer code to spawn concurrent container instances and cause a spike in the billing amount. Design your functions considering throughput, and when you have a genuine case to increase this limit to a value beyond 1000, do reach out to AWS support with your request.

Do keep in mind that while Lambda allows you to scale in terms of throughput, you might still be constrained with the external systems your Lambda function connects to. For example, you might be tempted to concurrently connect to your database, but the database server can only support a limited number of active connections at a time. So you will require to work within such limits.

If you require Lambda to access resources in your private VPC, you need to launch the Lambda in the VPC. This means, a Lambda instance will have an Elastic Network Interface (ENI) attached to it. Each concurrent invocation of a Lambda function requires a separate ENI and the concurrency settings set for a Lambda can easily use up the VPC ENI capacity leading to scaling issues.

Scheduling

The easiest way to schedule Lambda execution is using AWS Cloudwatch. AWS Cloudwatch CRON gives ‘minute’ level precision and it should suffice in majority cases.

Cold Starts

Keep in mind that cold start of a Lambda function can affect expected latency. Cold start duration of a Lambda in VPC is significantly high due to ENI attachment, and may not be an option for many. Lambda container instances are destroyed on being idle for about 10–12 minutes or so. Cold starts are normally not an issue with scheduled jobs. It is a choice between latency and cost in the end.

Forcing Restart

There might be a need for you to force resetting the state of containers. Say the external configuration that is loaded in your initialisation logic now needs to be refreshed. Any new Lambda container instances will load this new configuration, but what about the instances that are already hot? Remember, you coded your Lambda function to be stateful above. There are three basic routes one could take to enforce that all new requests will be routed to new container instances.

Redeploy Lambda

Redeploying the Lambda function will make sure that all new requests are routed to new Lambda container instances. The existing ‘hot’ Lambda container instances will eventually be terminated by AWS.

Modify Lambda configuration

A change in Lambda configuration such as modifying the memory allocated or adding a new environment variable via Lambda console can be as effective as Lambda redeployment to force Lambda restart.

Periodic Configuration Refresh

You can design your function logic to periodically force refresh configuration programmatically. This approach will not require an intervention or restart, but some (or a large number) of the container instances can run for a while with stale configuration.

We hope that this post is useful to you to make better use of AWS Lambda. If you would like to share your comments or suggest a best practice, please do leave a feedback.