Speedup AWS Lambda deployments with Layers

Photo by @gebhartyler on Unsplash

AWS Lambda Layers is a new feature announced by Amazon on Nov 29th.

It allows creating an environment which can be reused between Lambda functions.

In this post I will show you how you can improve deployment time of your Lambda function written in Java by reusing a layer. It will also improve an overall development cycle experience and decrease internet bandwidth usage.

Before layers, if you wanted to deploy a Lambda function you had to provide a “fat” zip file with all the dependencies included.

In my example application I am using the following dependencies:

dependencies {
compile(
'com.amazonaws:aws-lambda-java-core:1.2.0',
'com.amazonaws:aws-lambda-java-log4j:1.0.0',
'com.fasterxml.jackson.core:jackson-core:2.8.5',
'com.fasterxml.jackson.core:jackson-databind:2.8.5',
'com.fasterxml.jackson.core:jackson-annotations:2.8.5',
'org.eclipse.jgit:org.eclipse.jgit:5.1.1.201809181055-r',
'com.google.guava:guava:26.0-jre',
'com.mashape.unirest:unirest-java:1.4.9',
            // utils and aws client sdk
'com.amazonaws:aws-java-sdk-lambda:1.11.461',
'com.amazonaws:aws-java-sdk-core:1.11.461',
'com.amazonaws:aws-java-sdk-dynamodb:1.11.461'
    )
testCompile(
'org.junit.jupiter:junit-jupiter-api:5.3.2',
'org.junit.platform:junit-platform-launcher:1.3.2'
)
}

When the function is deployed the ZIP of 13.85 MB has to be uploaded to S3 bucket. If I want to test changes on AWS directly I have to redeploy the package several times. I have to wait about 10 seconds every time and it’s annoying.

After I moved all the dependencies to reusable layer the deployed ZIP was reduced to 27.41 KB and it takes long for the function to deploy only fir the first time, and any time I modify the dependencies because this is when the layer needs to be updated. In other cases the deployment is blazingly fast and I like it :)

I developed my Lambda function using Serverless Framework, which also has support for Lambda Layers already.

In my other post I explained what the example application does, so you can have a look if you need more introduction.

Here I will show you what to change to move dependencies to the layer.

First, let’s create a layer with library dependencies.

In Amazon docs you can check how the standard structure of libraries should look like for different execution environments.

For Java it should be a ZIP file with jars placed in java/lib/ path.

Such archive can be built with gradle. We need to modify the original build.gradle slightly:

task buildZip(type: Zip) {
baseName = "aws-java-github-webhook-gitstats"
from compileJava
from processResources
// into('lib') {
// from configurations.runtime
// }
}

In the Lambda function ZIP file we don’t need dependencies any longer so i commented out this fragment.

We need to move them to a separate folder which Serverless Framework will use to create a ZIP for layer, so we add a new gradle task:

task buildLayer(type: Copy) {
into "$buildDir/layer/java/lib"
from configurations.runtime
}

and add it as dependency to build task:

build.dependsOn buildZip, buildLayer

Next, we have to configure layer in serverless.yaml .

At the bottom of the file add:

layers:
dependencies:
path: build/layer

the path is a path to the folder where gradle will copy dependencies.

We also need to configure each Lambda function to use the layer:

webhook:
handler: com.serverless.ApiGatewayHandler
timeout: 30 # max API Gateway timeout
events:
- http:
path: webhook
method: post
cors: true
layers:
- {Ref: DependenciesLambdaLayer}

The Ref name is created from layer name configured in layer section in camel-case and appending LambdaLayer postfix.

That’s it. Now when you build the project and deploy we can see that Serverless will upload two archived. One large for layer and smaller for project classes, e.g.:

Serverless: Packaging service...
Serverless: Excluding development dependencies...
Serverless: Uploading CloudFormation file to S3...
Serverless: Uploading artifacts...
Serverless: Uploading service .zip file to S3 (27.41 KB)...
Serverless: Uploading service .zip file to S3 (13.9 MB)...
Serverless: Validating template...
Serverless: Updating Stack...
Serverless: Checking Stack update progress...
...............................................................

If we check the Lambda function created in AWS console we will see the layer dependency:

You can find the complete example on Github here in lambda-layer branch.

You will see that if you update any of your project classes and redeploy the function, it will only upload a few KB ZIP file.

In the next post, which I will publish soon, I will measure if adding the layer impacts the cold start of the Lambda function.

Follow me, if you want to get updates.