A Practical Guide to Surviving AWS SAM

Part 3 — Lambda Layers

According to the younger, less-experienced version of me, the next chapter of this series should be about Lambda Layers. So, let’s listen to him and get started.

As usual, you can find all the code on my GitHub page.

Valley of Fire Nevada — Fire Wave

The first time I heard about Lambda Layers, I was in Las Vegas sitting as a spectator at the AWS re:Invent 2018 — Keynote of Dr. Werner and I clearly remember that when Werner announced this new feature, in my mind I was like:

Oh great one month of thinking, discussion and attempts on how to share code between Lambdas lost.

Yes, during the first second or so, I was pretty angry because at the time, I was new to the serverless world and one of my biggest concerns during my first experiments was the huge amount of duplicated code that I was writing. So I started studying ways on how to solve this problem — Git submodules, CI/CD pipeline retrieving code from a common repository to create an aggregated package, custom libraries. These were only some of the possibilities.

Then, Werner Vogel arrived with this new service helping to reduce duplicated code and increase code sharing between Lambdas. But after another second, I realized that this service was the answer that I had been waiting for during that month.

A Layer is a ZIP archive that contains libraries and custom code that you can import at runtime to be used by your Lambda functions, you don’t have to create custom libraries nor create custom packages, all will be managed by AWS. There are two main uses for Layers:

• share libraries between Lambda
• share common code between Lambda

The first use case helps to manage libraries and dependencies across Lambda function reducing package size and increasing governance in managing version numbers. It also helps reducing build time in your CI/CD pipeline since you don’t have to Re-build each time libraries but will be imported at runtime.

The second use case instead helps to share custom code between Lambdas. Since a Layer is a ZIP archive you can add your common functions and reuse them among teams, accounts, organizations…lowering code duplication and coupling.

In an AWS SAM template, you can define a Lambda Layer in less than 10 rows. Besides AWS specification you only have to specify the folder where your code can be found and the compatible runtimes. If you don’t specify the name field, will be used the name of the resource (in this case MyLayer)

MyLayer:
  Type: AWS::Serverless::LayerVersion
  Properties:
    ContentUri: my_layer
    CompatibleRuntimes:
      - python3.8

Then your folder my_layer will look almost like a normal Lambda Function. In Python will be a .py file with your code and optionally a requirements.txt file. Keep attention that the Python folder is mandatory; otherwise the Lambda environment will not be able to locate your function (we will see later a way to avoid it).

And content of my_common_function.py

def say_hello():
    return 'Hello world!'

You can now use say_hello function inside your Lambda. You just need to add Layers field in your AWS SAM template under your Function properties…

HelloWorldFunction:
  Type: AWS::Serverless::Function
  Properties:
    CodeUri: hello_world/
    Handler: app.lambda_handler
    Runtime: python3.8
    Layers:
      - !Ref MyLayer

…and import it in your code. If you are using PyCharm IDE to get rid of the import error you can mark my_layer/python directory as source root like we did in the first part of the series.

import json 
# import layer function
# remember to mark my_layer folder as source directory from PyCharm
from my_common_function import say_hello 

def lambda_handler(event, context):
    return {
        "statusCode": 200,
        "body": json.dumps({
            "message": say_hello() # invoke layer function
        }),
    }

Pretty easy. Under the hood, after spinning up the container, AWS also unzips the Layer and loads it into the python folder so that the interpreter can find it. You can now share say_hello function among all your Lambdas and also define Layers as AWS SAM global property avoiding duplication. Keep in mind that each function can have at most 5 Layers attached to it and also that the Layers count as Lambda package size, so you can’t exceed the 250MB.

You can now deploy your template and check it on the AWS console. After executing sam build command, take a couple of minutes to check the structure of .aws-sam/build folder. In particular, look at .aws-sam/build/template.yml.

sam build
sam deploy --guided

After deployment, you should see your Layer on AWS Lambda Console under the Layers section. As you can see Layers are versioned so each time you edit your Layer a new version will be deployed, this gives you great agility in Layer management and is really helpful in case of disruptive modification, you can control when updating a Layer like a normal library.

You should also see your Layer attached to the Lambda function

And if you execute a test you should see the message Hello world retrieved from Layer function

Now that you have deployed your first Layer let’s see some tips and tricks that may help you.

We have seen how to create a Layer with custom code. If you want to create a Layer that also embeds dependencies, the AWS SAM template will look like this:

MyLibLayer:
  Type: AWS::Serverless::LayerVersion
  Properties:
    ContentUri: my_lib_layer
    CompatibleRuntimes:
      - python3.8
  Metadata:
    BuildMethod: python3.8

The only differences are the Metadata and BuildMethod fields, telling AWS SAM that we want to build our Layer using python 3.8 interpreter. So during the execution of sam build SAM will execute a pip install -r my_lib_layer/requirements.txt -t .aws-sam/build/MyLibLayer downloading our dependencies. This time we can get rid of the python folder but we need to add the requirements.txt file with libraries specification

In our case, we want to force boto3 to be version 1.17.56

boto3==1.17.56

It’s always a best practice to force boto3 version even if it’s already pre-installed on each Lambda for a better version management and reproducibility between environments.

Now after running sam build if you take a look at .aws-sam/build/MyLibLayer folder you will notice that the python folder has been automatically added by SAM and contains all the boto3 packages. Also if you check .aws-sam/build/template.yml you will notice that if you don’t specify Metadata field SAM will use your original source code instead of the code in .aws-sam folder. That’s why in the previous case, we had to add a Python folder

Now, you can attach the Layer to your function like before and all your dependencies will be retrieved by the interpreter at runtime without the need for a build.

The last tip that I want to share with you is how to share Layers between AWS accounts, this feature is really important if you want to maximize the reuse of your code.

You just need to add a LayerVersionPermission resource in your AWS SAM template specifying the ID of the AWS account to which you want to share the Layer. I will use a generic MyAccountB parameter.

MyLibLayerPermission:
  Type: AWS::Lambda::LayerVersionPermission
  Properties:
    Action: lambda:GetLayerVersion
    LayerVersionArn: !Ref MyLibLayer
    Principal: !Ref MyAccountB

And then your function in MyAccountB will look like this:

HelloWorldAccountBFunction:
  Type: AWS::Serverless::Function
  Properties:
    CodeUri: hello_world/
    Handler: app.lambda_handler
    Runtime: python3.8
    Layers:
      - !Sub "arn:aws:lambda:${AWS::Region}:${MyAccountA}:layer:MyLibLayer:1"

Instead of using the Ref primitive, you can use Sub to build the Arn of MyLibLayer created on MyAccountA.

That was the last tip about Lambda Layers. As you may have noticed, Layers really help to lower code duplication and increase code sharing all in a simple way that is included in the AWS SAM template.

See you in the next chapter of this series.

More content at bip.xTech