4 Tools for Easy AWS Lambda Deployment
Dive into the world of AWS Lambda deployment tools and select the perfect one for your project.
AWS is one of the leading Cloud providers hosting millions of applications daily. And one of its most frequently used services is AWS Lambda.
AWS Lambda provides a way to run code without managing servers. Lambdas are used frequently for microservices, data pipelines, DevOps automation projects, and many more.
However, before you start using AWS Lambda, you need to choose the optimal tool to help with the development and deployment of AWS Lambda functions.
In the last couple of years, the DevOps principles, IaC, and enormous investments into automation changed the way we are developing projects. Each Cloud provider’s resource is now deployed in a reproductive and monitored way. Thus, selecting the optimal tool is crucial, since it can speed up development or slow it down significantly. This choice will have consequences for you for a long time — changing it is never easy.
Sidenote: the benefits of AWS Lambda itself are widely known and discussed, so we won’t dive into detail here. But if you’d like to summarize your knowledge about the solution before reading my article, take a look at this AWS summary.
Now, let’s see some most well-known tools in action, so you can choose the one that will be most adjusted to your needs!
Lambda Deployment
Lambdas are deployed based on two types of artefacts:
- Zip packages (via direct upload or S3)
- Docker images (via ECR)
When to Use Which?
There is a simple rule here — it depends on the size of the Lambda.
If your uncompressed Lambda code with dependencies is over 250MB — use Docker Images. For smaller sizes, it’s more convenient to use Zip packages.
It is generally better to use S3 buckets as storage for Zip artefacts instead of direct upload. Storing artefacts in the S3 bucket enables fast rollbacks in case of errors. However, if you still chose direct upload, remember that only Zip files less than 50MB can be used this way.
Where Are the Codes?!
The project shows a few ways to deploy Lambda. There are four evaluated tools placed in folders:
- manual — bash scripts, AWS CLI
- terraform — terraform scripts
- serverless — serverless framework configuration files
- sam — AWS SAM deployment
I encourage you to run them — nothing helps to understand them more than hands-on code. In each folder, you can use a list of commands to run the examples and detailed descriptions of a tool.
Tools
Manual
This is a version that does not require knowledge of any additional tools other than bash and AWS CLI. The implementation uses Lambda Zip direct upload which is fast to develop but does not allow for artefact storage. It can be extended to support S3 bucket file upload.
#!/usr/bin/env bash
aws iam create-role --role-name bpk-lambda-role --assume-role-policy-document '{"Version": "2012-10-17","Statement": [{ "Effect": "Allow", "Principal": {"Service": "lambda.amazonaws.com"}, "Action": "sts:AssumeRole"}]}'aws iam attach-role-policy --role-name bpk-lambda-role --policy-arn arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRoleaws iam attach-role-policy --role-name bpk-lambda-role --policy-arn arn:aws:iam::aws:policy/AmazonS3FullAccess
rm code.zip
zip -X -r code.zip ../code
# Read README why used!
sleep 10
ROLE_ARN=$(aws iam get-role --role-name bpk-lambda-role | jq -r .Role.Arn)aws lambda create-function --function-name bpk-lambda-function --zip-file fileb://code.zip --handler code.handle.lambda_handler --runtime python3.8 --role $ROLE_ARN
The imperative approach to a solution. You define the set of actions that will happen. This is a great approach for educational purposes and allows you to better understand the mechanisms of other tools. It clearly shows the shortcomings and required features to look for when selecting tools. (For example, when the starting state of the system is different than expected, the set of actions will not work correctly). Another big advantage of this approach is simplicity. Generally, people who implemented this basic approach understand the Lambda deployment mechanism better.
Recommended: education only
Terraform
There are a few ways to deploy Lambda using terraform. We can use basic AWS provider resource modules and build the code ourselves using the CICD pipeline or the AWS supported module.
I really like this quick approach using one of the officially approved modules from the Terraform Registry: terraform-aws-modules/lambda
provider "aws" {}
module "lambda_function" {
source = "terraform-aws-modules/lambda/aws"
function_name = "bpk-lambda-function"
description = "My Lambda function"
handler = "handle.lambda_handler"
runtime = "python3.7"
source_path = "../code"
}
resource "aws_iam_role_policy_attachment" "lambda_basic_execution" {
policy_arn = "arn:aws:iam::aws:policy/AmazonS3FullAccess"
role = module.lambda_function.lambda_role_name
}The module was created with multiple AWS resources to simplify the Lambda build and deployment process. The building process supports both local and in-Docker builds. Lambda deployment in the provided example looks simple, but all magic is done underneath the hood. By default, the code in the given source path is zipped and placed in the builds folder. Then, the Lambda is created using direct Zip upload. If you want to use S3 as artefact storage, this is possible with this module — codes appear directly in the documentation. The module uses a hash function to only update the Lambda code when it changes. I strongly encourage you to review the terraform plan for more insights.
Layers are an important mechanism that allows adding resources like files or libraries to multiple Lambdas and manages them from a single point. Common usage is to create a layer with libraries or certificates and attach it to Lambda. Be aware that the layers’ size is included in the Lambda size limit for Zip deployment using S3.
Terraform is a great tool for deploying whole infrastructures, but does not support debug, log viewing, and local execution — features that might come in handy. On the other hand, Terraform exceeds deploying big infrastructures. Other functions can be supplemented by the AWS console or AWS CLI.
Recommended:
- selected IaC is Terraform,
- multi-cloud environment,
- do not care about Lambda development helpers — AWS CLI can be a replacement for most of them.
Serverless
The Serverless Framework is a tool that allows deploying
serverless functions on different Clouds and on-premise systems. While deploying on AWS, it uses CloudFormation to run the deployment process. The process starts from building an artefact in form of a Zip package or Docker Image, then uploads it to artefact storage (S3 or ECR). After the upload finishes, CloudFormation Stack is created and executed. Future updates can be done by redeployment or simple updates of configuration. Configuration changes are much faster than full redeployment. Deployment history is preserved in form of CloudFormation stacks.
Here is an example of a Lambda function with layer and IAM Role deployment specification:
service: bpk-lambda-function
provider:
name: aws
runtime: python3.8
region: eu-central-1
# you can add statements to the Lambda function's IAM Role here
iamRoleStatements:
- Effect: "Allow"
Action:
- "s3:*"
Resource: "*"
# you can add packaging information here
package:
include:
- ../code/**
exclude:
- node_modules/**
- ./*.json
layers:
myLayer:
path: ../layer
functions:
superfunction:
handler: code/handle.lambda_handlerThis tool has more benefits when used with the “Pro” version by providing monitoring features and a CICD pipeline. Build-in integration with different Lambda triggers is also a useful feature. It can generate an API Gateway for your Lambda backed Websocket with few lines of code. Adjusting generated infrastructure can be done using CloudFormation scripts. There are a lot of community-supported plugins ready; before you try to create something by yourself — check the plugin list.
Support for layers is also provided and is simple to configure. It’s worth using them as a single place for storing and managing common code, configuration and file resources.
The local artefacts are stored in the .serverless folder. There are both the zip file to upload and CloudFormation templates to create and update values. Be aware that there are two different templates — one for the creation and one for the update.
Recommended:
- integration with multiple Lambda triggers
- plugins are available for all many extensions
- focus on a tool to develop Lambdas (incl. local execution)
- multi-Cloud environments
- CloudFormation as IaC tool
AWS SAM
AWS SAM — Serverless Application Model is an extension of CloudFormations. This tool consists of an application specification and CLI to support the build/deployment/execution of Lambda functions. The application project is created based on templates. In most cases, standard AWS provided templates are enough. For bigger players, there is a way to define custom ones. Custom templates can help with applying company standards, CICD pipelines, and security checks across newly created projects. This gives teams kicking off a new project a quick start to begin developing business value. There is also a risk here — remember that template is applied only on project creation, it will have no impact on the existing project.
Application is a form of function grouping. This allows monitoring and managing the whole application at the same time in the AWS Console. This comes in handy for bigger applications and robust systems. We can define an application that consists of a few functions, databases and Lambda triggers like API Gateway in a template file. For the first deployment, it’s worth adding a “guided” flag — this way, CLI will help us create a configuration file, which will be used for future deployments.
AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: >
BPK Function Description
Globals:
Function:
Timeout: 3
Resources:
MyLayer:
Type: AWS::Serverless::LayerVersion
Properties:
ContentUri: ../layer
CompatibleRuntimes:
- python3.7
Metadata:
BuildMethod: python3.7
BpkFunction:
Type: AWS::Serverless::Function
Properties:
CodeUri: ../code/
Handler: handle.lambda_handler
Runtime: python3.7
Layers:
- !Ref MyLayer
Policies:
- Version: '2012-10-17'
Statement:
- Effect: Allow
Action:
- 's3:*'
Resource: 'arn:aws:s3:::*'
Outputs:
BpkFunction:
Description: "Deployed Lambda Function ARN"
Value: !GetAtt BpkFunction.Arn
BpkFunctionIamRole:
Description: "Implicit IAM Role created for Deployed Lambda Function"
Value: !GetAtt BpkFunctionRole.ArnAWS SAM is an extension of CloudFormation — when looking for examples or documentation do not omit CloudFormation documentation. CloudFormation documentation is much richer than AWS SAM examples when it comes to the creation of additional resources. There is a downside of this, however — adding resources outside of AWS (like in the hybrid-Cloud approach) will be a big challenge.
Temporary files can be found in .aws-sam. This folder is a good source of knowledge for troubleshooting deployment errors. There are a few important files there:
- code copy before packaging,
- template.yaml with resource description,
- build.toml with Lambda packaging and deployment information.
AWS SAM CLI supports code development, deployment and troubleshooting. During deployment and writing different Lambdas, there’s no need to jump back to the AWS console — all needed data and logs can be easily extracted from CLI.
Recommended:
- deployment of multiple lambdas as a part of a bigger application
- leverage of SAM application repository (public and private)
- focus on helpers for lambdas development (incl. local execution)
- no creation of resources outside of AWS
- CloudFormation as IaC tool
Summary
Long story short: there is no perfect tool for every project — there are only tradeoffs and developer preferences :)
The manual approach is good for teaching, but it is missing many production features.
The Terraform approach is worth considering when vendor lock-in on the IaC tool is not an option, and when missing Lambda development helpers are not a problem.
The Serverless Framework’s strength is a swift integration with multiple triggers, making it possible to deploy serverless functions on multiple clouds and the tools to develop and run serverless functions locally. Troubleshooting is quite hard and requires reviewing CloudFormation stacks for complex infrastructures.
The AWS SAM is a tool that works only for the AWS stack — moving the serverless function to another cloud is not possible. The option to create templates with best practices, CICD pipelines and security tools is a great function for big companies.
Choosing the right tool for the job is important. Consider the Pros and Cons of each of them to make sure to see the big picture. After all, this choice will impact your infrastructure creation and Lambda development capabilities.
For MLOps Hands-on guides, tutorials and code examples, follow me on Medium and contact me via social media.
