AWS CDK & SAM: Testing TypeScript Lambdas Locally Fast
Tutorial on how to speed up local AWS CDK + TypeScript Lambda development using a custom esbuild
script
I am all about optimizing my workflow to ship code fast, so you can imagine my surprise when I tried AWS CDK and realized that the AWS-recommended way of locally testing TypeScript Lambda functions is to build (cdk synth
) the whole CDK stack first before being able to run sam invoke local
on a specific function. This approach is totally fine if you don’t have too many Lambdas. However, quite often that is not the case, so waiting for 20 functions to build just to test a 2-line change in one of them sounds excessive.
So, let’s scope the problem.
The Problem
AWS CDK does not provide any out-of-the-box solution to rebuild a single TypeScript Lambda handler. Moreover, cdk synth
stores transpiled JavaScript code in seemingly random and ugly-named asset folders that get recreated on each new cdk deploy
with newly generated names (if the content changed).
So what do we want instead?
The Goal
Ideally, during local development, we want to bypass the cdk synth
step completely and work only with individual Lambdas. The ideal workflow is:
- Make code changes to some Lambda function
- Build it (and nothing else)
- Invoke it locally using
sam invoke local
So all we need to do is introduce a custom build step that will replicate the build from the cdk synth
command — but only for a single Lambda.
But first, let’s understand how AWS CDK works (or you can jump over to the Custom Build section if you just need the solution).
Understanding CDK synth
Generic Knowledge
When you run cdk synth
or cdk deploy
, CDK will generate the cdk.out
output folder by default.
There are two files of interest there:
CdkLambdaTemplateStack.template.json
— this is the CloudFormation template that gets deployed. All of the resources (functions, buckets, roles, queues, etc.) are defined here.CdkLambdaTemplateStack.assets.json
— this is the config file that links the transpiled code (.js
files) to the CloudFormation template.
Node Specific Knowledge
Now, this cdk.out
folder also contains a bunch of asset.abc123...
folders that, in turn, contain the transpiled JavaScript Lambda handler code. These folders are referenced in the CdkLambdaTemplateStack.assets.json
file because they will be later archived and sent to S3 during the deployment step.
Why is that important for us? This means that our custom build
step should point to these asset folders as the output, and we will need to map Lambda names to their corresponding asset folders by parsing the CdkLambdaTemplateStack.template.json
file.
Now, let’s do that! In the next step, we will prepare a dummy CDK project with a single Lambda function that we will be transpiling.
Requirements and Preparations
Prerequisites
You need to have the following software installed:
- AWS CLI;
- AWS CDK CLI;
- AWS SAM CLI — for running Lambdas locally. Also requires Docker for
sam invoke local
; - Node 16+.
Initial Setup
Let’s create an empty CDK TypeScript project:
mkdir cdk-lambda-template
cd cdk-lambda-template
cdk init app --language typescript
# this command will read the name of the folder it is in, so your variables
# and files may have different names if the initial folder is called something
# other than "cdk-lambda-template"
Define a new Lambda function in lib/cdk-lambda-template-stack.ts
:
// lib/cdk-lambda-template-stack.ts
import * as cdk from "aws-cdk-lib";
import { Construct } from "constructs";
export class CdkLambdaTemplateStack extends cdk.Stack {
constructor(scope: Construct, id: string, props?: cdk.StackProps) {
super(scope, id, props);
const lambda = new cdk.aws_lambda_nodejs.NodejsFunction(
this,
"SampleLambda",
{
runtime: cdk.aws_lambda.Runtime.NODEJS_18_X,
entry: "lib/Lambdas/sampleFunction.ts",
functionName: "SampleLambda",
}
);
}
}
Note: I am using aws_lambda_nodejs
module instead of the aws_lambda
module because the former one uses esbuild
under the hood, which simplifies (and speeds up) the build process considerably. However, this also means that we need to install the esbuild
package into the project:
npm install --save-dev esbuild
If you don’t install
esbuild
, CDK will fallback on using Docker for bundling the code. If for whatever reason you want to usetsc
-compiled code, then you should not use theaws_lambda_nodejs
module and revert to theaws_lambda
instead.
Next, create a Lambda handler file at lib/Lambdas/sampleFunction.ts
:
// lib/Lambdas/sampleFunction.ts
import type { Handler, Context } from "aws-lambda";
// run `npm install --save-dev aws-lambda` to add Lambda types
export const handler: Handler = async (_event: unknown, _context: Context) => {
console.log("Starting the function!");
return {
result: "Done!",
};
};
Validate the steps by running cdk synth
.
Custom Build
Now we just need to write a create a custom build
step that will replace the output .js
file in the asset.abc123...
folder. There are about a million ways of doing that, but I have chosen to write a Node script (since we are in a Node repo!). The script expects two arguments: Lambda name and path to its handler source file. It can be invoked from the project root via:
node ./buildLambda.js SampleLambda ./lib/Lambdas/sampleFunction.ts
Here is the ./buildLambda.js
code:
// ./buildLambda.js
// 1
const OUTPUT_DIRECTORY = "./cdk.out";
const CDK_STACK_NAME = "CdkLambdaTemplateStack";
// 2
const template = require(`${OUTPUT_DIRECTORY}/${CDK_STACK_NAME}.template.json`);
const esbuild = require("esbuild"); // 3
const lambdaName = process.argv[2]; // 4
const lambdaHandlerSource = process.argv[3];
if (!lambdaName) {
console.log("No Lambda function name was specified in argv. Exiting...");
process.exit(1);
}
// 5
let targetLambda;
for (let resourceName in template.Resources) {
const resource = template.Resources[resourceName];
if (
resource.Type === "AWS::Lambda::Function" &&
resource.Properties.FunctionName === lambdaName
) {
targetLambda = resource;
break;
}
}
if (!targetLambda) {
console.log(`No ${lambdaName} lambda found in the CloudFormation template`);
process.exit(1);
}
// 6
const assetSuffix = targetLambda.Properties.Code.S3Key.split(".")[0];
const buildPath = `${OUTPUT_DIRECTORY}/asset.${assetSuffix}/`;
// 7
esbuild.build({
entryPoints: [lambdaHandlerSource],
bundle: true,
format: "cjs",
outfile: buildPath + "index.js",
});
It is rough around the edges, but it does the job. Some comments about each of the marked lines:
- These variables are hardcoded, but they can be either added to CLI if you have more than one CDK stack, or changed inline if you are using a non-default output folder;
- We dynamically import the template JSON file.
import
statements do not allow non-literal strings, so that’s why we have to userequire
; - We will use
esbuild
for transpiling the code, just like the CDKaws_lambda_nodejs
module; - I am reading the positional arguments from
argv
for the sake of simplicity, but you can use something more dedicated if you wish; - This is the core of the script. We parse the CloudFormation template JSON to filter only for Lambdas (by comparing the CloudFormation resource type to the desired one). Then we filter for the CLI-provided Lambda name. If we don’t find it, we exit the script.
6. Once we find the CF template for the desired Lambda, all we have to do is extract the asset suffix. It is stored Properties.Code.S3Key
(check the image). Once we have the suffix, we can compose the build output path;
7. And the last step is to invoke esbuild
with the generated output path and the provided handler source code path. bundle: true
and format: "cjs"
are set to replicate the preconfigured esbuild
settings from the aws_lambda_nodejs
module.
Testing
The script requires the CloudFormation template to exist, so you must build the project at least once first:
npm run cdk synth
Now we can invoke Lambdas locally by running these commands:
# 1. build the specific Lambda using our script
# the syntax is as follows:
# node ./buildLambda.js function_name path_to_function_handler
node ./buildLambda.js SampleLambda ./lib/Lambdas/sampleFunction.ts
# 2. invoke the Lambda via SAM CLI
sam local invoke SampleLambda --no-event -t ./cdk.out/CdkLambdaTemplateStack.template.json
The sam local invoke
follows the exact same syntax as the recommended cdk synth + sam local invoke
approach. If everything goes well, you should see this output:
Streamlining
There are many ways of running these two commands as a single script. For example, in Unix-based systems, you could use environmental variables to pass arguments to each of the individual commands. If you define a single script in package.json
to run both of the commands above like so:
"lambda:build": "node ./buildLambda.js $LAMBDA $SOURCE",
"lambda:run": "sam local invoke $SAM",
"lambda:local": "npm run lambda:build && npm run lambda:run",
Then you can execute both of the steps as a single command:
LAMBDA=SampleLambda SOURCE=./lib/Lambdas/sampleFunction.ts SAM="SampleLambda --no-event -t ./cdk.out/CdkLambdaTemplateStack.template.json" npm run lambda:local
Since there are many ways of achieving this (and some of them are OS-specific), I leave it up to you to decide which one you like most.
Conclusion
The goal of this article was to show how to speed up the local Lambda development process by bypassing the full project build step. It is unfortunate that CDK does not provide a native way of doing this. With that said, I think that the current script is not the cleanest, but it is very easy to set up since it doesn’t have any external dependencies.
As an alternative, you could consider using cdk watch
with hot-swapping, if you are OK with redeploying your target Lambda on every change.
Another option is to use the Serverless Framework if you prioritize optimizing Lambda development workflow over architecture development workflow (Serverless uses raw CloudFormation with occasional syntactic sugar). When running serverless invoke local -f SampleLambda
, Serverless Offline Plugin will build the target function (either with webpack
or esbuild
) before invoking it. It also doesn’t require Docker! However, make sure to first consider using version 3 for work, as the newly-introduced version 4 now adds mandatory revenue-sharing for larger organizations among other features.
Hope you find this helpful! Check out my other related articles: