Lambda@Edge gotchas and tips

Lambda@Edge is a powerful tool that lets you customise CloudFront request and response handling. It is also extremely frustrating to use for first time users, and for those who haven’t used it in a long time.

Here’s a list of gotchas and tips I’ve found useful over my time with the service. Hopefully this’ll be of help to someone working with Lambda@Edge.

Without further ado:

Setup

When creating Lambda functions for use with Lambda@Edge, they must conform to these requirements:

• The function must be created in the us-east-1 region.
• The function must not have environment variables.
• The timeout can be at most 5 seconds. serverless defaults to 6s.
• The max memory can be 128 MB. serverless defaults to 1028 MB.
• The function can’t be placed inside VPC.
• You must use nodejs6.10 or nodejs8.10 runtime.
• The AssumeRolePolicyDocument must allowedgelambda.amazonaws.com to execute sts:AssumeRole
• You need to publish a new version after each code change. You can’t reference the$LATEST$ version when associating the function with one of the event types.

CloudFront can catch most of these issues in the validation step, but in order to avoid the frustrating 30 minute wait while updating CloudFront distribution, it’s better to make sure all these are in check before continuing.

Start by logging. Then log more.

One of the best tips I can give is to start any Lambda@Edge project by creating separate functions for each of the four handler types. In each of them, log the entire event to console and call the callback with the request or response you were given in the event.

Why all of them? To see how each one behaves. I get surprised by how different the viewer request and origin request events behave every time I try to do something. It’s likely you don’t even get the event type right the first — or even second time — you try to do something with Lambda@Edge.

While developing your function, add generous amounts of logging. console.log absolutely everything: event payloads, variable values, anything that comes to mind. There’s no debuggers here. Remove the excess logging once you’re confident everything works.

Find the logs

You’ll find soon enough that Lambda@Edge logs are not necessarily delivered to the same region as the function is created at (us-east-1). This is because CloudFront internally clones the lambda functions to multiple locations for low-latency invocations across the globe. Because of this, the logs are delivered to the AWS region closest to the CloudFront edge you’re hitting.

This makes accessing the CloudWatch logs for the Lambda@Edge functions somewhat painful, as you can’t be sure which CloudFront edge you’re hitting and what AWS region might be closest to that edge. The logs are there, you just have to find the proper region.

So, you’re up for some digging. Just open the AWS CloudWatch Logs console and go through the locations. It’s likely that if AWS has a region on your continent, the logs are in one of the continents regions.

Test locally

As CloudFront updates can be notoriously slow — it can often take 30 minutes for distribution updates to finish — it’s essential that you figure out a way to locally test the code.

Don’t try to create the functions in CloudFormation stack with inline code. Instead, use something like serverless to manage and deploy the functions and write normal unit tests for your functions using your favorite framework. From the previous logging step you should be able to figure out the event payloads, so testing the functionality should be easy. Just invoke a function, expect the callback to be called.

Headers in viewer vs origin requests

Sometimes choosing between viewer and origin request types can be hard. If you need to work with request headers in your Lambda@Edge function, these rules of thumb might come handy and guide you in choosing the right one:

Viewer request functions:

• See all headers sent by the client. No filters.
• Can add request headers to be forwarded to origin.
• Don’t see the Origin Custom Headers.
• Host header value is the requested host name.

Origin request functions:

• See only headers whitelisted in the cache behaviour.
• Can also add request headers to be forwarded to origin.
• Sees the Origin Custom Headers defined in origin settings.
• Unless Host is whitelisted, the value will be the origin’s host.

Passing original Host to origin request handler

In some scenarios it might be handy to know the original requested host in the origin request function. The only way to directly do this is to set up the cache behaviour to forward Host header. This is problematic, because you might not always want the origin to receive the request with the original host. And with S3 origins, you just can’t forward the Host header.

In this case, you could use viewer request function to addX-Forwarded-Hostheader to requests forwarded to the origin and thus also to the origin request function:

module.exports.viewerRequest = (event, context, callback) => {
  const request = event.Records[0].cf.request;
  const host = request.headers["host"][0].value;
  
  request.headers["x-forwarded-host"] = [{
    key: "X-Forwarded-Host",
    value: host
  }];
  callback(null, request);
};

And remember, you also need to add X-Forwarded-Host to the headers forwarding whitelist.

Faking environment variable support with Origin Custom Headers

As mentioned in the setup section, it’s not possible to configure environment variables with Lambda@Edge functions. CloudFront won’t accept those and will fail to create/update the distribution if any are configured.

This is a shame, because it’s ugly to hardcode this stuff in the function code. Further, it’d be nice to write reusable middleware functions that can be attached to all distributions. Often this requires some form of customisation on per distribution basis — be it toggling off a feature, configuring a domain name or user/password combination for basic auth, or something else.

What we’d like to have then is a way to parameterise the lambda invocation somehow per distribution, without having to hardcode host names and configurations in the lambda function code.

It’s possible to do so through Origin Custom Headers feature. A word of caution though: this is only an option if your use case is doable with origin request and/or origin response functions. The Origin Custom Headers, as mentioned above, are not forwarded to the viewer request function. This is simply because CloudFront, at the time of invoking the viewer request function, haven’t chosen the origin to connect with, and thus doesn’t know which origin’s custom headers it would forward.

In most cases though you can replace the use of viewer request function with origin request. And it might be best to do so for performance reasons anyway — remember, origin request function only runs on cache misses whereas viewer requests run on every request.

Anyway. Configure custom headers under the origin settings:

Then you can access them in your lambda function code via request.origin.s3.customHeaders or request.origin.custom.customHeaders object depending on the origin type:

module.exports.originRequest = (event, context, callback) => {
  const request = event.Records[0].cf.request;
  const customHeaders = request.origin.s3.customHeaders;
  const acceptedUsername = customHeaders["x-env-username"][0].value;
  const acceptedPassword = customHeaders["x-env-password"][0].value;
  const authorization = request.headers["authorization"][0].value;
  
  // Remove the authorization header to prevent it from being
  // forwared to origin
  delete request.headers["authorization"];
  if (!validAuthorization(authorization, acceptedUsername, acceptedPassword) {
    callback(null, { status: "401", statusText: "Unauthorized" });
    return;
  }
  callback(null, request);
};

My CloudFormation workflow

If you’re a good citizen and manage your CloudFront distributions via CloudFormation template, changes are you might be wondering how exactly you’re supposed to create the lambda functions, publish their versions and manage the distribution function associations.

Everyone has their preferences, but here’s how I do it:

1. Manage the functions separately with serverless. This will create the functions in separate stack.
2. Manage the CloudFront distribution in another stack parameterised by the full ARN of the lambda function version.

To update, I first runserverless deploy. This will update the lambda functions and publish new versions. Then, you can retrieve the full ARN of the lambda function using the following command:

aws lambda list-versions-by-function \
  --function-name SERVLESS_STACK_NAME-functionName \
  --region us-east-1 |jq ".Versions[-1].FunctionArn" -r

Now that you know the ARN of the latest version, you can do a separate deploy of the stack containing CloudFront distribution by usingaws cloudformation deploy and passing in theOriginRequestLambdaVersionARN parameter.

It’s easy to script this process into one shell script. And I’m sure you could hack serverless to export the latest function ARN in the stack outputs and then use CloudFormation Fn::ImportValue to import the value in your CloudFront stack.

And finally, some ideas on what you can do with Lambda@Edge

• Do a naked domain to www redirects. Or vice versa.
• Add a prefix to origin paths behind the scenes. For example, if you want to map / to S3 origin with key prefix site/ , you can do that here. Or let’s say you want to serve different site to visitors from other countries.
• HTTP Basic auth, session cookie validation, …
• Serve index.html when requesting /some/sub/folder(CloudFront has support for Default Root Object, but this only applies to the root path /, not subfolders)
• Serve different favicon based on requested host name.
• Host a simple web application. Just make empty S3 bucket the origin and always generate response from origin request function.
• Add Strict-Transport-Security, Content-Security-Policy, X-Frame-Options etc. headers to S3 origin responses to make your site a bit more secure