Enhance AWS Lambda Monitoring by enable Lambda Insights

4 min readAug 14, 2021

Monitoring your Lambda resource on AWS CloudWatch by enable Lambda Insights

I used to develop applications with GCE or GKE, it has some common metrics like CPU Utilization, Memory Utilization …, now I’m trying to use AWS Lambda to develop applications but can’t find these common metrics, so I start searching solutions and found that I need to using Lambda Insights to monitoring AWS Lambda.

In this article I will show you:

Background of my use case
What is Lambda Insights
Option A: enable Lambda Insights on AWS Console
Option B: enable Lambda Insights with Terraform
One more thing: what is the exact size of Lambda Insights Extension Layer

Background

Here’s a simple infrastructure intro of my application:

Use API Gateway with Lambda as an API Server
Use Terraform to manage all resources

Lambda provides some metrics in CloudWatch like ConcurrentExecutions, Duration, Errors, Invocations …, but I need more metrics to represents the overall service health like CPU usage, Memory usage and Network usage. I think the most important part is the Memory usage, with this metric I can understand if my current implementation has memory leak issue, or I can determine how much memory should I setup for my service based on the metric.

What is Lambda Insights

From AWS Documentation:

CloudWatch Lambda Insights is a monitoring and troubleshooting solution for serverless applications running on AWS Lambda. The solution collects, aggregates, and summarizes system-level metrics including CPU time, memory, disk and network usage. It also collects, aggregates, and summarizes diagnostic information such as cold starts and Lambda worker shutdowns to help you isolate issues with your Lambda functions and resolve them quickly.

Simple and straightforward, it gives you more visions for troubleshooting, looks quite beneficial but one thing need to be aware of is it may have addition costs.

Option A: enable Lambda Insights on AWS Console

There’s a simple instructions on AWS Documentation, I’ll show you with pics below:

(1) Navigate to your Lambda Function on AWS Console, and choose the Configuration tab, navigate to Monitoring and operations tools pane and click Edit button.

(2) Under CloudWatch Lambda Insights, turn on Enhanced monitoring.

(3) After enabled this configuration, testing your Lambda Function and wait for a while, the metrics will start showing up on CloudWatch / Lambda Insights / Performance monitoring page.

Option B: enable Lambda Insights with Terraform

Since I use Terraform to manage all AWS resources, I definitely want to enable Lambda Insights with Terraform, since Lambda Insights uses a new CloudWatch Lambda extension, which is provided as a Lambda layer, we need to do two things to achieve our goal:

(1) Add a layer to your Lambda Function Resource with arn of LambdaInsightsExtension

arn:aws:lambda:ap-southeast-1:580247275435:layer:LambdaInsightsExtension:14

Version 14 is latest one when I wrote this article, you can find latest version of extension here: https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/Lambda-Insights-extension-versionsx86-64.html

(2) Attach a role policy to your Lambda execution role with policy arn

arn:aws:iam::aws:policy/CloudWatchLambdaInsightsExecutionRolePolicy

An example of terraform file will looks like below

One more thing: what is the exact size of Lambda Insights Extension Layer

Now Lambda start populating metrics, all looks good. But one thing comes to my mind in a sudden, if Lambda Insights occupied a Lambda Layer, it will increase the Deployment package size, to preventing it hits the quota limitation of Deployment package size, I should know what is the exact size of it.

I start searching for AWS documentation about the size of Lambda Insight Extension but can’t find any, I decide to submit a support ticket for this technical question. Thanks for Guozhi’s help from AWS Support, he’s not only giving me answer but also some instructions to find the answer.

Lambda Insight Extension is about 4.32 MB in compressed, and it’s about 11.19 MB when uncompressed. Here’s the way to find it out:

(1) Run command to get layer information

$ aws lambda get-layer-version-by-arn --arn arn:aws:lambda:ap-southeast-1:580247275435:layer:LambdaInsightsExtension:14

(2) The response may looks like below

{
    "Content": {
        "Location": "https://awslambda-ap-se-1-layers.s3.ap-southeast-1.amazonaws.com/snapshots/580247275435/LambdaInsightsExtension-fd478df8-820d-456d-8315-3424d93b9698...",
        "CodeSha256": "...",
        "CodeSize": 4323428
    },
    "LayerArn": "arn:aws:lambda:ap-southeast-1:580247275435:layer:LambdaInsightsExtension",
    "LayerVersionArn": "arn:aws:lambda:ap-southeast-1:580247275435:layer:LambdaInsightsExtension:14",
    "Description": "CloudWatch Lambda Insights Extension.",
    "CreatedDate": "2021-01-14T16:54:45.130+0000",
    "Version": 14,
    "LicenseInfo": "Apache-2.0"
}

The CodeSize property show the compressed size of layer, and if you copy the value fromLocation property, paste it to your browser, it will start download the layer from AWS, this link is valid for 10 minutes after you run the command.

(3) Now you get the actual file of Lambda Insight Extension, uncompress it and check out whole folders information, now you know how big is it.

I remember that I was attended a conference before, a speaker from DigitalOcean said:

No monitoring, no production.

I always keep this in my mind after that conference.