Introducing LambdaGuard — a security scanner for AWS Lambda
We’ve built a tool which allows you to visualise and audit the security of your serverless assets — and now we’re open-sourcing it, too
Skyscanner is well known for building great products for travellers — but today I want to talk about something we’ve built for the cloud and security communities: LambdaGuard.
Some context: we are a mobile first organisation and provide our app on all major platforms. We operate our production services in Amazon’s Cloud, and in part rely on serverless architecture to deliver our products. Prior to developing LambdaGuard, and getting to where we are today, we had some challenges to overcome: we needed a way to get visibility of our serverless assets, and we had to be able to perform a security audit on those we had deployed in production.
To come up with some ideas for getting over these hurdles, we recently ran an internal security hackathon to brainstorm ideas. The product of this hackathon was LambdaGuard — an AWS Serverless Security auditing tool — which was recently open-sourced and presented at BSides Barcelona 2019.
In this article I will review how Lambdas work and describe the main security challenges around them. I will also show how LambdaGuard can help you understand the security posture around your Lambdas and recommend how you can protect them.
Lambdas
AWS Lambda is an event-driven serverless computing platform, and is responsible for implementing custom code for processing data in response to events from other components. Lambda code is automatically scaled with high availability, and runs in an isolated environment with a read-only filesystem - typically for a few seconds (with a 15 minute runtime limit).
Versions and aliases
Lambdas have versioning and aliases. You can upload multiple versions and create human-readable tags (called aliases) to reference each individual version.
Layers
Lambdas have layers, which is a way to avoid common code duplication across similar functions. For example, you could publish a layer with metrics tracking logic, and then simply reference this layer from several Lambda functions that all share this functionality.
Resource-based policy
Lambdas have an access policy (aka Resource-based policy), which is a document that specifies who can use a function and how they can use it. This is where event source mapping is configured.
Execution Role policy
Finally, and importantly, Lambdas have an execution role. This is an IAM role that controls function access to AWS resources.
Serverless services — examples
Here is a simple example from AWS documentation where the user uploads a file and that file is then automatically processed by a Lambda function.
The user uploads a file to an S3 bucket. That bucket has a notification configured to trigger Lambda code when a new file is uploaded. Triggered code runs with an execution role that gives access to resources, and has an access policy that specifies who can trigger it.
A more interesting example would be that of a microservice:
In order to set-up a serverless service developers must define which components are to be used (i.e. Lambda code, API, DNS, database, static web pages, etc) and define permission policies regarding how these components interact with one other. An important observation regarding how serverless tends to be deployed can be traced back to this definition process. Developers want to go fast, and AWS certainly allows them to do that. However, as a side-effect of speed we see that many developers make mistakes defining Resource-based and Execution role policies - usually resulting in an “allow all actions for everybody” approach. This is an easy shortcut to get serverless components to communicate with each other. Nonetheless, while it may be impractical to define a strict policy from the beginning in an agile development cycle, policies should definitely comply with a secure production standard when they are published.
AWS’ Shared Responsibility Model states that Amazon is responsible for the security of its cloud, but that customers are responsible for security in the cloud - meaning that it is your responsibility to setup IAM roles, configure access permissions and deploy secure code.
Policy configuration and maintenance is a unique serverless challenge. Lambdas that rely on insecurely configured components can be exposed to attacks and malicious content injections.
The following sections will give an outline of important policy elements and provide several common pitfalls that affect serverless services.
Policies
Definition
A policy is an AWS object that defines permissions for an associated identity or resource. The following key-value pairs define a policy statement:
Common pitfalls
Unrestricted Actions
Actions specify permissions. For example, the following Action explicitly refers to publishing messages to an SQS queue:
“Action”: “sqs:SendMessage”
Wildcards can be used to reference all Actions:
“Action”: “sqs:*”
Using wildcards when allowing Actions is insecure because it does not follow the principle of ‘least privilege’. If a service is only meant to be allowed to publish messages to a queue, the policy document should explicitly grant that specific permission only. It is not likely that a service would ever require permissions for all possible actions.
Unrestricted Principal
Principal specifies which entities are affected by the policy. For example, the following Principal specifies an IAM user:
“Principal”: { “AWS”: “arn:aws:iam::AWS-account-ID:user/user-name” }
Wildcards can be used to reference all possible entities:
“Principal”: “*”
Using Principal as a wildcard when allowing actions is insecure because it allows unauthorized access. This typically means that the service will be publicly accessible unless an additional condition is defined in order to restrict access.
Undefined Condition
Conditions specify when a policy takes effect and are optional, unlike other key policy elements. For example, the following Condition specifies that the policy takes effect when requests are coming from the given IP range:
“Condition” : { “IpAddress” : { “aws:SourceIp” : [“192.0.2.0/24”] } }
Not defining a Condition is insecure when allowing Actions with an unrestricted Principal because it allows unauthorised Access. Defining a Condition is useful in cases where there are too many Principals to specify explicitly, or when Principal + wildcard is the only option. The Condition can restrict access to an internal subnet or an otherwise logical subgroup of characteristics.
Other Pitfalls
There are many ways to misconfigure a policy:
- Allow NotPrincipal
- Allow NotAction
- S3 ACLs
- Using wildcards everywhere — there is really nothing stopping you from doing it
- And so on…
Serverless vulnerabilities — some examples
Public S3 buckets
S3 buckets are frequently misconfigured so that their contents are accidentally exposed to a wider group than was intended.
Let’s say you have a couple of files to share publicly via an S3 bucket. But instead of granting per-file permissions, access is granted to the entire bucket. Now anyone will be able to read all the files in the same bucket.
Another common misconfiguration is granting permissions to the Authenticated AWS Users group thinking that it refers to users in your own account. But it actually refers to ANYONE with an AWS account. This example then leads to similar vulnerabilities related to sensitive data exposure as in the previous example.
Writable buckets can be abused for storing and distributing malware, phishing, and in some cases client-side code injection attacks such as XSS, CSRF and so on.
The following policy will grant complete public access due to unrestricted Principal and unrestricted Action…
…but it can be secured by explicitly specifying Principal and allowed Actions:
Public SQS queues
Public SQS queues are less common since they are not as widely used as S3 buckets (and for this reason are often overlooked). Unless queues are correctly restricted, external unauthenticated users may be able to consume and publish items, among other things. Applications that use messages published to a queue for displaying content to the end-user can be attacked by directly publishing content of an unexpected type, such as injecting a raw Celery task for pickle attack (used in http://andresriancho.github.io/nimbostratus/pivoting-in-amazon-clouds.pdf). On the other hand, having the ability to consume messages from a public queue can lead to sensitive data disclosure in applications that queue personal information files for processing.
The following policy will grant public read access due to unrestricted Principal and an undefined Condition:
It can be secured by defining a source ARN Condition:
Public API Gateway
Publicly exposed API Gateway endpoints for internal services are also often a result of policy misconfiguration. API Gateway creates unauthenticated endpoints by default. So unless the endpoint is meant for public usage, this becomes a problem for services that are used by employees or are meant for backend service-to-service communication.
The following policy will grant public access due to unrestricted Principal and undefined Condition lines:
It can be secured by defining a SourceIP Condition:
LambdaGuard: AWS serverless security
LambdaGuard is an AWS Serverless Security auditing tool designed to provide asset visibility, illustrate service dependencies, and configuration checks from a security perspective.
LambdaGuard:
- Takes Lambda function ARNs as input
- Maps event sources that trigger function execution
- Maps resources used by the function
- Tracks demographics (runtimes, regions, etc.)
- Checks function policies and Execution Role policies
- Checks event source policy and configuration
- Checks resource policy and configuration
- Optionally runs static code analysis on function source code (using SonarQube)
- Provides JSON and HTML reports
LambdaGuard is made to be used as a CLI tool as well as a Cron job for continuous monitoring. JSON reports are made to be easily consumable by other services. HTML reports visually represent JSON data.
Reporting
JSON
JSON reports are made to be easily consumable by logging and analytic services.
HTML
LambdaGuard HTML reports provide a visual representation of JSON reports.
Discussion and conclusion
There are many ways in which LambdaGuard can be applied. Gain visibility of how many functions you currently have in production and confirm that they are running on officially supported runtimes. Identify policy and configuration vulnerabilities in event sources and resources that your function depends on. Identify hardcoded credentials and code injection vulnerabilities in function code by combining with SonarQube. And more.
We use LambdaGuard for detecting serverless vulnerabilities, and then convert them to CFRipper rules for preventing future insecure deployments to production. In this way we complete the security review cycle by identifying, mitigating and preventing threats within our serverless environment.
The source code for LambdaGuard is available at https://github.com/Skyscanner/LambdaGuard.
Join Skyscanner, see the world
Life-enriching travel isn’t just for our customers — it’s for our employees too! Skyscanner team members get £500 (or their local currency equivalent) towards the travel trip of their choice in 2019 — and that’s just one of the great benefits we offer. Read more about our benefits and have a look at all of our open roles right here.
About the author
My name is Artëm Tsvetkov and I am part of the Product Security Squad in Barcelona, where we are dedicated to securing awesome Skyscanner traveler products. I spend a lot of my time working with Amazon Web Services and sincerely enjoy hacking. Outside of work I am an avid CTF player, do kickboxing, play guitar, read and enjoy cultural activities like sightseeing, museums and expositions.
