Skyscanner’s products are powered by hundreds of services hosted on AWS. In order to deploy changes and new services to production with zero clicks, we have an automated pipeline that´s responsible for building, testing and deploying new code, and provisioning and configuring new infrastructure. Developers perform these changes by writing CloudFormation templates that model their service’s Infrastructure as Code (IaC).
CloudFormation is an AWS service that essentially allows developers to programmatically provision AWS resources. CloudFormation templates could have security issues similar to regular source code, such as hardcoded secrets, overly permissive permissions, and many more.
Our goal at the Security Automation team is to inject security into the pipeline as early as possible, and to make sure that the relevant security scans and audits are performed at every step, minimising the risk of any vulnerable code getting into production.
In the past, we introduced CFRipper, a Python tool we developed that aims to prevent those vulnerabilities from getting to our production infrastructure. You can read more about this tool in this article. While using CFRipper we stumbled across a few challenges on working with CloudFormation and making sure that our coverage was high.
CloudFormation dynamic syntax
CloudFormation, although not a fully fledged programming language, is rich and dynamic. In order to make templates reusable and customisable, developers use parameters, mappings, conditions, intrinsic functions and conditions sections in their templates.
For example, developers may want to use different configurations depending on the environment. The template below would attach a different policy; depending on the account where the cloud formation is going to be executed.
This flexibility allows developers to manage their infrastructure in a better way, making it easier to perform potentially challenging and time-consuming tasks.
CFRipper and other CloudFormation template scanners perform static analysis. During static analysis the CloudFormation resources are still not created or updated, and neither parameters, functions or conditions are evaluated.
Therefore, if the template scanner isn´t able to handle these sections or components, they will be ignored. This opens a large window for an attacker to bypass CloudFormation scanners.
Bypassing CloudFormation scanners with intrinsic functions
Let’s craft a simple template that grants anyone permission to perform any action on any resource in an AWS account, effectively making anyone an administrator:
Obviously, this template is flagged as non-conformant by most of the CloudFormation scanners available out there:
- Example of cfn-nag (another Open Source project, which inspired CFRipper):
- Other commercial scanners detect the issues correctly.
We can observe that scanners detect the issue in the CloudFormation stack. As an example of how easily a static CloudFormation scanner can be bypassed, let’s now write the same template but using intrinsic functions. You can use intrinsic functions in resource properties, outputs, conditions, etc. For example, using the sub function we can substitute variables in a given string.
In the template below, the wildcard character is defined within a sub function:
If we try to analyse the above template using the same scanners, no issues are detected:
- CFRipper without using the new “resolve” option:
- Other commercial scanners fail to detect the issue.
A template using intrinsic functions is valid; intrinsic functions are not a security issue and are regularly used by our developers. Therefore, not allowing them will undermine their capabilities to perform infrastructure changes at scale.
Nevertheless, our developers are continuously using constructions such as:
That will make AWS at compute time resolve it as ‘test-stack-lambda-role’, considering `AWS::StackName=test-stack`:
Do you see the problem? Without becoming aware, developers are bypassing CFRipper, undermining our efforts to stop insecure resources being created in our production environments — and therefore potentially exposing a greater “attack surface” without us even noticing it.
Since our first CFRipper article was published, we’ve been working on improving CFRipper to expand its coverage and make sure we prevent more insecure stacks to going into production. One of the main areas of work has been implementing our own CloudFormation template resolver in pycfmodel (https://github.com/Skyscanner/pycfmodel), which is used by CFRipper to model CloudFormation templates.
Pycfmodel now handles conditions, parameters and intrinsic functions out of the box. Before CFRipper analyses a template, pycfmodel will resolve all parameters, conditions and intrinsic functions, then generate a new model.
Using the new model, we can see how the same vulnerabilities are now reported (notice the CLI option — resolve):
Let’s take a quick look at the new features implemented by pycfmodel and CFRipper:
- CFRipper CLI: Besides running the tool as a linter step of their CI system, you can now run CFRipper on the command line to perform a quick scan on a CloudFormation stack. This makes the tool more accessible to a wider group of users.
- Intrinsic Function resolver + Smarter Condition resolver: In the past, we were blind when a function was used. Now, we are able to resolve most of them. This allowed us to improve the condition resolver, we decided to analyse both sides of an if / else statement. Other approaches just check the true section. But in the last update, we are able to resolve conditions at the very beginning, so we can just evaluate section that is going to be evaluated.
- AWS default parameters: AWS has some special parameters called pseudo parameters (prefixed by `AWS::`). These are automatically defined and don’t need to be passed. (A complete list can be found here: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/pseudo-parameter-reference.html)
- Secrets handling in template parameters: Instead of using a raw template parameter, users can get a placeholder to prevent exposing the secrets. Some new rules take advantage of this to warn the user if the value will be printed in AWS logs or not.
The current open-source version of CFRipper contains a set of built-in rules ranging from to. We’ll continue to add any new rules that we develop in-house, but we’ll also appreciate any rules that you might have developed while using CFRipper. Please read the contribution guide and submit a pull request.
About the authors:
Oscar Blanco Castan
Oscar recently completed the Graduate programme at Skyscanner as a software engineer. He’s passionate about security, and currently works in the Platform Security and Automation team, developing services to automate the security processes in Skyscanner.
Xavi Mendez is a principal security engineering at Skyscanner,
leading the security automation team that builds security in our DevOps pipeline and our AWS and Kubernetes infrastructure.
Join Skyscanner, see the world
Life-enriching travel isn’t just for our customers — it’s for our employees too! Skyscanner team members get £500 (or their local currency equivalent) towards the travel trip of their choice in 2020 — and that’s just one of the great benefits we offer. Read more about our benefits and have a look at all of our open roles right here.