Using AWS X-Ray to achieve Least Privilege Permissions

Run-time monitoring via AWS X-Ray can help with achieving Least Privilege IAM Permissions. We recently released a new tool to monitor X-Ray traces and generate Least Privilege IAM Policies.

6 min readJan 20, 2018

I’m pleased to announce that we’ve recently released a new open source project “AWS Least Privilege”. The project aims to streamline the process of collecting run-time resource usage information from X-Ray and reaching a Least Privilege IAM Policy for a given application. Project is available on GitHub at: https://github.com/functionalone/aws-least-privilege.

AWS X-Ray provides in-depth information about service API calls executed via the AWS SDK. Using this information, it is possible to build a profile of the AWS resources and actions that are actually used by an application (or part of an application, such as a Lambda function) and generate a precise policy document reflecting what resources are used and how they are accessed.

In this article, I’d like to take the chance to walk through the process of enabling X-Ray and using our new open source tool to reach a least privilege IAM policy. We’ll start with a demo Serverless application from Serverless Stack, which uses Lambda and DynamoDB for the back-end. The application with a full tutorial is available at: serverless-stack.com. I won’t go into the full setup process as this is documented very nicely at serverless-stack.com.

Once setup. you will have 5 lambda functions defined each using a single shared IAM role named: notes-app-api-prod-us-east-1-lambdaRole. The role has an in-line policy which uses the following allow statement to grant DynamoDB access:

{
 "Action": [
  "dynamodb:DescribeTable",
  "dynamodb:Query",
  "dynamodb:Scan",
  "dynamodb:GetItem",
  "dynamodb:PutItem",
  "dynamodb:UpdateItem",
  "dynamodb:DeleteItem"
 ],
 "Resource": "arn:aws:dynamodb:us-east-1:*:*",
 "Effect": "Allow"
}

This IAM configuration has a few issues. First the allow statement grants “*” access to all DynamoDB tables and second the role is shared between multiple Lambda functions. Creating a shared role is the default option when using the serverless framework, but is not best practice when trying to achieve a least privilege setup. With least privilege, the goal is to run each Lambda function with a role granting only the minimal permissions required for the function to operate and avoiding a wide set of permissions that is common to all functions. Let’s see how we can improve this using run-time information from X-Ray.

Enabling AWS X-Ray

First step for collecting run-time information is to configure AWS X-Ray. Enabling X-Ray is a two step process:

First, enable “active tracing” configuration on the Lambda functions. This can be done via the AWS Console UI as documented here: https://docs.aws.amazon.com/lambda/latest/dg/enabling-x-ray.html. But, since the project is using the Serverless Framework, I suggest doing this as part of the serverless.yml setup. You will need to add to the serverless.yml file the permission statement:

Then there is need to add a Resource configuration for each function to enable active tracing:

Note: there is also a plugin which makes the configuration even easier when working with a large number of functions. You can check it out here: https://github.com/alex-murashkin/serverless-plugin-tracing.

Second, we will need to install the X-Ray SDK and initialize X-Ray. Installing X-Ray is done via npm:

npm install --save aws-xray-sdk

Then we need to initialize X-Ray in the code to capture all AWS SDK calls. This can be done by adding the following code snippet during function startup (I added it to dynamodb-lib.js):

import AWS from "aws-sdk";
import XRay from 'aws-xray-sdk';
XRay.captureAWS(AWS);

To see all the changes together, I’ve created a branch with the changes which is available here: https://github.com/glicht/serverless-stack-demo-api/tree/add-xray.

After doing these changes there is of course need to re-deploy the functions using:

serverless deploy

We now have X-Ray configured and tracing the back-end application. After using the application we can go to the X-Ray console and see the service map which is generated:

We can also view specific traces. For example, here is the trace for the “update” function:

From looking at the trace it is obvious that the update function uses only the “UpdateItem” action on the “notes” table and doesn’t require the full permissions granted in the serverless.yml file. Now let’s see how we can use this information to create least privilege IAM policies.

Scanning X-Ray to achieve Least Privilege

First we will need to install the aws-least-privilige npm:

npm install -g aws-least-privilege

This will install the command line tool: xray-privilege-scan.

We can now run the tool to scan X-Ray traces and generate IAM policy documents per Lambda function. We can run the tool with no parameters and it will scan the last 60 minutes of X-Ray traces (for a full list of options run: xray-privilege-scan -h). Running the tool will produce output of the form:

$ xray-privilege-scan
Completed running xray scan.
Generated IAM policies based upon xray scan:
arn:aws:lambda:us-east-1:xxxx:function:notes-app-api-prod-delete - 0bdcf1c154bddd684294aec1334a9b72.policy.json
arn:aws:lambda:us-east-1:xxxx:function:notes-app-api-prod-create - 593020e43d8bf0737daecb471c81b698.policy.json
arn:aws:lambda:us-east-1:xxxx:function:notes-app-api-prod-list - 2e166200fc364a6552f5934489b512dc.policy.json
arn:aws:lambda:us-east-1:xxxx:function:notes-app-api-prod-update - 711a58cf1c53a4f14d9193a0b8c8fd4d.policy.json
arn:aws:lambda:us-east-1:xxxx:function:notes-app-api-prod-get - c577c60d2945d3b7989bd3d538fd0cf5.policy.json

The tool will create a policy doc named <unique-id>.policy.json per Lambda function. We can take a look at the policy doc for the “update” function:

We now have a policy document which describes the resources and actions used by the Lambda “update” function. The policy document can be used directly in the AWS console, but take note that it contains an extra Description field which needs to be removed if doing a copy-paste. Since our sample application is using the Serverless Framework, it is probably best to incorporate the generate policies in to the serverless.yml file. You can check out the following serverless.yml file which I’ve done this for here: https://github.com/glicht/serverless-stack-demo-api/blob/least-privilege/serverless.yml.

A few notes about how the xray-privilege-scan tool works:

The tool takes into account multiple traces for the same Lambda function and will merge the resources accessed (and associated actions) into the generated policy.
The tool will try to group together resources of the same type which are accessed by the same set of actions. For example, if two S3 buckets are both accessed with actions GetObject and PutObject a policy statement of the following form will be generated:

{
  "Effect": "Allow",
  "Action": [
    "s3:GetObject",
    "s3:PutObject", 
  ],
  "Resource": [
    "arn:aws:s3:::test-bucket1/*",
    "arn:aws:s3:::test-bucket2/*"
  ]
}

What’s Next

We’re just starting with this project and we would love to hear suggestions and feedback. Please feel free to open an issue on the Github project page with suggestions, bugs, and questions.

Update (Mar 17, 2018): There is now a follow-up, part 2, to this post: