Using IAM Roles Anywhere To Access S3 With The Golang SDK(2023)

Background

The Release of IAM Roles Anywhere

On July 6, 2022 Amazon Web Services (AWS) announced a new addition to their Identity and Access Management (IAM) service called “Roles Anywhere”.

Then, in the July 14, 2022 update to the IAM Security Best Practices, AWS added the recommendation to “Require workloads to use temporary credentials with IAM roles to access AWS”. This recommendation was meant to end the practice of issuing longstanding access key IDs and secret access keys to external workloads for them to interact with AWS services. Further, this recommendation states that “For machines that run outside of AWS you can use AWS Identity and Access Management Roles Anywhere”.

Additionally, AWS added this recommendation to the Security pillar of the Well Architected Framework, directing users to “Use temporary credentials to sign programmatic requests to the AWS CLI, AWS SDKs, or AWS APIs”.

Clearly the use of temporary credentials and Roles Anywhere is important to AWS security, but what exactly is it? Why is it important? How can we use it to access AWS services programmatically with our external workloads?

What is IAM Roles Anywhere?

IAM Roles Anywhere is a service from AWS that allows external workloads to acquire temporary credentials to interact with AWS services. If you’ve ever written an API, database driver, data pipeline, or any code that interacts with AWS but is not hosted on AWS, you should be using Roles Anywhere.

Roles Anywhere works by exchanging X.509 certificates between the external workload and AWS to establish trust. In order to do this you need to register a Root CA on AWS as a “trust anchor”. This can be done using an AWS Private Certificate Authority or by providing your own external certificate bundle. Using public certificate authorities for this is not permitted. For this guide, we’ll be providing our own. Once trust has been established between your workload and AWS, you can configure a role and attach it to a profile to provide permissions through the temporary credentials that are granted.

Why Should We Use IAM Roles Anywhere?

The AWS re:Inforce 2022 Conference had a great talk hosted by Bridgid Johnson and Matt Luttrell on security best practices in IAM where they explain the benefits of using temporary credentials. I recommend watching this talk all the way through not just to learn about Roles Anywhere but to hear about all of the best security practices to use when using AWS.

The main benefits highlighted in this talk included the following:

1. Temporary credentials have limited lifetime and automatic expiration
   This fact is important as it greatly reduces the potential attack surface compared to long standing credentials provided by traditional access key IDs and secret access keys. Malicious actors can only do so much harm with credentials that expire quickly compared to those that last indefinitely. This also helps with the principle of least privilege as workloads are only given permission to do what they need to do when they need to do it. There is no reason for an application to have these permissions when they are not using them. This process also makes it easier to track activity within an AWS account as the system records when credentials are requested, the success of credential requests, and other data that make it easier to audit activity.
2. Temporary credentials eliminate the need for credential distribution and storage
   This is important for any person or organization that uses AWS on a regular basis as it greatly simplifies the process of key rotation. We no longer need to keep track of all our workloads’ long standing keys and when they expire in order to effectively replace them year after year. This also makes applications more secure as access keys will no longer be stored on local servers in config files or within code. This means that even if someone were to breach the codebase they wouldn’t automatically have access to your AWS services.

Using Roles Anywhere To Access AWS S3 Using the Golang SDK: A Practical Example

In this example we’ll use Roles Anywhere to gain temporary credentials to access S3 in order to read a list of buckets that are stored on the account.

Prerequisites

This guide assumes you already know how to generate X.509 certificates for your applications including your own Root CA. If not, check out my other blog post here. It also assumes you have an existing Go project set up with the AWS SDK installed.

Creating S3 Buckets

We are going to create 3 S3 buckets for this example: rolesanywhere-tutorial-1, rolesanywhere-tutorial-2, and rolesanywhere-tutorial-3. Keep in mind that although buckets are created within a region, the names must be unique within the global namespace. Each of these buckets will have the following settings:

1. Region: us-east-1 (N. Virginia)
2. ACLs disabled (the recommended setting) as we will be granting permissions via policies rather than access control lists
3. Block all public access enabled as these buckets should only be accessible by our own account
4. Bucket versioning off for this example though it is typically recommended to turn it on in order to recover easily from application failure and unwanted user action
5. S3 managed keys for encryption as opposed to KMS managed keys to reduce calls needed to KMS. For other application you may want to use KMS managed keys in order to have all of your account keys in the same place

Creating a Trust Anchor

As mentioned before, a trust anchor is simply a certification authority that will be used to establish trust between your workload and AWS as well as to encrypt communications.

Sign into your AWS account and navigate to the IAM console. From the sidebar, click “Roles” and then under Roles Anywhere click “Manage”. From here, under Trust Anchors click “Create a trust anchor”.

For the initial settings we are going to create the trust anchor under the same us-east-1 region for simplicity, though it’s not mandatory to have the trust anchor in the same region for the services with which it interacts. We will name it rolesanywhere-tutorial-ca and specify that we will provide an external certificate bundle.

Below, we will paste the PEM encoded contents of our certificate file into the provided field. Then, scroll down and click “Create a trust anchor”.

Creating a Policy

Typically, you should use AWS managed policies wherever possible. However, for our case we will be creating our own policy because none of the AWS managed policies are restrictive enough to provide least privilege of just listing bucket names. This obviously is not practical in a real-world sense but it does highlight the importance of the principle of least privilege.

From the IAM console we’ll click Policies on the sidebar and then click “Create policy”. Within the policy editor we’ll provide the following JSON:

{
 "Version": "2012-10-17",
 "Statement": [
  {
   "Effect": "Allow",
   "Action": "s3:ListAllMyBuckets",
   "Resource": "*"
  }
 ]
}

This policy will allow any entity that has this policy attached to list the S3 buckets on the account, but nothing else. We will name the policy rolesanywhere-tutorial-s3-read-buckets and create the policy. Now we’ll configure the role that will assume this policy.

Configuring a Role

For the purposes of this example, we are going to create a new role that allows access to an entity for reading bucket names and nothing else based on the principle of least privilege.

Navigate back to the Roles page of the IAM console and click “Create role”. Under “Trusted entity type” we will specify that we are going to provide a custom trust policy. The trust policy specifies which accounts, services, and other entities are allowed to assume this role.

Under the Custom trust policy section we will provide the following JSON:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
          "Service": "rolesanywhere.amazonaws.com"
      },
      "Action": [
          "sts:AssumeRole",
          "sts:TagSession",
          "sts:SetSourceIdentity"
      ],
      "Condition": {
          "StringEquals": {
              "aws:PrincipalTag/x509Issuer/CN": "Tutorial Root CA"
          },
          "ArnEquals": {
              "aws:SourceArn": "arn:aws:rolesanywhere:us-east-1:************:trust-anchor/ab4bb7e8-9de4-435d-a3f6-235ca018d256"
          }
      }
    }
  ]
}

This policy provides the Roles Anywhere service permission to assume this role as well as permission to tag the session and set the source identity. The importance of the latter two permissions becomes evident when we look at the conditions.
The first condition checks the tag attached by Roles Anywhere relating to the common name (CN) of the certificate issuer (in our case Tutorial Root CA). This ensures that when a request is made to obtain credentials with an X.509 certificate, it will check that the issuer of the certificate matches the provided name.
The second condition checks the source identity to ensure that the request for the role came from the trust anchor that we set up before. If both of these conditions are met then Roles Anywhere will be allowed to assume the role. Ensure that you replace the ARN from this policy with your own trust anchor’s ARN as all of the resources shown here will be deleted before the publication of this article so they won’t work.

From the roles list we’ll attach our previously created rolesanywhere-tutorial-s3-read-buckets policy. We’ll name the role rolesanywhere-tutorial-s3-read-buckets-role and finally create it.

Creating a Profile

A Roles Anywhere profile defines the roles that will be assumed by the external workload once it has been authorized and authenticated. Back in the IAM console, we’ll go to Roles and then under Roles Anywhere we’ll click “Manage”. Under the profiles section we can click “Create a profile”. We are going to name this profile rolesanywhere-tutorial-s3-profile. Under roles we’ll attach our previously created rolesanywhere-tutorial-s3-read-buckets-role. Optionally, you can attach a session policy to further restrict the actions that are allowed by the external workload through Roles Anywhere but we can skip this since we created a very restrictive role in the first place. Now that the profile has been created we can begin the process of obtaining temporary credentials.

Setting Up Credential Helper

The AWS credential helper is a tool that is used to obtain temporary credentials from Roles Anywhere. Using the AWS shared config file we can specify the credential_process to run this application with parameters that we provide in order to get credentials back in JSON format. You can download the official credential helper tool for any operating system here.

Once we have this downloaded, we’ll store it in the same folder as the default config file path. For Windows this is %USERPROFILE%\.aws\config. Alternatively, you can modify the AWS_CONFIG_FILE environment variable if you want to use a different path. In this folder, we’ll also create a certificates directory for simplicity when detailing the config file. We then provide the config file with the following contents:

[profile tutorial]
credential_process = %USERPROFILE%/.aws/aws_signing_helper.exe credential-process --certificate %USERPROFILE%/.aws/certificates/tutorial.crt --private-key %USERPROFILE%/.aws/certificates/tutorial.key --trust-anchor-arn arn:aws:rolesanywhere:us-east-1:************:trust-anchor/ab4bb7e8-9de4-435d-a3f6-235ca018d256 --profile-arn arn:aws:rolesanywhere:us-east-1:************:profile/3d1f265d-fb8c-4fe2-b9a4-08e8df29f77c --role-arn arn:aws:iam::************:role/rolesanywhere-tutorial-s3-read-buckets-role

This creates a config profile called tutorial that sets the credential process to the signing helper tool. We then provide the location of of the certificate signed by the trust anchor root CA and the corresponding private key. We also specify the trust anchor, profile, and role that should be used in the request for credentials. Now that we have this set up we can write the code to retrieve the bucket names.

Accessing S3 Buckets With Golang SDK v2

To fetch the bucket names using the credential provider we’ve set up we can use the following code:

package main

import (
 "context"
 "fmt"

 "github.com/aws/aws-sdk-go-v2/config"
 "github.com/aws/aws-sdk-go-v2/service/s3"
)

func main() {
 cfg, err := config.LoadDefaultConfig(context.TODO(), config.WithRegion("us-east-1"), config.WithSharedConfigProfile("tutorial"))
 if err != nil {
  panic(err.Error())
 }

 creds, err := cfg.Credentials.Retrieve(context.TODO())
 if err != nil {
  panic(err.Error())
 }
 fmt.Printf("Access Key ID: %v\n", creds.AccessKeyID)
 fmt.Printf("Secret Access Key: %v\n", creds.SecretAccessKey)
 fmt.Printf("Session Token: %v\n", creds.SessionToken)

 s3Client := s3.NewFromConfig(cfg)

 output, err := s3Client.ListBuckets(context.TODO(), &s3.ListBucketsInput{})
 if err != nil {
  panic(err.Error())
 }

 for index, bucket := range output.Buckets {
  fmt.Printf("Bucket %v: %v\n", index+1, *bucket.Name)
 }

}

This code retrieves config information from the default config file location using the us-east-1 region and the tutorial profile we set up in the config file.

To check that this was successful we use the credential provider from the config to retrieve the Access Key ID, Secret Access Key, and Session Token and print them out.

Then, we create a new S3 client from the config to connect to the S3 service with the temporary credentials and from that client we retrieve a list of bucket names.

When we run this code we get the following output:

Access Key ID: ASIAT646NT3JQP4TOJ2J
Secret Access Key: O/vY/vAxxsZVhABnj+B3ZwvysUVCoDUEs+7YYgjA
Session Token: IQoJb3JpZ2luX2VjEEAaCXVzLWVhc3QtMSJHMEUCIQCrZ9LzL9Ej7Bo2H7rLHnFSlR/NdqcF1PEPTfhAB8SIHwIgEBx7Cnntd+BZ02bGsmVFuYEe6qVuZdkgGLwycjwTTh4qvQQIeRAAGgwyNzI1MjU3OTUwMjciDB+wML80rnUTcg2YgiqaBCiadcYW8CMH+6igP1GWq5zYZSt5VfztgoL4e2fVzBNJJ7Zkmv0SpF3IO2ZNKQZCkHkns9AQjO8pTyTfIpTfWjIhyh0ZHlHlaUW7TwIQj1Jv1wTXHefn1bGzp0hrv5f/iBWraqUOqYk8VaLCoGDEoXUlWFh3OcbS5cN1c/FFVW+LgfP2aI3b3VPLPahFBziZYIHwg8wiWxPFElW36CL2sk/PhWbkL3Vl0qPbx1Y+16otZEX0yaDT++RfizIsjhY/PxSSICv3kG8HjHNy5zJFOemaue8KtfPYSOVnMLAULPd4bD0mYoWq2vYqHV/fY+GsZJndCcDBmsX5AAcGD2lO7wUTiI8dEjbQ0CILK3vVdNAyDXgq6VZ8Flt6F9Jr0/VEbIpjelvhX/6XfKUmBJt1nc9xkRwcVAvWyoZLYu5oDCrpxm8EGitrZFhA+Nkw/VBQ7Q2UkNX6lOISSPYfxGDZMaicwYvfEKskqyAIWyrNXDYS2R+qS9cdMH6xXmVibvDh4KuiI+vfn59+sV4qJSLDXrtVJVrLA5sRqIxcQe+DEb4fXJGH3MoCF5EQRX+KxD6oWV0xeCh4ka5NwFHlP06ZuZREkeJLo51UCwH3R8L0WgyZHR8YGHxlXCRHVFXsBFOcKU1GUctY/SiXtVtTJLlOgafESNE0Fk+AY2axcvc0W225xJ76kkFAoso4y07JML7pyvWqEueq5LNQwGcwm/meqgY6kwFVSQPbBW2ZAK1FcgRHPYqv3E4aXp9uz+Egm83QzMt4XkIJrATNTr9lktJpxh5ANvCl9b3TVtr1p7BnEuCDTHl9SBZc/8IWetUQfDTwIatQb5fZO9lYS2c3vMiSmbhmlZBKyw1AkLb3+1KJ9tIO1hU///KPFtS+fTl3xtdJnNAQ/IushTtAAhOmWEvp6X6XS1LFDLc=
Bucket 1: rolesanywhere-tutorial-1
Bucket 2: rolesanywhere-tutorial-2
Bucket 3: rolesanywhere-tutorial-3

These credentials will be accessible for the maximum session duration defined by the role. By default, this is 1 hour.

Now, we have successfully connected to S3 with temporary credentials provided by Roles Anywhere!