Exchange AWS Credentials for GCP Credentials using GCP STS Service

This is a sample procedure that will exchange a long term or short term AWS credential for a GCP credential.

You can use the GCP credential to access any service the mapped principal has GCP IAM permissions on.

This article and repo is the first part that explores how to use the workload identity federation capability of GCP which allows for external principals (AWS,Azure or arbitrary OIDC provider) to map to a GCP credential.

The two variations described in this article and repo will acquire a Google Credential as described here:

• The “Automatic” way is recommended and is supported by Google
• The “Manual” way is also covered in this article but I decided to wrap the steps for that into my own library here github.com/salrashid123/oauth2/google which surfaces the credential as an oauth2.TokenSource for use in any GCP cloud library.

NOTE: the library i’m using for the “manual” way is just there as an unsupported demo of a wrapped oauth2 TokenSource!

You can certainly use either procedure but the Automatic way is included with the supported, standard GCP Client library.

This repository is not supported by Google salrashid123/oauth2/google is also not supported by Google

For OIDC based exchanges, see: Exchange Generic OIDC Credentials for GCP Credentials using GCP STS Service

You can find the source for this article here:

salrashid123/gcpcompat-aws

Workload Federation — AWS

GCP now surfaces a STS Service that will exchange one set of tokens for another using the GCP Secure Token Service (STS) here. These initial tokens can be either 3rd party or google access_tokens that are downscoped (i.,e attenuated in permission set).

Basically what happens is that the AWS token is sent to GCP which inturns does the validation and mapping to the federated principal (i.e, the AWS token the client _initially_ sends to GCP’s STS service is validated by it calling AWS APIs). Once the AWS credentials are validated and mapped, the federated token for the new identity is returned to the client.

To use this tutorial, you need both a GCP and AWS project and the ability to create user/service accounts and then apply permissions on those to facilitate the mapping.

Again, the two types of flows this repo demonstrates:

• Manual Exchange: In this you manually do all the steps of exchanging AWS_ACCESS_KEY_ID/AWS_SECRET_ACCESS_KEY for a federated token and then finally use that token
• Automatic Exchange In this you use the google cloud client libraries to do all the heavy lifting. << This is the recommended approach

It is recommended to ty the manual first in this tutorial just to understand this capability and then move onto the automatic

AWS User/Session/Roles → GCP Identity → GCP Resource

This tutorial will cover various source AWS identity for federation and how they map to a GCP principal:// and principalSet://.

AWS User/Roles/Sessions can all be mapped to various degrees to GCP subjects (principal://) or as a group (principalSet://). There are four variations this tutorial will cover:

a) AWS User --> GCP principal://

In other words:

"Arn": "arn:aws:iam::291738886548:user/svcacct1"

which maps to

principal://iam.googleapis.com/projects/$PROJECT_NUMBER/locations/global/workloadIdentityPools/aws-pool-1/subject/arn:aws:iam::291738886548:user/svcacct1

b) AWS User --> AWS Assumed Role --> AWS Named Session --> GCP principal://

"Arn": "arn:aws:sts::291738886548:assumed-role/gcpsts/mysession"

which maps to

principal://iam.googleapis.com/projects/$PROJECT_NUMBER/locations/global/workloadIdentityPools/aws-pool-1/subject/arn:aws:sts::291738886548:assumed-role/gcpsts/mysession

c) AWS User --> AWS Assumed Role --> GCP principalSet://

AWS User assumes Role arn:aws:sts::291738886548:assumed-role/gcpsts

which maps to principalSet://iam.googleapis.com/projects/$PROJECT_NUMBER/locations/global/workloadIdentityPools/aws-pool-2/attribute.aws_role/arn:aws:sts::291738886548:assumed-role/gcpsts

d) AWS EC2 --> AWS Assumed EC2 Role --> GCP principalSet://

AWS EC2 Role arn:aws:sts::291738886548:assumed-role/ec2role

which maps to "principalSet://iam.googleapis.com/projects/$PROJECT_NUMBER/locations/global/workloadIdentityPools/aws-pool-2/attribute.aws_role/arn:aws:sts::291738886548:assumed-role/ec2role"

Option (d) is likely the most common usecase since it allows a group of EC2 instances to collectively map to a GCP identity.

a) is used to map a given user (aws-pool-1/subject/)

b) is used to map a user to a named session as a subject (i’m not sure where this is used)

c) is used to map a user that assumes a role (aws-pool-2/attribute.aws_role)

Manual Exchange

On the AWS side, you need to configure a user, then allow it to AssumeRole to derive a short-term token. You do not need to go the extra step to assumeRole but this example shows best-practices for short-lived tokens.

1) Create AWS user

In this case, the user is "arn:aws:iam::291738886548:user/svcacct1" with uid=AIDAUH3H6EGKDO36JYJH3

Using local env-var based bootstrap on your laptop:

$ export AWS_ACCESS_KEY_ID=redacted 
$ export AWS_SECRET_ACCESS_KEY=redacted
$ aws sts get-caller-identity
{
    "UserId": "AIDAUH3H6EGKDO36JYJH3",
    "Account": "291738886548",
    "Arn": "arn:aws:iam::291738886548:user/svcacct1"
}

2) Define Role

Allow the user to assume an AWS role (gcpsts)

In this case, we are assuming a role and a named session arn:aws:sts::291738886548:assumed-role/gcpsts/mysession

3) Verify Role change

Use the assumed roles token to confirm the change

GCP

Switch to the GCP account

1) Create Service Account

Create a service account the AWS one will map to and grant this service account permissions on something (eg, gcs bucket)

From here, you can test using principal:// or principalSet:// as described above

Using (principal://)

The following commands below we are specifically mapping an arn to a subject. That is, it will match for exactly

• "arn:aws:iam::291738886548:user/svcacct1" (user)
• "arn:aws:sts::291738886548:assumed-role/gcpsts/mysession" (session)

a) First create a workflow identity pool

gcloud beta iam workload-identity-pools create aws-pool-1 \
    --location="global" \
    --description="AWS " \
    --display-name="AWS Pool"

b) Define aws-provider

Define the aws-provider associated with that pool using your AWS AccountID (in this case its 291738886548).

The attribute-mapping= sections are the default mapping that does the actual translation from the AWS getCallerIdentity() claim back to a GCP principal.

You can define other mappings but we're using the default mapping

gcloud beta iam workload-identity-pools providers \
  create-aws aws-provider-1  \
   --workload-identity-pool="aws-pool-1"  \
   --account-id="291738886548"   \
   --location="global"

c) Grant WorkloadIdentity Pool to use SA

Now grant the mapped identity permissions to assume the actual GCP service account.

In the example below, we’re allowing a user (arn:aws:iam::291738886548:user/svcacct1) and specific sessonName (arn:aws:sts::291738886548:assumed-role/gcpsts/mysession) will be allowed to impersonate aws-federated@$PROJECT.iam.gserviceaccount.com

Using (principalSet://)

Use (principalSet://) to map any AWS system that assumes a role. For example, if you have a set of AWS EC2 instances that all can assume a specific Role name, use principalSet://

Note the command below we are specifically mapping federation on TWO Roles that we will demo shortly. The first one is for a user to assume that role; the second one is for ec2 itself.

• "arn:aws:sts::291738886548:assumed-role/gcpsts"
• "arn:aws:sts::291738886548:assumed-role/ec2role"

a) Define identity-pool

We will create a new identity pool here just to test this separately from (a)

gcloud beta iam workload-identity-pools create aws-pool-2 \
    --location="global" \
    --description="AWS " \
    --display-name="AWS Pool 2"

b) Define aws-provider

gcloud beta iam workload-identity-pools providers \
   create-aws aws-provider-2  \
   --workload-identity-pool="aws-pool-2" \
   --account-id="291738886548"   \
   --location="global"

(note, we are again using the default mapping attribute-mapping of "google.subject=assertion.arn" and attribute.aws_role=<AWS Role>)

c) Grant WorkloadIdentity Pool to use the service account

You should end up with IAM bindings on the service account that is used for impersonation. The bindings below shows both principal:// and principalSet://

Automatic Exchange

With the automatic exchange, the GCP cloud auth libraries do all these steps for you.

In this tutorial, setup an EC2 VM, install golang and configure it such that you can assume the role ec2role automatically on the VM

For this to work, you must have previously setup the aws-federated@$PROJECT_ID.iam.gserviceaccount.com service account and gave it permissions to the GCS object. You should have also configured the aws-provider-2 configurations and use principalSet://

First step is to generate the client library helper file which will act as the APPLICATION_DEFAULT_CREDENTIAL

It should look something like this in sts-creds.json:

What that basically states is for GCP to look for AWS federation credentials from the EC2 metadata server or from the AWS env variables in the current shell. Once the AWS credentials are acquired, the client library will perform the STS exchange and enable the GCP client library access.

Copy the sts-creds.json file to the EC2 instance

On the EC2 instance, make sure it has a role binding:

Now Make sure you’ve previously mapped the Assumed Role defined for aws-pool-2 as a principalSet://

Test Automatic

Finally, on the EC2 instance, invoke the client provided in this repo:

export GOOGLE_APPLICATION_CREDENTIALS=`pwd`/sts-creds.json
$ go run main.go    --gcpBucket mineral-minutia-820-cab1  \
  --gcpObjectName foo.txt  \
  --useADC
2021/03/10 22:05:36 >>>>>>>>>>>>>>>>> Using ADC
 FOOOOO

the FOOOO is the contents of our file on GCS!

If you are on your laptop and have the AWS Environment variables, you can also use the automatic flow

export AWS_ACCESS_KEY_ID=redacted 
export AWS_SECRET_ACCESS_KEY=redacted
export AWS_REGION=us-east-1
export GOOGLE_APPLICATION_CREDENTIALS=`pwd`/sts-creds.json
$ go run main.go   \
 --gcpBucket mineral-minutia-820-cab1  \
 --gcpObjectName foo.txt    --useADC

TODO: update google storage client library to pickup PR482

Test Manual

For manual testing, we are using main.go which simply wraps the flow on your laptop (note the automatic was done on an EC2 instance)

• Flags:

Run as user:

Using Federated or IAM Tokens

GCP STS Tokens can be used directly against a few GCP services such as (IAMCredentials and GCS (more to come later)).

What that means is that you do not need to enable iam impersonation on these tokens to access these services; the STS token can be used directly.

Basically, that means you can skip step (5) of Exchange Token

This not only saves the step of running the exchange but omits the need for a secondary GCP service account to impersonate.

To use GCS, allow either the Assumed Role or AWS User access to the resource. In this case storage.objectAdmin access (yes, i'm granting the permissions below on the project, you should grant on the bucket only):

To use Federated tokens, use remove the --useIAMToken flag and allow the federated identity _direct_ access

Set UseIAMToken: false in the go code

NOTE: the GCP “Automatic” libraries use impersonation by default…However, if you DELETE the service_account_impersonation_url entry in the sts-config.json file, the ADC library will end up using federated tokens!

Logging

Depending on the mode you used UseIAMToken flag in code, you may either see the IAM service account impersonated then access the GCS resource, or the AWS principal directly.

• UseIAMToken: true: In this mode, the AWS credential is exchanged for a GCP STS and then the GCP STS is again exchanged for a GCP ServiceAccount Token. AWS Creds -> GCP STS (workload pool) -> GCP IAM (service_account) -> GCS

The net result is you see the iam exchange but the original AWS caller is hidden in the GCSlogs The following shows the logs emitted if using AssumeRole

• UseIAMToken: false: In this mode, the AWS credential is exchanged for a GCP STS creds and then directly against a GCP Resource AWS Creds -> GCP STS (workload pool) -> GCS

The following logs shows the dataaccess logs when accessed directly as arn:aws:iam::291738886548:user/svcacct1:

UseIAMToken=false only works on certain GCP resources.

Direct AWS Credentials

If you want to directly pipe in the AWS credential object ("github.com/aws/aws-sdk-go/aws/credentials") into GCP, you will need to use the 'unsupported' library and bootstrap AWS creds first and then pipe it into GCP creds:

In the following, we are using AWS credentials as the source with the ARN of --awsRoleArn arn:aws:iam::291738886548:user/svcacct1

If you want to see this work, uncomment the appropriate sections in main.go,

GCP Organization Policy

Finally, you can restrict which AWS projects are allowed federation to GCP by specifying a restrict

Finally, if you’re just looking to write you own STS service for whatever reason, see Serverless Secure Token Exchange Server(STS) and gRPC STS credentials