Connecting AWS and GCP workloads with Workload Identity Federation (part 1/2)

https://unsplash.com/photos/low-angle-photography-of-highrise-building-p-rN-n6Miag

The world has gone multi-cloud and since the ultimate goal is to get the best out of the different cloud providers, it has become a common practice to have shared workflows between them. The way that access between clouds used to be managed was by using access keys and storing them in some place “safe”.

Experience has taught us that no place is safe enough when it comes to security, that’s why Google introduced Workload Identity Federation. This service allows us to authenticate identities from different cloud providers to our GCP organization using short-lived credentials.

In this two-part article we’ll explain how to set up Workload Identity Federation to connect an AWS CI workload to deploy GCP resources. Part one — this one — will focus on Workload Identity Federation configuration steps, which can be applied to any AWS workload. In part two, we’ll dive deeper into our use case and explain how to configure kubernetes-based TF providers from AWS to GCP .

Understanding the use case

There are many CI alternatives to choose from when deploying GCP resources. Here we want to focus on how to do it using a self-hosted orchestration agent like Terraform Cloud, Spacelift, Env0, Scalr or Cloudify running on an EKS, an ECS cluster or an EC2 instance.

Use case example diagram

Be aware that the agent configuration, how to connect it to your git repository and the pipeline workflow will entirely depend on the provider you’re using. Please make sure you have followed their instructions to have the agent working properly.

Configuring Workload Identity Federation

On GCP, first thing we need is to enable all the necessary APIs:

gcloud services enable iam.googleapis.com sts.googleapis.com iamcredentials.googleapis.com \
  --project <PROJECT_ID>

Create the workload identity pool:

gcloud iam workload-identity-pools create "aws-identity-pool" \
  --project="<PROJECT_ID>" \
  --location="global" \
  --display-name="AWS identity pool"

Create the workload identity provider.

Here we recommend as a security best practice to control the scope of identities that can access your GCP environment. You can use any of the following roles accordingly:

For workloads running on EKS use the cluster service role.

For workloads running on ECS use the task role from your ECS task definition.

For workloads running on EC2 use the EC2 instance IAM role.

Now, run:

gcloud iam workload-identity-pools providers create-aws aws-identity-provider \
  --project="<PROJECT_ID>" \
  --location="global"  \
  --workload-identity-pool="aws-identity-pool" \
  --account-id="<AWS_ACCOUNT_ID>" \
  --attribute-mapping="google.subject=assertion.arn,attribute.account=assertion.account,attribute.aws_role=assertion.arn.extract('assumed-role/<AWS_WORKLOAD_ROLE_NAME>/')"

Replacing AWS_ACCOUNT_ID with your AWS account ID number and AWS_WORKLOAD_ROLE_NAME with the name of the role your workload is using.

Next, create a service account that will represent the AWS workload in GCP :

gcloud iam service-accounts create ci-agent-sa --project <PROJECT_ID>

Grant the service account access to the resources you want. In our case, we want the CI agent to manage compute resources, so we’ll grant Compute Admin role:

gcloud projects add-iam-policy-binding  <PROJECT_ID> \
  --member="serviceAccount:ci-agent-sa@<PROJECT_ID>.iam.gserviceaccount.com" \
  --role=roles/compute.admin

Finally, allow the workload identity principal access to impersonate the service account:

gcloud iam service-accounts add-iam-policy-binding "ci-agent-sa@<PROJECT_ID>.iam.gserviceaccount.com" \
  --project="<PROJECT_ID>" \
  --role="roles/iam.workloadIdentityUser" \
  --member="principalSet://iam.googleapis.com/projects/<PROJECT_NUMBER>/locations/global/workloadIdentityPools/aws-identity-pool/*"

All set! Now let’s see how to authenticate our workload.

Authenticating AWS workloads

Next step is to generate a valid short-lived token that can grant the workload the access it needs to perform its tasks. To achieve this, we’ll start by generating an external credentials configuration file and subsequently utilize it for authentication.

As an example, we’ll go back to our use case to explain how the configuration can be done.

We are setting up a self-hosted CI workload agent that uses Terraform code to deploy infrastructure resources. This means there are two pieces we need to put together: the pipeline workflow and the terraform providers configs.

In the pipeline workflow, we’ll include a pre-deployment step that generates the credentials file. Depending on the agent you’re using, this will look something like:

...

- |
gcloud iam workload-identity-pools create-cred-config \
projects/<PROJECT_NUMBER>/locations/global/workloadIdentityPools/<POOL_ID>/providers/<PROVIDER_ID> \
--service-account=<SERVICE_ACCOUNT_EMAIL> \
--aws \
--output-file=creds.json

...

Now, configure the google terraform providers to use the output file:

data "google_client_config" "provider" {}

provider "google" {
  credentials = file("${path.module}/creds.json")
}

provider "google-beta" {
  credentials = file("${path.module}/creds.json")
}

All done! You can start running your workloads from AWS to GCP.

Conclusion

https://unsplash.com/photos/selective-focus-photography-of-chess-pieces-G1yhU1Ej-9A

Workload Identity Federation is a recommended and secure solution that helps reduce distances between different cloud services and definitely helps simplify its management. Here’s another example of how you can use it with GitHub Actions and stop pushing your service account keys to GitHub.

If you want to know more about the example use case we presented, checkout the second part of this article: Configuring kubernetes-based TF providers from AWS to GCP with Workload Identity Federation (part 2/2).

Curious about other GCP tools? Visit our MediumPublication to learn more about different GCP solutions and how they can help you deploy applications and solutions.

And, stay tuned for part 2!