Streamline CI/CD: Secure GCP Deployments with GitHub Runner Sets on GKE & Workload Identity Federation

In this blog, we’ll guide you through establishing a robust, secure CI/CD pipeline that leverages the power of Google Kubernetes Engine (GKE) self-hosted runners, GitHub Actions workflows, and the security of Google Cloud Platform (GCP) Workload Identity Federation (WIF).

This combination eliminates the need for long-lived service account keys, significantly reducing security risks while providing granular access control and scalability.

Why This Approach Matters ?

• Enhanced Security: WIF replaces static credentials with short-lived tokens, minimizing the risk of unauthorized access.
• Granular Control: Precisely define permissions for your GitHub Actions workflows, limiting their scope to necessary actions.
• Scalability: GKE’s self-hosted runners offer on-demand scaling, ensuring your CI/CD processes can handle varying workloads.

Prerequisites

• Active GCP project with billing enabled
• GitHub repository
• Google Cloud SDK (gcloud)
• kubectl
• helm (version 3.0+)

Steps

1. Configure Workload Identity Federation (WIF):

Workload Identity Federation is a critical component for secure authentication between GitHub Actions and GCP. By establishing a trust relationship, you can grant temporary, scoped access to your GCP resources without the risk of exposed credentials.

Follow the Google Cloud documentation to set up WIF, linking your GitHub repository with your GCP project. This establishes the trust relationship required for authentication.

How does the GCP Workload Identity Federation work with Github Provider?

To use the GitHub Actions auth action, you need to set up and configure Workload Identity Federation by creating a Workload Identity Pool and Workload Identity Provider:

Create a Workload Identity Pool :

resource "google_iam_workload_identity_pool" "main" {
  project                   = "test-project-001"
  workload_identity_pool_id = "gh-wif-pool"
  display_name              = "gh-wif-pool"
  description               = "Workload Identity Pool for Testing"
  disabled                  = false
}

Create a Github Provider :

resource "google_iam_workload_identity_pool_provider" "github" {
  project                            = "test-project-001"
  workload_identity_pool_id          = google_iam_workload_identity_pool.main.workload_identity_pool_id
  workload_identity_pool_provider_id = "gh-provider"
  display_name                       = "gh-provider"
  description                        = "Workload Identity Pool Provider managed by Terraform - GitHub"
  attribute_condition                = null
  attribute_mapping   = {
    "google.subject"       = "\"github::\" + assertion.repository + \"::\" + assertion.ref"
    "attribute.actor"      = "assertion.actor"
    "attribute.aud"        = "assertion.aud"
    "attribute.repository" = "assertion.repository"
    "attribute.ref"        = "assertion.ref"
  }
  oidc {
    allowed_audiences = []
    issuer_uri        = "https://token.actions.githubusercontent.com"
  }
}

Create a Service Account:

resource "google_service_account" "ghe_repo_sa" {
  project    = "test-project-001"
  account_id = "sa-ghe"
}

Allow Workload Identity User Role:

resource "google_service_account_iam_member" "ghe_wif_iam" {
  service_account_id = google_service_account.ghe_repo_sa.email
  role               = "roles/iam.workloadIdentityUser"
  member             = "principal://iam.googleapis.com/${google_iam_workload_identity_pool.main.name}/subject/github::${var.ghe_repo_full_name}::refs/heads/main"
}

Replace var.ghe_repo_full_name with GITHUB_ORG/REPO_NAME .

This Terraform code snippet grants specific GitHub actions (from the main branch of a designated repository) the ability to use a Google Cloud service account. This allows your GitHub workflows to securely interact with and manage Google Cloud resources.

2. Prepare Your GKE Environment :

• Create a GKE cluster.
• Enable the APIs.
• Create Namespaces : Create two namespaces in your GKE cluster — one for deploying the ARC and one for deploying the runner set

kubectl create namespace ghe-runner-controller
kubectl create namespace ghe-runner-scale-set

• Create a GitHub Token : Generate a GitHub Token with the repo and admin:org scopes (or similar permissions). This allows GitHub runner set to register to GitHub organization.
• Store the token as a Kubernetes Secret: Create a secret in the ghe-runner-scale-set namespace to securely store your GitHub Token.

kubectl create secret generic ghe-token \
--namespace=ghe-runner-scale-set \
--from-literal=github_token='<GITHUB_TOKEN>'  

3. Install the Actions Runner Controller (ARC)

ARC is a Kubernetes operator that manages your self-hosted runners. Install it using Helm

INSTALLATION_NAME="ghe-runner-scale-set-controller"               # Replace with your desired installation name 
NAMESPACE="ghe-runner-controller"               
helm upgrade --install "${INSTALLATION_NAME}" \
--namespace "${NAMESPACE}" \
oci://ghcr.io/actions/actions-runner-controller-charts/gha-runner-scale-set-controller

• Validation Check

Ensure pods are running

kubectl get pods -n ghe-runner-controller

ARC pods running

Ensure CRDs are installed

kubectl get crds | grep actions.github.com

Custom resource definition (CRDs)

4. Deploy Runner Scale Set

Note : To deploy a runner scale set, you must have ARC up and running

• Install GHE Runner Scale Set

INSTALLATION_NAME="arc-runner-set-in-00"       # Replace with your desired installation name 
NAMESPACE="ghe-runner-scale-set"               # Replace with your created namespace
GITHUB_CONFIG_URL="https://github.com/<ORGANIZATION_NAME>"
CONTROLLER_SERVICE_ACCOUNT_NAMESPACE=""        # Replace with the controller service account
CONTROLLER_SERVICE_ACCOUNT_NAME=""             # Replace with the controller namespace
helm install "${INSTALLATION_NAME}" \
--namespace "${NAMESPACE}" \
--set githubConfigUrl="${GITHUB_CONFIG_URL}" \
--set githubConfigSecret=ghe-token \
--set containerMode.type = "dind" \
oci://ghcr.io/actions/actions-runner-controller-charts/gha-runner-scale-set

Verify the Deployment

Verify by checking your GitHub organization’s settings under Actions -> Runners. The deployed runner set should be listed and online.

5. Github Action Workflow

Let’s create a basic workflow that leverages Workload Identity Federation (WIF) for secure authentication with Google Cloud, giving us the ability to seamlessly interact with Google Artifact Registry (GAR).

• Create a workflow file (e.g., .github/workflows/main.yml) that defines your CI/CD steps.
• Utilize the google-github-actions/auth action for authentication, ensuring you reference your WIF configuration.
• Since we want our workflow to interact with Google Artifact Registry (GAR), we’ll need to give it the appropriate permissions Artifact Registry Writer (roles/artifactregistry.writer) to the service account associated with the WIF configuration. This will enable our workflow to push, pull, and manage artifacts within GAR.

name: Actions WIF Demo

on:
  push:
    branches:
      - 'main'
  pull_request:
    branches:
      - 'main'
  workflow_dispatch:

jobs:
  GitHub-Actions-on-GKE:
    runs-on: arc-runner-set-in-00
    permissions:
      id-token: write
      contents: read
    steps:
      - name: 'Git repo checkout'
        uses: actions/checkout@v4
        
      - id: 'auth'
        name: 'Authenticate to Google Cloud'
        uses: 'google-github-actions/auth@v1'
        with:
          create_credentials_file: true
          token_format: 'access_token'
          workload_identity_provider: 'projects/test-project-001/locations/global/workloadIdentityPools/gh-wif-pool/providers/gh-provider'
          service_account: 'sa-ghe@test-project-001.iam.gserviceaccount.com'
       
      - name: 'Configure Docker Login for GAR'
        run: |
          gcloud auth configure-docker us-central1-docker.pkg.dev

      - name: 'Build Docker Image'
        run: |
          cd docker && docker build -t us-central1-docker.pkg.dev/YOUR_PROJECT_ID/YOUR_REPOSITORY/alpine-custom:latest .
      
      - name: 'Push Docker Image'
        run: |
          docker push us-central1-docker.pkg.dev/YOUR_PROJECT_ID/YOUR_REPOSITORY/alpine-custom:latest

Conclusion

By following this guide, we can successfully set up a self-hosted runner on GKE and configured a GitHub Actions workflow to create a GCP resource using Workload Identity Federation.

This setup enhances security by eliminating the need for long-lived service account keys and provides scalable, secure CI/CD pipelines.