The Ultimate Step-by-Step Guide for Transferring Multiple Images from AWS ECR to GCP Artifact Registry Using Shell Script

Introduction

In today’s multi-cloud environment, managing container images efficiently is crucial for seamless operations and collaborative development. AWS Elastic Container Registry (ECR) and Google Cloud Platform (GCP) Artifact Registry are two powerful tools for managing container images. This guide will walk you through the process of transferring multiple images from AWS ECR to GCP Artifact Registry using a shell script, enabling you to leverage the strengths of both platforms and streamline your workflows.

Step-by-Step Process

Setting Up AWS ECR for Image Transfer

Accessing AWS ECR Console
1. Log into the AWS Management Console.
2. Navigate to the ECR service.

Creating a Repository for Images
1. Create a repository in AWS ECR.
2. Use this repository to store your container images for transfer to GCP Artifact Registry.

Pushing Images to AWS ECR
1. Push container images to the ECR repository using Docker commands.
2. Ensure images are tagged correctly for easy identification during the transfer.

Setting Up GCP Artifact Registry for Image Transfer

Accessing GCP Console
1. Open the Google Cloud Console.
2. Navigate to the Artifact Registry service.

Creating a Repository in GCP Artifact Registry
1. Create a repository in GCP Artifact Registry.
2. Use this repository to store incoming images from AWS ECR.

(Make sure your docker daemon is running already**)

Writing the Shell Script

Create a shell script that automates the transfer of multiple images from AWS ECR to GCP Artifact Registry.

Configuration Section

# Configuration
AWS_ACCOUNT_ID="your-aws-account-id"
AWS_REGION="your-aws-region"
AWS_ECR_REPO="your-aws-ecr-repo"
GCP_PROJECT="your-gcp-project-name"
GCP_PROJECT_ID="your-gcp-project-id"
GCP_ARTIFACTORY_REPO="your-gcp-artifact-repo"
AR_LOCATION="your-gcp-artifact-registry-location"

This section sets up the configuration variables needed for the script:

• AWS_ACCOUNT_ID: Your AWS account ID, which identifies your AWS account.
• AWS_REGION: The region where your AWS ECR (Elastic Container Registry) is located.
• AWS_ECR_REPO: The name of your AWS ECR repository that contains the Docker images you want to transfer.
• GCP_PROJECT: The name of your Google Cloud Platform (GCP) project.
• GCP_PROJECT_ID: The ID of your GCP project.
• GCP_ARTIFACTORY_REPO: The name of your Google Artifact Registry repository where you want to transfer the Docker images.
• AR_LOCATION: The location of your Google Artifact Registry

Authentication Section

# Authenticate with AWS and GCP
aws ecr get-login-password --region $AWS_REGION | docker login --username AWS --password-stdin ${AWS_ACCOUNT_ID}.dkr.ecr.${AWS_REGION}.amazonaws.com
gcloud auth login
gcloud auth configure-docker $AR_LOCATION-docker.pkg.dev

This section handles authentication:

• The first command logs into AWS ECR. It retrieves a password using aws ecr get-login-password, then pipes (|) this password to docker login to authenticate Docker with AWS ECR.
• The second command configures Docker to use GCP Artifact Registry using gcloud auth configure-docker.

Fetching Image Tags

# Fetch the list of images from AWS ECR
IMAGE_LIST=$(aws ecr list-images --repository-name $AWS_ECR_REPO --region $AWS_REGION --query 'imageIds[*].imageTag' --output text)
echo "IMAGE_LIST: $IMAGE_LIST"

This block fetches the list of Docker image tags from the specified AWS ECR repository:

• aws ecr list-images retrieves a list of image IDs (including tags) from the ECR repository.
• --repository-name $AWS_ECR_REPO specifies the repository name.
• --region $AWS_REGION specifies the AWS region.
• --query 'imageIds[*].imageTag' extracts only the image tags from the response.
• --output text formats the output as plain text.
• The result is stored in the IMAGE_LIST variable and printed to the console for verification.

Loop Through and Migrate Images

# Loop through the images and migrate them
for IMAGE_TAG in $IMAGE_LIST; do
    if [[ -n "$IMAGE_TAG" ]]; then
        echo "Starting migration for image tag: $IMAGE_TAG"
        
        AWS_IMAGE_URI="${AWS_ACCOUNT_ID}.dkr.ecr.${AWS_REGION}.amazonaws.com/${AWS_ECR_REPO}:${IMAGE_TAG}"
        GCP_IMAGE_URI="${AR_LOCATION}-docker.pkg.dev/${GCP_PROJECT_ID}/${GCP_ARTIFACTORY_REPO}/${IMAGE_TAG}"

        echo "Migrating image $AWS_IMAGE_URI to $GCP_IMAGE_URI"

        # Pull the image from AWS ECR
        docker pull $AWS_IMAGE_URI

        # Tag the image for Google Artifact Registry
        docker tag $AWS_IMAGE_URI $GCP_IMAGE_URI

        # Push the image to Google Artifact Registry
        docker push $GCP_IMAGE_URI

        # Optionally, remove the local image to save space
        docker rmi $AWS_IMAGE_URI
        docker rmi $GCP_IMAGE_URI

        echo "Migration completed for image tag: $IMAGE_TAG"
    else
        echo "Skipping empty image tag"
    fi
done

This block migrates each image from AWS ECR to GCP Artifact Registry:

• The for loop iterates over each IMAGE_TAG in IMAGE_LIST.
• The if [[ -n "$IMAGE_TAG" ]]; then condition checks if IMAGE_TAG is non-empty.
• AWS_IMAGE_URI and GCP_IMAGE_URI are constructed using the image tag and appropriate repository URLs.
• The script prints a message indicating the start of migration for the current image tag.
• docker pull $AWS_IMAGE_URI: Pulls the Docker image from AWS ECR.
• docker tag $AWS_IMAGE_URI $GCP_IMAGE_URI: Tags the pulled image for GCP Artifact Registry.
• docker push $GCP_IMAGE_URI: Pushes the tagged image to GCP Artifact Registry.
• docker rmi $AWS_IMAGE_URI and docker rmi $GCP_IMAGE_URI: Optionally removes the local images to save disk space.
• A message is printed indicating the completion of the migration for the current image tag.
• If the IMAGE_TAG is empty, the script prints a message and skips the iteration.

Full Script

Here is the entire script with explanations embedded:

#!/bin/bash

# Configuration
AWS_ACCOUNT_ID="your-aws-account-id"  # Your AWS account ID
AWS_REGION="your-aws-region"          # Your AWS region
AWS_ECR_REPO="your-aws-ecr-repo"      # Your AWS ECR repository name
GCP_PROJECT="your-gcp-project-name"   # Your GCP project name
GCP_PROJECT_ID="your-gcp-project-id"  # Your GCP project ID
GCP_ARTIFACTORY_REPO="your-gcp-artifact-repo"  # Your GCP Artifact Registry repository name
AR_LOCATION="your-gcp-artifact-registry-location"  # Your GCP Artifact Registry location (e.g., asia-south1)

# Authenticate with AWS and GCP
aws ecr get-login-password --region $AWS_REGION | docker login --username AWS --password-stdin ${AWS_ACCOUNT_ID}.dkr.ecr.${AWS_REGION}.amazonaws.com
gcloud auth login 
gcloud auth configure-docker $AR_LOCATION-docker.pkg.dev

# Fetch the list of images from AWS ECR
IMAGE_LIST=$(aws ecr list-images --repository-name $AWS_ECR_REPO --region $AWS_REGION --query 'imageIds[*].imageTag' --output text)
echo "IMAGE_LIST: $IMAGE_LIST"

# Loop through the images and migrate them
for IMAGE_TAG in $IMAGE_LIST; do
    if [[ -n "$IMAGE_TAG" ]]; then
        echo "Starting migration for image tag: $IMAGE_TAG"
        
        AWS_IMAGE_URI="${AWS_ACCOUNT_ID}.dkr.ecr.${AWS_REGION}.amazonaws.com/${AWS_ECR_REPO}:${IMAGE_TAG}"
        GCP_IMAGE_URI="${AR_LOCATION}-docker.pkg.dev/${GCP_PROJECT_ID}/${GCP_ARTIFACTORY_REPO}/${IMAGE_TAG}"

        echo "Migrating image $AWS_IMAGE_URI to $GCP_IMAGE_URI"

        # Pull the image from AWS ECR
        docker pull $AWS_IMAGE_URI

        # Tag the image for Google Artifact Registry
        docker tag $AWS_IMAGE_URI $GCP_IMAGE_URI

        # Push the image to Google Artifact Registry
        docker push $GCP_IMAGE_URI

        # Optionally, remove the local image to save space
        docker rmi $AWS_IMAGE_URI
        docker rmi $GCP_IMAGE_URI

        echo "Migration completed for image tag: $IMAGE_TAG"
    else
        echo "Skipping empty image tag"
    fi
done

echo "All migrations completed."

This script provides a systematic way to transfer multiple Docker images from AWS ECR to GCP Artifact Registry. Each block is designed to perform specific tasks to ensure a smooth migration process.

Conclusion

By following this guide, you can seamlessly transfer container images from AWS ECR to GCP Artifact Registry, leveraging the strengths of both platforms. The provided shell script automates the process, ensuring efficiency and consistency in your multi-cloud operations.

Happy Migrating :)