Connect to Compute Engine using SSH keys and Cloud Build

Avoid SSH keys repetitive creation

This article was created after solving this problem with the great help from David Spenard of DoiT International. DoiT International, a strategic partner of Google Cloud Platform and Amazon Web Services, tackles large-scale, complex problems in the cloud.

my graphic designer is off…

Silent error you might face

If you are already using Google Cloud Build to connect to Google Compute Engine for a CI/CD process or any other process: Read this article.

Because, if you are already using Google Cloud Build to connect to Google Compute Engine, you might see these lines into the build details:

gcloud compute ssh server_name --zone=europe-west2-c 

WARNING: The private SSH key file for gcloud does not exist.
WARNING: The public SSH key file for gcloud does not exist.
WARNING: You do not have an SSH key for gcloud.
WARNING: SSH keygen will be executed to generate a key.
This tool needs to create the directory [/builder/home/.ssh] before being able 
to generate SSH keys.

Do you want to continue (Y/n)?  
Generating public/private rsa key pair.
...

The build is creating an SSH key, at, every, execution…

It creates the SSH key and store it to Compute Engine metadata.

Have a look at this section, you might have hundreds of SSH keys.

If so, nothing terrible, you can just delete all of them every X months, but it’s not ideal.

Sooner or later, you will see this message:

The following SSH key will be removed from your project because your SSH keys 
metadata value has reached its maximum allowed size of 262144 bytes.

If this situation occurs, you won’t be able to access the instance…

To avoid this situation there is an other way: Create an SSH key-pair, store it in Google Secret Manager and use it at each build.

That’s what we see here.

The correct way

If you want to automate deployment with Google Compute Engine and Cloud Build, you should first read this article (brilliant but incomplete):

Automate deployment with Google Compute Engine and Cloud Build

You have to:

• Create a trigger
• Connect your repository
• Create a YAML file

After creating these elements, you will face the build message we just saw in “The wrong way”.

For the correct way, there are additional steps:

• Generate (locally) an SSH Key Pair
• Store this key-pair into Secret Manager
• Authorise Cloud Build access
• Allow the SSH keys to access the instance
• Get and use the SSH Key Pair

Generate (locally) an SSH Key Pair

For UNIX and UNIX-Like Systems:

ssh-keygen -t rsa -f ~/.ssh/cloud_build_ssh_key -C cloud_build_ssh_key -b 2048

Follow the clear-enough instructions until the end.

For windows, go here.

Store this key-pair into Secret Manager

Be sure to have gcloud CLI installed and Secret Manager enabled.

And run these commands to store public and private keys:

gcloud secrets create cloud-build-ssh-key --replication-policy="automatic" --data-file=/path_to_the_key/cloud_build_ssh_key

gcloud secrets create cloud-build-ssh-key-pub --replication-policy="automatic" --data-file=/path_to_the_key/cloud_build_ssh_key.pub

You can also create Secrets using GCP UI.

Authorise Cloud Build access

Keys are, by default restricted, but you can easily authorise access from Cloud Build.

To get the service account email used by Cloud Build, you can go to the Cloud Build settings page and replace SA_EMAIL by the email you see after Service account email.

gcloud secrets add-iam-policy-binding cloud-build-ssh-key --member="serviceAccount:SA_EMAIL" --role="roles/secretmanager.secretAccessor"

gcloud secrets add-iam-policy-binding cloud-build-ssh-key-pub --member="serviceAccount:SA_EMAIL" --role="roles/secretmanager.secretAccessor"

You can also authorise Cloud Build using GCP UI.

Allow the SSH keys to access the instance

To authorise the key to access the instance, go to Compute Engine instances list.

Select your instance.

Click Edit.

Go to the SSH keys section.

Add the public key (you can read your public key using this command: cat /path_to_the_key/cloud_build_ssh_key.pub )

This will authorise the key to access the instance.

Get and use the SSH Key Pair

Once we have our SSH key pair, it time to update the YAML file:

(As a reminder, how to use this file is explained here)

steps:
  - name: 'gcr.io/cloud-builders/gcloud'
    id: Connect to instance
    entrypoint: /bin/sh
    args:
    - '-c'
    - |
      mkdir -p ~/root/.ssh && \\
      gcloud secrets versions access latest --secret=cloud-build-ssh-key > ~/root/.ssh/id_rsa && \\
      chmod 600 ~/root/.ssh/id_rsa && \\
      gcloud secrets versions access latest --secret=cloud-build-ssh-key-pub > ~/root/.ssh/id_rsa.pub && \\
      chmod 600 ~/root/.ssh/id_rsa.pub && \\
      set -x && \\
      gcloud compute ssh server_name --ssh-key-file=~/root/.ssh/id_rsa --zone=europe-west2-c

I think it’s clear enough?

Feel free to ask any question to help me to improve this article.

Bie.