Multi Region Load Balancing with GO and Google Cloud Run — Part 2
In part 2, we will provision and configure the required infrastructure for our multi region load balanced Cloud Run service.
Introduction
So in part 1, we built a simple golang API service, built a Dockerfile to build our project and then compiled it and uploaded the final image to Google container registry. We also created a terraform service account that we can use to provision the required infrastructure in this chapter. By the end of this article we will have all the required infrastructure in place and have our new service up and running in four regions. Then we will be ready to test our service in the final instalment of our tutorial in part 3.
Code and Documentation
You can find all the code and documentation used in this article on GitHub.
https://github.com/robcharlwood/multi-region-cloud-run-terraform.
Creating the infrastructure
A couple of notes before we get started…
- UPDATED 2020–09–03 : The code in this repo has been updated to work with Terraform 0.13.2.
- Recommended practice is to store terraform state (the current state of the infrastructure that terraform provisions and tracks) into a Google Cloud Storage bucket. However, for simplicity, this tutorial will just keep state on your local machine. Please see the Terraform and google provider documentation for more information on this.
- UPDATED 2020–09–01: Serverless Network Endpoint Groups are now fully supported in the
google-betaterraform provider. The accompanying codebase has been fully updated to make use of this new functionality and the following tutorial has been updated to match.
Let’s get ready to rumble!
So with that out of the way, let’s get setup ready to create our infrastructure.
- Clone the terraform repository to your desired location and change into that directory
git clone git@github.com:robcharlwood/multi-region-cloud-run-terraform.git2. Copy your terraform service account key from part 1 into a .keys directory in the repo root. Don’t worry, .keys is added to .gitignore so you can’t accidentally commit this file into source control.
3. Update the terraform.tfvars file with details applicable for your project. For an example see below:
project = "multi-region-cloud-run"region = "europe-west1"image_name = "multi-region-cloud-run"image_version = "0.0.1"registry = "eu.gcr.io"domain = "example.com"
What are we actually going to build?
We are now setup and ready to configure our infrastructure. But before we do, let’s run over the resources that are created during the process, why we need them and what they do.
Custom Cloud Run Service Account
Whilst the default cloud run service account is fine for our purposes, we are going to define our own service account. This will allow us to add extra permissions as the app grows or lock down the permissions should we find there are some that we never use.
Static IP address
Our infrastructure requires a static IP address. This gets assigned to our load balancer and ensures that we have a persistent IP address that we can point our domain name’s A records at. Without this, the load balancer would be provisioned with an ephemeral IP address which could change at any time— not great for a high availability service! :)
DNS Records
In order to access our new service via it’s domain name, we need to provision some DNS records via Google Cloud DNS. We will point the domain’s A records at our new static IP address which is assigned to our new load balancer.
Please note, that in order for these DNS changes to work, you’ll need to login to your domain registrar and point your domain’s name servers to the name servers that are assigned to your DNS zone once we’ve applied our terraform. More on this later.
SSL — Google Managed Certificate
Cloud Run requires HTTPS and so we have to provision an SSL certificate that can be applied to our domain name. The code in this repository will provision a Google managed SSL certificate and then assign it to our load balancer for a fully secure connection. You can use your own SSL certificate here if you wish, but that is beyond the scope of this article.
Compute — Cloud Run
Next, we need to provision our Cloud Run resources. The code in this repository will provision instances of our service in europe-west1, us-east1, us-west1 and asia-northeast1. It will run the services under the service account described above and will make the service publicly accessible. It will also set the instances up with some basic CPU and memory allocations.
Serverless Network Endpoint Groups and Load Balancer
The final part of our infrastructure is the Load Balancer and Serverless Network Endpoint Groups.
We utilise four of Google’s new serverless NEGs in this repository. Each NEG is bound to a region e.g europe-west1 and points at the relevant cloud run service running in the same region.
These NEGs are then attached to a global backend. This backend is assigned to our load balancer which has the SSL certificate and static IP address bound to it. A global forwarding rule is then created to forward all traffic from the static IP on port 443 to our backend.
For the curious among you, you can find the meat of what makes multi region load balancing possible here.
Let’s apply ourselves
Now we know what our terraform is going to do and have taken the required setup steps, let’s put our infrastructure in place and get our service up and running on Google Cloud Platform!
terraform init
terraform plan
terraform applyOnce these commands are complete you should be able to see all infrastructure in Google Cloud Console with the exception of the serverless NEGs. This is because this functionality is still in beta and so Google have not yet made it available via the UI.
Domain changes
Great! So all our infrastructure is now in place and we now have instances of our service running in four regions across the globe! How cool is that?!
However, we can’t currently access our application via the load balancer as our domain’s name servers do not yet point to our Google DNS zone’s name servers. We could try and access it via the static IP address, however this would most likely fail since Cloud Run requires an HTTPS connection and we don’t have an SSL certificate for the IP address, only the domain name.
So to get this working, head to the Cloud DNS page in the Google Cloud Console, click on the zone that terraform created for us and grab the 4 name servers from the list — these are normally in the format of ns-cloud-aXXX.googledomains.com where XXX is a number from 1 to 4.
Once you have these name servers, you need to head to your domain registrar and update your domain’s name servers to point to these addresses. My domain was with Google domains, so this change was a breeze!
This change could take anywhere from 24 to 48 hours to fully propagate so don’t panic if your service is not immediately available.
Once the name server changes have fully propagated, you’ll need to check that your SSL certificate has been successfully provisioned. You can see this from the Network Services -> Load Balancing -> Advanced Menu -> Certificates. Click on your certificate and you should get its status.
Once the SSL certificate is fully provisioned, it might take the load balancer a further 10 minutes or so to pick up the changes and serve your service over HTTPs without any errors.
We should now be able to hit our service at our domain name! I am using Postman in the screenshot to hit my service.
Let’s wrap it up!
Wow! There was a lot covered here! In this part we ran some terraform to provision all the required infrastructure for our multi region cloud run service — including the load balancer and serverless network endpoint groups. We also pointed our domain’s name servers to those requested by Google.
Join me again in part 3, where we test and prove that our service is load balancing across all the available regions and serving requests from the region closest to the user by default. See you there! :)
