Cloud Run and Load balancing: go beyond your own project!

guillaume blaquiere
Jan 15 · 9 min read
Image for post
Image for post

Routing and load balancing are the pillars of the Internet and its scalability. On Google Cloud, these aspects are great due to a global networks and anycast IP deployed on Global HTTPS load balancers.
Like the other cloud services, serverless compute products (App Engine, Cloud Functions and Cloud Run) have been getting the load balancing capacity few month ago.

One of the most interesting load balancing feature is the capacity to route the traffic from the Load Balancer to the deployed services the closest to the user location. And thus to have the best latency, wherever the users are.

Serverless NEG for serverless load balancing

For this, and as described in the documentation, blog and articles, you have to use the serverless NEG (Network Endpoint Group) and to add the services, deployed in different region, to it.

Use the serverless NEG as load balancer backend and enjoy!

This situation is perfect when all the services and the load balancer are deployed in the same project. And the serverless NEGs are only compliant with this case

Load balancers using serverless NEG backends must be created in the same project as the Cloud Run (fully managed), App Engine, or Cloud Functions services pointed to by the NEG.

However, the reality is often different. Sometimes, you need to reuse services from other projects for different reason:

  • Billing constraint: each project need to have its own billing because labels aren’t enough in some use cases.
  • Team organisation and deployment velocity: typically a data scientist team that serves a model usable by one or several web teams
  • Security isolation: Least privilege principle with IAM policies

And it’s especially true with Cloud Run because more and more services that we developed are hosted on it.

How to use a load balancer with Cloud Run services deployed in other projects?

Internet NEG solution

Recently, a new type of NEG has been introduced: Internet NEG. Its purpose is to route the requests from the global HTTPs Load Balancer to an internet endpoint, defined by its IP or its fully qualified domain name.

So, that can fit my use case! Instead of calling internally a Cloud Run service with a serverless NEG, we can use an Internet NEG to call it externally!

Image for post
Image for post

Let’s go!

Create an hello world Cloud run

  • In the menu, go to Cloud Run
  • Click on CREATE SERVICE
  • Name it, choose your region. In the step 2 use the example container
Image for post
Image for post
  • Finish by allowing unauthenticated connection, for easier tests

So, now, get the service URL and test it if you want. Keep this URL, we will use it later.

Create an Internet NEG

  • In the menu, go to Compute Engine -> Network endpoint groups
Image for post
Image for post
  • Click on CREATE NETWORK ENDPOINT GROUP
  • Name it, and choose an Network Endpoint Group (Internet) type
Image for post
Image for post
  • Then, set the port 443 by default and select Fully qualified domain name in the Add through section.
Image for post
Image for post
  • Fill in with your Cloud Run service URL (without the https://) and click on create

Perfect, now the latest, and the biggest piece, let’s configure an HTTPs Load Balancer.

Create the global HTTPS Load Balancer

  • In the menu, click on Network services
  • Click on Create a load balancer
  • Click on Start configuration for the HTTPS Load Balancing
Image for post
Image for post
  • Expose from Internet, click on continue in the following screen
  • In the Backend configuration, select a Backend service and Create a backend service
Image for post
Image for post
  • On the new screen, name your backend and, as type, select Internet network endpoint group
Image for post
Image for post
  • Choose the protocol HTTPS to call your Cloud Run service and the name of your previously created Internet network endpoint group
Image for post
Image for post
  • Click on Create on the bottom

That’s over for the backend! Let’s continue the configuration
Let the Host and path rule as-is and go to the frontend configuration

  • Name it and select your protocol. You can choose HTTPS, but HTTP also work and the configuration easier (no certificate and DNS to manage).
Image for post
Image for post
  • Use the simplest configuration for you and click on create.

Now the load balancer is creating; wait few minutes to finish and to dispatch the configuration all around the globe.

Time to test! Go back to your Load Balancer and copy the IPv4.

Image for post
Image for post

Paste it in your browser and.. FAIL!!! Yes FAIL, this solution doesn’t work!
You should have a 404, and that won’t change.

The Cloud Run routing

The issue comes from the Cloud Run routing mode. The URL provided by Cloud Run service isn’t bind to an IP or a CNAME, the routing is based on the host value of the request’s header.

It could be surprising, let’s validate this and have a try with curl directly on the root domain of Cloud Run: run.app. And we will add only the Cloud Run service fully qualified name (URL without https://) in the host header

curl -H "host: <Cloud Run fully qualified name>" https://run.app

It works!! And yes, you can reach any Cloud Run services with this solution, you only have to know the exact URL name of the service.
In fact, it’s not really surprising because Kubernetes and Knative works on the same way.

Now that we know the routing mode, we can go back to the previous configuration and have a fresh look on the issue:

The host in the request’s header is the load balancer IP address (or your own domain name if you set up it).

Therefore, to solve this, we have to override the host header in the backend configuration of our Load Balancer with backend custom header.

Image for post
Image for post

Backend custom header

  • Go back to the load balancer, edit it and go to your backend.
  • On the bottom, click on Advanced configuration
  • Then click on Add header
  • And add host as key, and the Cloud Run fully qualified name of your service as value
Image for post
Image for post
  • And click on Update
  • And again on Update in your load balancer edit page

Here again, wait few minutes for the configuration advertising. And test again your IP.

Boom! Now that works!!

That’s great for 1 service, but how to do with several Cloud Run services?

Multi Cloud Run service routing

When you have several Cloud Run services to call with a load balancer like this, in the same other project or in different project, the immediate idea is to repeat the same pattern for each service

  • Create an Internet NEG
  • Create a backend
  • Customize the host and path rules

Actually, you don’t need to create an Internet NEG per service, you can cheat!

Indeed, because all the Cloud Run services requests land on the https://run.app URL, you can create only one Internet NEG, and use it in multiple backend definition where you customize only host request’s header to route the traffic to the correct Cloud Run service.

Image for post
Image for post

Let’s deploy this!

Create a new service

Start by creating a new Cloud Run service, as previously done. You can use the same example container image.

Get the URL, we will need it later

Update your “generic” Internet NEG

  • Go back your internet NEG
  • You can’t update the current endpoint. So, select it and click on Remove endpoint; Confirm the deletion
  • And click on Add network endpoint.
  • Add simply the value: run.app
Image for post
Image for post
  • And click on Create

And nothing appear. Wait few seconds, the end of the creation and your endpoint appears! It’s an UI glitch.

Add a second backend

  • Go back to your load balancer, clieck on Edit and go to the backend part.
  • Create a backend service and, as previously done, select the Internet NEG type, the HTTPS protocol, select your generic Internet NEG and go to Advanced option to override the host header with the Cloud Run service fully qualified name of your 2nd service.
Image for post
Image for post
  • And hit Create

Customize the host and path rules

This customization can be complex here. But here my requirement:

  • I want to reach the 2nd Cloud Run service on the path /hello2
  • The second Cloud Run service doesn’t have a path /hello2. So I need to rewrite the path to /

Let’s configure this!

  • Select Advanced host and path rule radio button and click on Add host and path rule
Image for post
Image for post
  • Firstly, define the host name filtering. In my case, don’t care. I set *
  • Then, click on the pencil of the default path, and select the backend by default. This one which answer on the / path on the load balancer
Image for post
Image for post
  • And Save
  • Then, click on Add path rule to configure the second service
  • Add the path, in my case /hello2 and select my second backend.
  • But, I need to rewrite the path. For this, click on Add-on action and, on Path prefix rewrite set /
Image for post
Image for post
  • And click on Save
  • And click on Update to save the new configuration of your load balancer

Wait again few minutes and test your IP on the / path and on the /hello2 path.

Congrats!!! You have a load balancer with several Cloud Run services deployed in different projects!!

Routing to the closest location

This configuration is good, but you loose one of the main advantage of the a global HTTPS load balancer: the capacity to route the request to the closest location.

In this case, you have to combine

  • The Serverless NEG: for the multi-region service as backend of a HTTPS load balancer and all in the same project
  • The Internet NEG: to access externally to the load balancer of the multi-region Cloud Run services.

Because of the constraint of the Serverless NEG, the solution works only if the multi-region Cloud Run services are deployed on the same project.
In other words, you can’t have configuration with a project per region.

To achieve this, 2 main steps:

  1. Firstly, create a HTTPS load balancer in the multi-region project. Use standard Serverless NEG. Here, it’s important to use the HTTPS protocol for the frontend configuration.
    It’s the documented configuration.
  2. Then, create another HTTPS load balancer, in another project, with an Internet NEG that use the IP or the domain name of the multi-region HTTPS load balancer.
    This time, no need to override the host request’s header value because you don’t reach directly the Cloud Run services, but only another load balancer which will perform this operation automatically through the Serverless NEG.
Image for post
Image for post

Because the communication between the Internet NEG and the multi-region HTTPS load balancer is over the Internet, HTTPS is strongly recommended in the documentation

We strongly recommend you use HTTPS or HTTP/2 as the protocol when configuring a backend service with an internet NEG, so that communication between the external HTTP(S) load balancer and your backend is encrypted and authenticated when transiting the public internet.

Go beyond the limits!

Even if these solutions seem easy to implement and use, it’s important to keep in mind that each hop (through a load balancer) implies:

  • Latency, even few milliseconds.
  • New possible point of failure in your architecture
  • Additional cost for the load balancers

Eventually, the project-only limitations of Serverless NEG are not limits thanks to Internet NEG capacity. And with a composition of load balancers, request headers and URL mapping, it’s possible to address all the use cases!

Google Cloud - Community

Google Cloud community articles and blogs

guillaume blaquiere

Written by

GDE Google Cloud Platform, scrum master, speaker, writer and polyglot developer, Google Cloud platform 3x certified, serverless addict and Go fan.

Google Cloud - Community

A collection of technical articles and blogs published or curated by Google Cloud Developer Advocates. The views expressed are those of the authors and don't necessarily reflect those of Google.

guillaume blaquiere

Written by

GDE Google Cloud Platform, scrum master, speaker, writer and polyglot developer, Google Cloud platform 3x certified, serverless addict and Go fan.

Google Cloud - Community

A collection of technical articles and blogs published or curated by Google Cloud Developer Advocates. The views expressed are those of the authors and don't necessarily reflect those of Google.

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app