Routing and load balancing are the pillars of the Internet and its scalability. On Google Cloud, these aspects are great due to a global networks and anycast IP deployed on Global HTTPS load balancers.
Like the other cloud services, serverless compute products (App Engine, Cloud Functions and Cloud Run) have been getting the load balancing capacity few month ago.
One of the most interesting load balancing feature is the capacity to route the traffic from the Load Balancer to the deployed services the closest to the user location. And thus to have the best latency, wherever the users are.
Serverless NEG for serverless load balancing
Use the serverless NEG as load balancer backend and enjoy!
This situation is perfect when all the services and the load balancer are deployed in the same project. And the serverless NEGs are only compliant with this case
Load balancers using serverless NEG backends must be created in the same project as the Cloud Run (fully managed), App Engine, or Cloud Functions services pointed to by the NEG.
However, the reality is often different. Sometimes, you need to reuse services from other projects for different reason:
- Billing constraint: each project need to have its own billing because labels aren’t enough in some use cases.
- Team organisation and deployment velocity: typically a data scientist team that serves a model usable by one or several web teams
- Security isolation: Least privilege principle with IAM policies
And it’s especially true with Cloud Run because more and more services that we developed are hosted on it.
How to use a load balancer with Cloud Run services deployed in other projects?
Internet NEG solution
Recently, a new type of NEG has been introduced: Internet NEG. Its purpose is to route the requests from the global HTTPs Load Balancer to an internet endpoint, defined by its IP or its fully qualified domain name.
So, that can fit my use case! Instead of calling internally a Cloud Run service with a serverless NEG, we can use an Internet NEG to call it externally!
Create an hello world Cloud run
- In the menu, go to Cloud Run
- Click on
- Name it, choose your region. In the step 2 use the example container
- Finish by allowing unauthenticated connection, for easier tests
So, now, get the service URL and test it if you want. Keep this URL, we will use it later.
Create an Internet NEG
- In the menu, go to
Network endpoint groups
- Click on
CREATE NETWORK ENDPOINT GROUP
- Name it, and choose an
Network Endpoint Group (Internet)type
- Then, set the port 443 by default and select
Fully qualified domain namein the
Perfect, now the latest, and the biggest piece, let’s configure an HTTPs Load Balancer.
Create the global HTTPS Load Balancer
- In the menu, click on
- Click on
Create a load balancer
- Click on
Start configurationfor the HTTPS Load Balancing
- Expose from Internet, click on
continuein the following screen
- In the
Backend configuration, select a
Create a backend service
- On the new screen, name your backend and, as type, select
Internet network endpoint group
- Choose the protocol HTTPS to call your Cloud Run service and the name of your previously created Internet network endpoint group
- Click on
Createon the bottom
That’s over for the backend! Let’s continue the configuration
Let the Host and path rule as-is and go to the frontend configuration
- Name it and select your protocol. You can choose HTTPS, but HTTP also work and the configuration easier (no certificate and DNS to manage).
- Use the simplest configuration for you and click on create.
Now the load balancer is creating; wait few minutes to finish and to dispatch the configuration all around the globe.
Time to test! Go back to your Load Balancer and copy the IPv4.
Paste it in your browser and.. FAIL!!! Yes FAIL, this solution doesn’t work!
You should have a 404, and that won’t change.
The Cloud Run routing
The issue comes from the Cloud Run routing mode. The URL provided by Cloud Run service isn’t bind to an IP or a CNAME, the routing is based on the
host value of the request’s header.
It could be surprising, let’s validate this and have a try with
curl directly on the root domain of Cloud Run:
run.app. And we will add only the Cloud Run service fully qualified name (URL without
https://) in the
curl -H "host: <Cloud Run fully qualified name>" https://run.app
It works!! And yes, you can reach any Cloud Run services with this solution, you only have to know the exact URL name of the service.
In fact, it’s not really surprising because Kubernetes and Knative works on the same way.
Now that we know the routing mode, we can go back to the previous configuration and have a fresh look on the issue:
hostin the request’s header is the load balancer IP address (or your own domain name if you set up it).
Therefore, to solve this, we have to override the
host header in the backend configuration of our Load Balancer with backend custom header.
Backend custom header
- Go back to the load balancer, edit it and go to your backend.
- On the bottom, click on
- Then click on
- And add
hostas key, and the Cloud Run fully qualified name of your service as value
- And click on
- And again on
Updatein your load balancer edit page
Here again, wait few minutes for the configuration advertising. And test again your IP.
Boom! Now that works!!
That’s great for 1 service, but how to do with several Cloud Run services?
Multi Cloud Run service routing
When you have several Cloud Run services to call with a load balancer like this, in the same other project or in different project, the immediate idea is to repeat the same pattern for each service
- Create an Internet NEG
- Create a backend
- Customize the host and path rules
Actually, you don’t need to create an Internet NEG per service, you can cheat!
Indeed, because all the Cloud Run services requests land on the
https://run.app URL, you can create only one Internet NEG, and use it in multiple backend definition where you customize only
host request’s header to route the traffic to the correct Cloud Run service.
Let’s deploy this!
Create a new service
Start by creating a new Cloud Run service, as previously done. You can use the same example container image.
Get the URL, we will need it later
Update your “generic” Internet NEG
- Go back your internet NEG
- You can’t update the current endpoint. So, select it and click on
Remove endpoint; Confirm the deletion
- And click on
Add network endpoint.
- Add simply the value:
- And click on
And nothing appear. Wait few seconds, the end of the creation and your endpoint appears! It’s an UI glitch.
Add a second backend
- Go back to your load balancer, clieck on
Editand go to the backend part.
Create a backend serviceand, as previously done, select the Internet NEG type, the
HTTPSprotocol, select your generic Internet NEG and go to
Advanced optionto override the
hostheader with the Cloud Run service fully qualified name of your 2nd service.
- And hit
Customize the host and path rules
This customization can be complex here. But here my requirement:
- I want to reach the 2nd Cloud Run service on the path
- The second Cloud Run service doesn’t have a path
/hello2. So I need to rewrite the path to
Let’s configure this!
Advanced host and path ruleradio button and click on
Add host and path rule
- Firstly, define the host name filtering. In my case, don’t care. I set
- Then, click on the pencil of the default path, and select the backend by default. This one which answer on the
/path on the load balancer
- Then, click on
Add path ruleto configure the second service
- Add the path, in my case
/hello2and select my second backend.
- But, I need to rewrite the path. For this, click on
Add-on actionand, on
Path prefix rewriteset
- And click on
- And click on
Updateto save the new configuration of your load balancer
Wait again few minutes and test your IP on the
/ path and on the
Congrats!!! You have a load balancer with several Cloud Run services deployed in different projects!!
Routing to the closest location
This configuration is good, but you loose one of the main advantage of the a global HTTPS load balancer: the capacity to route the request to the closest location.
In this case, you have to combine
- The Serverless NEG: for the multi-region service as backend of a HTTPS load balancer and all in the same project
- The Internet NEG: to access externally to the load balancer of the multi-region Cloud Run services.
Because of the constraint of the Serverless NEG, the solution works only if the multi-region Cloud Run services are deployed on the same project.
In other words, you can’t have configuration with a project per region.
To achieve this, 2 main steps:
- Firstly, create a HTTPS load balancer in the multi-region project. Use standard Serverless NEG. Here, it’s important to use the HTTPS protocol for the frontend configuration.
It’s the documented configuration.
- Then, create another HTTPS load balancer, in another project, with an Internet NEG that use the IP or the domain name of the multi-region HTTPS load balancer.
This time, no need to override the
hostrequest’s header value because you don’t reach directly the Cloud Run services, but only another load balancer which will perform this operation automatically through the Serverless NEG.
Because the communication between the Internet NEG and the multi-region HTTPS load balancer is over the Internet, HTTPS is strongly recommended in the documentation
We strongly recommend you use HTTPS or HTTP/2 as the protocol when configuring a backend service with an internet NEG, so that communication between the external HTTP(S) load balancer and your backend is encrypted and authenticated when transiting the public internet.
Go beyond the limits!
Even if these solutions seem easy to implement and use, it’s important to keep in mind that each hop (through a load balancer) implies:
- Latency, even few milliseconds.
- New possible point of failure in your architecture
- Additional cost for the load balancers
Eventually, the project-only limitations of Serverless NEG are not limits thanks to Internet NEG capacity. And with a composition of load balancers, request headers and URL mapping, it’s possible to address all the use cases!