Accessing Google APIs via Private Service Connect(PSC) and Private Google Access (PGA)

Introduction

Applications running in Google Cloud within a VPC network in resources like VMs, GKE etc may need to interact with other GCP resources like Cloud Storage, Big Query etc. When your applications within your Google Cloud VPC need to communicate with Google APIs (e.g., BigQuery, Cloud Storage, etc.), the data leaving your VPC to the internet typically incurs egress charges. Also traffic flowing via the internet will incur additional latencies apart from cost.

GCP provides two solutions Private Service Connect (PSC) and Private Google Access(PGA) , to enable applications within the GCP VPC network to access the GCP specific APIs within google network. This will enable the traffic to flow within the Google network thereby reducing costs, enabling better performance and keeping the network transmission more secure and reliable. In this article, we will look at PSC and PGA for accessing Google APIs privately.

Solution

Private Service Connect

Private Service connect enables applications to connect to Google APIs via private endpoints thereby minimising costs, providing better performance and reliability List of Google APIs supported by PSC. Lets go through the detailed steps of setting up a private service connect to access “Google Cloud Storage API to List Buckets”

SSH into a VM instance running within a VPC in GCP and access the below cloud storage api via CURL. When you perform traceroute storage.googleapis.com you will notice that the traffic travel via the internet by default.

curl -H “Authorization: Bearer $(gcloud auth application-default print-access-token)” https://storage.googleapis.com/storage/v1/b?project=<PROJECT_ID>

Trace route

The above trace route shows that it traverses via Internet.

To Route the traffic internally via Google network configure a PSC endpoint as shown below.

Create PSC endpoint

PSC endpoint for all Google APIs

As shown above choose “All Google APIs” and specify an IP address from the range. Also create a Service Directory Namespace(purpose of this will be described below). Click on Add Endpoint to create the PSC endpoint.

A PSC endpoint also results in a Service Directory endpoint within the namespace as depicted below. Service Directory provides a common service endpoint directory to group the endpoints of different services.It serves as a single place to keep track of all the services you need to build robust, distributed applications. Inter service communication can happen based on these endpoints. Detailed discussion of Service directory is outside the scope of this article.

Service Directory created for psc endpoint (10.1.1.1)

Service Directory is integrated with Cloud DNS, and can automatically populate DNS records as services are added to it. Here the DNS entries are created for the service directory namespace as shown below. Each PSC endpoint that is added as a Service Directory endpoint can be resolved as DNS subdomain.

Cloud DNS Entry

Access via PSC

curl -H “Authorization: Bearer $(gcloud auth application-default print-access-token)” https://storage-testgoogleapi.p.googleapis.com/storage/v1/b?project=<PROJECT_ID>

NOTE: Cloud DNS entry is created for googleapis.com APIs alone. For other google apis like gcr.io, *.cloudfunctions.net etc., the entries have to be created manually

If the Cloud DNS entry does not exist or if you want to use an API from from googleapis.com like gcr.io, then perform below steps

1. Create an A record for the domain (zone) name itself; for example, googleapis.com or gcr.io. Point this A record to the IP address of the PSC endpoint created above. If you're using Cloud DNS, see adding a record.
2. Create a CNAME record for all of the additional domain's possible host names by using an asterisk and a dot followed by the domain (zone) name; for example, *.googleapis.com or *.gcr.io. Point this CNAME record to the A record in the same zone. For example, point *.googleapis.com to googleapis.com or point *.gcr.io to gcr.io.

Private Google Access

Private Google Access (PGA) enables users to access the Google APIs privately and securely within the google network, without routing to the internet. It exposes a specific set of IP addrress ranges as published in the documentation to which we can route the Google APIS by creating DNS specific entries. For e.g you can decide to route only cloud storage GCP APIs via PGA by creating a DNS entry as below.

With this any request to storage.googleapis.com will be routed via PGA within the GCP network. To route all google APIS you can replace storage.googleapis.com with googleapis.com (*.googleapis.com). Refer documentation on creating DNS entries and the IP address ranges of PGA.

Note: Though traffic is not route via Internet PGA requires that the default route to Internet Gateway is present in the route table either for all IP address (0.0.0.0/0 default route) or for the PGA IP ranges specific address. Refer

curl -H “Authorization: Bearer $(gcloud auth application-default print-access-token)” https://storage.googleapis.com/storage/v1/b?project=<PROJECT_ID>

The above curl command should be routed via PGA IP address range once its configured. You can verify this with traceroute storage.googleapis.com and it should print one of the 4 IP address range that is configured in Cloud DNS for PGA (119.36.153.8, 119.36.153.9, 119.36.153.10, 119.36.153.11)

Conclusion

Private Service Connect(PSC) and Private Google Access(PGA) are options that GCP provides to access GCP APIs and they should be leveraged for security, privacy and cost factors. Also PGA requires default Internet Gateway route to be present which may not be allowed in a lot secure environments. Hence these scenarios PSC can be chosen.

Also another aspect is PSC entails changes in the code which invokes the Google APIs as the endpoint is changed, whereas in case of PGA the code that invokes the Google APIs is agnostic of this and the routing is taken care via Cloud DNS entries to route the call via PGA IP ranges.

# Replace ‘your-psc-endpoint’ with your actual PSC endpoint URL

client = storage.Client(api_endpoint=”your-psc-endpoint”)

Both options can also be used from on prem or from a different cloud provider such as AWS, Azure to connect to Google APIs