GCP VPC SC with Shared VPC Network

Quick Summary (TL;DR)

This article describes the limitation and workaround solution of using VPC Service Controls (VPC-SC) with Shared VPC in Google Cloud Platform.

Issue arises from the fact that VPC-SC observes service project VMs’ as resources of Shared VPC (instead of service project resources). On one hand if VPC-SC is enabled using only service project, then its own VMs are treated as resources outside perimeter. On other hand, if Shared VPC is included to same VPC-SC along with service project then VMs of other service projects using that Shared VPC are also included in same perimeter. Thus, other service project VMs are not able to connect to resources. These issues are explained in more detail in sections below.

Workaround solution is by creating VPC-SC only on service project and then stitch GCP VMs’ access by adding public IPs to Access Context Manager. More details in sections below.

What is VPC-SC (VPC Service Control)

In layman language, it is firewall for GCS, BigQuery, Bigtable and other supported services. It gives information security teams peace of mind that no-one can access data contained in these services from unauthorized networks. In other words it disables data exfiltration. Google Cloud documentation here.

What is Shared VPC Network

Shared VPC as its name suggests is a VPC Network created within a GCP project and is shared to multiple other projects. The project containing Shared VPC is called Shared VPC Host and project using its network are called Service Project. GCP documentation here.

Issue in implementing VPC-SC with Shared VPC

Often with a Shared VPC, multiple independent workloads/applications used same Shared VPC. While one of these teams require VPC-SC but other teams don’t.

Below diagram describes how VPC-SC observes the current setup.

Now in above diagram, let us suppose only Project A has sensitive data in GCS and BQ and therefore VPC-SC is required. This can be achieved in following ways:

1. VPC SC perimeter with Shared VPC Host project + Service Project A + Access Context Manager (ACM) with public IPs of external network. Drawback is Service Project B VMs will not be able to talk to GCS or BQ in its own project.
2. VPC-SC with Shared VPC Host + All Service Projects + ACM of external network. Drawback is some products are not compatible with VPC-SC (documented here) like AppEngine, Cloud Functions, Dataflow etc. This can restrict other teams who do not require VPC-SC but still want to use serverless services. Second drawback is, lack of multiple VPC-SC for each team. This increases attack surface as breach in perimeter can cause more damage.
3. VPC-SC with Service Project A + ACM of external network + ACM of shared network VMs’ public IPs. While this fits most of the use cases, drawback is maintaining list of VM’s public IPs in ACM as new VMs can spun up anytime. (we will fix this drawback in solution).

Solution for using VPC-SC with Shared VPC

One of the simple solution is to create VPC-SC with Service Project A + ACM with IPs of external network + ACM with public IPs of shared network VMs. However to make it practically useful we need to mitigate its drawback which is need to maintain list of VM’s public IPs in ACM as new VMs can spun up anytime. Workaround for above drawback is use NAT Gateways.

Using ACM we can whitelist public IPs which are allowed to access VPC-SC perimeter. Thus, we create an ACM list containing public IPs of NAT. Thus any VM hosted on shared network will be allowed communicate with resources inside VPC-SC.

Broad steps to implement this solution are:

1. Create NAT Gateways in Shared VPC project using compute engine VMs. Assign static external IPs to NAT VMs. Create required routes. Step by step implementation for NAT Gateway is described here.
   NOTE: Cloud NAT does not works along with this workaround yet. (authoring date: June 25, 2019).
2. None of the service project VMs should have external IPs instead they should be using NAT Gateway only.
3. Create Access Context Manger (ACM) for whitelisting NAT IPs of shared network (plus public IPs of any other authorized network).
4. Create VPC-SC only with Service Project and link it to ACM above.

What is NAT Gateway

It allows us to use one external IP address to send traffic from multiple virtual machine instances but only expose a single virtual machine to the Internet. Other benefits of NAT Gateway is ability to easily monitor traffic escaping to internet.