Increase security and reduce costs through VPN connections between AWS and GCP step by step — (1) How to connect to VPNs

12 min readAug 6, 2023

VPNs in a Multi-Cloud Environment

Maintaining efficient data communication and security in a multi-cloud environment requires a secure and reliable connection between each cloud. Virtual private networks (VPNs) play an important role to do this. VPN allows you to securely transfer data by creating an encrypted connection between multi-cloud environments such as AWS and GCP. I’m going to explain how to connect a site-to-site VPN between AWS and GCP very easily, and besides this post, I also introduce how to query domains in each cloud environment in the following post, and it would be good if you read it in order.

Benefits of VPN Connectivity

Secure Connection: Site-to-site VPNs protect communication between networks by creating encrypted tunnels over the Internet. This keeps your data safe while it is being transmitted over the Internet.
Data Security: Site-to-site VPN encrypts data transmission to prevent unauthorized access and data leakage from outside. This improves the security of business data and protects sensitive information from exposure.
Efficient Data Communication: The isolated VPCs of each Cloud Service Provider (CSP) can be linked to enable direct private communication and direct communication without using a public IP when communicating with the CSP’s Managed Service.
Cost savings: Introducing VPN for security can also bring cost-saving benefits.

The AWS-GCP architecture above is an example of when an AWS server sent a request for the Google API (googleapis.com ). (In this post, we will explain only how to make VPNs between AWS and GCP, and I will explain one by one in detail in the next post.)
In some cases, all workloads are operated from one Cloud Provider, but in many cases, multi-cloud is used to meet the needs of the enterprise.
For example, workloads for customer services are run by AWS, and workloads for analysis are run by GCP, using multiple Cloud Providers for each workload.
VPN connection and DNS forwarding operations between each cloud service allow for requests to AWS -> GCP api, requests to GCP -> AWS api, or direct private communication between servers in different cloud environments. Therefore, separate network environments configured on AWS and GCP can communicate as if they were in one network environment.
If you have a private dns zone in AWS Route53 and GCP Cloud DNS, by default you can only do domain lookups on connected VPCs. But I’ll go into more detail in other blog posts about how to look up domains from AWS to Cloud DNS Private Zone and from GCP to Route53 Private Zone.
Basically, in the case of VPN connection between AWS and GCP, since there is a physical distance, the VPNs must be configured in the regions as close as possible to CSPs to prevent unnecessary communication delays.
e.g) AWS ap-northeast-2(Seoul) — GCP asia-northeast3(Seoul)
In AWS, VPC is a resource with limited Region, and GCP VPC is a global resource. In addition, in AWS, Subnet is a resource with a limited Zone, and GCP Subnet is a resource with a limited Region. Therefore, GCP can have multiple regions of subnets within a VPC.

AWS to GCP VPN Interworking Scenarios

The advantages of connecting AWS and GCP have been briefly described above, but I will tell you the advantages of comparing them before and after the VPN linkage through a brief hypothetical scenario.

[Cost Savings] AWS runs the service, but sends large amounts of data to GCP periodically

Suppose AWS sends a large number of data from AWS to GCP for analysis by GCP BigQuery, as shown above.

If VPN is not linked, traffic will flow through the path of AWS VM -> NAT Gateway -> Internet -> GCP BigQuery whenever data is sent to BigQuery.
The network traffic related expenses incurred in the above traffic flow are as follows (based on Seoul).
Charges per Data handled by NAT Gateway: $0.059 (per GB)
Outbound data transfer fee: $0.126 (per GB)
However, if a large amount of data sent for analysis is egressed through the NAT Gateway to the Internet environment, the data size charges Outbound data transfer fee $0.059 (per GB) and NAT Gateway charges $0.126 (per GB) are duplicated. If you send data in tens to hundreds of TB or more, you’ll be charged a fairly large amount of data.

However, as shown above, when using Google APIs on AWS, if traffic travels through the VPN section rather than through the NAT Gateway, the network traffic-related costs associated with sending to BigQuery will be reduced.
With VPN interworking, whenever data is sent to BigQuery, traffic will flow through the path of AWS VM -> TGW -> VPN Tunnel -> GCP Private Service Connect (PSC) -> GCP BigQuery.
The network traffic related expenses incurred in the above traffic flow are as follows (based on Seoul).
Charges per data processed by TGW: $0.02 (per GB)
Outbound data transfer fee: $0.126 (per GB)
Of course, Outbound data transfer rates remain the same, but Gateway changes to NAT GW -> TGW, reducing the processing costs on Gateway by nearly one-third (if you want to reduce data transfer rates, consider Direct Connect, not VPN).

[Increased security] GCP needs to communicate directly with AWS server

As shown above, you may need to access the api of a server built on AWS from a GCP server, and let’s assume that the api is an api that can only be used in the same AWS VPC environment.
If you do not make the server available directly in a public Internet environment or expose only a specific api path that you want to use to the public Internet environment as an exception, there is no way for GCP to use it.

If VPN interworking allows private communication with the GCP subnet, the APi of the AWS server can be used on the GCP server without publicly exposing the APi that is used separately and privately.
And traffic travels through VPN tunnels to ensure secure communication.
In addition, even if the domain for that api is in the AWS Route53 Private Hosted Zone, which can only be queried by the associated AWS VPC, you can also make GCP VM to enable domain queries to AWS Route53 with setting up a few things in GCP ans AWS.

VPN Interworking Method

How to set up AWS VPN

1. Create GCP VPN Gateway

Because AWS Customer Gateway settings require external IP settings of the counterpart (GCP) Gateway, you first create a VPN Gateway in GCP.

Check the external ip issued automatically.
Be careful not to expose the external ip to the outside.
Permissions for that VPN service are controlled so that only the person responsible for the network can verify them.
If you manage vpn settings in code such as terraform, we recommend that you put public ip in secret manager and use it as a reference to that resource in code.

2. Create AWS Customer Gateway

Create a Customer Gateway in AWS VPC.
The BGP ASN is configured as the GCP-side ASN.
You can find it on the GCP Cloud Router.
Because GCP ASN can only select numbers between ‘16550’, ‘64512 and 65534’, and ‘4200000000 to 4294967294’, if you do not already configure GCP Cloud Router and set up Customer Gateway, set the ASN that is not overlapped with the ASN used by the network in the mentioned range.
IP address is set to the IP allocated by the GCP VPN Gateway.
Commercial environments create two Customer Gateways for high availability.

3. Create VPN Connection

Create one VPN Connection per Customer Gateway. Therefore, create a total of two VPN Connections.
Target Gateway types include Virtual Private Gateway (VGW) and Transit Gateway (TGW). Let’s set TGW as Target Gateway here.
If you work with VGW, even if you have two VPN Connections, you must use them as Active/Standby.
Interworking with TGW allows VPN Connection to be used as an active/active structure. However, you will be charged separately per GB ($0.02) for the data processed by the TGW. (Note)

All VPN Connections have been created, and two tunnels per connection have been set.
As each tunnel is created, one outside IP address is assigned, just as the GCP VPN Gateway was created above.
The public ip must also be controlled so that only the network administrator operating the VPN can see it.
In addition, an inside ip of /30 is assigned per Tunnel.
It’s obvious, but the GCP hasn’t done Tunnel Setting yet, so the IPSEC Tunnel is down.
You can also set detailed options for each tunnel, but we won’t do any additional settings because it is for interlocking tests.
Click ‘Download configuration’ in the upper right corner to download the configuration file for the VPN connection.

4. Save VPN Connection configuration

You can download the configuration by selecting VPN Connection and clicking ‘Download configuration’.
I don’t work with specific On-Premise equipment, so Vendor will choose and download the famous and familiar Cisco equipment. A ‘.txt’ file with the VPN ID name will be created on your PC.

If you check the corresponding txt file, you can see ‘pre-shared-key’ as well as ‘Outside/Inside IP Address’ that you just checked on the AWS console. Save all of these values because they are used to create the GCP VPN Connection.
Be careful not to leak the pre-shared-key because it must also be managed with a secret value.

How to set up a GCP VPN

1. Create Peer VPN Gateway

Just as AWS created the Customer Gateway, I also create the ‘Peer VPN Gateway’ in GCP.

Enter the outside IP address assigned when you created the AWS VPN Connection.
I created 2 CGWs on AWS, and when I created a VPN connection for each CGW, I created 2 tunnels for each VPN connection, so I need to enter 4 IP addresses allocated for each tunnel.

2. Create Cloud Router (If you already have one, skip it)

![](https://i.imgur.com/5Mk5zau.png

You must have a Cloud Router to create a VPN Tunnel.
If there is a cloud router that was previously operated in the VPN linkage region, you don’t have to create it separately, but if there is no cloud router, please create it yourself.
You can set ‘Advertised routes’ to advertise all connected subnets, and you can customize the advertising of other GCP bands other than Subnet.

3. Create VPN Tunnels

Configure the VPN Tunnel as ‘ADD VPN TUNNEL’ on the first VPN Gateway you created.

3–1 Create VPN Tunnels

Select the AWS vpn gateway that you have already created as Peer vpn gateway.
Select ‘Create 4 VPN tunnels’ because High availability works with AWS.
For Cloud Router, enter the Router you just created in the previous course, or select it if you already have a Cloud Router.
You must enter information about the 4 VPN Tunnels. Refer to the AWS VPN Connection configuration file and enter the required values.
In ‘Associated peer VPN gateway interface’, enter the interface that you entered when you created the GCP peer VPN gateway one by one.
For IKE version, enter the value you set when you created the AWS VPN Connection.
Enter the IKE pre-shared key by referring to the configuration file.
It may be confusing because there are four connections, but try to enter the correct value for each connection.

3–2. Configure BGP Sessions

After you configure Tunnel, you must configure BGP sessions for each Tunnel.
You have to set it up for each tunnel.

Enter the ASN used by AWS because Peer ASN means AWS-side ASN.
When creating AWS VPN Connection, I entered ‘Transit Gateway’, but if you check the details of the TGW, you can check the set ASN value.
BGP IPv4 address refers to ‘Inside IPv4 CIDR’ in the /30 range assigned when creating Tunnel for ‘AWS VPN Connections’.
Since it’s an IP in the /30 band, you can actually allocate two IPs, but since AWS has configured the corresponding BGP IP (‘Inside IPv4 CIDR’), set the smaller number ip to AWS IP (‘BGP peer IPv4 address’).
Set the remaining ip(larger one) to GCP IP (‘Cloud Router BGP IPv4 address’).

I referred to the inside IPv4 CIDR band of the first Tunnel in AWS VPN Connections.
Sets the remaining BGP sessions daily as well.

Check VPN Tunnel status

You can check the GCP VPN Tunnel/BGP Status.

The same can be found on AWS.

Communication test

You can set up a vm for testing on aws subnet and gcp subnet one by one to proceed with the communication test. However, if we proceed with the above configuration only and conduct communication tests immediately with ‘ping’ between vm’s, communication may be possible. I will summarize the parts that I often miss or make mistakes below.

Things that need to be checked and additionally configured before the communication

1. Set static routing to TGW for GCP CIDR band in aws subnet route table

Through the VPN linkage above, AWS TGW understands the routing path because it receives the band from GCP for the GCP CIDR band, but there is no separate target for the GCP CIDR band in the Route Table of VPC Subnet, which requires actual communication testing. Therefore, communication will be possible only when the GCP CIDR band is designated as Destination and the target is designated as TGW in Subnet’s Route Table, which requires communication.

And if you have multiple GCP CIDRs that need to be communicated, you can easily manage them by registering and managing the AWS Managed Prefix List (Note).

2. GCP Firewall, AWS Security Group/ACL, etc. allow counterpart cidr list

If you set up the VMs that need to be tested for communication on AWS and GCP, you will also have firewall settings. The corresponding firewall settings must allow the other side CIDR band to perform the communication test normally.

Communication Test (General Internet vs. VPN)

1. General Internet (from aws to gcp)

Results of pinging gcp vm in Seoul/Oregon/Iowa from awsec2 in Seoul (General Internet)

seoul -> seoul avg: 3.3ms
seoul -> oregon avg: 119.1ms
seoul -> Iowa avg: 174.9ms

2. VPN (from aws to gcp)

Results of ping to gcp vm in Seoul/Oregon/Iowa from awsec2 in Seoul (VPN applied)

seoul -> seoul avg: 5.4ms
seoul -> oregon avg: 119.4ms
seoul -> Iowa avg: 175.4ms There was little difference in speed, and on the contrary, when we tested communication with GCP -> AWS, it was similar.

We’ve learned how to link AWS-GCP VPN, and you can template it into Terraform, an IaC tool, and set it up quickly. Of course, just like this, AWS-GCP VPN integration does not mean that the workload between AWS and GCP is fully connected. When AWS actually calls the GCP server, it will call domain-based rather than ip-based. So, I will also introduce how to do domain lookup in the other DNS service in a separate post. And with this VPN connection, AWS can call GCP Managed Service using only private ip (Private Service Connect), or GCP can call AWS managed Service only private ip (VPC Endpoints). Let me summarize this in a separate post.