Connect GCP and AWS with Cloud VPN HA
The last Cloud Next was rich in announcements, many new features and products. We will focus on one specific thing in this article: Cloud VPN is available in HA with a 99,99% SLA coverage. This is the right moment to update my previous blogpost on “How to build a multi-cloud Infrastructure” by converting it to Terraform 0.12+ version and implementing VPN in HA!
Prepare the environment
We need to prepare the environment, be sure to have:
- A way to authenticate to GCP (usually with GCloud CLI with a keyfile or impersonation)
- A way to authenticate to AWS (with AWS CLI and a pair of Access Key and Secret Key)
- Terraform 0.12+
The Goal
The overall objective is to deploy two VPCs on AWS and GCP, and connect them through a managed highly available VPN.
As you can see in the above diagram, deploying two tunnels is not sufficient, because HA on AWS is done through the redundancy of the Customer Gateway, and each GW provides two tunnels. So, to accomplish HA we need to deploy 4 tunnels to connect AWS and GCP.
Deploy Networks and VPCs
First of all, we need to create a VPC on each cloud provider, you can skip this step if you want to reuse your existing VPC.
This part isn’t complicated on its own, so I suggest you to get inspiration from the official Terraform documentation. You will have to create a GCP Network with at least one subnetwork and an AWS VPC with their subnetworks. Both must be deployed in the same region as your VPN connections.
Additionally, we have to create a Cloud Router on the GCP side. This router will be connected to your global GCP route table and will manage BGP sessions for all of your VPN tunnels.
resource "google_compute_router" "main" {
name = "router-main"
network = google_compute_network.main.self_linkbgp {
asn = 65273 # you can choose any number in the private range
}
}
For the ASN (Autonomous System Number) you can choose any number in the private range 64512 to 65534 and 4200000000 to 4294967294.
On AWS there isn’t a Cloud Router, but you have to ensure that your route table will propagate the routing informations received from BGP sessions.
resource "aws_route_table" "main" {
vpc_id = aws_vpc.main.idpropagating_vgws = [aws_vpn_gateway.vpn_gateway.id]
}resource "aws_route_table_association" "main" {
count = 3subnet_id = element(aws_subnet.main.*.id, count.index)
route_table_id = aws_route_table.main.id
}
With that code, we have activated the propagation from the VPN gateway, and have associated the route table with our subnets.
Deployment of the VPN
This is the most interesting and complicated part, so I decided to split it in two parts: AWS and GCP.
VPN on AWS
I decided to begin with AWS because this the easiest part. We do not connect to anything, we just expose the VPN entrypoints and wait until the other side connects.
We start by creating three resources: the VPN gateway itself (remember this component will be connected to your VPC), and two customer gateways for the high availability.
resource "aws_vpn_gateway" "vpn_gateway" {
vpc_id = aws_vpc.main.id
}resource "aws_customer_gateway" "customer_gateway_1" {
bgp_asn = google_compute_router.main.bgp[0].asn
ip_address = google_compute_ha_vpn_gateway.target_gateway.vpn_interfaces[0].ip_address
type = "ipsec.1"
}resource "aws_customer_gateway" "customer_gateway_2" {
bgp_asn = google_compute_router.main.bgp[0].asn
ip_address = google_compute_ha_vpn_gateway.target_gateway.vpn_interfaces[1].ip_address
type = "ipsec.1"
}
You can note that we are referencing a resource not yet declared. Do not worry, Terraform graph will handle this perfectly and will avoid the cyclic dependency.
The BGP number on Google side is read from the Cloud Router resource. It would be possible to set the BGP for the AWS side on the VPN gateway resource, but we decided to let the default value (unlike Google’s side which do not provide a default).
Then we create a VPN connection attached to each customer gateway. This setup will provide 4 tunnels because each VPN connection comes with two.
resource "aws_vpn_connection" "cx_1" {
vpn_gateway_id = aws_vpn_gateway.vpn_gateway.id
customer_gateway_id = aws_customer_gateway.customer_gateway_1.id
type = "ipsec.1"
}resource "aws_vpn_connection" "cx_2" {
vpn_gateway_id = aws_vpn_gateway.vpn_gateway.id
customer_gateway_id = aws_customer_gateway.customer_gateway_2.id
type = "ipsec.1"
}
VPN on GCP
This part is slightly more complicated, we have to create the following resources:
- 1 HA VPN Gateway
- 1 External VPN Gateway
- 4 VPN Tunnels
- 4 VPN Interface on Cloud Router
- 4 Peers on Cloud Router
The gateways are a new kind of resources on GCP and are still in Beta (for the moment).
resource "google_compute_ha_vpn_gateway" "target_gateway" {
provider = "google-beta"
name = "vpn-aws"
network = google_compute_network.main.self_link
}resource "google_compute_external_vpn_gateway" "aws_gateway" {
provider = "google-beta"
name = "aws-gateway"
redundancy_type = "FOUR_IPS_REDUNDANCY"
description = "VPN gateway on AWS side"interface {
id = 0
ip_address = aws_vpn_connection.cx_1.tunnel1_address
}interface {
id = 1
ip_address = aws_vpn_connection.cx_1.tunnel2_address
}interface {
id = 2
ip_address = aws_vpn_connection.cx_2.tunnel1_address
}interface {
id = 3
ip_address = aws_vpn_connection.cx_2.tunnel2_address
}
}
As we said, we chose to create 4 interfaces to connect to AWS. You can see that the first 2 are connected to the first AWS VPN connection, and the others 2 are connected to the second VPN connection. Remember this order, otherwise the link will not work and/or the BGP sessions will never establish.
And now the redundant part, for more simplicity I will paste here only the first and the last resource, you will have to adapt (or simply paste the complete code from my Github repo linked at the end of this post).
resource "google_compute_vpn_tunnel" "main-1" {
provider = "google-beta"
name = "vpn-tunnel-1"
vpn_gateway = google_compute_ha_vpn_gateway.target_gateway.self_link
shared_secret = aws_vpn_connection.cx_1.tunnel1_preshared_key
peer_external_gateway = google_compute_external_vpn_gateway.aws_gateway.self_link
peer_external_gateway_interface = 0
router = google_compute_router.main.name
ike_version = 2
vpn_gateway_interface = 0
}[...]resource "google_compute_vpn_tunnel" "main-4" {
provider = "google-beta"
name = "vpn-tunnel-4"
vpn_gateway = google_compute_ha_vpn_gateway.target_gateway.self_link
shared_secret = aws_vpn_connection.cx_2.tunnel2_preshared_key
peer_external_gateway = google_compute_external_vpn_gateway.aws_gateway.self_link
peer_external_gateway_interface = 3
router = google_compute_router.main.name
ike_version = 2
vpn_gateway_interface = 1
}
You can note that I have only changed:
- the resource name (obviously)
- the shared secret
- the peer external gateway (from 0 to 3, remember the order on the GCP external gateway)
- the VPN Gateway interface (from 0 to 1, corresponding on the AWS VPN Connection resource)
Then let’s create the router interface for all of these tunnels.
resource "google_compute_router_interface" "main-1" {
name = "interface-1"
router = google_compute_router.main.name
ip_range = "${aws_vpn_connection.cx_1.tunnel1_cgw_inside_address}/30"
vpn_tunnel = google_compute_vpn_tunnel.main-1.name
}[...]resource "google_compute_router_interface" "main-4" {
name = "interface-4"
router = google_compute_router.main.name
ip_range = "${aws_vpn_connection.cx_2.tunnel2_cgw_inside_address}/30"
vpn_tunnel = google_compute_vpn_tunnel.main-4.name
}
Here again, I have only pasted two interfaces. The things that will vary are :
- the name (what a surprise)
- the IP range used (given by the AWS tunnel)
- the tunnel itself
And to finish, do not forget to configure BGP for each interface with the peer configuration.
resource "google_compute_router_peer" "main-1" {
name = "peer-1"
router = google_compute_router.main.name
peer_ip_address = aws_vpn_connection.cx_1.tunnel1_vgw_inside_address
peer_asn = aws_vpn_connection.cx_1.tunnel1_bgp_asn
advertised_route_priority = 100
interface = google_compute_router_interface.main-1.name
}[...]resource "google_compute_router_peer" "main-4" {
name = "peer-4"
router = google_compute_router.main.name
peer_ip_address = aws_vpn_connection.cx_2.tunnel2_vgw_inside_address
peer_asn = aws_vpn_connection.cx_2.tunnel2_bgp_asn
advertised_route_priority = 100
interface = google_compute_router_interface.main-4.name
}
Some parts have changed also:
- the name
- the peer IP address (AWS side)
- the peer ASN
- the interface on Cloud Router
We do not need to change the priority because we want to share traffic equally between all tunnels.
And that’s it, with this you can securely connect AWS and GCP with an HA VPN connection.
terraform init
terraform apply
---An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
+ createTerraform will perform the following actions:
[...]
Plan: 30 to add, 0 to change, 0 to destroy.
Review our setup
After a couple of minutes from applying, everything should be up and looks like this:
Let’s start by checking our tunnels states on GCP and AWS.
Well, everything seems ok, all the tunnels are up (I did not take a screenshot of the second customer gateway on AWS, but it works too).
Now let’s take a look on the GCP Cloud Router.
All BGP sessions are connected, so the routes exchange should be fine. We cannot see this part on AWS.
The last part is to ensure that routes are propagated on the routes tables directly.
Everything looks good! The AWS subnet is propagated to the GCP route table, and the GCP network is propagated to AWS.
To finish
If you want to test this setup on your own, and do not want to copy/paste the above code, take a look at my GitHub repo implementing this: https://github.com/bcadiot/multi-cloud.
Thanks for reading!
