Moving from AWS to Google Cloud Platform
Mighty recently transitioned off of Amazon Web Services (AWS) as its primary cloud infrastructure provider onto Google Cloud Platform (GCP). Here’s how we did it.
tl;dr: Create a VPN between AWS and GCP, set up a DB migration job, switch your DNS settings to point to GCP hosted services.
Why did we switch?
AWS and GCP are both totally capable of handling Mighty’s infrastructure. Our decision was based on developer opinion that GCP offered tools, services and documentation which was more convenient and accessible to developers.
This should also be helpful in moving from GCP to AWS if that’s the direction you’re headed.
Starting point in AWS
Mighty is essentially Flask servers, a PostgreSQL database, and a React front-end.
On AWS, we deployed our servers on Amazon Elastic Compute Cloud (EC2) and hosted our database instances with Amazon Managed Relational Database Service (RDS), all contained in a virtual private cloud (VPC). We used Amazon Simple Storage Service (S3) to host static assets along with Amazon CloudFront as a content delivery network (CDN). Amazon Route 53 was our domain name system (DNS).
Destination in GCP
GCP and AWS have very similar product offerings, so most of the services we chose to use in GCP were the direct AWS analogs. For example:
EC2 -> App Engine (very similar to AWS Elastic Beanstalk)
RDS -> Cloud SQL
S3 -> Cloud Storage
CloudFront -> Cloud CDN
One service we did not choose to move to GCP was Route 53. Given the risk in migrating DNS, and given Route 53 is an adequate DNS, we left it in AWS instead of migrating to Cloud DNS.
Setting up a new GCP environment
Most of the products in GCP have great documentation and quick-starts, so we won’t talk too much on specifically how to deploy a new App Engine service.
Some not-so-obvious things which will make this transition easier:
- Create a new VPC for your application resources. Your project will come with a default VPC, but we found it easier to apply specific configurations on a new one.
- Some products, like App Engine and Cloud Functions, require serverless VPC connectors to access the private network and talk to other hosted services like the database. E.g. here’s how to do that for Python3 apps running on the App Engine standard environment.
Connecting AWS to GCP
It was important for us to be able to transition from AWS to GCP piecemeal and with zero downtime. To do that, we decided to connect the two environments for a hybrid cloud configuration during the transition.
Using a hybrid cloud, we were able to deploy App Engine services to hit internal resources on AWS, which allowed us to move our database independently from our application instances.
There are a ton of ways to connect services in AWS to services in GCP. The solution that made the most sense for us was a VPN. A VPN was the right choice for us because our AWS configurations had very restrictive and hard-to-change security groups which prohibited most public traffic. A VPN allowed us to configure resources in GCP to connect to resources in AWS securely via private addresses.
There are also a ton of ways to set up a VPN. At Mighty, we used a configuration very similar to what’s described here. Google also has documentation for how to set up a high availability VPN from AWS to GCP.
Moving the database
If you use Google’s Database Migration Service, migrating from RDS to Cloud SQL is fairly simple so long as the your source database instance is configured correctly. For us, that meant updating a few things in the parameter groups and restarting the RDS instance.
The db migration service creates a new replica of the RDS instance which you can promote and point your applications to when you’re ready.
Migrating traffic
Once all systems have been created in the new environment, you can point traffic to the new instances and de-provision the old ones.
For App Engine services, this means adding new records to your domain registrar and deleting the old records. DNS changes can take time to propagate, so it’s important to monitor traffic to make sure old servers are no longer serving requests before shutting them down.
