Cloud Migration
My Experience with Migrating AWS EC2 instances to GCP with Velostrata.
The purpose of this post is to assist those of you who wish to migrate your cloud infrastructure from AWS to GCP. I will provide here a step-by-step tech guide followed by print screens and relevant links to support best your migration process.
In general, cloud migration can be really easy and straight-forward. The smaller your app is the easier it would be for you to migrate.
The more vast and complex your app is, the more challenging this mission becomes — when you have many microservices and you are running many instances with large volumes- when you use many cloud services in multiple availability zones. This kind of DevOps task becomes not so straight forward. Mastering new skills in the new cloud platform is a must, doing so while trying to migrate instances, services, and DBs — with zero interference to production — makes this task close to impossible.
A basic app on AWS can be composed of multiple cloud services: EC2 instances/Lambda functions, S3-storage, Route-53, multiple VPCs, Load balancing, DBs (DynamoDB, RDS, ElastiCache, etc..):
In this article the main focus will be on migrating EC2 instances from AWS to GCP using Velostrata, a solution to migrate other cloud services does not exist at this point (although migrating DBs like MySQL and Redis should be supported in the near future) so you have to move the data and rebuild these services yourself in GCP (more about these processes in the next post).
Wish you could just drag-n-drop instances from one dashboard to another right?
While exploring some really impressive migration solutions I’ve decided that Velostrata is the way to go to migrate my EC2 instances.
Before you start anything, I would advise you to invest a little more time (if you have it of course) and containerize your solution (assuming you are not already doing that) — it will simplify many things for you and enable you to use GCP engine and run Kubernetes clusters with all of its known advantages.
Also, if you are not familiar with Amazon VPC I highly recommend watching this webinar:
Velostrata
Velostrata is a software that helps organizations move workloads to and from public clouds. It was recently acquired by Google and it is free if you want to migrate your resources to GCP.
I documented all the necessary steps to configure Velostrata and successfully migrate virtual machines from AWS to GCP. The last section is dedicated to problem-solving and troubleshooting issues that should help you with the process. Believe it or not, most came from the support group in Velostrata as I could not find any solutions online. Follow these steps and you’d be fully migrated in no-time:
- Connecting an AWS VPC to GCP over VPN is pretty straight forward — follow websiteops/JumpCloud guides to implement. I assume you already have a VPC in GCP, if not I recommend not using the default and create a custom VPC. note: Custom VPC is a recommendation I’d give to anyone who is using AWS as a cloud platform. This creates the opportunity to establish VPC peering between availability zones once your application grows and scale.
- In your AWS dashboard go to VPC=>Site-to-Site VPN connection and make sure that your VPN Tunnels status are ‘UP’ (in Tunnel Details)
3. To use Velostrata, you also need to have an organization ID. If you do not already have one, create one from your GCP dashboard:
- set up a cloud identity account
- verify domain ownership — since AWS managed the domain you can use Route 53 to do that
- add users to your cloud identity account
4. If you already had a project in GCP (more common across startups) it means that now your project is under ‘no organization’, so in this case, you need to migrate your project to the new organization you have created. if you feel you are not familiar with the resource’s hierarchy on GCP I recommend to do some reading about it first before you start migrating machines or before creating your custom VPCs and subnets in GCP.
5. Deploy Velostrata from GCP marketplace in your custom VPC
6. Creating the Velostrata IAM group for AWS migration is defined nicely in the docs — follow these steps and add the IAM user to the group created by the cloudFormation script, this will establish all of the necessary permissions to Velostrata in the AWS side. keep the ‘access key id’ and ‘secret access id’ — you will need it for later.
after all that, you pretty much stand for all of the Velostrata prerequisites from the AWS side. yes!
7. GCP roles and service accounts can be created easily via cloud shell. And now both of your environments are ready!
8. Configure AWS as a source in Velostrata dashboard
9. Create cloud credentials with Velostrata dashboard
- login Velostrata Management Appliance with “apiuser”
- click on “Target Cloud”=> “Cloud Credentials” =>”Create”=>”GCP”
- select the JSON with the GCP credentials and complete the setup
10. Build your Cloud Extensions — these nodes provide back up to one another and serves as a conduit for VM storage between two hosting environments. make sure you define each node in a different availability zone.
11. Create your cloud extensions — click on “Cloud Extensions” in the upper left corner and then click “Create”
For the tags, keep the default values for simplicity. Here is a list of those basic default values:
Network settings:
GCP Credentials: gcp-source
Project: your-project
Region: your-region
VPC: your-VPC
Default destination project for workloads: your-project
Default Service Account for Workloads: velos-mgmt-sa
Default service account for velostrata Worker: velds-worker-sa
Cloud extension
Cloud extension name: -your Cloud Extension name-
Service account for edge nodes: -your service account-(you can use a default service account and add Storage Object Admin permissions)
Cloud extension size: small (num of VMs is ~10–15)
12. Generate runbook:
Go to the Velostrata dashboard and choose ‘migration wave’ and then choose ‘Generate Runbook’
Source Cloud Details is a ‘drop down’ with your configured source cloud AWS.
Add your key-values tags from AWS to include the VMs you wish to migrate.
Target Cloud Extension is also a ‘drop down’ (available after completing ‘Build your Cloud Extensions’: step 10)
After your Velostrata_Runbook.csv file is downloaded you need to edit it:
- RunGroup — edit rows with -1 — represents machines that are being skipped in the migration wave (-1 is the default value)
- TargetInstanceType — fill in machine type that fits your needs
13. Choose ‘New Wave’ and include your newly edited .csv file
hit ‘save’ and now you can validate the configurations and make sure your cloud extensions are ready for a migration wave (‘Actions’=>’Validate’). This process takes about 2 minutes and after that ‘Validation Status’ should be ‘Passed’
Migration of Linux machines
SSH to your future-to-be migrated machine, download and install the driver. This will enable the instance to be booted up successfully in GCP after the migration is complete
- wget https://storage.googleapis.com/velostrata-release/V4.0.0/Latest/velostrata-prep_4.0.0.deb
- sudo dpkg -i velostrata-prep_4.0.0.deb
- sudo apt-get install -f -y
After you initiate a migration you will be able to see:
- Velostrata importer created on the AWS EC2 dashboard:
- Velostrata edge-node-a/b instances in GCP
Migrating…
From this point Velostrata starts migrating your instances between clouds, if all goes well, you will be able to run your app within 3–4 hours in GCP (should the wave consists of 3–4 normal-sized machines) — might be more for larger waves.
After a successful migration, log into your virtual machine and make sure that everything works as you expected. if all is well at this point I advise you to stop the machines and create backup images from the persistent disks. saved images and disks snapshots are the only way to retrieve these instances in case someone accidentally deletes the instance (believe me it can happen) — without it, you will have to migrate your instances again from AWS.
Q&A
In this section, some of the errors I’ve encountered are logged and what I did to handle it. hopefully, this will make your life easier should you encounter these errors as well:
- ‘Full Migration Failed’:
Logs indicate an Error during startFullMigrationToGcp: No working CloudEdge found
solution: make sure your CloudEdge components are well defined and make sure your firewall is configured correctly in GCP. Make sure the tags we used in the cloud edge extension configuration are consistent with your VPC’s firewall rules)
2. [Failed to create target cloud instance] (The user does not have access to service account ‘velos-mgmt-sa@your-project-name.iam.gserviceaccount.com’. User: ‘velos-mgmt-sa@your-project-name.iam.gserviceaccount.com’. Ask a project owner to grant you the iam.serviceAccountUser role on the service account)
solution: go to ‘IAM and Admin’ in your GCP dashboard and add “Storage Object Admin” to service account Roles in GCP.
3. weird behavior during a ‘full-migration’- the process stuck on 90%:
In your GCP dashboard under VM instances you will see that the machine is stuck on reloading:
Eventually, an error message will appear in the wave logs that indicate that the instances fail to perform boot correctly:
First thing is to verify that your OS is supported by Velostrata. Then either the problem might be that the OS reboot, in fact, fails as the message indicates, or that Velostrata fails to verify the successful boot then falsely initiating ‘move back’ process which removes the instance from GCP.
solution: this can be solved by
- Installing the Velostrata driver in the Linux machine (see above ‘migration of Linux machine) if not already installed
- Disable the cloud-init option in GRUB file on the machine in the source cloud before running the migration:
a. sudo vi /etc/default/grub
b. on the GRUB_CMDLINE_LINUX= line add the parameter cloud-init=disabled
c. save and quit
d. run: sudo update-grub
e. reboot the system
f. perform ‘Offline Migration’
4. [Failed to start Prepare to Detach task] (Task of type ‘Prepare to detach’ for vlst-23 can’t be started, due to an already running task (type: ‘Stop Cloud Extension’, id: t-173)) — happens after trying to migrate a machine with large volume (~1500GB)
solution: in this case simply try to run the job again from Velostrata “migration wave” dashboard, with the same configurations:
SSH to the migrated machine
After your machine had migrated successfully to GCP, assign it with a public IP to enable SSH connection.
To attach an external IP address, follow these steps:
- Click on the machine name in GCP “VM Instances” page to open the “VM instance details” page-
- Click Edit button
- On “Network Interfaces” click the Edit/pencil icon for edit it.
- Under “External Ip” select “Ephemeral” and click “Done”.
- At the bottom of the page click “Save”
- Go back to the “VM instances”, you can now see the Public IP assigned
In order to achieve an SSH connection for your migrated machines, complete that process from your local terminal, as you would have using AWS. If you’d like to do this via browser, you’re in luck. After speaking with the Velostrata team I hear it will be supported soon.
At this point, you should have your instances copied in GCP, and your instances ‘Stopped’ in AWS. Velostrata does not turn the instances back on in AWS after the process is completed. so make sure you ‘Start’ them manually (if needed) only after the ‘Offline-Migration’ process is finished and not before.
5. Offline-migration is successful but you cannot achieve an SSH connection to the machine- in this case, view the instance details on GCP console and then click “Serial port 1” under Logs. If you see: “The disk drive for /data is not ready yet or not present” it means that your disks were somehow corrupted, At this point, remove the machines, and restart the migration job again.
For other errors and troubleshooting, I encourage you to log into your ‘System Settings’ in the Velostrata dashboard with username: “localsupport”. Choose ‘logs’ => ‘download support bundle’. Then upload it to google-drive and share it with the Velostrata support team (GCP support console).
This will enable them to view the logs from your migration process and to enable productive troubleshooting. Throughout the migration, the Velostrata team spent a lot of time helping me troubleshoot, and was very responsive. So definitely use them as a resource.
Minimum interruptions to AWS environment
Use the following steps to create minimal interruptions to the env in AWS:
- create an image for the machines and volume snapshots(if needed)
- deploy the instance from the AMI — this replica will be migrated and so when the instance will be stopped by Velostrata during the migration process there will be no interruptions to any on-going live services.
Summary
Congrats, you’ve made it! While any cloud migration has many more steps to it, I hope this post has helped give you a good overview of the migration process using Velostrata.
Here, we’ve covered the general migration process, as well as some of the common issues and errors I experienced throughout.
I encourage everyone working on a similar project, to comment and share their experiences below.
Happy Migrating
cheers.
