Lift and Shift: The Stuff We Don’t Talk About When Moving to the Cloud
By: Jesse Mandel
While “build your existing datacenter in the cloud” shouldn’t ever be your long-term solution, sometimes there are reasons to lift and shift. AWS Server Migration Service easily moves servers to the cloud so you can continue your data center shutdown plan and tackle some of the legacy systems later.
When TrueCar decided to migrate from a self-managed data center to AWS, we decided to take the time to rewrite everything in the cloud instead of just moving our servers. TrueCar is not a simple system, but it was easy to see the path: take a bunch of APIs that power truecar.com, dealer tools, and many partner sites and rewrite them into a new scalable monolith API. Similarly, write a new unified UI in React that allows business logic to be reused and still allows us to provide a custom look for our partner sites. Very straightforward. But for ALG, it wasn’t as clear.
What is ALG?
ALG is a subsidiary of TrueCar. While we share an office and both work with the auto industry, we have different products and clients. ALG was founded in 1964 and our core service offering is predicting the future value of vehicles. Our data is used in a variety of ways: guiding OEM decisions through consulting, evaluating lease and loan portfolios of lenders, powering desking tools, and setting leasing programs. If you’ve ever leased a vehicle, ALG’s data was probably used along the way.
When ALG started out, we published physical books. We now have many databases and a warehouse of data, a lot of internal tools and processes to manage our data, plus web applications and client deliverables. Over the course of 30 years this all grew organically into a pretty complex system with many dependencies.
We are working toward rewriting the entire ALG system to take advantage of a modern tech stack in the cloud and using our new tools, like Armatron and Spacepods. We’re building out a new data lake and data warehouse with AWS Glue, S3, and Redshift. For relational data we’re using PostgreSQL, and our API uses a managed ElasticSearch cluster.
Today ALG publishes weekly data that our clients depend on. So we needed to keep our current legacy system running, even though the data center was getting shut down before we could rewrite it. New servers had been spun up and configured where possible, but there were a few we couldn’t get to. I am embarrassed to say we still run a Microsoft SQL Server 2005 server and a few SQL Server 2008s. Not only are these versions not supported by Amazon RDS for SQL Server, we use Replication, Database Mail, BULK INSERT, SSIS packages, and other features that are generally not supported. Furthermore, Microsoft doesn’t have an easy migration path to newer versions of SQL Server, so we couldn’t spin up a new 2017 SQL Server and migrate to that, either.
We were actively working to shut off these legacy systems, but in the meantime we needed to get out of the data center.
Server Migration Service
Amazon made it easy to move existing servers with their Server Migration Service. It still took a few steps, but the process went smoothly.
- In our data center, SQL Servers run on bare metal, so the first step was to virtualize them. First, turn off the SQL Service to prevent changes to the data, then use a Hyper-V server to create a virtual machine. When that’s done, shut down the physical server and boot up the new virtual machine to make sure it works.
- Next, install and configure the Server Migration Connector onto the Hyper-V server.
- Log in to the Server Migration Service in the AWS Console and import the server catalog. Once the virtual server shows up, start a migration job to create an AMI.
- When the migration to the AMI is complete, shut down the virtual server and spin up an EC2 instance with the new AMI; then test the server now running in AWS.
The new server found our Active Directory servers and joined the domain seamlessly. The only difference was an IP change. All the connections based on the servername continued to work. Additionally, now that it’s in AWS, expanding the hard drive or upgrading the hardware takes just a few clicks. Having all our servers in one place significantly reduces latency as we move data from the legacy systems to the new ones we’re building.
We made this move with four legacy SQL Servers last December and ALG has been running fully in the cloud ever since.
We are hiring! If you love solving problems, please reach out. We would love to have you join us!