Cloud Migration Factory
Migrating to Cloud at scale, securely with speed and efficiency
Today, every company is striving to become a technology company, whether it’s a manufacturer, retailer, auto company, or a financial services company. And for the right reasons, the ones not started on that journey have already started to lose out to disruptors. Cloud adoption is one of the important aspects of digital transformation and it’s a game-changing decision for an organization to make. To meet the constantly changing customer needs and market demands, it propels agility, flexibility, and scale to business.
Migrating on-premise applications and associated workloads (databases, servers, microservices) to Cloud necessitates a well-thought-through strategy, planning and execution roadmap. It could be an organization-wide transformational program to migrate the majority of apps to Cloud or could be done by different verticals of organizations in silos at their own pace and priority.
Some organizations (e.g. banks) have hundreds of applications, servers and databases to be migrated to Cloud and many have been built over the last 20+ years (i.e. legacy applications). Migrating one application at a time would be an eternity. What is needed is a strategy to migrate all these applications in a series of migration waves, a few hundreds at a time in one wave. What is needed is a Cloud Migration Factory. Before we talk about the factory, let me explicate a bit on migration strategy…
Migration Strategy
Migration strategy involves a zillion things to consider depending on the nature of business, business objectives and the scale of migration. This itself is a separate topic to talk about at some point, but I wanted to highlight a few relevant points here. At a high level, you have business objectives to meet, i.e. cost efficiency, scalability and performance improvement, resiliency and business continuity, speed of innovation and time-to-market, to name a few. You also need to figure out what all applications need to be moved (discovery), how mature they are to be able to migrate (assessment for readiness), what type of migration it would be for each application, and so on. All of these data points would drive your design for what components you need to build for your migration factory.
Type of Migration — Just a quick note on the 6 types of migration paths for Cloud migration (for details, look for AWS 6R types). Rehost is basically lifting and shifting from on-premise to Cloud without making any change. Replatform is where you might have to make a few changes to use some capability of Cloud-native resource. For example, changing from a VM hosted mySql database to managed mySql service of the underlying Cloud provider. Re-Architecting involves redesigning of an application for maximum leverage of Cloud-native resources. For instance, changing a Java WebSphere monolithic app to microservices-based architecture and host them as containers on Kubernetes platform. I will leave the rest three for you to read up yourself; they are not important for this migration topic.
Migration Workflow
How awesome life would have been, if I could just push a button on Migration factory and it would deploy all workloads from on-premise to multi-Cloud.
Alas, we are not there yet, but maybe one day we will attain that level of automation. Tesla Giga factory would awe that :-)
But realistically, a simplified view of Cloud Migration workflow would look like the one below:
Some of the steps in this end-to-end process can be fully automated and a few would require manual intervention with key decisions and approvals needed from different stakeholders in the organization. And we will dig into those next…
Factory Design
As with any system design, there is no single ‘right way’ of designing the factory because it is driven by scale, objectives and expected business outcome, to name a few. Below is one design for the factory:
Migration factory has five main components —
Strategy & Mobilization: This is the first step in building the migration factory. It discovers the inventory of applications and workloads to be migrated, assess candidates for migration, check risk and compliance conformity, and identify apps for the pilot phase to be migrated in a series of migration wave.
Automation Engine: As the name suggests, it provides a great degree of automation for building core infrastructure in Cloud. You should have a Terraform module for pretty much every resource application team plan to use on Cloud, and it should be built with flexibility for teams to configure parameters suited for their deployment, i.e. image AMI, m/c type, etc. At a minimum, you should have ready-to-use modules for VPC, subnets, Networking, Kubernetes clusters, Cloud storage, load balancers, etc. It’s important that you thread the organization's security policies into these modules so it gets applied across all applications in a consistent manner.
Landing Zones module would provide a baseline environment for projects with multiple accounts, IAM policies, agreed blueprint pattern for n/w design, governance, security, etc.
Security Policies & Guardrails would not only apply the org policies agreed with the CSO office, but will also check if policies are violated at build and run time. For instance, Terraform Sentinel allows you to check policies before you ‘apply’ changes, and you can deploy Prisma defenders for runtime (after deployment) compliance checks. Chef InSpec is another tool you can use to check runtime checks.
Authn & Authz module takes care of certificate management, Service account creations and role-binding.
Image Bakery uses tools like packer pipelines to build OS images (Ubuntu, window server, etc.) hardened as per organization policies. App teams are only allowed to use these images for their VMs and K8s clusters.
Artifactory manages all the artifacts created internally by teams or external downloaded libraries, packages & frameworks etc.
CI-CD pipeline allows a team to deploy all terraform modules, policies, and images using tools like GitHub action workflows or Jenkins. It can be fully automated with various actions supported by GitHub like deploy on pull request merge into the main branch.
Cloud Service Provider specific migration services — We talked about different migration types for workloads earlier. All Cloud providers like Google, AWS and Azure provide tools for Database, Server and Application migration out of the box. These are especially helpful for Rehost migration, but are also needed for Re-Architect, Replatform as well, where a database has to be migrated on homogeneous or managed databases on Cloud.
Verification — This module allows you to check all functional & non-functional test cases and ensure production readiness before you cutover to production. This could be fully automated with tools like Terratest & Test Kitchen.
Operation and Control — Operation & control modules keep monitoring infra resources to make sure they are operating as expected, frequent health checks, alert raised for security violations and help manage incidents and diagnose the issue.
Why do you need Factory?
Scale — Many organizations have multiple on-premise data centers (DCs) with thousands of machines hosting workloads and would want to migrate most of it to Cloud in a phased manner. CMF allows them to move most of these workloads as id (i.e. Rehosting) in a very efficient and cost-effective manner. Applications that need Re-Architecting would need re-design and re-architecture and eventually build in a Cloud native way. CMF can help to provide the necessary infrastructure as a readily available platform.
Governance — Having one central CMF team responsible for building and maintaining the platform enforces consistency across design and architecture and adherence to approved blueprints and patterns. Any deviation from approved blueprints goes through the Cloud Design Authority approval process, ensuring the application team is not violating the organization's set guidelines and compliance.
Consistent Security Policies — The CMF team works closely with Cloud Security office of the organization to define security policies, ensuring consistent policies are applied across on-premise and multiple Clouds being used. CMF security module checks both build time and run time for any violation and can put a hard constraint on proceeding for deployment, if necessary. It periodically checks all the infrastructure and application resources running in Cloud and alerts for violations with defined SALs to fix them.
Multi-Cloud/Hybrid — One uniform, consistent way to deploy across multiple Clouds.
Automation — A majority of components and processes for migrating a Rehost application is automated, I would say somewhere 75–90%. There are certain workflow steps requiring manual decision making, selection or approval for migration, specifically database and data migrations.
ROI/Cost Savings — CMF once build and operational, requires very few resources/engineers to maintain and enhance as you go along. It provides significant cost savings and ROI for medium to large migration programs.
Lastly, who owns the factory? It should be built and owned by the Cloud Platform team or Cloud Center of Excellence (CCoE) group in an organization that serves to hundreds of application teams across business functions.
Conclusion — Is it good for everyone ?
Cloud Migration Factory is not a panacea for every Cloud migration activity and for every organization. It is the right choice for organizations having thousands of workloads to be migrated in a stipulated time driven by business strategy. It's quite efficient and cost effective (ROI) for Rehost & Re-Platform migration cases, especially for organizations plagued by decades of old legacy applications. Its services can be used for Re-Architect migrations as well, especially database migration needed as part of the re-architecture of an App.
Another variation of CMF is where you split some building blocks (Terraform modules, Security, Image Bakery etc) into a separate Cloud Platform team and let CMF focus on migration strategy and pipeline. It really is a matter of choice the Cloud leadership makes in an organization. Nothing is perfect and the right solution…
As they say, don’t let perfect be the enemy of good enough!
