Simplifying AWS Infrastructure Using AWS Control Tower: Migrating Away From a Monolith AWS Account Structure

Photo by Brett Jordan on Unsplash

Managing AWS infrastructure efficiently and securely is vital for organizations in today’s technology-driven landscape. The monolith AWS account structure poses several challenges, including limited developer autonomy, complex access control, inaccurate cost attribution, and scalability issues. To address these challenges, organizations are increasingly turning to AWS Control Tower’s multi-account structure. This article explores the SSENSE journey and the reasons why we migrated from a monolith account structure to AWS Control Tower. We will dive into the benefits of AWS Control Tower and provide guidance on the migration process, enabling software developers and architects to simplify their AWS infrastructure while enhancing scalability and manageability.

Context Around AWS Account Structure at SSENSE

SSENSE adopted AWS as a cloud platform approximately a decade ago. During the initial adoption phase, some of the workloads were migrated from on-prem infrastructure to a couple of legacy AWS accounts (let’s call them “Legacy-QA” and “Legacy-Prod”). However, account-level segregation proved too complex to manage at the time. This was mainly due to the complexities of managing cross-account connectivity between different AWS native services (SNS to SQS, S3 to Lambda, etc.) as well as managing different VPC peerings. It also didn’t help that the infrastructure management was fully manual at that stage, meaning the infrastructure changes had to be applied on multiple levels in order to achieve the desired result. So, after a couple of years, the decision was made to move away from separate AWS accounts to a new monolith AWS account (let’s call it “Monolith”) that was going to host all environments and applications. That migration was not fully completed due to several reasons including complexities with managing dependencies, changes in business priorities, etc. So, the old legacy accounts still existed as unmanaged infrastructure. Meanwhile, the “Monolith” AWS account grew substantially. In addition to the migration of the workloads from on-prem and legacy accounts, all the new projects were launched inside it.

Managing a Monolith AWS Account

The decision to consolidate production and QA environments within a single “Monolith” AWS account introduced significant complications. Firstly, this created a big blast radius: even minor mistakes could cause cascading failures that would eventually impact the production environment and SSENSE customers. This risk led us to create an organizational structure where all AWS infrastructure management was to be entrusted to a single centralized team, which quickly became a bottleneck in the software delivery process. The structure also resulted in the need to create and manage a complex IAM access control system that was also managed by the same centralized team. The dependence on the centralized team, the big blast radius, and the complex access control system resulted in limited developer autonomy, which created silos within the Tech department.

Additionally, accurately attributing costs within the “Monolith” AWS account structure proved to be a daunting task. AWS tags help to slice and dice to some extent, but there are always AWS services that cannot be analyzed by tags (example: “AWS Data Transfer” cost). Without granular visibility on cost allocation, it became challenging to track and manage infrastructure expenses associated with specific projects, teams, or environments. The lack of cost visibility also hindered developers from accurately analyzing the infrastructure cost of their products.

Last, but not least, the existing AWS infrastructure at SSENSE suffered from scalability limitations (with AWS limits applying for the whole account, not per environment) and the absence of easily replicable configurations. Manual configurations and undocumented processes resulted in a complex environment that obstructed the ability to manage and scale resources efficiently and reproduce infrastructure reliably. This lack of scalability created obstacles to the rapid growth and evolving needs of the organization.

Multi-Account Structure and AWS Control Tower

In response to the challenges faced with a “Monolith” AWS account structure, SSENSE recognized the need for a more scalable and manageable approach. After a few rounds of internal brainstorming, as well as discussions with our AWS partners, AWS Control Tower service was chosen as the solution. AWS Control Tower offers a comprehensive solution that streamlines AWS account management and provides numerous benefits including:

• Landing Zone Setup: A landing zone is a well-architected, multi-account AWS environment based on security and compliance best practices. AWS Control Tower automates the setup of a new landing zone using best-practice blueprints for identity, federated access, and account structure.
• Account Provisioning using Account Factory: AWS Control Tower simplifies the creation and management of AWS accounts. It establishes a standardized account structure, ensuring consistency while allowing flexibility for individual teams or projects. This structure enables improved isolation and reduces the blast radius associated with shared resources.
• Governance and Compliance: With AWS Control Tower, organizations can enforce predefined preventative and detective “Guardrails” aligned with AWS best practices and compliance standards. These guardrails help maintain a secure and compliant environment, reducing risks and ensuring consistent security across multiple accounts.
• Dashboard: AWS Control Tower provides a centralized dashboard for monitoring member AWS accounts. It offers a unified view of the compliance status of AWS accounts with respect to enabled guardrails.
• “Customizations for AWS Control Tower” Solution: This AWS solution allows customers to easily add customizations across all AWS accounts in AWS organizations using an AWS CloudFormation template and service control policies (SCPs). You can deploy the custom template and policies to individual accounts and organizational units (OUs). More to come on this topic in part 2 of this series, stay tuned.

Migration Considerations

To guide the migration process to an AWS Control Tower multi-account structure, consider the following steps:

• Assess Current Infrastructure: Evaluate the existing AWS infrastructure, identifying dependencies, security requirements, compliance considerations, and any customization needs specific to your organization. This assessment will lay the foundation for a successful migration.
• Plan the Migration: Develop a comprehensive migration plan that defines clear goals, account structure, and organizational units (OUs) tailored to your organization’s requirements. Collaborate with software developers, architects, and stakeholders to ensure a smooth transition.
• Execute the Migration: Carefully execute the migration plan, provisioning new accounts within AWS Control Tower and migrating existing resources to their respective accounts. Employ best practices for resource tracking, testing, and validation to minimize disruption during the migration process.
• Implement Governance and Compliance: Leverage AWS Control Tower’s guardrails to enforce security policies, compliance standards, and access controls across the new multi-account structure. Customize these guardrails to align with your organization’s requirements, ensuring a secure and compliant AWS environment.

Let’s take a high-level look at the migration flow from the initial stages to the present. The following diagram provides an overview of the journey so far, highlighting the key steps and milestones in the migration process.

Conclusion

Migrating from a “Monolith” AWS account to AWS Control Tower’s multi-account structure is a significant step towards simplifying and enhancing the management of AWS infrastructure. The SSENSE experience highlights the challenges associated with a monolithic account structure and the compelling reasons for adopting AWS Control Tower. By embracing AWS Control Tower, organizations gain improved scalability, enhanced security, better cost visibility, and simplified management. In the upcoming parts of this series, we will dive deeper into each phase of the migration process, providing detailed insights and practical tips for a successful migration to AWS Control Tower’s multi-account structure. Stay tuned to unlock the full potential of your AWS infrastructure.

Editorial reviews by Catherine Heim & Mario Bittencourt

Want to work with us? Click here to see all open positions at SSENSE!