Seamless Cross-Account Migration of AWS RDS at Scoutium: Leveraging Automated DevOps Practices

About the Customer

Scoutium is a leading sports technology company that specializes in providing innovative scouting and talent management solutions to professional football clubs worldwide. With a deep understanding of the modern football industry and a passion for leveraging data-driven insights, Scoutium empowers clubs to make informed decisions in player recruitment, team performance analysis, and strategic planning.

With a customer-centric approach and a strong focus on delivering value, Scoutium is dedicated to providing its clients with a comprehensive suite of solutions tailored to their specific needs. The company’s team of experienced professionals combines their deep knowledge of football with technical expertise to deliver innovative products and services that revolutionize the way clubs approach talent identification and player evaluation.

Through its advanced technology and commitment to excellence, Scoutium has become a trusted partner for numerous football clubs, enabling them to uncover hidden talent, optimize player development strategies, and drive success on and off the field.

Customer Challenge

As a fundamental part of the system design, the usage of AWS RDS service in the infrastructure was essential. The data being stored was constantly expanding due to active users of the system, and the development efforts on the code side required frequent reshaping. Therefore, it was necessary to ensure that the up-to-date data and designs from the production environment were consistently replicated in the staging environment during each development cycle, making them readily available for application usage. Throughout this process, the architecture had to be cost-effective, user-friendly for management, and optimized to avoid any potential errors during the development process.

Furthermore, it was essential for this infrastructure to be managed through automation and equipped with preventive measures against potential errors that could occur during regular repetitions. The architecture needed to be designed in a way that supported seamless integration of new developments and allowed for the removal of specific enhancements without compromising the entire system.

Partner Solution

As the Bestcloudforme team, in the designed architecture:

We established a backup structure using the native snapshots provided by AWS RDS to facilitate the movement of data. To enable the usage of these backups across accounts, we created a design using Lambda and EventBridge for sharing the snapshots in the production account.

Through this design, the generated snapshots were automatically shared with the test environment without incurring additional costs. In the second part, it was necessary to make modifications within the database based on the requirements for automatically provisioning these snapshots in the test environment. Data that needed to be different from the production environment in the staging environment included sensitive information such as user credentials and newly developed application tables.

To handle this, we utilized CodeBuild to extract and store this data in an S3 bucket before the restoration process. Leveraging the serverless and pay-as-you-go nature of CodeBuild eliminated the need for keeping additional resources active during idle periods and prevented unnecessary costs. In the next stage, we implemented AWS Step Functions, triggered by CodeBuild. AWS Step Functions was designed to manage the controlled execution of Lambda functions. It provided error handling and retry policies, enabling us to handle potential problems and incorporate solution scenarios. With the help of the Lambdas managed by AWS Step Functions:

• The encrypted snapshot was decrypted.
• An RDS database was provisioned from the decrypted snapshot.
• The readiness of the RDS database was checked to ensure it was in an active state.
• The CNAME record in Route53 was updated to include the new RDS database in the network.

• CodeBuild was triggered to continue the process. In the final stage, CodeBuild was triggered again to execute SQL scripts for external backups or data cleansing internally. Through this entire design, we were able to migrate the staging environment from the production database within a total duration of 30 minutes, adhering to defined standards and requirements. Additionally, the entire process can be automated and scheduled at specified intervals.

Results and Benefits

The implementation of this architecture yielded significant results and brought numerous benefits to the organization:

1. Efficient Disaster Recovery
2. Cross-Account Snapshot Sharing
3. Fast Environment Provisioning
4. Cost Optimization
5. Reliable Execution with AWS Step Functions
6. Automation and Scheduled Execution

Overall, the implemented architecture resulted in enhanced disaster recovery capabilities, streamlined operations, improved collaboration, and optimized costs. The organization now has a robust and efficient process for migrating the production database to the staging environment, enabling faster application development and testing while ensuring data integrity and security.

Thank you so much for taking the time to read this article. I hope you found the information helpful and valuable. If you have any questions, comments, or feedback, please feel free to reach out.

Contact E-Mail: hello@bestcloudfor.me
Project Team:

• Ahmet Uçan
• Görkem Aktaş
• Alperen Ruhbaş