Bringing about a paradigm shift in development & deployment
Co-authors:
- Prashant Mishra (Technical Architect @ Tata 1mg)
- Vimal Sharma (Technical Architect @ Tata 1mg)
- Aman Garg (Associate Technical Architect @ Tata 1mg)
Introduction
At Tata 1mg, our mission revolves around providing a seamless experience for those seeking healthcare services. The platform is an amalgamation of world-class services and a wide variety of products, all integrated by technology to provide convenience to all our users. At the backend, it is essential to enhance developer productivity and satisfaction by addressing the diverse obstacles that arise during the development and deployment phases. We are dedicated to overcoming these challenges to create a more seamless and enjoyable experience for our developers.
Tata 1mg redefines development, ushering code from local to the cloud with precision. Welcome to a containerized future.
Problem Statement
In the rapidly evolving landscape of software development, ultimate challenge is to strike a balance between faster release cycles and maintaining impeccable quality.
As developers, we’re often familiar with the importance of local development, debugging in crafting efficient and refined code. Yet, the surge in microservices has rendered the ‘run-everything-locally’ approach impractical, often leading to architectural mismatches and intricate dependencies.
The sheer number of services and dependencies have rendered local development unsustainable.
How do we address this problem and empower teams to collaborate seamlessly while upholding high standards?
Unlocking Seamless Local Development Through Containers
Local development and debugging are keystones to effective coding. However, in an era dominated by microservices, running every service locally becomes impractical due to resource constraints and diverse machine architectures.
Picture a world where local development is as effortless just a few clicks away. This vision becomes a reality with containerized local development environments.
Containerization: By encapsulating each service within a container, we ensure that what works locally will harmoniously integrate with cloud environments. This eliminates the age-old disparity between development and deployment environments.
- Dockerfiles: Located within service repositories, it enables service owners to define dependencies without disrupting others. This simplifies local service setup, streamlining onboarding and facilitating cross-team collaboration.
We have developed a utility that facilitates seamless one-click setup on a local environment. The primary objective of this tool is to streamline the installation of all essential dependencies like LocalStack in accordance with the underlying machine architecture and to perform the necessary networking processes.
Our local development solution has been constructed around the Visual Studio IDE, leveraging the Docker debugging framework. Comprehensive information regarding this setup can be accessed at. https://code.visualstudio.com/docs/containers/debug-common.
Sample Tasks.json file
Sample launch.json file
Efficient Shared Environment Access: To mitigate the strain of running everything locally, applications including AWS Resources running on shared environments are locally accessible. Developers can seamlessly point to these services from their local environments, eliminating the need for extensive local resources.
We have established a domain that is accessible both through VPN and the office network. This domain serves as an entry point for accessing APIs, which have been made available to developers for consumption. Similarly, various AWS resources such as RDS are also exposed.
Furthermore, we’ve devised a Golden Environment in which only the trunk (master) branch is operational for all services. This configuration enables developers to align their local development setups with a production-like environment.
LocalStack: Simulate the various AWS services (SNS, SQS, S3, etc) on local machines. This will help in testing a piece of code that accesses such AWS services without requiring the actual AWS resources.
Developers now build and debug locally, assured that success here translates seamlessly to the cloud, where containers reign.
Devtron: Orchestrating DevOps Dreams to Solve Cloud Environments
Our ace up the sleeve is Devtron, a powerful Kubernetes and shared infrastructure management tool. Devtron turns the intricacies of cluster and namespace management into a visual delight, eliminating the need for terminal acrobatics.
With role-based access control (RBAC), we’ve transformed shared environment access into a fine-tuned endeavor, replacing broad access with team-specific privileges.
A game-changer, Devtron’s software delivery workflow redefines CI/CD pipelines. We’re envisioning a future where intricate knowledge of Kubernetes becomes secondary.
SSH access to pre-staging/staging environments becomes GUI-enabled, while ssh access for developers on environments can eventually be revoked. Database alterations are channeled through CI/CD pipelines, ensuring controlled, automated changes via schema change management tools like Dbmate.
Each team will possess complete autonomy over their environment, and if necessary for collaboration, developers from other teams can be granted access through a GUI.
Streamlining Branch Management for Seamless Development: A Strategy Unveiled
In the intricate dance of development, the rhythm of branches can either orchestrate harmony or lead to disarray. At our core, we believe in simplicity and efficiency. That’s why we’re introducing a strategic branch naming convention to amplify our development processes.
- Precise Naming for Clear Progression: We’re adopting a straightforward approach: the pre-staging environment is represented by ‘pre-stag’, the staging environment by ‘stag’, and the production environment by ‘master’.
- Master: Our master branch is a sacred ground. All other branches whether hotfixes, features, or more will be branched off from master. Once these offspring are production-ready, they’ll gracefully merge back into the master realm.
- Intermediary Branches for Pipeline Support: Intermediate branches, bolster our CI/CD pipeline and tackle shared infrastructure deployments. However they can grow obsolete or even become corrupted. Simply refresh in such cases. When an intermediate branch like pre-stag or stag loses its sheen, we’ll simply bid it adieu and recreate it from the master source. It’s a pragmatic, easy-to-manage approach that ensures the pipeline stays clear and robust.
- A Hands-off Approach to Master Merges: A cardinal rule is that pre-stag and stag branches won’t be merged into the master branch, they serve as intermediaries branch.
- The power of Automation: Automation comes to the rescue by automatically reconstructing intermediary branches from the master. Aligning them with the pace of progress, they’re lagging behind or require a fortnightly refresh, automation has our back.
- Embracing Flexibility: Teams decide the lifecycle of pre-stag branches for their services, whether it’s a day, a week, or aligned with sprints.
- Optimized Release Paths: If linked branches are prepped in pre-stag, but only one needs release, it’s a direct merge into master. For testing, we turn to stag or the golden environment with CI/CD on master branch.
- Lifecycle of Intermediary Branches: Intermediary branches serve their purpose briefly, championing CI/CD. Teams aiming for extra assurance can road-test on stag or the golden environment.
- Seamless Service Stability: Our system ensures service consistency across shared environments. Pre-stag environments undergo CI/CD on their specific branch, creating a safety net. It’s simple, if it is working in one, it will work in all.
- The Magic of Automation: Introducing a new service? Our automation ensures its presence in all environments through CI/CD and the DevOps pipeline.
- Timeless Strategy: In this holistic scheme, lower environments never gather dust. Developers are spared the labyrinthine task of maintaining services across numerous environments.
Simplicity reigns, harmony prevails. With one branch to oversee shared environments, we quell complexities, streamline development, and prevent the stagnation of infrastructure. At every turn, our strategy seeks to optimize, empower, and innovate.
Conclusion: A Glimpse into the Future
In the nexus of development and deployment, innovation prevails. Containerization, Devtron, meticulous branching, and an empowered DevOps culture stand as our pillars. We’ve witnessed the transformation of local development from a solitary endeavor to a harmonious symphony, where quality and velocity work seamlessly.
As the journey unfolds, we’re poised for more triumphs. Exposing cloud infrastructure to local setups and further codifying workflows are on the horizon. Devtron’s prowess will reveal new depths of configuration, and tools like EKS Anywhere beckon a future where local clusters mirror the cloud. Database synchronization methods are primed for evolution.
