Developing Infra Features without an Infra Team
In the last year CyberArk has launched a new SaaS offering, which uses a microservice architecture and involves multiple teams.
Whether you’re an Architect or an R&D leader, you’ve probably faced the following Infra related questions:
- How is it best to develop Infra features? E.g., authorization between services, audit, etc.
- How can we be certain every service follows our company guidelines (secured coding, CI/CD pipeline, etc.)? Moreover, how can we lower the development effort for every service?
- What is the best way to share code between teams, making sure we limit the waste?
Today, I’ll walk you through our journey and show you how we approached these questions.
How we addressed Infra related questions
Some companies might have a dedicated Infra team that can help answer all those questions. These teams are usually in charge of shared Infrastructure, and provide solutions while consulting with other teams.
Instead of forming an Infra team, we decided to use a more open source model that enables teams to influence the decisions and changes we make.
We created a shared codebase, Java Service Template, developed by the following company guidelines:
- CI pipeline — full pipeline including scanning tools like Sonar, Checkmarx, and more
- Container support
- Installation
- Security (hardening, secured coding, etc.)
- and more…
Then, to get started, we used our Backend Guild for meetings and development. The guild meets once every 1–2 weeks and includes representatives from all teams — allowing us to regularly collaborate across the company.
Java Service Template — behind the scenes
Java Service Template is a simple Spring boot application that contains two main parts:
1. Infra layer — including logic relevant to all services
2. Sample “service” application — in our case it’s an account management app using the Infra layer
The sample app is developed using CyberArk best practices and guidelines. All other services can use it as a reference, and see what we expect in terms of:
- Automation
- Security
- CI
- and more…
It’s also a good starting point, as it contains best practices and examples that can help newcomers get up to speed quickly.
The Infra part provides a better way to develop common logic. All services (including the sample app) consume Infra layer jars, like any other 3rd party dependencies. This way, getting new Infra functionality requires a simple version increment. Moreover, we use the sample app to test Infra changes and provide a working demo to all other services. It acts like a playground where we can test new functionality and verify that nothing breaks.
Quick Wins & Challenges
Fast forward a few months, we already have a few successes under our belt:
- Created several services that use the developed Infra functionality
- Lowered service ramp-up time — from no code to basic service with full CI pipeline in days instead of weeks. This helped to focus teams on business logic instead of developing their own infra
- Created uniformity across teams
- Implemented several Infra features, like:
- Inter-service authorization
- Service level automatic password rotation
- Tomcat hardening (e.g., limiting supported ciphers and protocols)
- Multi-tenant support
- API versioning management
- Public API documentation
- Monitoring metrics
- and more…
Despite the successes, we also faced some challenges along the way:
- Ownership — who should handle bugs found in production?
- Development — the Guild helped us to get started, yet, it’s hard to develop when we only meet once in 1–2 weeks. Guild members should invest extra time (in addition to guild time) in adding features to Java Service Template for the benefit of other teams
Wrapping up
Going back to the questions raised at the beginning, we can say that Java Service Template is our answer. It provides us with better communication, sharing, collaboration and answers the Infra features development.
Initially, we used our Backend Guild for developing Java Service Template. Moving forward, the Guild might be a good place for discussions, but not for development. Whenever a team needs a new feature, it will develop it in their service and then open a pull request in Java Service Template. Backend Guild leads will review the pull request and either merge it to master or require additional changes. This way we will keep it updated, make sure it meets our requirements and share changes with other teams.
One last thing left without a clear answer, is ownership. There is no owner for the template’s Infra code, so once an Infra bug is found in one of the services, it is the service’s responsibility to fix it in Java Service Template.
While we might decide to form an Infra team in the future, this model seems to work and meet our expectations.
