Developing a monolithic service that can break down into microservices
Microservices architecture is a method of developing software applications as a suite of independently deployable, small modular services, like tiny lego building blocks. And if you combine well-running modules together, you can make an entire service.
In some cases, at the design time, we agreed that microservices architecture is suitable for our software. However, when we started the development, it seems that microservices cause more technical problems than solving the problems. And after that project started to miss the delivery target.
Microservices are often 10 times larger and require more resources than monolith applications. Here are some points to consider before starting microservices development
- Size of the development team: Developing a production-grade running service requires many tasks to be completed, i.e. code base, CI, deployment, error handling, resiliency, etc. (see full list below in Concerns for a single service). Imagine that you have to implement those for each service, it requires more resources than developing a single service (or monolith)
- Team expertise: Unless you have enough developer to fit in each position on each service, a microservices developer requires to have a broad set of skills from software design, coding, database design, deployment, DevOps, etc.
- State of the project: It would be much easier to design the overall system and splits into small modules for a mature project which already has set of firm technical requirements in place, a project that recreated from an old monolithic service with microservices. On the other hand, if requirements are not set and still evolve over time, each change may need to modify multiple services that invoke more works like code reviewing, testing, deploying, etc.
In short, if you have a relatively small to medium size team and starting a new MPV project, developing a full microservices would stress your workloads pretty much. What if, you already foresee that the service would need more flexibility and scalability in the coming future. How can you achieve that and also manage to deliver a project on time?
Start with monolithic, but keep microservices principle
When implementing a module, keep the following point in
1. Use the onion model to design each module. Core business logic is at the center of the onion and wraps other concerns as middleware (decorator pattern) around the business logic. There are three major layers: service, endpoint, transport. Interaction between modules are made through the transport layer, which can be HTTP or RPC (GRPC)
2. Use Clean Architecture design for each module. This helps coding easy to organize and adaptable to changes. The central rule of the Clean Architecture is the Dependency Rule, which says source code dependencies can only point inwards.
3. Avoid direct dependency between modules. Instead of make a direct reference to another module, using the transport layer from #1 to interact between modules.
4. Unit test suite: test all logic in the code
5. E2E test suite: E2E test suite help to check that all service is up and running as designed. If your service provides public APIs, you use postman collection to test them. see more in starman
6. CI: quick deployment, integration, and testing
Transforming monolith to microservices
After several development iterations, services start to be mature or require to scale out. You can spin out each module to be a standalone service.
- Enhance startup script and configurations to deploy modules as a service, and update other services the new endpoint for splitting service. Incorporate service discovery service may help to decouple service configurations.
- Add load balancer in front of services for scalability if requires
- Run full E2E test suite to make sure that all functionalities are still working fine after transformation. If the service provides public APIs, the E2E test suite help identifies breaking APIs.
Concerns for a single service
List of questions that would need to have answers for before a service could be considered production ready
ref: Sean Treadway, SoundCloud
- Service name
- Programming language(s)
- Programming paradigm(s)
- Architectural choices
- Integration pattern(s)
- Transport protocols
- Authentication
- Authorization
- Reporting
- ETLs
- Databases
- Caching
- Platform libraries
- Service dependencies
- CI Pipeline dependencies
- 3rd party library dependencies
- 3rd party service dependencies
- Security threat model
- License audit
- Compliance audit
- Capacity plan
- Provisioning plan
- Cost reporting plan
- Monitoring plan
- Maintenance process
- Backup and Restore process
- Secret management
- Secret rotation
- On-Call schedule
- Configuration management
- Workflow management
- Alerts
- Log aggregation
- Unhandled failure aggregation
- Operations and incident response runbooks
- API documentation
- Source Code Repository
- Humane Service Registry
- Service Discovery Registry
- Distributed Tracing Registry
- Monitoring Dashboard Registry
- Build Artifact Repository
- CI pipeline(s): Build, Test, Publish
- Integration tests
- Contract tests
- Canary
- Deploy
- Post deploy tests
References
- Microservices vs. Monolith Architecture
- Go + Microservices = Go Kit — Peter Bourgon
- The Clean Architecture
- starman let you do Postman collection with the way the developer love it !! ✨🌟⭐️ 🚀🍾🧚🏼♀️💫.
