How we moved from a giant monolithic system to microservices
@ Rebel Foods (Part 1/2)
Defeat them in detail: The Divide and Conquer Strategy. Look at the parts and determine how to control the individual parts, create dissension and leverage it. — Robert Greene
With this being said, we at Rebel Foods tech, strongly believe in the idea of breaking down a large monolith system into the small sets of microservices. Here I would be describing the purpose and the journey behind it.
Overview of the monolith giant system of Rebel Foods:
When we started in 2011 with one brand FAASOS the requirements were quite clear to us — “Build a system that can handle 15k transactions per day” and 15k was just the number of orders expected daily, behind this Rebel Foods manages the complete lifecycle of on-demand food right from the inventory to our supply chain to content management to user channels like our mobile apps, websites and our third-party partners to our cloud kitchens to finally making the last-mile fulfilment. Looking at the fast-paced market we had to deliver a system that can produce a flawless experience while being 99% fault tolerant. We came up with a central core system design that would interact with our main database and would expose the APIs to perform various operations required to satisfy the above needs. We called this system the V1 architecture of Rebel Foods.
The V1 & V2 architecture:
We used AWS cloud to manage and deploy our systems and the code base was built majorly in NodeJS and Ruby on Rails.
We divided the complete architecture mainly in three modules:
- Routing
- Computing
- Storing
User channels like iOS/Android mobile applications, websites and third-party channels were using our APIs that were deployed and managed in a single monolithic system.
Routing: This part involved the AWS Route 53 where DNS resolution happens, Cloudfront was added for content delivery with API caching and AWS WAF to protect from various attacks like DDoS
Computing: A central auto-scaled system deployed using EC2 instances on SPOT fleet for achieving on-demand cost optimized scaling. Various peripheral services like our food ordering Alexa bot was deployed as AWS Lambda function.
Storing: AWS RDS with master-slave replication has been used to store the complete data we had, in addition to this, data caching has been done using AWS Elasticache with Redis.
V1 and V2 architectures were essentially the same with V2 having some additional code base optimization which mainly affected the computing component of our system and the same architecture was being used.
Benefits we had with this architecture:
- Due to being monolithic in nature, the systems were very simple to develop because by default all the tools and IDEs supported that kind of application.
- The system was very easy to deploy because all components were packed into one bundle.
- It was easy to scale the whole application.
Challenges we soon started to face:
The above architecture worked flawlessly for 5 years and gave minimum hiccups to us but soon Rebel Foods witnessed a major change with the idea of changing our business model from being just one brand to have multiple brands operating from the same cloud kitchen. The initial brands that we launched were Behrouz Biryani, Ovenstory Pizza and Olivetrails. Each brand has to have its separate inventory, separate products, a separate website and mobile applications along with multiple integrations on third-party channels and soon we started to feel that it’s high time to evolve!
The monolithic architecture itself had many drawbacks of its own like:
- Adapting to any new technology was very challenging.
- Very difficult to maintain its CI/CD pipeline.
- One component failure will cause the whole system to fail.
The Solution: Thanos of Rebel Foods
We started to break down our system in different sets of independent microservices which are unique and are built to perform their specific tasks quite similar to what the infinity gems are in the MCU. This was the whole idea behind calling the V3 architecture as Thanos- mad titan harnessing energy from all the infinity gems.
But before going further into the details, let’s see what microservice architecture is:
In microservice architecture, various loosely coupled services work together as a system with each service focuses on a single purpose.
The benefits of microservices
- Faster deployments since each service can be deployed independently of other services also this makes it a lot easier to deploy new versions of services frequently.
- Easier to scale development and can also have performance advantages.
- Easier to work on new technologies and removing dependencies on the old stack.
- Decoupled services are also easier to recompose and reconfigure to serve the purposes of different apps (for example, serving both web clients and public API).
The drawbacks of microservices
- We must deal with the additional complexity of creating a distributed system.
- Deployment complexity. In production, there is also the operational complexity of deploying and managing a system comprised of many different service types.
- As you’re building a new microservice architecture, you’re likely to discover lots of cross-cutting concerns that you did not anticipate at design time.
But with proper planning and vision, we can overcome these drawbacks.
The V3 architecture:
The major change in this architecture was the further division of computing unit into different sets of microservices which have their own database. This strategy also helped us in breaking down our single database and removing the complete dependency on a single data source. For instance: (assuming) due to heavy load the reporting service became unhealthy but this won’t affect the revenue since the order service can work independently and would not depend on any other service and leaving the revenue unaffected. Whereas in our veteran architecture this would have impacted the revenue due to very tight coupling and a monolithic system i.e load used to be equally impactful on the overall system.
In Thanos, each service communicates with each other using the pub-sub implementation in Kafka. Essentially each service can take the role of both producer and consumer based on the events they generate or consume. This enabled us to improve on the response time of our APIs since TCP(transport layer) communications are faster than the HTTP(application layer) communications in our case.
In the next part of this blog, I would be sharing with you the working and implementation of each component we added in our V3 architecture. Thanks for reading. You can connect with me if you have questions or want to discuss more on how we started adopting the microservice architecture.