Scaling Stories of Tech Startups — Journey from EC2 to Kubernetes Cluster
After being the champion of the Alola Region, Ash Ketchum came up with an extraordinary idea of teaching the basics of Pokemon. With this idea in mind, Ash Ketchum, with his friend Brock Harrison came up with Oak School, an ed-tech platform for teaching the basics of Pokemon training, so that every Pokemon trainer can complete their dream of becoming a Pokemon Champion in this place.
Ash and Brock hired Misty Williams as a tech Founder.
Misty, with his sister Daisy, came with a Monolithic(All Code in one place) application for the backend and the Frontend application. The monolithic architecture was mainly preferred as they had to ship tons of features and shit with many todos. Misty and Daisy started shipping features in a short time. Now, it was time to deploy the product to the Alola region and show the power of Oak School to the world.
Misty spun off two virtual machines in Amazon Web Service and an RDS instance and deployed both codes manually. Next, Misty bought two domains, “oakschool.co” for frontend and a backend domain “api.oakschool.co”, and connected them with their respective instances.
After the successful launch of the oak school, the oak school went viral within 3 Months, and daily active users scaled from 10 to 1000 Users. One hundred active Instructors started their respective courses at oak school. With this, Misty started discovering anomalies in working on the website. The site was loading too slowly, and students and instructors complained about performance. Misty tried scaling of Backend Instance vertically by increasing the capacity of the EC2 instance(8GB Ram to 32GB Ram, 4 Core to 16 Core). The site generally started working until the scale increased from 1000 to 5000 users.
After extensive research, Misty came to a new horizontal scalable architecture for backend instances. Requests were forwarded to different instances running the same code but connected to the same database.
With this approach, Misty can quickly scale up for any number of users, Misty can increase the number of instances running Api Instance, and Amazon load balancer does its jobs and shifts all requests to different instances in such a way no instance is overburdened. But such a large number of instances are running. So misty can’t just manually deploy code and expect them to run in a very similar approach.
Misty needs a reliable solution, so the code behaves as planned on every machine. Misty came up with containerisation, a way of packaging code with their requirements and libraries. Misty used Docker for containerising the backend instance. Docker helps them to resolve the issue of “it works on my machine”.
Docker helped them maintain similar code working across different instances, but the pain of deploying the code in every machine persisted. So again, Misty came up with Jenkins, an automation server which automates deployment. Jenkins builds and deploys docker image to required EC2 instances. Jenkins pipeline looks like the following.
Above pipeline is also called a Continuous Delivery(CD) Pipeline. With this, Misty solved significant issues related to scaling to deployment.
Team Update: The team has scaled from 2 members to 30 members. The team includes Frontend engineers, Backend engineers, ML Engineers, Data Scientists, DevOps engineers, etc.
As a result, future features like Virtual Pokemon battles, combat analysis, and other features in development cannot be grouped into a single monolithic due to task-specific requirements like the need for a GPU for simulation and costing. Misty decided to preserve the monolith by adding various helper microservices that enable the monolith to execute the necessary functionality.
With this many services running on different machines, Misty required an orchestrator, someone who helps maintain different services and machines and makes them more reliable(Remove unwanted failures such as network and instance failures).
Kubernetes came to the rescue. The Orchestrator platform developed by google maintains the required state by spinning instances when required. The required deployment state is provided to Kubernetes as a code, and then Kubernetes tries to maintain this state by increasing or decreasing instances when required. Kubernetes also helps in scaling instances with ease based on request and demand. Kubernetes solves significant issues related to deployments, scaling as well as maintenance.
TLDR: Now Every One sleeps in peace, including Misty with her Togepi.
Our DevOps Series overview has come to an end. In many following articles, we’ll learn more about Kubernetes, from its inner workings to how to maintain cloud-native applications effectively.
