From Agile to Serverless

Looking back. And forth.

The microservices architecture was born as a technological answer for the iterative Agile development methodology. At the early days of microservices, many companies were doing a form of Agile (XP, Scrum, Kanban, or a broken mixture of these) during development, but the existing software architectures didn’t allow an incremental way of design and deployment. As a result, features were developed in fortnight iterations but deployed every six to twelve months iteration. Microservices came as a panacea in the right time, promising to address all these challenges. Architects and developers strangled the monoliths into tens of services which enabled them to touch and change different parts of the system without breaking the rest (in theory).

Microservices on its own put light into the existing challenges of distributed systems and created new ones as well. Creating tens of new services didn’t mean they are ready to deploy into production and use. The process of releasing and handing them over to Ops teams had to be improved. While some fell for the extreme of “You Build It, You Run It”, others joined the DevOps movement. DevOps meant, better CI/CD pipelines, better interaction between Devs and Ops and everything it takes. But a practice without the enabling tools wasn’t a leap, and burning large VMs per service didn’t last for a very long. That led to containers with the Docker format which took over the IT industry overnight. Containers came as a technical solution to the pain of microservices architecture and DevOps practice. With containers, applications could be packaged and run in a format that the Devs and Ops would both understand and use. Even at the very early days, it was clear that managing tens or hundreds of containers will require automation, and Kubernetes came from the heavens and swept all the competition with a swing.

Now, we do Agile development and use microservices architectures. We have one-click build and deployment pipelines with a unified application format. We practice DevOps and manage hundreds of microservices at scale, do blue-green and canary releases with Kubernetes. This was supposed to be the time of long-lasting technological peace where we can focus on the business problems and solve them. But it was not.

From Agile to Serverless and Beyond

With Kubernetes, Devs can develop and Ops can deploy in a self-service manner. But there are still only one or two company-wide Kubernetes clusters that occasionally run out of resources. The cluster is now the center of the universe. And it is even more critical than the monolith was before as it runs the build server, and the git server, and the maven repository, the website, MongoDB, and a few other things. So a promise for a serverless platform came from the clouds. It came even before the containers reached into production. With serverless, every service would have its own cluster, a cluster where resources are limited only by the limit of the credit cards. It is a platform to which you tightly couple, and bet the existence of your company for the price of going faster. So it started all over again.