This week I am beginning a series of articles talking about a software stack that I’m working on for my own projects. I have a set of goals that are driving this effort and I want to help other developers get around the challenges that are typically faced when trying to bootstrap a web application.
My Software Stack Goals
Software Development is a touchy art. If you do it well, you can build something that will be admired and viewed as a good standard to build upon. If you do it poorly, you will end up with something that may be endlessly ridiculed and held up as a standard of what NOT to do. If you want to do it right, you need to start with a good stack, and finding that stack can be like searching for the Fountain of Youth.
I don’t claim to be an expert on picking the best stack, but I do champion a lot of patterns that are generally considered to be “best practices”. These patterns have helped me and teams I’ve worked with successfully launch large enterprise applications processing millions of transactions a day.
My real goals (you could call them my 10 commandments) when picking a stack are:
- A stack should be easy to assemble once and then reused in future projects. I shouldn’t have to be starting completely from scratch every time.
- A stack shouldn’t have pieces that are all required all the time. I should be able to add things when needed but leave them out when they are not.
- A stack shouldn’t require expertise in every aspect of the stack. I should be able to jump in and start using it even if I don’t know entirely how it all works. I can use it more effectively if I do know how it all works, but it shouldn’t be required for a basic “hello world” application.
- A stack should preferably use open-source solutions whenever possible. Not only does this save me money, it also allows me to benefit from the support of communities of developers who are continually improving and fixing the pieces.
- A stack shouldn’t force a square peg into a round hole. It should be flexible enough that the right tools for the right tasks can be used and I don’t spend effort trying to make something work for a task it wasn’t intended for.
- A stack should be able to evolve. Although this may be the right way to do things this year, new methodologies can appear that will make parts of the stack irrelevant or cumbersome compared to the new methods. I should be able to adapt those new methods in without starting over.
- A stack should allow me to optimize for production but also allow me to effectively develop without having multiple configurations. I should be able to easily debug and develop the code locally and then publish the code to run in an optimized way that will run the same on my machine as it does in production (i.e., no environmental drift).
- A stack shouldn’t be full of proprietary frameworks that make it difficult to migrate to different solutions. Although I may need to use a proprietary solution for some of my functionality, I should be able to replace it without tearing down my entire stack.
- A stack should be able to easily scale as I move from a small number of users to a large number of users. I shouldn’t be starting from scratch when volumes get big.
- A stack should be fault tolerant. If something goes wrong, it should be able to recover quickly or direct traffic elsewhere so that my users have no downtime. This also applies to intentional downtime such as for regular maintenance.
These goals are important to me because I have been on projects where not all of these goals were attained and changes in requirements caused painful changes to the stack I was working on. Sometimes the changes were so significant that rebuilding from scratch was less work. It shouldn’t be that way — you should be ready for any kind of change.
Microservices, Containers, and Serverless
The Linux kernel has some controls (namespaces and control groups) that allow developers to limit access to resources, such as memory and CPU, so that processes can be prohibited from using more resources than they are given access to. Docker created a platform so that developers could easily use these controls to build simple applications (microservices) that could be spun up quickly in what came to be known as containers.
Developers began to write smaller pieces of functionality to run in these containers, and it was found that applications would scale better when different pieces of functionality were run in separate containers. Containers could quickly be created and removed, and so services with high demand would use more containers than ones that had low demand. This helped utilize hardware resources better, and infrastructure costs went down.
This scaling down of functionality was so effective that developers began writing single-function services that could be easily scaled up and down to meet demand. Cloud platforms like AWS created ways to easily write these Serverless functions and expose them so that anyone could use them. Advanced container platforms like Kubernetes made managing all of these containers a snap.
So Why am I Telling You This?
I am telling you this because this revolution in software development is making it possible to achieve the goals of the ideal software stack. Many development teams in enterprise companies have already started utilizing these tools to accelerate their development and a lot of them now release software every single day. Some of them do it many times a day. Developers can join the team and deploy code to production on the first day.
Sound too good to be true? Well, stay tuned for my next article where I will start going over the stack I’m building and talk about how it meets the goals that I listed above.