DevOps hierarchy of needs

People often ask how Qubell is different from Chef, Puppet, OpenShift, Docker, Heat, Ansible, Mesos, Kubernetes or some other hip product du jour. Like any comparison of apples to oranges, the answer involves a surprisingly deep investigation into botanical conventions, differences between cultivars and species, and even a venture into dangerous waters of “what is fruit, really?”. Intellectually stimulating, but not that useful from the nutritional standpoint.

However, over time it became clear that it is much easier to phrase the question differently. The right question is “I got Chef/Puppet/etc, what’s next?”

So I raise you the following model (dibs on the name “Klimow’s Pyramid”):

One by one, from the bottom up:

Infrastructure-as-a-service is a basic need, a recognition of the fact that IT processes do not work without infrastructure. That does not necessary mean servers, it’s just a way of mining basic resources to keep the lights on. (Some people say that access to infrastructure-as-a-service is now a basic human right in former Siam, guaranteed by Constitution.)

Cluster management is my best attempt to give a name to the category of cattle management tools designed to take thousands of pets, braid them with numbers and relieve the operators from the need to come up with names for each of them. Pretty much every tool that call themselves “configuration management” fall into this category: take a server, assign it to a role, administer electric shock every time it tries to get over the fence. I do not call them “configuration management” for the reasons that will soon become evident — but hey, “cattle management” is not half bad.

So at this point in time I have Chef/Puppet, what next? Service orchestration is what’s next. What is service orchestration? Well, everyone has something different in mind, but the goal is the same: we have higher-level bricks now, how do we make a system out of it? Clearly, any non-trivial system will contain distinctly different parts. Application tier needs to talk to the database etc etc. Some systems will let the former look up the later, some will feed the dependency from the outside, but the net result is an emergence of some kind of a network of runtime dependencies that can handle non-uniform scaling, zero-downtime upgrades and the other goodies that we came to expect from a robust system.

Well — this is paradise, right? What could possibly be better that? Nothing, if you came from the future and all you need are my jacket, boots and motorcycle. The puny humans among us, though, request that the resulting service network provide some guarantees to its overlords. This is where the service-level configuration management kicks in: we want the service network to converge to a given revision of a known state, we want it to scale based on human (business) metrics, and we want the development environment to run on CentOS and Tomcat, because, you know, licenses. Oh, and never, ever, ever, we want the application tier of one customer to inadvertently hook up with the database of another. Put simply, orchestrators are the traffic controllers and stop lights, but policy is the vehicle codes that enable us not to have a traffic controller at every junction.

Now we have the system that is perfectly automated, governed and efficient. What’s left? Well, the piece that is often left out is the human touch. The resulting system will grow with its users, operators and developers, it will have to change and adapt, and it cannot rely on a priviliged caste of priests to mediate the interaction. I wanted to call this layer “self-service” — as a nod to self-actualization —but it did not fit, so I put a wider goal in there — enabling the just-in-time IT by trivializing change management. Oh, you want a new Hadoop cluster? a feature branch deployed? somewhere to run your tests? dark launch of a new feature? scrubbed production data for analysis? You got it, buddy. Let me shake this system for you and get you what you need. Just hang in there.

So there you have it. Personally, the most amazing thing is how quickly the needs get realized once a level is cleared. There’s a ton of knowledge about building the higher levels, but it used to be a 1% occupation until the foundation became ubiquitous and commoditized. These new needs is where Qubell is focused, this is where exciting stuff will be happening in the next few years.

Oh, and if you’re at DevOps Summit this year, @vlivschitz, @ornatsky and myself are all gonna be there. Come say hi.