AppsOps: The Forgotten Discipline and Answer to the NoOps Fallacy

I’ve watched the emergence of the NoOps terminology with some amusement, but also some fear as I see the potential for all the cultural advancements of the DevOps movement to be quickly erased in a short bout of ignorance. NoOps is the sort of demeaning and elitist terminology that’s bred from a lack of understanding and empathy, and even worse it’s being spouted by enterprise software companies at their customers. Software companies need this empathy, and it’s the kind of empathy one only gains by having lived the lives of our buyers.

I’m sure every software developer has had the negative experience of going to IT Ops, requesting a machine so they can setup a development environment, and gotten the runaround of process after process obstructing them from getting the basic tools required to do their job. It is certainly incredibly frustrating, and that experience I’m sure can lead to the idea that life would be better without having to deal with obstructionists. It is also near certainty that the emergence of Cloud, Server-less Computing, Containers & Orchestration, and other advances in computing and IT will allow for developers to minimize their interactions with these sort of processes and people, allowing for minimal interactions with operations: NoOps.

However, the idea that we no longer need an Operations department is a total fallacy. It’s run of the mill for us, online companies and products companies, who are generally the thought leaders in terminology and methodology, to espouse that their methods are the true way forward. Certainly, we’ve brought most of the technology and emergent trends over the last decade, but we also lack an understanding of the challenges of our customers and peers, running businesses on technology, but who are not technology companies.

In a major online services company you are likely to be operating a handful of applications that serve millions of customers. Size and scale are massive challenges. You are an incredibly agile company because your marketplace demands that you be fast to market. However, those applications have a fraction of the surface area of an ERP system for example. In another business, you are likely running dozens or hundreds of applications, all with incredibly complicated workflows that have been customized over years to embed your businesses specific processes into the software. Some of these systems are off the shelf, but you are also running dozens if not hundreds of custom software applications built in-house. Your life looks pretty similar to the online services company in that regard. You need to be agile, you need to get software to production quickly to be competitive, but instead of size and scale of a few applications, you’re dealing with how to cover all the applications that you need to keep running to keep every area of your business functioning and how to make sure they can all talk to each other without problems.

In a traditional business running dozens of applications, the idea that the minimal number of on-staff or contract developers are going to take on the load of also taking every production support call is ludicrous. Some applications users use on a daily basis don’t even see updates for months on end and have no developers even assigned to them! This isn’t bad architecture or something that even needs to be solved, it’s just the reality of living in a business world where your business wasn’t created 5 years ago. Every one of those applications will have issues, and every one of those applications will need someone to triage and understand more than the average Service Desk person will be able to understand. Sending every error that a user experiences from the Service Desk directly to an engineer who wrote the code will create a huge hole in the middle of the organization.

I know this world well. Before I came to the products side for a software vendor, I was Director of Application Operations for a telecommunications company. I lived and breathed this world day in and day out for years. We had dozens of custom software applications, including our CRM and POS system which was completely written in house. I also owned the Configuration Management function, which 5 years later I would have classified as DevOps, and before I owned it, it was sitting in Development. I like to think we were on the leading edge of organization and how we thought and deployed software, although I very much wish the advancements in CI/CD, Microservices, etc, had been around 5 years ago because we certainly felt all the pain those movements hope to minimize.

Some of the time AppsOps was about pure outages, or about brown outs of the services, but most of the job was understanding the business and the processes of the business. It was understanding about data and interactions amongst systems, so we could triage whether conditions we were seeing, errors which were showing up in production, were expected or unexpected. We were the day to day representation of the business in IT. We were far fewer than the developers, they outnumbered us 5 to 1 at least, but I would hope years later that those that I worked with would absolutely say that we added value and did very little to impede the developer’s interaction with their production software. They still saw all the defects. They still wrote all the patches. They just had a filter, an expert at a lower tier in the organization who operated their systems for them every day so they could focus on development. I think if you asked them, NoOps would have been a huge step back.

We can mostly all agree that the days of treating your systems like pets should be over, and we no longer need to have a large number of people in your IT organization racking and stacking servers and working ticket queues to provision physical and virtual machines for users. Automation is the future, and CI/CD has shown us the way to moving code to production reliably, quickly, and with high quality. However, NoOps is a fallacy, and while you may not need server admins, you definitely need Application Admins and Operators who will run the services that run your business. They are face of Operations to the business on a day to day basis, and they will maximize your development team’s time for moving the business forward while minimizing the time they spend keeping the lights on and the doors open.