The Evolution Of DevOps And Why We Are Here
So as to not disappoint, this article is not about the fundamental question “why are we here” and evolutionary theory. But in saying that, almost as mysterious is the question about what is DevOps… and indeed, why are they here?
What was there before DevOps? Is this a new fancy title, originating from the deep seated undertow of Sys Admin’s wanting to be acknowledged? Or is it a genuine skill, borne by the evolution of Captain Automation and his trusty sidekicks CI & CD?
To be honest, the answer is probably both. Firstly, let’s look at the role of a Sys Admin and how over time, their responsibilities needed to morph into what we today call a DevOps Engineer.
The primary role of a System Administrator was to ensure that systems and processes to keep those systems running at their optimum, was adhered to, generally around the clock, ensuring maximum uptime. Traditionally you were either a Linux or Windows based Sys Admin, where you were well versed in the nuances of either platform. Windows Sys Admin’s for instance would be well versed in the need for regular reboots. Awash with potential for memory leaks, security flaws and service crashes in the tightly integrated Windows eco-system, meant that scheduled reboots for Windows servers were the norm. Linux, being a much more modular premise meant that it could practically run for years without so much of a peep. But often when things went wrong, well they were catastrophic.
Change is coming
So there it was, you were either a Linux or a Windows Sys Admin, and never the twain shall part from that. Or would it? What was relatively static in responsibility, started to become a little less so rolling around the time of the early 2000’s. Our first taste of cross platform development was front and centre thanks to the massive popularity of development languages like Perl & PHP.
Microsoft’s strategy was simple. Cater for their bread and butter developers, already active in the Microsoft Windows eco-system. But almost as importantly, attempt to cater for developers toiling away in their Linux-Java-Perl-PHP-CGI world. Capture that market. They did this by providing support for running ported runtimes. Perl started supporting Windows O/S from 1999. Java’s JIT compiler was released for Windows as early as 1997. A little later was the release of PHP for Windows in 2004.
So you can see, there was a real push to make these server-side development languages truly cross-platform as early as 1997- which which incidentally coincided with the initial release of Microsoft Active Server Pages (ASP)and Internet Information Servers 1.0 (IIS).
Technically speaking, at this point in the early 2000’s, if you were a developer in the Linux world, the world was your oyster. You could write applications for any employer and host them, regardless of whether they were to run behind an Apache server or Microsoft’s IIS. Same couldn’t be said for Microsoft developers however. Visual Basic, C# and later F# were all executed on runtimes that were tied down to Microsoft’s main source of income: licensing. Microsoft weren’t going to open source their operating system, that would be corporate suicide.
Ok, but what happened to our Sys Admin?
So, you may be wondering what happened to our Sys Admin? Good question! An upside for our Windows Sys Admin is that they were forced to become accustomed to development languages and frameworks not native to Windows. Troubleshooting server, and in particular, web server issues, meant they needed to be able to not just debug and troubleshoot the IIS execution pipeline, but debug and troubleshoot the PHP or Perl execution runtime. Suddenly they started picking up skills with other languages and frameworks more accustomed to a Linux world. In the meantime, not much had changed for our Linux Admin friends. They didn’t have to learn anything about the nuances of serving a Microsoft Web Form application for instance. But watch this space.
At this point, our Sys Admin, particularly in a Windows world, was forced to expand their skillset to accomodate the additional development languages and frameworks that they were to encounter day to day.
The turning point
This is where the turning point comes for our favourite Sys Admin. Gradually the relationship between application development environment and hosting environment became a tighter and tighter eco-system. Supporting these applications in the wild when a website was down, was usually the responsibility of a Sys Admin. They needed to understand not only the hosting environment and infrastructure behind it, but the application environment and how it runs. With modern day uptime requirements and time to restore becoming more and more business centric, Sys Admin’s needed to fill a gap of being able to not only triage and treat hosting, network and platform anomalies, but application anomalies also. They were relied upon to become increasingly adept at being able to isolate application issues and recommend to the application authors, the source of the problem and recommendations, in some cases, for a restoration of services.
From this, the DevOps Engineer was practically born. As the name suggests, an Engineer that has a sound knowledge of not only the operational aspects of the application or services runtime environment, but also had a solid Developmental understanding of the application running in that environment. This covered what I like to call the “handball over the wall” gap. For years it was common for the software lifecycle process to include the shipping of code from developers to Sys Admin’s to production. Once this was done, developers often wiped their hands of any issues occurring post deployment because, well, it “passed fine in testing”. This is the handball over the wall. Sys Admin’s becoming DevOps Engineers closed this gap and shortened the feedback loop when problems arose.
One of these is not like the other (thanks Dave Grohl)
So are DevOps Engineers more like Sys Admin‘s’ or more like Developers? That’s another good question and the answer can vary. I believe to be a truly competent DevOps Engineer, you must have a solid understanding of software development fundamentals and design. Making sense of production issues, especially in a micro-service architecture where there are a multitude of moving parts, requires a core understanding for how software services “hang together”. But you also need to have a keen understanding about how the underlying infrastructure impacts upon said applications/services. When issues arise, sometimes infrastructure can impact upon application/service performance, sometimes it doesn’t. Being able to determine this quickly is a critical skill of a DevOps Engineer.
Let’s face it, technology is advancing at a rapid rate. Whether it’s stack or infrastructure tooling, frameworks or new buzz word developer languages — yeah I’m looking at you Haskell. There’s no stopping the community generated drive to write a new piece of kit that plugs a gap in operational efficiency or software “coolness”.
DevOps Engineers today
Nowadays, a DevOps Engineer operates in a hodge-podge of declarative/imperative languages and tools. There are no right or wrong choices for languages and tools. Personal preference plays the joker hand in the type of tooling and language you choose for writing, testing, deploying and serving your applications to the masses.
In addition, over the better part of the last 10 or so years, there’s been the astonishing push of Platform-as-a-Service providers and the huge uptick in businesses wanting to adopt Cloud First strategies. By nature of this, additional tooling and frameworks have entered the fray for standing up, configuring and maintaining our application hosting environments. Making the creation of these environments more declarative and making doing so more “development like”.
This plays directly into the hands of a DevOps Engineer. It also continues to bridge the gap between what a traditional Sys Admin does and what a Developer does. It also fundamentally increases the opportunities to identify areas and processes that can be automated. We’ve all heard of Continuous Integration and Continuous Delivery. The ability to do both, reliably and declaratively, gives businesses the competitive edge. Develop, test and deploy more often and more reliably, enabling product and project owners the ability to iterate reliably at rapid rates. What’s not good about that?
The Future of DevOps
DevOps Engineers today can provide great hands on value in so many areas of a business. But with the advent of increasingly “developer friendly” infrastructure tooling frameworks, such as Terraform, Ansible, Chef and the native configuration providers from your favourite cloud providers, the line between the responsibilities of a Developer and a DevOps Engineer does become slightly blurred.
Often, time and resources might dictate that developers should take it upon themselves to create their own application environments and they are now able to do this in a consistent and uniform way, still driven declaratively. So if the developers can do this now, then who needs DevOps anymore?
That is a question that is being asked in a lot of organisations, especially organisations where development mandate calls for autonomy in terms of technology, toolset and framework choice. An organisations end goal should naturally be to enable developers to quickly and easily deploy applications and service without having to know about all the “under the covers” infrastructure to do this.
But heres the thing. The underlying infrastructure, whether its a monolithic application running on oversized and less than elastic compute or a best of breed Kubernetes cluster with all the bells and whistles, are not infallible. You can’t rely on your hosting or cloud provider to tell you that you have problems and where they are. DevOps Engineers still play a key role here in understanding all the moving components and identifying areas of concern.
Further, there can be a lot to learn from DevOps Engineers in terms of best practice regarding scalable infrastructure design, Security Operations (least privilege) and observability. All key parts of a DevOps Engineers day-to-day responsibilities. So it’s a great mix of hands-on doing, mentoring and evangelisation of modern best practice.
Being a DevOps Engineer can be a rewarding and enriching career choice, giving you major exposure to a plethora of technologies, platforms and frameworks. And if you love problem solving, there is probably no better choice of job.
