What is DevOps?
I was originally inspired to write this post after spending some time talking to recruiters for a company looking to hire a DevOps engineers. They hadn’t done this before and were curious as to what kind of skills these applicants needed. Sometimes I wonder what exactly DevOps is as well, so I thought I would try to put into words what I believe it to be. Let’s start with what Wikipedia says…
“DevOps is a software delivery process that emphasizes communication and collaboration from concept to market, including product management, software development, and operations professionals. DevOps also automates the process of software integration, testing, deployment and infrastructure changes. It aims to establish a culture and environment where building, testing, and releasing software can happen rapidly, frequently, and more reliably.”
I think that is a great explanation, but to put it even more simply, I believe it to be the missing link between code being runnable somewhere and that same code running in production. BIG difference, which unfortunately is frequently overlooked when looking at what it takes to create a successful product. I’ll save that whining for another post, though.
To better understand what DevOps Engineers might do, lets look at the lifecycle of software. In my mind, there are 5 main parts which include planning, development, testing, deployment, and maintenance, although my experience has shown me that most of a DevOps engineer’s time is spent in planning and maintenance. Keep in mind, the stages don’t always come in this order, and frequently this process is embedded in itself as you perform the same tasks for features that are part of a larger project.
The stage where the team (feature developers, qa, product managers, etc) define the goals of a project. They might start deciding on an overall architecture for the software.
The important piece of this stage that must be performed as a DevOps engineer involves using their technical knowledge of their platform and how they can bring it all together into a single coherent system. Once the initial system is laid out and implemented, it then becomes how they can integrate a given technology into the existing stack. This might be planning how to implement an orchestration layer, how to deploy a new database, or how the he/she might support a new microservice infrastructure. They also need to think about how these things can be maintained and automated. Essentially, DevOps engineers should always be looking for ways to get rid of their job through automation.
Common questions at this stage are…
- How will two separate services communicate?
- What communication protocol will we use?
- Do I need to worry about hardware or am I in a fully virtualized cloud?
- What do engineers need to produce in order for me to help them get it into production?
- What is a production-ready service?
- How will all of the software we need going to be run?
- Are all of our dependencies able to be monitored and do we understand them well enough to debug issues?
- Buy vs build?
- Can this be automated?
- How will I support this into the future?
This is where much of the feature work is done. Developers collaborate, code, drink coffee, and spit out features.
The main purpose of a DevOps engineer at this stage is to get out of the way! By this, I mean that they need to enable developers to do their best work, as fast as possible, and in a way that is most compatible with the end goal, production.
This is where some of the tooling comes into play. Let’s say the DevOps engineer spent their time putting together this awesome infrastructure that might be self healing, autoscaling, <insert more buzz-adjectives here>, but it loses a lot of it’s splendor if the software that is being built by developers is being done in a different environment.
Obviously, we cannot ensure that every developers local environment is exactly like production — that’s just not feasible, but we can make it darn close. Containers are a big reason this is possible now, although prior to now we could do some of this in virtual machines.
Questions DevOps engineers encounter here…
- How do we allow developers to use the tools they love while being in a production like environment?
- How can I enable developers to be more efficient?
- How can I teach developers about the environment they are building for?
Developers and QA and whomever else test. Get code ready to be integrated back with the mainline code.
They will setup the tooling and scripts to get this done in an automated fashion, but at the end of the day they are still required to manually run them on their local machines. What if we want to test in production or a production-like environment? What if the test suite is too slow or actually takes control of the host machine in order to work properly? Those tests cannot be run locally, they need to be done in a more automated and consistent environment. DevOps to the rescue!
DevOps engineers play the role of defining and implementing infrastructure that can run these tests in a repeatable way. They would setup something like Jenkins, Bamboo, or Drone, to name a few. These are Continuous Integration (CI) tools that take much of the complexity out of setting up continuous testing. Actually, this would probably be done in the planning phase as well, but I thought I made more sense to put it here since these tools are usually a staple for DevOps engineers.
- How do we create client environments that are repeatable?
- How do we know what version of a service the test is running against?
- How do we keep track of test history so that we can see trends?
- How do we alert developers of their build statuses?
- Where do we get test data from?
Get the code out to an environment, let’s assume production for this article.
How does that feature code get to a production environment that can eventually reach our end-users? That is what deployment is about. DevOps engineers usually use similar tools to the CI tools listed earlier in order to accomplish this task.
Some of the main tasks include answering the following questions…
- When is a build declared ready to deploy?
- How do I deploy a service without my customers noticing?
- How do I ensure that the newly deployed service didn’t cause a regression?
- How do I automatically deploy?
- How do I insert manual steps into an automated deployment process?
- How do I build the correct artifacts so that they are ready to be deployed?
- How do I deploy in a repeatable way?
DevOps engineers may spend quite a bit of their time in this stage, but I believe it should rather be spent in the next stage. They are going to spend much more time with code in production than they are going to spend actually deploying code to production, so that’s where they should focus their efforts.
I believe the vast majority of a DevOps engineer’s time is spent here. In fact, Google essentially has a role in their company fully devoted to it! They are called SREs. I like one of the quotes on that site.
“Our work is like being part of the world’s most intense pit crew. We change the tires of a race car as it’s going 100 mph.”
This stage is essentially all of the work and tooling that it takes to make a system that is highly available and performant. This includes questions like…
- How am I made aware of production level bugs?
- How do I cascade different failures to the correct teams when a bug happens in production.
- How do I solve production infrastructure bugs?
- How do engineers solve production bugs in their services?
- How do I keep track of the health of all services?
- How can our services be made in such a way that we are self healing and autoscaling to different failures and loads, respectively?
DevOps is the amalgamation of all of these roles. Essentially a developer and operations engineer put together. It blends feature work with infrastructure. It is the missing link between code being runnable somewhere and that same code running in production.
Let me know if you disagree or if I’m spot on. @ingshtrom on Twitter