Derek the DevOps Dinosaur
Before I joined the Red Hat Open Innovation Labs team, I was a developer working for a large System Integrator. While there, someone asked me to explain what a pipeline is - referring to some build automation I had created. The person asking me was a Partner and had very limited technical knowledge. He wanted to know what a pipeline is in simple language that he could understand and relate to when talking to his customers. His questions were fundamental; such as what does does one look like and what should it do? While thinking of ways to describe a pipeline in a simplified, relatable way I kept thinking if I could explain it to a 3 year old, I could probably explain it to him. And so was born Derek the DevOps Dinosaur.
Let’s forget about Software for a minute…
Imagine for a moment we’re not building software. We’re not configuring Jenkins, dealing with shell, Ansible or any other automation tool. Let’s imagine we’re building dinosaurs! Big, scary, tough, monstrous and ferocious ones with lots of teeth! Close your eyes and imagine the scary dinosaur for yourself. Maybe you’re imagining of some kind of hybrid Jurassic Park dinosaur. Think about the parts of the dinosaur you’d want to build, how many teeth does it have? How many arms and legs? When I think of my scary dinosaur I think of Derek.️
So how do we know Derek is as big and scary as I need him to be? Let’s start with his parts. First we might analyse each of the individual parts of our dinosaur. Give them a quick check over and ensure they meet the standard we set. For example, do I have 2 arms and 2 legs for my dinosaur. Has he got enough teeth? If it all looks good; we can then pop the parts in the️ Dino-Constructor 5000™. With the Dino-Constructor 5000™ complete, we should hopefully produce our dinosaur, Derek.
How do I know my dinosaur is fierce enough?
So we’ve got a Dinosaur. But remember we’re here to build ferocious scary dinosaurs that are tough and fit. How do we know Derek is tough enough? Well, we could put him through a series of obstacles. Let’s build an obstacle course for Derek.
We’ll start him on a climbing wall, much like one you’d see recruits on in an army movie. Then if he’s lucky enough to get over that hurdle, he’s on to the next obstacle where he must jump over some spikes, Indiana Jones style! Next we check how fit our dinosaur is, if he’s able to run fast on the treadmill he gets to pass to the next step. Here he must try swimming past some fishes that are trying to nibble on him. Once through that perhaps he has to jump through a ring of fire. If Derek is capable, and makes it through the obstacles he can then run towards the pen — however, if Derek is not careful, he may be stopped by the swinging blade which was menacingly looming over him the whole time, like something from a Mario level. At any time, the blade could drop and stop Derek dead in his tracks. Let’s for a moment assume Derek was careful and has made into the pen where the other dinosaurs are.
Derek can now live out his days with the other dinosaurs in the Dino Petting Zoo, Danny and Debbie. However, unfortunately for Derek, Debbie the dinosaur is quite mean to him. She keeps stealing all of the precious grass and water that Derek likes to eat (Derek is vegetarian in this metaphor). So in order to give Derek the isolation and correct amount of things he needs to be strong and healthy, the Zoo Keeper comes along and moves him to a new pen.
Derek as it turns out is quite a popular Dinosaur at the Zoo, so the Zoo keeper decides to make more of him and puts them in a pen with all of his clones. Here he is happy and has enough of all the things he needs to survive.
But wait — we’re building software not dinosaurs!
Sorry to shatter the illusion — but yes, we’re (sadly) not in the business of making dinosaurs. We are here to build applications. What we have just done to our Dinosaur is the same thing we do to our code base on every commit. We build our code, run it through a series of obstacles and then deploy it for our users to consume it. This is a pipeline, it’s quite simple really!
The obstacles are the things we do to further assess our code quality. The initial hurdle Derek must get over could represent some unit testing. It is important that these obstacles are tough enough of a challenge while also not so easy as they provide no value. For example, if Derek can make it over the climbing wall with ease, then it’s probably not testing all the parts of him. Imagine for a moment, we decided to add another arm to Derek. We now have a terrifying three armed dinosaur! If we were to ask him to climb the wall again, he would find it much simpler than before. In this regard it is important to increase the difficulty of the wall, perhaps widen the gaps or make it steeper so it presents more of a challenge. Thinking back to code for a moment, the logic still holds. As we introduce new features to our application, we therefore have to improve the testing coverage to include this. Writing tests is not a one time thing, they must continue to evolve alongside our application.
The other obstacles represent additional testing types. The small piranha pool Derek must swim in to get to safety could represent some early integration tests. The treadmill he must run on might be some kind of performance testing. The final obstacle he must pass by unscathed it the giant blade hanging above him. Constantly looming, this testing type is in my eyes often the one that gets forgotten about. Derek might think he is free and runs towards the pen when the blade drops on Derek and he can go no further….. This is security testing. Often forgotten about until the last minute and can be a show stopper in a lot of cases.
Once Derek arrives to the dinosaur pen, he has to share the space with some other dinosaurs. Perhaps the code has been deployed to a public cloud or to a VM where there is competition for shared resources. Hopefully the Ops teams have noticed the app running out of memory or the lack of compute available. To combat this, the team might automate the containerisation of the application. Once our code is in a box it becomes shippable. We can move it between cloud providers or even just between environments as the code is packaged up with all of the dependencies it requires to run. This ability to move things without the need to rebuild and test can be achieved safely by building immutable container images and by versioning the application configuration separate to the built software. This means we can also horizontally scale our software easily by running more instances based on user demand.
A final thought on building dinosaurs
All of these testing types can be, and should be, automated as part of a software pipeline. On each code check-in an automated process should execute a build, tests and deployment if each proceeding step is successful. Through this process teams can deliver new features faster. They can introduce new code without fear of regression. Container platforms such as Red Hat OpenShift and Kubernetes can ensure the application always exists in the desired state. They can also be used to run our software pipelines, using build tools such as Jenkins to runs the stages. Dynamic provisioning of test tools like Zalenium to execute our browser tests and creating Jenkins agent nodes to execute our builds makes creating pipelines repeatable and reusable.
By automating all the things in a pipeline like this, we ultimately can get the awesome work by the dev and ops teams into the hands of users quicker.