Continuous Delivery and What the Heck Happened to QA?
Continuous Delivery (CD) is often considered a stakeholder’s dream. No waiting for long QA and UAT cycles, think of it: get it built: ship it. And you know what? Continuous Delivery is really pretty cool. But it’s not a free-for-all, as some people may purport. What it really requires is far, far more responsibility and discipline — and most importantly, collaboration — than what was inherent in traditional models.
I am currently implementing this process right now, and it has been an intense learning process. Recently while working on automating deployments with Jenkins, I was listening to a radio show and it became clear to me WHY CD works.
The program was talking about how people become engaged in their work, and how the assembly line and many facets of the industrial (factory) age optimized output while entirely disenfranchising people (workers). Often in my core-dev days, there were times I felt like a factory worker in a classic systems development life cycle. Now, as a technology leader, I have the blessing, even the mandate, to make sure my crew feels engagement and ownership.
Classic SDLC
“I develop code, and throw it over the fence. QA reviews, tosses it back or sends it forward. Operations asks questions. UAT sends it forward or throws it back. In traditional development, we have “industrialized” software production much like a factory approach turns making a simple pen into a fourteen step process, each step done by a different person.”
In this classic model, a developer is involved in the various stages of the life cycle, but often doesn’t feel committed beyond their invisible wall — the hand off to QA. How can we help them feel committed to all phases?
Why I Think CD Works
I have been a fan of Agile for years. And then CD came along. I thought I knew how to run the process. I thought I knew how to get products shipped. And I did. I do. CD doesn’t change shipability or quality. It just shortens the Agile cycle. CD elevates my contribution, my commitment, and my responsibility.
How does it do this? Let’s look at a few more questions to get the answer.
What the Heck Happened to QA?
Once upon a time there were developers… They wrote code. They copied the code to a server. If there were issues, they fixed them on the server. If they were careful, they remembered to copy the same changes back down to their own computers. (then we invented source control)
As it turned out, we broke stuff more often than our customers liked! Thus, the position of QA was created. Someone who was responsible for making sure the code really did what the developer claimed, before it went out to the public audience.
Then we decided UAT was an important step so we added that. So now, once QA made sure the code worked as advertised, UAT checked to make sure it worked as it was intended.
At some point, we realized developers didn’t always make good admins. So, we made sure we separated those roles and pushed developers back to writing code. And what did they do? They monkeyed around. They got complacent. They started throwing code over and hoping for the best, relying on QA and UAT to sort out and solve the problems.
Fast forward to the present… If you have been following the evolution of CD, you might have noticed the role of traditional QA engineer shrinking, and in many environments, simply not even existing. What is happening? Does management no longer care about quality? Or developers?
If we’re doing Continuous Delivery right, quite the opposite. The CD approach attempts to RE-enfranchise the developer. To increase their ownership of their own work — from typing in lines of code to making sure it all does what it’s supposed to.
So What’s Our Job Again?
Yes, under CD we still have product owners. And yes, in some cases it still makes sense to have QA (although the role has changed dramatically). But by giving developers the power to go to UAT (or even production) on their own assertion of completeness and quality, we have given them back power, and (cliche alert) with power comes responsibility.
There has been a gradual realization on the part of developers that they need to care again. Want to care again. Their obligation to quality goes all the way to the front door, the customer. QA isn’t going to catch problems and nobody wants to be the one who caused the production outage, or the million-dollar null pointer exception.
So we need to write more tests. More types of tests. Front and back end tests. And we need our peers writing tests for our work so they can understand it as well as we do. Shared ownership = team spirit (we hope!).
Under CD we also need to perform proactive in-stream code reviews (not after-the-fact-the-system-is-complete code reviews). We need to interact with the product team more closely, feature by feature, not sprint by sprint. Peer reviews gain greater importance in this system with the level of collaboration spiking dramatically compared to the traditional models. This is why CD works. It also equals cross training, which is quite nice.
We also need to BE Ops, or work closely alongside Ops, to advise and promote. We need to devise more intelligent systems so there’s less overall operational work (centralized configuration, service discovery, deployment automation) — and in the end, for doing this — we get to feel like we did something. And that’s the key. That’s the trick.
At the end of the day our work goes live, and that’s the re-enfranchisement, the hidden promise, in the seeming ‘insanity’ that is Continuous Delivery to the un-initiate.
Yeah That’s What We Do!
I love to look at the reports of number of deployments in a week now. Instead of .02, it’s 3, or 5, or even more. It’s fun to talk about “when can we ship that” in terms of a couple of days or a week, not months or more. That complacency mentioned earlier wasn’t laziness — it was the perception that what I might do today just doesn’t matter all that much — I can work a little harder tomorrow maybe, or solve some bugs when QA files them.
In the stream of life we must have goals, a purpose, and it is fine for those to be six month, one year, five year goals. But when we break them down and then down again and then down again into a schedule that says “In two weeks we’ll be ten percent done toward this six month goal!” it can be difficult to keep a good pace. On the opposite side of the spectrum we can go into our weekend with some one or two day goals and attack them with vigor — often achieving them. Let us apply this philosophy, this human physiology of dopamine shots when achieving small short term goals to our lives as engineers.
Sure, lots of advancements in automation systems, virtualization, and the like have helped make this possible. But don’t let it be lost on you that the reason it CAN work, that it DOES work, is because being a developer doesn’t mean being a code monkey, it means being an engineer, taking pride in your work because you see the results in a contextually relevant amount of time, it means being an owner. That’s what we do.
STATEMENT: These opinions are those of the author. Unless noted otherwise in this post, Capital One is not affiliated with, nor is it endorsed by, any of the companies mentioned. All trademarks and other intellectual property used or displayed are the ownership of their respective owners. This article is © 2017 Capital One.
For more on APIs, open source, community events, and developer culture at Capital One, visit DevExchange, our one-stop developer portal. developer.capitalone.com/
