Ever since my professional career shifted from development to DevOps (meaning I stopped coding in Java and started working with CI/CD pipelines and IaC), I’ve had the chance to learn a few “dos” and “don’ts” of DevOps culture adoption/implementation, on existing teams and projects.
Well, if there was one insight I got out of reading Jim Collins’ “How the Mighty Fall”, it was that it pays more to look at what people have done wrong, and avoid it, than at what was done right, only to end up asking yourself “how come it did not work? I did everything right!”
So, without further ado, here is my list of “things you should never, ever, ever do, when adopting DevOps”:
Mixing up “DevOps” and “The Cloud”
Or, put in a different way, mistaking technology for culture, that is, approaching something that is essentially a concept as if it was a matter of start using a set of tools.
On a parallel universe where no one came up with cloud computing, there are still people adopting DevOps, and benefiting from it; on another one, where no one thought about DevOps, you still have people opening support tickets to infra, asking them to setup new database servers for them. It’s just that, there, sysops run scripts, instead of adding servers to racks in data centers.
As the name implies, “DevOps” will impact on how at least two very distinct groups of people do their work; some responsibilities will be shared, others transferred between them altogether. Granted, learning new technologies, like Jenkins pipelines and Terraform, is a challenge on its own, but hardly the most difficult one, for technically oriented people; it’s the mindset transformation hardships that screw you.
Here’s the example I always come back to, to drive home the point of how different “culture” is from “technology”: do not expect developers, that never had to care about infrastructure costs, will suddenly realize they should not spin environments just because they can, after you give then that amazing, fully automated, just-push-this-button, IaC enabled, interface. It is like giving a bazooka to a child, then blaming the child for what happens afterwards.
You have a team with five people, working on the same sprint, albeit in different stories/features, that are constantly stepping on each others toes because they are all deploying to the same stage environment. From their point of view, it may seem perfectly reasonable to create other environments; if doing so would be unacceptable because of cost related issues, do not expect they will realize that on their own.
If anything, they are probably thinking that, by spinning new environments, they are optimizing their development process and actually cutting down costs.
This is just an example of one, among many things, that could go wrong. What you should have done, in the first place, was evaluating the impact adopting DevOps would have across your enterprise, anticipating which new responsibilities and concerns each team member (or team member profile) would now have, and train them accordingly.
Everything else is more or less a side effect of this mistake, in the sense that not realizing the distinction between “culture” and “technology” leads to poor planning.
Assume the development team will catch on the technology on their own, without proper training
Just because the technology is not where you should start, does not mean you should underestimate it. For example, it is becoming increasingly common to find developers that are familiar with containerization, but not so much that you can simply assume anyone could properly configure a Kubernetes deployment, in ways that are both safe and resource correct (that is, neither allocating more resources than needed, generating unnecessary costs, nor too little of them, causing pods to starve and crash).
It pays to support key team members in learning new technologies, so they can in turn help spreading the knowledge to everyone else. In my experience, poorly understood new technologies are more damaging than the ones you know absolutely nothing about. What you don’t know, you learn; what you think you know, you do wrongly.
Learning-as-you-go has its benefits, but do tend to make people only catch up to something when the need arises; which sometimes is only after something has gone terribly, terribly wrong.
There are lots of online training platforms out there, and lots of certifications. Help your key team members get them.
Ignore your context, history and team members profiles
Say you have an amazing development team: your architects seem to feed on technology books and articles, and are always on top of everything that is going on in IT. Your developers code as if their fingers are dancing away carelessly on their keyboards.
Still, shit happens, and sometimes it happens in production. On Sunday. At 4:00AM.
And, because you adopted DevOps, you now believe whoever builds the code, runs it. In the old days, an alarm would go up, or someone would notice the issue, and sooner or later your operations team would get cracking on it. Because this is how things used to be done back then, they would most likely be sysops.
Now, you are playing a different game. Now, the “DevOps guys” are responsible for monitoring, and they will handle it. Problem is, that super development team you have abhors support. They believe, most of the time, when something goes wrong, it is not because their superb code has a bug, but because the network had a glitch, or a database has to be scaled up, or anything else they don’t care about.
Well, you say, they should care, because now we are doing it “the DevOps way”. But maybe they are thinking don’t have to care, at all. They just have to find a new job.
It does happen, you know. People get unhappy about their current job and move on to new ones. I have never worked on a company at which turnover was not, to some extent or another, a concern.
Cultural transformations can be traumatic events. It pays off to set the game rules, before your roll them out. Sometimes people will be ok with them, as long as they are consulted first, and have a chance to agree on what you expect of them, and organize accordingly. Sometimes you will know, in advance, that you need to hire new profiles, to cover the bases your team is currently missing.
Starting DevOps as a technical initiative, without executive sponsorship
Because people — all kinds of people, all across the enterprise — see DevOps as something inherently technical, sometimes they miss how important it is to get executive sponsorship, first.
Say you want to start doing continuous delivery as part of your project, because you know this will help with POs complaining your team does not seem to be able to keep up with the competition: if they can have a new feature up and running in just a few days, how come it takes you weeks?
So, you work overtime, write a pipeline and tell your team to start doing Trunk Based Development. Presto! You now can deploy new features in production not in days, but hours.
- Your scrum master still wants stories broken down in a way that does not grant you the granularity you would need to deploy features independently from one another, and won’t agree to releases, except at the end of each sprint;
- When you go to your PO for help, they won’t hear any of it: delivering is your problem, they are already busy enough seeing where the competition is at, doing research to understand what the clients want, or simply coming up with as many ideas as they can, that they need you to finish by yesterday;
(To all the product owners out there: sorry. You know I am not talking about you, right? But we all know that guy, don’t we?)
- Management can’t or won’t approve overtime, and your team can’t or won’t work out of regular hours to learn everything they need to run the new process, so you are seeing a sudden decline in performance;
- Stuff is breaking up in production a lot more often than it used to.
You can’t mobilize people for change, if they think the change in question is out of their scope of work, and it can be hard to explain why DevOps is their problem, too. Your best bet is to seek executive sponsorship.
If you can get people above you to understand the value of what you are trying to do (and if you can’t, maybe you should consider whether there is actually value in it), you won’t have to do the same to people around you, with their own problems and concerns. It’s not their job to see the benefits you see, but yours to create a context in which transformation can happen and each person receives the conditions they need to be able to commit to delivering what is expected of them.
And last but not least…
Assume “adopting DevOps” is a one-shot initiative
Just like you would never assume your product has reached perfection and will never need another feature added to it, never assume your DevOps efforts have reached their peak, and are finished.
If I was given the opportunity of having a whole department to stop to “work on setting DevOps up” for a fixed period of time, then go back to their business, I would never take it, not in a million years. I would probably suggest to direct 10% of the available resources to it, but for an indefinite amount of time. It this was not approved, I would bring it down to 5%, and try again, then work on building success cases that might justify increasing the investment.
Just like everything else, we are talking about a learning process. At first, things will go wrong, problems will be misunderstood and underestimated. Things you never thought of will appear out of nowhere. It’s not like shooting an arrow at a known target; it’s more like driving to a destination based on spoken instructions (ok, maybe written down on a piece of paper), without a map (or, these days, without GPS) and without knowing the road conditions (we are probably talking about an apocalypse-like scenario here).
So, you want a long-lived commitment on something everyone agrees, and understands, will not be perfect at first, but will improve, given time.
I dare say that, if you do not make any of the mistakes above, no matter how you are approaching the adoption of DevOps at your enterprise, you have a reasonable chance of success. Each context will be different and demand a different approach, but these errors seem to be pervasive.
To join our community Slack 🗣️ and read our weekly Faun topics 🗞️, click here⬇