DevOps Transformation & Cognitive Overload — Part 2
Continuing on from part 1, here’s how to make a DevOps transformation successful when so many end in failure
How to avoid a NoOps culture & avoid overload
Many company leaders undoubtedly see the value in DevOps. The industry is trending this way in an effort to not get left behind. This typically comes as a “top-down” mandate for the Engineering department to accomplish as quickly as possible. While this is great to get support from leadership, it needs to not be “led” by leadership. If that is confusing, I will clarify — instead of company leadership dictating an implementation, you will get better results by challenging engineering and tech leads to come together to pragmatically think about the best way to improve value flow by implementing a DevOps framework for the company. These meetings should be the cornerstone for what actually needs to be done and can greatly help your teams for the necessary foundations. DevOps is done slightly differently everywhere — there is not a “silver bullet” that solves all needs.
Many places tend to lean too much on development during this transition, and worse, form patterns leading to a place of NoOps. An operations skill set is a necessary function of any Engineering department, and removing it is the wrong idea. Relying on an already-stretched development team to handle everything the operations team was working before the transition is a recipe for burnout and lower morale.
Be careful as you transform, to make sure the value of all teams is present and accounted for. While it is true that it may take fewer individual contributors to do the work you once had, there is still a need for the entire team, and it would be better to elevate individuals to make sure all the new processes remain seamless and you don’t create skill gaps. We will start by creating a roadmap strategy and how we get from today to the end goals you have.
Product Management support is a prerequisite
Your DevOps transformation is doomed without adequate support from many different sections of the business. Most importantly is product management. As an engineering department, you can usually get away with smaller, internal initiatives. Anything larger will absolutely require support from the product management department. Product Management support and buy into the transformation is one of the key tenets of avoiding failure and cognitive overload on your engineers. Product management is important because they usually control the upstream workflow coming to engineering from the business side. If they refuse to transition to a new model, slow feature requests, or circumvent priority shifts then your transformation will be incredibly hard. With product management in agreement to support a transformation, then it can feel like a catalyst to the entire process. This is especially true at the senior leadership levels, where if it is seen that product and engineering are in agreement then confidence remains high and potentially more funding/resources could be put in place to assist.
“Product Management support and buy-in to the transformation is one of the key tenets of avoiding failure and cognitive overload on your engineers.”
Creating a Plan
Step 1 — Assess where your teams are.
- Where is your Engineering department currently at? If you haven’t yet, take inventory
- Do you have any Continuous Integration / Continuous Delivery already in place?
- Review all manual processes
- Shadow different team members
- Where are the bottlenecks?
- How long does it take a new developer to “ramp-up”?
- How long does it take to deploy from developer machines to production?
- Do you have QA environments, staging, etc? How are they used?
- How often do QA/POs have to review and iterate with development?
These are just a sample of questions you could ask. Use your own if you’d like. The goal is to find and alleviate all the bottlenecks first.
Step 2 — Compile all findings
- Ultimately, you will probably find many things you knew, and many more things you had no visibility into.
- Compile all findings into a list so they can get prioritized.
- Make sure the right representation is in place at the meetings to give adequate sizing for each effort of change.
Step 3 — Initially prioritize ways that elevate engineers
- These should start off simple. For example, free up minutes each day from every person, eliminate or minimize manual work, automate mundane tasks, and integrate systems for more transparency
- Once in the priority of your choosing, begin to execute the high-priority items that require the least effort.
- Done well, you can elevate employees and alleviate cognitive overload from having to context-shift too often. This helps with burnout and morale.
Step 4 — Creating the Roadmap
While the previous steps are underway, leaders of each department in engineering should work together with tech leads and senior engineers to draft a roadmap. Create 0–3 month, 3–12 month plans. The reason for this cadence is to give adequate time for each section to accomplish the work at hand. Here is a template that most companies can accomplish in these timeframes.
In 3 months
- Be able to assess your teams
- Create the findings document
- Draft a roadmap
- Research and choose the tools you would like to bring in
- Formalize one or more DevOps teams
- Implement some smaller low-hanging fruit items (stretch goal)
The next 9 months will begin the more complicated pieces, not only technically, but culturally.
3–12 months
- Put all items from findings into a backlog and prioritize them into sprints
- Implement, or at least begin implementing, a cloud approach.
- Establish a Continuous Integration pipeline
- Ensure development teams start to adhere to standards around committing and pushing code to repositories
- Begin tracking and baselining test coverage
- Decide upon branching strategies for the repositories
- Review an agile process for SDLC, to make sure DevOps matches and makes the most sense for your organization
- Start to use metrics in many parts of the organization to show the value stream
- Help teams start to think in a product team mindset (where it makes sense)
- Integrate your first set of DevOps tools into the pipeline or at least have a proof of concept (POC) demonstration
- Script, automate and document most manual processes
- Treat infrastructure as code.
- Ensure strong communication inside the company, with an awareness that the business is on board with the change completely
- Realize that DevOps > Product development during this period
At the end of 12 months, your goals should be formalized, as a department, and be tracked. You should be seeing significant reductions in “wasted” time, and improvements in team morale — before and after surveys are helpful to highlight these changes. You may begin to see speed to production increased, as well.
Part 3 will cover what to expect after 12 months. The first 12 months are key, because if done right, the transformation will be understood and accepted easier.
Stick to principles for success
Over the years, I have collected a set of many principles I follow — much of the time not even realizing what I was doing. Compiling my thoughts, I have narrowed this list down to 5 tenets.
- Stay pragmatic, do what makes sense for your organization.
- Constantly re-evaluate, pivot, continuously improve.
- Any and all ideas should be heard, always.
- Empower your team to be responsible from end to end.
- Limit meetings, get shit done.
There are many ideas, techniques, buzzwords, fancy tools to implement around the Software Development Life Cycle, in relation to DevOps these days. Do not buy into all of them or even most of them. Do what works well for your company. Every place works slightly different. Every place wants to deploy to production 100 times a day like Netflix or Google, but you are neither of those places. They spent 1000s of days failing and figuring out what works until they figured it out. Review what they did, but find your own flavor and techniques to implement that work for your organization and your team. A circle might fit into a square, but not completely — the same is true for how another company solved their problems when compared to yours.
Pivot, pivot, pivot. If an implementation is “fighting” to get done, step back, evaluate and decide if you should waste any more time. Time-boxing POCs is your best friend; listening to the implementor’s recommendation at the end of the timebox is incredibly valuable. Don’t decide on tools until after a POC. As such, always review existing completed products and processes to see if they can be improved upon (hint: they always can ).
Company offices should be a safe place. Meaning any and all ideas are allowed and heard. Well thought out or not, having ideas is better than not. Encourage your team(s) to want to speak freely all the time, you never know where the conversation will go. Avoid pulling rank or talking over others. Encourage younger, “greener” individuals to speak up.
Team(s) that feel secure will be more inclined to take on more responsibility. So when you work on empowering a development team, they will have more confidence to take on the new work. Empowering in a DevOps practice is about more than responsibility though, it is about truly getting buy in from your development teams so they see and care about the process of building, deploying, and supporting what they have built for all their consumers.
Last but not least, DevOps encompasses an incredible amount of process, tools, culture change, etc. I have seen organizations have a ton of meetings around all of these things and think they are getting something done. Initially, it is ok to have more meetings, but a few weeks/months into the process, plan on cutting back meetings and making sure actual implementations and POCs are making traction. Since so much of the transformation can be abstract to many, it is ideal to begin as early in the process to make as much concrete as possible.
Grassroots change is a necessity
Your DevOps transformation is dead in the water if you don’t have buy-in from the individual contributors. The higher planning sessions are essential, but grassroots support is just as important. While planning these changes, make sure general support begins to spread organically at the lowest levels, and that everyone understands what is at stake and how big the change is. Encourage everyone to explore tools on their own and see what the current ecosystem looks like. I guarantee, everyone will come back with new ideas they didn’t know existed that could help them.
Choose your tools with diligence
There are so many tools to help you on your adventure. In Part 3, I will explore many of these tools in detail, the pros and cons of each, and which sets I recommend. I will outline a number of different approaches and how certain organizations may want to adopt one technology over another.
We will dive deep into what goes into a pipeline, when to adopt microservices, etc.
Remember — Start small, prove the value
Just like with the grassroots approach, some of your initial changes need to be small and incremental. Don’t start more than you can finish. As you begin to work smaller tasks and changes, these will snowball into more time becoming available for more change and more growth. While this is happening, it will also become a reinforcement mechanism for showing enormous value. Use your initial findings in the plan to find that low-hanging fruit and tackle those items first. Your team will be ecstatic to fix all of those workarounds and inefficiencies.
Next steps
This is quite a bit to accomplish but doesn’t have to be overwhelming. Remember, follow the plan, roadmap the milestones, and work toward the shorter objectives after the plan has been created. You will stumble and fail, but ultimately, if you persevere, you will eventually find success. This change is a fundamental part of succeeding as a business today.
In part 3, I will outline tools for CI/CD, and discuss how to create a pipeline and process to get to the next stages of deploying continuously. We will then explore what the future looks like in 2020, and what to expect inside this movement and in the industry as a whole.
Currently, I am a DevOps Enterprise Architect, consulting with the Air Force on implementing a DevOps transformation.
Check us out @ https://centil.io
Like this? Feel free to reach out to me at https://www.linkedin.com/in/dwetteroth/
