DevOps Dojos at Extreme Scale
DevOps dojos are an emerging trend in companies large and small looking to create full stack teams capable of releasing software continuously. I had the good fortune to join one of the largest, earliest, and most visible efforts in this space, Target’s DevOps Dojo. Did I mention large? 12 concurrent teams, a dozen product and technical coaches, 24 learning events per week, 150+ whiteboards, and infinite stickers: so, yeah, large.
Over the course of a couple posts, I’ll share my story along with a few tips for creating meaningful dojo experiences at scale.
What do you mean by “dojo?”
Dojos, as a concept, have been around for quite a while now. The word dojo is a Japanese term for any formal training place, most commonly one for martial arts. Classic coder dojos are a proven approach to creating a learning environment based on deliberate practice. Developers find a place to gather, select one or more practices (e.g. test driven development, refactoring, continuous integration), and run through one or more exercises to gain hands on experience.
Exercises are called “katas.” There are many ready-made katas: score a 10-pin bowling game, Conway’s Game of Life, make a program that can’t lose a game of Tic-Tac-Toe, communicate a solution architecture for a given domain, etc. Kata’s present relatively simple problems as a canvas for exploring and leveling up on a technical practice. I continue to use katas in workshops, interviews, and as a way of building and maintaining my programming skills.
Classic dojos and katas are useful, but the Target DevOps Dojo is a different animal. The objective of Target’s style of dojo is to get teams up to speed on continuous delivery in six weeks (12 iterations) and for those teams to carry forward the work after leaving. Target calls this a “dojo challenge.” We were hired to help scale this approach, which I’ll cover in a minute. First, I’ll dissect the anatomy of a dojo challenge at the time of our landing.
A dojo challenge centers on a product team — product manager, coach, and engineers — working together toward a goal of continuous improvement. The challenge bit comes in when we compress iterations to twice weekly, which turns up the intensity of work. During this time the team is supported by one or more expert coaches who bring skill and experience with the tools and practices required to satisfy the challenge. Technical coaches are playing coaches. They are there to pair, ask meaningful questions, provide answers, pull in other specialists when things get esoteric, and ensure skills transfer. Their job is to become redundant.
I think of the Target Dojo’s challenges as more accelerators, but I try not to get too hung up on terminology. The point of DevOps isn’t to get good at Jenkins and Git. The point of DevOps is to create tight feedback loops connecting developer to user. Continuous deployment is a technique, valuable because it speeds up learning.
Pipeline + Product = Better Pipeline
When I landed with Joel Tosi, my partner for the gig, we noticed that most teams in the Dojo were creating CI/CD pipelines. Not that creating a pipeline isn’t a valuable goal in itself, but we suggested framing a team’s time in the dojo around the delivery of a small product impact. That is, we introduced product thinking to their already-in-progress dojo model, a subtle but important twist.
When teams base their CI/CD pipeline development on a real effort, their pipeline gets much more realistic and useful. This tactic prevents over engineering and waste due to speculation. There’s a great side benefit: teams start learning how to balance continuous improvement while shipping value to their customers. In our experience, the former comes at the expense of the latter, especially in the large enterprise.
Moving into practice we introduced DevJam’s collaborative framing technique (video). During framing, we gather a prospective dojo team and ask a series of questions:
- What’s your team name and elevator pitch?
- What delivery and learning goals do you have?
- How do we know we’re meeting our goals in a six week challenge?
- Who cares about value/usefulness/feasibility?
- What skills are required? Can we pull this off?
- What are our standards for workflow, quality, and availability?
- Are you fully committed to the 6-week challenge?
Once down this path, we saw the dojo open up to different kinds of product teams. For example, we ended up working with Target’s Learning & Development team to build training and other other educational resources to support their company-wide agile transformation. It was a rewarding experience: focused by a frame, the close quarters collaboration and quick feedback loops of the dojo’s twice a week iterations yielded a much better product in a shorter amount of time.
150+ Whiteboards & The Power of Place
The Target Dojo itself occupies an entire floor in one of the buildings on their main campus. The space is dedicated, intentionally designed, and cavernous.
I remember the day the whiteboards arrived. All hands on deck! A half dozen of us wheeled the assembled whiteboards across campus, from one building to the next. From a bird’s eye view we must have appeared as ants hot on the trail to the nearest picnic.
The whiteboards served a few purposes. They created walls between many concurrent teams. We’d use a whiteboard to place the frame from the beginning of the challenge in plain view. We’d use a whiteboard to self-organize a minimal process appropriate for the work that particular team was doing. We’d use a whiteboard to sketch and refine designs. All the while remodeling and wallpapering the team room for focus and purpose.
In terms of team success, moving into a new space engenders focus and opens folks to coaching. They are making time in their busy schedules, coming into the dojo with commitment, intentionally. Teams are ready to do work.
There are a few powerful effects in having a dedicated space that should not be underestimated. The sheer scale of the space sends a clear message: we are all in on the dojo approach. Their leadership bought in, engaged, and supported the effort. From my vantage point, this support wasn’t just the key, it was the keystone.
The dojo is a product backed by a solid team.
My favorite thing about this dojo approach is the close collaboration with the internal coaches and support staff at Target. Close and energizing. Energizing and effective. Experiments went into practice at a startup-like speed. This pace is unusual for large corporations like Target. We were given a lot of latitude to ideate and test rapidly, everyone throwing their hypotheses and hard work into the mix. Like all good products, success comes down to the people involved: their skills, their involvement, their collaboration, and their follow through and commitment.
In the next post I’ll talk about the positive effects of thinking of the dojo as a product in its own right. I’ll share a number of hacks that drove demand, throughput, and ROI. I’ll also detail the lightweight pull system we put in place to help scale the dojo up to 12 concurrent teams, along with the changes we put in place to keep our pace sustainable.
Stay tuned and subscribe for updates!