Based on people I’ve met over the last few years across many organizations, I usually hear a similar articulation of the same state. Teams automate to a point (sometimes called “test”, “pre-prod”, “integration”, “staging”, etc…). I almost never hear about full automation to production. When pressed, it typically comes down to teams being scared about what would happen if they make a mistake. This is a ramification of not having a blameless culture, which you can read some of my thoughts or Etsy’s fantastic writeup.

For teams that do have automation to production, it usually happens as a terminal event at the end of a sprint or they have canary deployments that last days to weeks. If you shipped to production at the end of every two week sprint, that’s a maximum of 26 deploys a year. Subtract out for the winter holidays, summer holidays, and throw in a few failed sprints, you’re down to about 20 deployments a year. In modern services, 20 deploys a year isn’t very much. As a benchmark, Facebook deploys twice per day, every day, and that was in 2013. That is at least 730 deploys per year, or 36,500% more deploys than deploying once per two week sprint. If we want to build truly modern software focused on fast iteration and fast learning, we can all do better.

Going to the gym

The metaphor I often use is that shipping software is like going to the gym. If you don’t go to the gym regularly and you’re like most people, once January rolls around every year, we all make a new year’s resolution to go to the gym. But then it comes time to go, and we start putting it off, we start skipping days, we’re too busy. And then you finally go, and because you never go, you do everything. Run, bike, lift every type of weight possible. And then the next day, everything just hurts. And hurts. And hurts. For days. Then next time you should go to the gym, you remember how much it hurt and pretty soon you stop going.

On the other hand, if you go to the gym every day, it’s just what you do. Mondays is running, Tuesdays is arms day, Wednesday is riding the bike, Thursdays is for legs, etc… It’s easy, it’s consistent, and it’s a habit. You don’t hurt the next day. You don’t put it off for days and weeks to avoid going. And once this is just a habit, you might end up looking like the Rock…

How does this apply to software?

If we think about how this applies to shipping software, there’s a strong correlation of the behaviors.

1. Going to the gym is a habit and shipping is just a habit. You have to do it every day (or every commit).
2. When you go every day, you do a little bit at a time. That’s how you get comfortable shipping software all the time is each release is a small change set instead of a massive, scary, painful change that’s going to hurt from trepidation in the weeks before and hurt from reality in the weeks after you ship.
3. Every time you go it gets easier. When you go to the gym every day, it becomes less and less of a big deal because you build muscles, you build cardiovascular. You get in shape. Shipping software is the same thing. Shipping needs to be muscle memory and you only get there by working at it every day and repeating.

How do you get there?

The simplest advice that I’ve seen work is to stop trying to ship at the end of your sprint. When your team is working on all different items and then the team tries to ship together at the end of the sprint, I think of this like going to the gym once every two weeks and trying to run/bike/swim/weights/abs all at the same time. You’re still not getting in shape this way.

Instead, what I coach teams to do is to change what it means to be “done” with a sprint item. Done == running in production. That’s it. So if I want to mark my scrum task as done, I have to have it deployed to production.

It’s a simple rule that drives a number of great behaviors. Every time anyone finishes a task, they have to merge it into main right away and push out a deploy. Now your production deploy is a small change (like using the treadmill on Mondays).

It helps the team identify all the reasons they can’t easily do this. Most teams still have more than zero manual steps in their deploy process. In modern software, machine are faster than people and people represent an opportunity for mistakes. So if you want to deploy hundreds of times a day, you need to take people out of the process. Having a scrum team try to deploy several times a day for various small changes will naturally motivate them to work that friction out of the process.

Next, this also forces teams to build more flexible software. Often, new features will be broken up into many tasks (e.g. front-end, service, data work, notification, mobile, etc…). If I’m the dev on building the service, how can I ship without all the other pieces already up and running? And if I can’t ship, I can’t mark the task as done. So the engineers adapt. They learn to build software that has configuration settings that can be easily changed. They learn how to deploy new versions of an algorithm side by side. They learn how to deploy code paths that try the new approach and gracefully fail back to the previous version. These are all great behaviors that provide a better user experience.

Lastly, everyone loves to ship. Why not start with this simple change so your team can ship all the time?

Love it or hate it, PowerPoint is a cornerstone of the modern corporate world. I know that I spent the first 15 years of my career attempting to master my PowerPoint ninja skills. And now I can put most of those skills on hold since I’ve seen the light of banning PowerPoint as a tool for internal discussions. I still use PowerPoint or Keynote for public presentations and internal all-hands meetings, but I do my best to avoid it whenever I can. When I look back, it seems almost crazy to calculate how much time I feel was wasted sitting in every meeting of every day and they were all orchestrated via PowerPoint.

Bryan Dove on Twitter

Quick article on why writing is critical for managers: Bad Managers Talk, Good Managers Write @idonethis http://blog.idonethis.com/managers-write/ #leadership

My first corporate experience without PowerPoint was when I changed jobs to a culture that didn’t use it at all. And while no one explained to me how to think about the adjustment, I’ve come to some realizations on my own that make it hard for me to ever go back and instead I’ve come to rely almost exclusively on documents. To see a well-written post on the power of writing, follow the link I shared via Twitter a few weeks ago.

To replace the barrage of internal PowerPoint meetings, I now simply ask my team to prepare a document that covers the relevant information about the topics they’d like to discuss. I’m not particular about the document structure so long as it covers the key points. Ultimately, most meetings with presentations are trying to do three things: 1) educate on a topic, 2) make a recommendation, and 3) possibly ask for an endorsement of the recommendation.

My ask is just that any document be distributed two days before the meeting and that everyone is expected to have read the doc in advance. Then, when everyone joins the meeting, we already have context and we focus on the discussion instead of losing time to educate everyone on the recommendation.

I rationalized my personal “Death to PowerPoint” mantra with these 6 points:

1. You‘re already telling a story

When you are creating a presentation, you are planning out your story to tell in your head. “I’ll cover these three bullets, verbally provide details on X and Y, use a datapoint to reinforce Z, and then move to the next slide.” We walk into the meeting, ready to tell our story and take the group through an education, a recommendation, and then ask for an endorsement. Except…

2. You always get interrupted by Slide 2

Never fails. There’s always someone in the meeting that starts firing questions at you before you’ve gotten to that part of your story. You start trying to handle the questions like you’re catching golf balls at a driving range, “But wait, that’s on slide 7!” or “Hold on, I’ll cover that on slide 12!” You’re forced to jump around like a trained animal instead of being able to tell your story the way you intended. Now, you’ll be lucky to get out of the room with an agreement to have another meeting and not have your battleship sunk right in the room.

3. Miss the meeting? You’re Screwed

Another problem with PowerPoint is that all the real information and wisdom is in the presenter’s head. It’s not written down anywhere. Trying to read typical slides after the meeting holds close to zero value. Plus, the number of people I’ve met that actually write their entire talk track into the slides notes I can count on one hand. So if you weren’t at the meeting, there is almost no way for you to know what information was shared, what recommendation was made. The only part you’ll hear about is the decision and then you’re asked to take action on that decision when you don’t have all the context. And that never ends well.

On the other hand… Imagine if you replace the presentation with a document that was shared with the attendees in advance. A few amazing things happen:

4. No More Interruptions

By writing your proposal down in a document and expecting everyone to read it before the meeting, your entire proposal (education, recommendation, ask for endorsement) gets consumed without interruptions. No more jumping to slide 12 when Bob asks a question. No more jumping around like a trained animal. You get to communicate your point of view the way you intended.

5. Everyone Gets Better

A funny thing happens when everyone on your team is communicating by docs instead of PowerPoint. Everyone’s writing gets better. The team’s ability to present and consume information get better. The collective ability to analyze new information improves. By forcing the team to write, read, and analyze, it makes the team better.

6. Can’t Attend; Send Notes

Remember what happens if you miss the meeting with the presentation? You never received the context of the discussion, only the result. But when everyone writes a document, all you need to do is read your email. In fact, when the documents are sent ahead of time, you can send your feedback ahead of the meeting too. You simply reply-all with your questions and now your questions can be answered by email (in writing again) or they can be part of the discussion. Not only do you have the context, you get to contribute to the discussion even if you can’t make the meeting.

I don’t expect PowerPoint to go away anytime soon. As much as I may not want to use it for my internal discussions with my team, it still has its place for facilitating a presentation. Somewhere along the way, presentations became a viable substitute for communicating complex topics and that’s what I wish would go away. Hopefully this sounds interesting enough for at least a few people to try out a different way to run a meeting and write it down instead. There’s not a lot to lose, and there’s a ton to gain. If you try it, please drop a comment back and let me know how it worked out for you.

If you’re interested in a change of scenery and want to come work in an environment like I’ve described in some of my posts (e.g. here and here), my team is always looking for people to join us in our offices across Europe and Asia. Please reach out to me directly (twitter.com/bdove99) or check out our jobs site (skyscanner.net/jobs).

The second part of this article it to convert philosophy into specific instructions and help everyone envision what a “great” operations review looks like. I wish more organizations would talk about the mechanics of how they run their reviews so that we can all learn from each other. How will anyone ever know if they’re doing it “right?” How will anyone ever learn about a better set of mechanics for how to create and improve this critical function of every modern software company?

I’m still learning about our current model in my new company so I can’t share our specific details, but I can share what I believe are the mechanics of a great operations review. This is certainly not the only way, but if I had to start from scratch, these tenets would be my starting point.

The only goal is learning — These meetings are expensive to put on the calendar. Any operations review will involve a number of people spending a meaningful amount of time together on a regular basis. If you add up the hourly wage of everyone involved and multiplied by fifty-two weeks in a year, these meetings cost millions of dollars for even a moderately sized organization. It’s important to reinforce the intended ROI is broad learning throughout the team. If all we cared about were inspection and measurement of each service, we could all sit at our desks and look at dashboards or read post mortem emails when we had time. Dashboard and post-mortems are simply tools to allow us to pursue our goal.

Executive Presence — If attention == importance, then there is no stronger sign of the criticality of this process than having your engineering leadership team in attendance and engaged every week. After seeing this done successfully elsewhere, I now expect the senior leaders of an organization to attend every week and to participate in the discussions. For an engineering team to see their most senior leaders participating and asking a lot of questions has a substantial influence on the culture of the product teams.

Write things down — For learning, writing trumps verbal. Every time.

“Written communication to engineering is superior because it is more consistent across an entire product team, it is more lasting, it raises accountability.” -Ben Horowitz

Verbal is a one-time event, or if it’s repeated, it loses details and focus between each iteration. Written forms can be consumed asynchronously and are far more durable. New person on the team? Onboard them by having them read the most impactful post mortems. They will be learning by their first afternoon on the team and it reinforces the team’s culture and lessons learned. The more people that learn these lessons early, the lower the probability of repeating the same mistake again in the future. This is how you change the future. (More on importance of writing — good read and read the deep links for Bezos here and here, and Horowitz’s classic on product managers here)

Attendance — Send the invite out to your entire engineering org. Anyone who wants to learn can attend. Remote offices? Online meeting, with cameras as appropriate, screen sharing, and clear audio. The only required attendance is that each service team must have at least one representative at each session. If you’re doing it well, you’ll have much more attendance than the bare minimum because people are learning and find it a valuable use of their time.

Cadence — Weekly. Our services run every hour of every day, there is no downtime for customers, and if our customer-focused culture is reinforced by ensuring we are constantly getting better at serving our customers, we need to be consistent about dispersing knowledge throughout the organization.

Action items actually get done — Too often, it’s easy to say you’ll take some follow up action and then you never do it and the broader team has moved onto ten other problems and forgotten about this. You don’t need ultra complex tools, but you do need a simple, public tracking system that flags overdue items in a public way. An example would be to create simple lists that are shared across your company and write a script to email your entire company for any item overdue by 30+ days. Action items are a contracts between the to each other that everyone is agreeing to do their part. When you have a simple, no exceptions rule policy and some healthy public awareness, the strong culture you’ve been building will naturally reinforce getting these action items completed.

Meeting Agenda — This is where it all comes together. I believe there are only three important elements to a successful, culture-changing operations review.

1. Ask the audience to share anything cool from the last week. No rules / guidelines / examples, just whatever anyone thinks is relevant to the audience.
2. Teams present their post-mortems. Up to 15 minutes per team. The sequence is any Pri-1 (top priority) events since the last ops review. If your event was 3+ days ago, you’re expected to have completed your post mortem. If the event was last night, be able to talk it through. Then, for the rest of priorities, teams will nominate the lessons they believe most applicable to the broader organization. Quality of self-selection will improve over time. This step is the single most important step of the process and it has to happen every single week. The most important part is that you go through the same process for big issues and small issues. Hopefully, you’re lucky enough to start with the small issues. When you can build the culture of learning and not blaming on simple things, then when big issues come up, it just feels like the same learning routine instead of an opportunity for blame.
3. Any time left? Dashboard lottery. Find a method to randomly choose a team and ask them to present their dashboard. This is why it’s mandatory each team has at least one person at the meeting. “Present their dashboard” means to walk through it, explain why they’ve chosen these metrics as their most critical, and the audience may see interesting anomalies, like “why is there a strange spike on your requests per second from last Saturday?”

So Now What?

Spend a minute to think about what your operations culture says about you. Are you really showing your customers the utmost respect and earning their trust every day? Remember, without them, you don’t have a business

If you see any ideas that you think could help you from this post, try them out. We’re all in the business of helping each other get better at serving our customers.

If you have other ideas that you think would make this process better, post them here in the comments. I’d love to get feedback that can improve beyond the framework here.

Long story short, I chose to take a role to Amazon Web Services S3 instead of Facebook. However, I walked away from the interview with more than a job offer — I learned a lesson that has really stuck with me. In one of my interviews, the interviewer only had one question:

“True or false, the culture of an engineering organization is dictated by your ops review?”

So I thought about it for a minute, and said “yes, I believe it does.” We then talked about why I thought that, positive and negative experiences I’ve seen along the way, how to build a stronger operations culture, etc… The reason this question stuck with me was that it wasn’t until a few months later, while working at Amazon, that it really sunk in how insightful and accurate the statement was. There are a number of different sources that talk about the Amazon way and their rabid attention to details (e.g. book, NYT article) so I won’t delve into that here, but what I can say is that it was the first time I observed how directly linked the ops review and the engineering culture were. What I want to focus on is some of the various ways this single posit can manifest itself inside any organization.

Why do I believe this statement is true?

1. Earning Customer Trust — Our customers expect that the services they rely on work twenty-four hours a day, seven days a week. There is no shortage of choice, so when a service doesn’t work, customers will find an alternative that they can count on. Customer trust is earned over a lifetime and lost in an instant. By having a rabid focus on operations and ensuring our production services are always functioning and always available, we are earning customer trust inch by inch each and every day. Our customers don’t care about what we are working on next if what we’ve already shipped doesn’t work. We owe it to them to keep our services available and functioning as that is the only way that we have earned the right to deliver them something new. Without that rabid operations focus, we are just waiting to see our customer trust erode.
2. Attention == Importance — Our leaders illustrate the most valuable parts of the business as a reflection of where they invest their attention. When your most senior leaders have never attended an operations review in their life, you can make an assessment about how strongly they value it. If leaders don’t demonstrate what they care about with their attention, why should anyone else care? In contrast, when you have a weekly operations review and your engineering leaders are there every week and participating in every discussion, that sends a different message to the team. That is leading by example.
3. A Blameless Culture — There are a number of articles and posts about how transformative blameless cultures can be. My personal favorite technology example is described in the Etsy article from a few years ago. I won’t dive deeply into creating a blameless culture, but focus on the ramifications of integrating it into your operations review. By running your operations review as an environment of learning and one that is devoid of blame, you reinforce a culture that values learning over everything else. A learning culture recognizes we can’t change the past, but we can change the future. The only path to changing the future is to avoid our past mistakes, and the only way to avoid repeating those mistakes is to learn from them and disseminate that knowledge as far and wide across the organization as possible. Running an operations review that spans your entire organization and focuses on learning from our collective missteps creates a team that will avoid learning the same lesson over and over again and enable us to provide reliable, always available services that our customers have come to expect.
4. Pride of Craftsmanship — As a team, we work hard to build our products. Once we have shared our products with the world, we owe it to our customers and to ourselves to ensure that they continue to function as we intended and provide a fantastic experience. Not investing the necessary time to provide care and feeding for your services in production is like building an intricate landscape at your house and then never watering the plants or mowing the lawn.

Each of these items are reflections of a strong, customer-focused, learning culture. The weekly operations review is the drumbeat that reinforces these cultural tenets, or it becomes the river that erodes them week after week.

Click here to read Part 2 about specifics details of how you could implement an operations review.

Your Ops Review == Your Engineering Culture… Part 1 was originally published in Skyscanner People on Medium, where people are continuing the conversation by highlighting and responding to this story.

When I ask someone about the first time they ever built anything and felt proud of what they’d done, they usually tell me about a time when they were a child. For me, I can remember from when I was about 6 and building a small house-like structure. My tools were Legos. I thought my house was the greatest possible thing I could build, and I was amazed that I could touch this thing in the real world that resembled what was in my head just moments earlier. I sat there, staring at what I’d created, simply smiling to myself. Beaming with a pride of creation I had never felt before that moment. That’s when I fell in love with the idea of building.

Maybe for you it was a different set of tools. You may have expressed your creativity using wooden toy blocks, erector sets, crayons, paint brushes, not to mention the hundreds of others toys from around the world I’ve never heard of. Take a minute to think back to the first time you ever created anything and felt that pride of creation. That feeling is what I want to talk about.

At the end of the day, we work in this industry because we all love to build. It’s what drives us every day and we should all remember how fortunate we are.

If you think about it for a minute, we get paid to come to work every day and do the same thing we used to do as a child: build based on our imagination. Sure, we use different tools and, yes, not every day is a walk in the park, but when you add it all up, this is a great time to be working in a great industry. Unless your team/project/manager sucks and it makes you dread going to work everyday. If that’s the case for you, there’s a link at the bottom of this story to help you out.

If there’s a downside to all of us being builders from early on, it’s that we can get completely caught up and focused on the daily progress towards our goals. Building and delivering features to our users, fixing bugs, prioritizing the backlog, monitoring production, and so on. While all of those things are important, more often than not, we forget to spend time on building the most important part of our success: our team. Building a team is not the recruiter’s job; it’s all of our jobs.

Everyone has a role in constructing our team. This starts from helping to get the word out about how kick ass your company — sparking a potential candidate’s attention — to their first interaction with an engineer already working there during an interview, and then every day during their entire tenure with the company. You are in charge of how amazing your team and your company will be. Why wouldn’t you take charge of that and surround yourself with other people that have the same passion and drive to work together and deliver great products for your customers?

“But I don’t know what to do”

A typical response is that you don’t know how to do this recruitment thing, or you don’t know anyone who’s looking right now, or you aren’t extroverted enough to reach out to someone you barely know, or it feels sleazy to ask a friend for other references.

You have to get this straight in your head — you are trying to do them a favor. You’re not selling them a used car or a bag of beans. You’re trying to help a friend take a step forward in their career, work on an exciting project, and work with a great team. You’ve picked this company for your own career, so I’ll assume you think it’s pretty damn awesome. Why wouldn’t you want to share that awesomeness? By telling others about the problems your team works on and the great company you work for, you are helping others understand what makes your company so unique and fantastic to work for. When other talented engineers hear your enthusiasm, it’s contagious. And maybe they’ll be interested and ask how they can join. Or maybe they’ll tell their three friends that are miserable working at another company down the street. Educating people on what makes your company great is the same as someone else sharing why their boss sucks and they hate their job. There’s nothing wrong with it, and if someone is interested in joining your team, then everyone wins.

You help your new teammate learn and improve their skills and build something new. They get to teach you and your other teammates new skills they’ve picked up along the way. And, these new skills help everyone build better, faster, cooler, new software that delights your customers. And that’s why most of us got started in this industry in the first place, because we love to build and we beam with that pride of creation when we get it into our customers’ hands. The time you invest in bringing others into your team reaps exponential returns. By strengthening your team, customers get a better product and everyone on the team gets better by helping and teaching each other.

Something else to remember — this is YOUR team. Not your manager’s team, not her manager’s team, not the CEO’s team, but YOUR team. It’s your team because you have the ability to help your teammates and they have the ability to return the favor. You have a say in who joins your team, you have a choice in how you welcome new people to the team, and you shape how the team acts simply by being an exemplary practitioner each and every day. Your team collectively makes the choice to succeed as a team, or fail as individuals. Inviting another great engineer to join your team strengthens it and helps both of you succeed.

So Now What?

You work at an awesome company on an awesome team with the opportunity to build amazing products for your customers every single day. Sounds great, doesn’t it? Don’t you think your friends deserve an invitation to join you on this exciting adventure as everyone gets to learn and grow together? And your friends of friends? And that guy that sat next to you at the meetup last week? And that amazing engineer who taught you everything in your first job who’s stuck working in some shitty job? It almost seems selfish not to invite all of them in. So go ahead and do it. Tell your friends about the cool stuff you get to work on everyday. Tell them about the great environment. Tell them about the last time you got stuck on a problem and you just popped a question into the slack channel and a flurry of help came back at you. Tell them about how you have the power to build the team and make it stronger. Most importantly, ask them to join you and work side by side on the journey.

Let’s all go build great teams and incredible products for our customers.

Oh, and if you don’t work on an awesome team at an awesome company building incredible products for your customers, my team is always looking for people to join our team and come to work every day and build. Please reach out to me directly (twitter.com/bdove99) or check out our jobs site (skyscanner.net/jobs).