Design Sprints: Great, but not as simple as the recipe
A UX Researcher’s review of Sprint: How to Solve Big Problems and Test New Ideas in Just Five Days
Disclaimer: The views expressed are my own and not of my employer or anyone else. I worked with the authors at Google. I participated in some sprints they ran while they were developing the methodology. I was not compensated or asked for this review. In fact, I actually went to a bookstore and bought their book at full price.
It’s been almost almost three years since the release of Sprint: How to Solve Big Problems and Test New Ideas in Just Five Days by Jake Knapp with John Zeratsky & Braden Kowitz. Since then, design sprints have taken off in popularity. Businesses and organizations both large and small have adopted the methodology. People can get certified as sprint masters and some organizations even have teams specifically setup to organize design sprints. I thought it be worth reviewing the original book as a UX researcher who has participated in several design sprints.
WHAT I LIKE ABOUT SPRINTS
Sprints are ultimately about taking action on ideas. This is my favorite thing about sprints. They address the fatal flaw of traditional brainstorming. More often that not, the ideas generated from brainstorms are never acted on. Brainstorming is more an act of idea theater rather than making progress towards anything tangible or actionable.
The two situations where I think design sprints are really good for:
Decision paralysis: These are engrained, long standing problems that teams have tried addressing in the past but failed or haven’t tried since they seem so daunting. Often, these problems have a vast array of potential solutions or options, ranging from simple quick fixes to super complicated overhauls. More times than not, teams simply don’t anything because of the uncertainty of what to do, plus the unknown amount of time, effort, and resources involved.
Blindly throwing darts: Almost the exact opposite of decision paralysis. Teams try to solve a problem by hurriedly thinking up ideas, doing what sounds best or from whoever is the loudest, then finding out it didn’t work. Repeat and fail again and again.
In both cases, it’s a primarily a problem of team focus. As the authors point out, sprints force teams to focus on the most pressing questions, which increases the chances of better results.
WHAT I LIKE ABOUT THE BOOK
Finally, a UX book that actually has good UX. The book is organized by the five day week long sprint schedule (“Monday, “Tuesday”, etc). It’s easy to go back and a get refresh on each step. The authors being designers also helps. There are plenty of drawings and pictures of examples to visually understand the steps and outputs of the sprint process.
THINGS ABOUT DESIGN SPRINTS AND THE BOOK THAT I QUIBBLE WITH:
Design sprints are not a grab and go methodology
Since the book’s publication, people who have conducted design sprints have advocated customizing the sprint process to meet your own team’s or organization’s needs. Essentially, take the parts that work for you and then change or disregard the rest. I would say that is really matters what you choose to keep or disregard. For example, one of the panelists on this CHI 2017 panel, recalled a time when all her entire sprint team changed over in between days. Meaning an entirely new set of people showed for the next day of the sprint. Keeping a consistent team is an example of something you can’t change about the design sprint process and expect it to be successful.
I would recommend trying out the process as detailed in the book at least one time. It’s easier to make changes and adjustments based on some experience first.
It’s a little optimistic
I think the books does overlook the dysfunction, mismanagement, disorganization that are present in many teams, especially those who could really use design sprints. If I were to suggest an additional chapter to a second edition of the book, it would be called “Convincing your team or leadership to get their heads out of their asses and admit there is a problem.” Team dysfunction or disorganization is not a criticism in itself of the design sprint process. But that dysfunction can derail a sprint. For instance, team members stop showing up half-way through or expert input is ignored because it presents hard truths the team is not willing to hear.
“The bigger the challenge, the better the sprint”
The authors boldly claim that “No problem is too large for a sprint.” In their defense, they almost immediately mention how absurd that sounds. And they do walk the claim back later when they recommend scoping the sprint down to one problem for a target group of users. I wouldn’t count on a design sprint to be the savior if you trying to solve nationwide child obesity or de facto school segregation. A design sprint might come up with a solution that might work in with a specific group or in one location. Huge, multifaceted, vastly complex, systematic problems weren’t created overnight. They are likely not going to be solved with a one week design sprint or even a couple of sprints.
I would recommend maybe a smaller problem to tackle first, especially if you are trying out the sprint process for the first time. Design sprints for bigger problems are more likely to successful with more experience and credibility of the sprint process with your team.
Sprints are not free
The authors repeatedly point out that a sprint is just five days. Which is far less time many teams spend spinning their wheels or investing solutions that don’t work. That’s true, but design sprints are not free. Given the five day time frame, the authors suggest doing more sprints in case you can’t cover everything you wanted in one or if the resulting deliverable from a prior sprints ends up not being effective. In many companies and organizations, it is very difficult to get a sustained five day long time commitment from key team members. Even if your team is all on board with a design sprint, the rest of the company or organization may not be. They may expect your team to work on current projects and make deadlines in spite of the sprint.
A poorly executed or unsuccessful design sprint is even more costly. Not only does it fail to produce an impact product or solution wise, it poisons people’s willingness to commit another five days to future sprints.
Deciders can make crappy decisions
The authors really push that sprint teams should have a “Decider.” It’s the person who will ultimately pick which solution, after much critique and input, to move forward with. Typically, it’s a leader like a CEO, product manager, department head, or whoever is empowered to make decisions that will be put into action. If a “Decider” can’t be there for all of the sprint, a representative empowered by the Decider can step in. It makes sense if you want move ideas into action. Any solutions without the Decider’s support are likely not to go anywhere.
In my experience, the Decider role is the biggest departure teams do in sprints compared to what the process laid out in the book. Many teams vote by consensus. It’s the exact opposite of the idea of a Decider.
My biggest problem with the Decider is that it has the potential of empowering poor decision making. The poor decision making is not the obvious kind. I am not talking about a situation where a person has to pick water or gasoline to put out a fire, and they choose the later. It’s more like there are several possible solutions that would help. The Decider then picks one of least impactful because it’s the cheapest, quickest, or the one they thought of. Often teams have to do design sprints to fix problems created by past poor decision making. I have little confidence in a Decider to pick a good solution if they are the same person who made the past poor decisions that created or exacerbated the problem.
One prototype does not rule them all.
The authors strongly suggest that Keynote is a suitable prototyping tool for almost every case. They do point out other tools for hardware or physical prototypes. One thing they overlook is that many designs may require personalization. In fact, personalization can be the key component of the experiences involving recommendations (ex. shopping, music, restaurants, places), social media/networks, personalized plans or routines. In my experience, testing a prototype and asking people to imagine the experience is personalized to them is an ineffective way of evaluation. Some workarounds I have found are:
- If there is enough time to prepare in advance, gather the relevant personal information from your test participants ahead of time. Your designers can then create personalized prototypes for each person. The downside is that prototyping takes longer. Your designers have to create, in this case, five prototypes instead of one. In addition, the test participants may be tipped off about what they might see in the prototype based on the information you ask for beforehand.
- Transpose the participant’s information into the prototype during the session. Before showing the prototype to the participant in the testing session, do a quick interview on the relevant information that will be in the prototype. For example, let’s say your prototype recommends local events or stores with reviews from people they know. Ask the participant to talk about what local events or stores they have visited recently. Then ask what people they know also went to those events or stores. When the participant sees the prototype, you can ask the participant to replace the stores, events, people in the prototype with the specific ones they mentioned earlier or similar ones. Having specific examples to talk about will likely yield better feedback than just simply asking people to imagine the prototype content relates to them.
- Make the prototype content the most accessible to the most people. If you only have time to do one prototype, then make it accessible to a wide range of people. If your prototype has things like movies, stores, restaurants then pick well-known examples (ex. The Avengers, Target, Cheesecake Factory) from a range of categories. Hopefully, your test participants are aware of at least some of the examples. Your test sessions are for people to react, provide feedback, and find problems with your prototype. You don’t want to distract from that by spending too much time explaining obscure or niche example content in the prototype.
MY OVERALL VERDICT
I definitely recommend reading Sprint if you hadn’t had a chance to check it out yet. Also, don’t be afraid to do a design sprint if you feel the situation is right for one. But just remember like all things, it’s never as simple as it is in the book.
