Piloting a GV Design Sprint: Will Sprinting Work for Us?
We had an hour. I had the framework. It wasn’t clear where to start. So we just started.
This probably isn’t the best way to dive into your first GV Design Sprint, but when you’ve never led one before and there’s no history of it at your organization, it’s as good as any.
I’d done my homework — read the Sprint book, been through training with Jake Knapp, researched examples on the web — so I knew how you were supposed to do it. But, like most people out there, I couldn’t simply run it by the book — I had to adapt it to our specific circumstances, people, and evolution along the path toward great product design. That meant compressing the process (one-hour sessions) with people who were remote and adapting on the fly.
In the room on the first day were six people: the product owner, two designers, a product manager, a developer, and me, the player-coach or designer-facilitator. We all work on the product side for an online academic publisher called PLOS (the Public Library of Science). Our team builds the online portals, tools, and platforms for science researchers to submit, review, and publish their work in our seven academic journals. Over our 15-year history PLOS has used these tools to publish more than 200,000 articles and reviewed twice that number.
The reason our team was in that room was to solve a problem for our editors who manage the submissions to our journal. They needed a simpler way to find potential reviewers for the work and formally invite them to review a manuscript. A design sprint seemed like a promising, new process to tackle this problem.
So we started.
As the facilitator, I cherrypicked what I thought were the most important pieces of each phase and led the team through them — in one-hour sessions.
Here’s what we did.
DAY ONE — The Map
Time: 1.5 hours
Because we had already done storymapping with stakeholders, we started the session by generating How Might We’s from the details of the storymapping. Then we grouped them so we could see the themes and heatmapped them.
And then we got stuck.
Why did we generate these HMWs? What do we do with them now? The next step was far from clear. It felt like the HMW exercise might have been a false start that led us into a cul-de-sac.
So what did we do? We started over. I apologized to the group that I wasn’t sure how the HMWs were going to help us and that our best course was to set them aside and start somewhere else. Fortunately, the team understand that this was a pilot effort and understood.
We shifted gears to making the Map — the diagram of the actors, outcomes, and the steps to get there. This proved infinitely more fruitful. The team clearly understood how to diagram these steps and focus in on the most important target. In addition, they were able to diagram a couple sub-maps that helped us see other use cases and know what point on the map would have the most leverage. The best part of making the Map is that it gave us a jumping off point for the next session.
DAY TWO — The Map (Part Deux)
Time: 1.5 hours
Before our second session, I realized two issues. First, while we had drawn the Map, we hadn’t picked the Target as firmly as we should have — it needed more review. Second, three of the five people in this session would be remote.
So I turned to Zoom and MURAL, an online collaborative whiteboard. Signed up for a trial session and gave it a go. It worked like a charm.
I transcribed the post-its from the first day into digital post-its, recreated the map and sub-maps and invited the team to join. We interfaced through Zoom and explored the MURAL on our individual machines. Using the nifty Voting feature on the MURAL board, we heatmapped the target, discussed the votes and picked the winner.
On a separate part of the board, I made a form for the Goals, Questions, and Metrics. After a quick discussion, we were able to identify those as well. This online collaboration was really working. We were ready for Sketching.
Since we had 45 minutes left, we dove in the search for inspirations and Lightning Demos. In 20 minutes, all five of us had plenty of examples to share and we presented 3–4 of our best. As the facilitator, I couldn’t draw the thumbnail for each while they were describing it — all I could do was take bullet notes and ask them for a screenshot later. This worked pretty well — overnight I was able to use the notes and screenshots to create digital thumbnails in MURAL and lay them out so they were easy to review the next day. That is where we had to stop.
DAY THREE —Sketching
Time spent: 1 hour
In this session, our goal was to produce five individual solution sketches. Step one was the Crazy 8s exercise. Step two was “working alone, together.”
Again, we were a remote team for this one. I didn’t think this would work well because people get easily distracted when they’re remote and working silently. But it actually did. We had a defined time (one hour), a defined task (sketch a finished solution), and some nice background music.
By the end of the session, everyone had a sketch of a series of screens to present next time.
DAY FOUR — Decision+Storyboarding
Time spent: 1 hour
It was time for a decision — what would we build for the test?
We Zoomed in again and reviewed the sketches on MURAL. Instead of following the prescribed process, we had each person walk us through their example. Then we heatmapped what we liked.
The next step was supposed to be visual storyboarding where we collectively make use of the sketches we’ve already drawn to map out the prototype. But that didn’t seem realistic to me.
Instead, I asked each person to look at their sketch and list out the high-level steps involved in their process. How would the user use their idea? I wanted to compare these lists and see what the commonalities and differences were.
This turned out to be very useful. It highlighted two things: one, most of us had a very similar idea of how the process would work so there was a lot of overlap in the solutions even though they didn’t look exactly the same. Two, one of us came to the process with a completely different assumption about where to start — something that turned out to be really important later.
The main use of this, though, was to spell out the steps involved in the prototype we needed to build. From these five lists, I was able to synthesize them into a user flow that we could build.
DAY FIVE — Prototyping
Time spent — 11 hours by one person!
This phase was where the sprint veered a bit off course. For a variety of reasons, I took it upon myself to craft the prototype. I imagined I could make a fake app through Keynote, Sketch, or some other simple software I was familiar with. And I imagined it would be easier to do it on my own.
I was wrong.
I spent a couple hours flailing around with Keynote and Sketch and even Justinmind before I knew I had hit a wall. We had a major feature that couldn’t be faked easily with slides or images — we had to create a realistic search. I didn’t realize this at the beginning, but our prototype was founded on the user interacting with data — a lot of data. An image-based prototype was unrealistic.
I swallowed my pride and called in a designer/developer on the team to help. We worked through a number of ideas for connecting a database to a prototype, but finally settled on a duct-taped javascript solution that could be integrated in an HTML prototype.
Yes, an HTML prototype. I know we were supposed to go lightweight and get this done in one day, but there seemed no way around it — we had to use actual code.
Our solution was to find a shopping cart template project on CodePen, a great source for actual functioning code examples. I forked a project (thanks Alex!) and laid the list.js script over the top of it. List.js simply searches the content on the page and returns certain list items. So I had to hardcode the entire database, complete with correct CSS classes, onto the page. This involved generating data from Mockaroo, carefully cleaning and formatting it in a Google Sheet, and pasting the clean HTML in CodePen. A laborious process but, at the end of the day, it worked.
DAY SIX — Testing
Time spent — 2.5 hours over two days
The user tests were exactly what they were meant to be — eye-opening. I won’t go into the litany of details that we gathered from the 30-minute interviews with five people, but I will say that there’s nothing like real users to get you out of your imagination and back to solving the real problem in front of you —and checking any ego you have about your great ideas.
Here’s what we learned from the user tests:
- Choose your testers carefully. Go beyond the surface demographics and really think about the roles these people play in their work. We selected five people from a department in our organization and didn’t really pay attention to their different roles. That bit us in the end — three of the people weren’t our target users. We didn’t realize until it was too late that our target user was a very specific role and a very specific task. We needed five people who faced that exact situation every day.
- Have the interviewer be one step removed from the prototype. I did the interviews and the problem this created was that I knew too much about it — I knew what we wanted users to do and, despite my best efforts, I asked leading questions which revealed more about my thinking and less about their understanding.
- One part that worked really well for us was to have the test machine broadcasting over Zoom and have the rest of the team watching from another room or remotely. They were able to watch the person, see what they did on the screen, hear the conversation, record notes, and even ask the tester questions at the end.
DAY SEVEN — Recap and Reflection
Time spent — 2.5 hours over two days
The recap day, though not listed in the Sprint book, was critical for us. We needed to collectively understand what we had learned about both the prototype and about sprinting in general.
Our original team of six gathered. We also invited a handful of people who will be instrumental at using the insights gained about the challenge and about how design sprinting can help other parts of our organization.
The first step was transcribing all our notes to Trello. On our board, we made lists for each of the prototype screens (splash, list, compare), for general prototype comments, and for general thoughts about the sprint process.
Once we had the notes up there, I opened up the conversation at the beginning: did we answer the original questions we started with?
The answer was a resounding “no” but not for the reason we expected. It was a “no” because by the end of this exercise it was clear these were the wrong questions. For example, our first question was “Will users understand how to compare reviewers?”
It turns out it doesn’t matter — these users weren’t trying to compare reviewers in the way we thought, and some of them weren’t trying to compare reviewers at all. This question completely missed the mark.
Although it was painful to realize it, our prototype had completely missed the mark. We had set out in a particular direction based on a particular set of assumptions which we based on the pre-interviews we had done. These assumptions we wrong. We hadn’t understood what these particular users really needed and didn’t realize that. So we designed something they didn’t need.
Does that make this sprint a total loss? Not at all. In fact, it brought the nature of the problem into a much clearer relief. We now know much more about the particular users and what they are trying to do. We know more about the different roles people have that are involved in this process. We know more about the types of information these people need to do their work. We know much more about how difficult this problem is to solve because its scope is much larger than we thought.
Our product owner summed this up perfectly: “Our description of the problem in our slide deck and story map that we made before the sprint are still valid. It’s just that now I know much more about the problem we need to solve.”
I’ll add that this benefit didn’t just accrue to our product owner; it accrued to all of us on the team. The fact that we had learned all these lessons together was powerful moving forward. Our team’s communication around this issue will be greatly enhanced both in understanding and efficiency. And our solution will be that much better in the end.
The group felt this way as a whole about the sprint:
If this process gives us answers to solving the right problem early on, it saves us sprints and sprints of development work.
If we had done it the normal way, we’d spend so much time on writing user stories and communicating around tickets. This way we save that time by working collaboratively at the same time.
Having people who see different parts of the problem share the same physical space and thinking space makes it so much easier to communicate and work together.
The sprint exceeded my expectations.
At the end of the day, we are looking forward to doing more design sprints. The missteps and time taken away from other tasks were outweighed by the benefits of understanding the problem together. It will take some practice to get the process right for us: deciding which processes work best for us, how much time is needed, how to frame the right size challenge. But it is a way forward in the evolution of our product design process.
FINAL STATS — 19.5 hours over nine days
Mapping — 2.5 hours over two days
Sketching — 1.5 hours over two days
Decision + Storyboard — 1.0 hours
Build Prototype — 11 hours (oops!) over two days
Testing — 2.5hours over two days
Recap — 1.0 hours
Many thanks to Jake Knapp, Xander Pollock, and Gitta Salomon for encouraging me to try out design sprints in our organization!
