Google Ventures Design Sprint Considered Harmful

A few weeks back Instabridge participated in the online Google Ventures Design Sprint Week. I’m always eager to improve our ways of working so I was looking forward to trying it out. But at the end of the week I came out unimpressed.

Kids: this is an actual book, you know, like made of dead trees.

A quick summary on the GV Sprint Week if you haven’t read the book: you lock a team of designers, developers, managers or whoever is relevant to what you’re doing in a room for a week and work together on hashing out a prototype for a problem you’re trying to solve. The week starts with the team figuring out the problem space, continues to let the team be creative around solutions and eventually test a prototype on a few test users. On paper it all sounded like a sound process.

Our prototype wireframe

So why didn’t it work for us? Let’s start with the good parts:

1. Part of the process is bringing in “experts” from your company or external experts, to verify you’re solving an actual problem and early on identify problems you might not have foreseen. The process around this is good and time efficient: the team schedules interviews with a list of experts and then helps each other keep notes on important issues that pop up.
2. Part of the interviews is the “how might we”-part where the team formulates problems as questions. Eg an expert you are interviewing might say that users will not understand the value of your app. Then the team turns that into the question “How might we ensure users understand the value of our app?”. This was a useful technique that fosters positive thinking and helps turn difficult problems into interesting challenges.
3. A third part is looking at competing solutions or just design patterns from any other product that might be useful. This was very helpful. Everyone has seen something they like and good to bring that all up on the table for discussion.
4. The week was a team building exercise. Six of us spent a week together working on one particular project from 9–5. It creates buy-in and empathy for everyone’s ideas and expertise.

It’s rumoured that 3M’s stock price is directly linked to the number of design sprints being run

But it only leads to one prototype

What I didn’t like is that the process only leads to one user tested prototype with no room for iteration. In my mind, this is the old way of doing product work.

At Instabridge we pride ourselves in being data driven and favour quick short sprints over long deliberations — nothing beats having real users test your product. To solve a particular problem we prefer creating a few different concepts and then iterate on them based on user tests, analytics and prototypes. Sprint Week has no data driven process and the end result is only judged on user testing. I think this is a huge issue.

User testing is notoriously unreliable, regardless of how life like you make the user tests (we built up a fake café in our office to mimic a real life situation where our prototype would be used). The only thing you can trust with 100% certainty in user tests, at least when it comes to apps or web sites, is problems people have with the UI (eg do people understand where they can swipe or push a button).

The second reason why verifying your prototype by just user tests is dangerous is because it tells such a small part of the whole story. As quickly as possible you need to get an MVP out to actual real live users, A/B-tested against competing solutions and instrumented with proper analytics.

My biggest gripe though was with its biggest advantage: while having six people in a room was great for team building it also consumed a lot of resources. Six people working together for a full week on just one prototype is too much resources for too little output. And it doesn’t really make sense — why do you need six people interviewing experts or creating prototypes? That sounds like the job of a UX engineer or a product manager to me. And if your PM is not doing that today you have another deeper problem that a design sprint won’t solve.

Don’t get me wrong — it helps being a whole team interviewing experts instead of just one or two but that time and energy can be spent so much more efficiently.

Results from a school using a GV Design Sprint to come up with their new name

At the end of the week I was left feeling like Sprint Week comes off as a way to “hack” the product process. Despite it being a fun experiment, I prefer a data driven product process any day of the week.