Design Sprint by the Book: Stumbles and Strides
Tophatter’s culture has historically revolved around engineering and data. We heavily skew towards quantitative data and having metrics dictate our decisions. While numerical data is seductive, it gives no insights into people’s real needs, desires, or motivations. While Tophatter excels with the numbers, we needed to find a way to better connect with our end users. We needed both data and rich qualitative insights.
Fortunately, the team at Tophatter is nimble and open to experimentation. At the beginning of Q4, we decided to embark on Google Ventures’ five-day Design Sprint. We got the green-light from the executive team, and were greeted with support and excitement. Launching a Design Sprint was something completely different for Tophatter — it symbolized a shift in paradigm to building new features with a customer-centric approach. The team read Jake Knapp’s book Sprint in three days and the next week, we were heads down running Tophatter’s first Design Sprint.
What was the Design Sprint about?
The Onboarding team faced a big challenge this quarter: how do we get first-time users to understand how Tophatter works, get excited to try it out, and trust us?
For those who haven’t heard of Tophatter before, we’re a discovery shopping marketplace that holds 90 second auctions. If you’re curious about discovery shopping, think about visiting a Bed Bath & Beyond store. Last time I went to Bed Bath & Beyond, I came in with a specific product in mind, but had much more than a quick in-and-out experience. I ended up browsing the store and doing everything from comparing soft blankets to weighing college mugs in my hands. That sense of exploration and discovery is what Tophatter tries to capture.
The first time a user visits Tophatter, they may be confused about auction marketplaces and unfamiliar with this discovery shopping mindset. If they’re familiar with auctions, they may have a different mental model for how they work, or consider us to be one of the less trustworthy penny auction sites. For first time users, there is a lot happening and little guidance.
With this in mind, these are the questions that framed our first Design Sprint:
- How might we show first-time users that we’re different, exciting, and fun?
- How might we elevate our company profile so that people trust us when they have not heard about us before?
- How might we educate new users about discovery shopping and the mechanics of Tophatter without being intrusive?
Heading into an intense week, we had a diverse and dynamic sprint team that consisted of a Product Designer (myself), Product Manager (Christina), VP of User Experience & Design (Ali), Visual Designer (Sheng), User Researcher (Matt), iOS Developer (Mike), Android Developer (Gabriel), and Full-Stack Engineer (Hani).
Our Design Sprint Process
As a first-time Sprint Facilitator, I followed Sprint religiously. My goal was to follow the process verbatim for the first sprint and then, based on the team’s feedback, figure out how we can better customize the process to fit our needs.
No Phones, No Laptops, No Problems
Like most Silicon Valley tech employees, we’re usually glued to our devices. I was concerned that five days without our phones/laptops would be a disruption, however, I found my team was receptive and the lack of distraction helped us to all align and be focused.
Day 1 — Map
Yes, and we’re Disney Princesses
To kick off the sprint, we actually began with a “yes, and” improv exercise that was not mentioned in Sprint. Someone in the room started the first sentence of a story and the person next to them continued the story with “yes and…” By the end of the exercise, we were all fictitiously dressed up as Disney princesses and laughing together. Afterwards, we found that we had gotten into the mindset of supporting and building off each other’s ideas.
Defining the problem
“A brilliant solution to the wrong problem can be worse than no solution at all.” — Don Norman
It’s easy to start defining the solution and much harder to define the problem.
Repeated divergence and convergence is important in determining the right problem to solve and finding the best way to solve it. Sprints strike a great balance of encouraging free exploration but also holding people to a tight schedule.
To keep people on schedule, we used the Time Timer. I had my reservations, but it ended up proving to be instrumental: conceptualizing the time remaining helped us all remain structured, built confidence in the sprint process, and allowed us to keep up with our no devices rule.
Thinking ahead
The goal for the first day is to set the stage for the sprint: plan long term goals, clearly outline the problem, and get alignment on the direction.
We started by thinking about what we want Tophatter to convey: delight, trustworthiness, empowerment, and a sense of community and belonging. We also wanted Tophatter to come across as rewarding and welcoming. With welcoming, we wanted to have a greeting like “Be Our Guest!” from Beauty and the Beast — a connection that surely came from our improv exercise.
Leveraging our brainstorm, our long-term goal evolved to “getting first time users to understand how Tophatter works, be excited to try it out, and trust us.” We circled back to this goal continually throughout the sprint.
With our goal defined, we thought of any setbacks that would prevent us from succeeding. Will we be able to educate first time visitors without deterring bidding and engagement? Will we be able to build an onboarding experience that’s modular and flexible? Can we actually convince people who would normally drop-off to stay?
Drawing from our goal and potential setbacks, we created a customer journey map that showed first-time visitors on the left and the completed goal of paid item on the right. The flowchart in between shows how first-time visitors interact with Tophatter.
When in doubt, ask around
To get a holistic understanding, we had an “Ask the Experts” section where we talked to ten employees, ranging from Head of Buyer Support to VP of Engineering. We thought of ourselves as makeshift journalists aiming to uncover the root of the problem from different perspectives.
While the experts were talking, we listened and jotted ideas on sticky notes in the “How Might We” format that turns problems into opportunities. We found that writing on post-its with black marker was ideal because it forced us to be efficient and concise.
We organized and voted on what “How Might We’s” we wanted to solve for and moved these notes over to the relevant section of the map. From there, we were able to see a heat map of the main problems. We decided to focus on the mobile experience to improve conversion rate from install to pay rate.
To make sure all stakeholders felt engaged, we brought in Tophatter’s co-founders, Ashvin and Chris, to review the map/target and provide any feedback or insights. Once we got the green-light, we were ready to start ideating.
Day 2 — Sketch
Working alone, together
Knapp pitches that the schedule should be Monday to Friday and warns of loss of continuity if a weekend disrupts a sprint. However, due to scheduling reasons, our sprint ended up being Friday to Thursday.
We found that setting up the problem on Friday, having a weekend to rest, and coming back fresh to brainstorm on Monday was an ideal workflow for us.
Learn from similar problems in different industries
When we came back from the weekend on Monday morning, we started with competitive benchmarking. We reviewed the current Tophatter onboarding flow and key takeaways from eCommerce competitors. However, it’s limiting to look only at competitors. In fact, according to Sprint, the best ideas come from similar problems in different industries. As a result, we put energy into reviewing design thought leaders as well as taking an in-depth look at how the gaming industry does onboarding since our audiences overlap.
Diverge on a solution
After we converged on a target, we started to ideate and diverge on the solution. Sprint emphasizes individual brainstorming and the benefits of not falling into groupthink. We individually did the four step sketch where we took notes, jotted ideas, did rapid iterations of our idea in the “crazy 8” format, and then finally, spent one to two hours creating a solution sketch.
Many aspects of our solution sketches converged since we pulled insights from the competitive benchmarking and experts notes (same pool of knowledge). However, because we worked on our own, everyone had their own unique approach to the solution.
Day 3 — Decide
We decided and we rumbled
We taped our solution sketches on the wall and displayed them like an art gallery. We reviewed the sketches individually and voted on the standout ideas from each sketch with small sticky dots, creating a heat map of great ideas. The decision on what solutions to build ultimately came to the “Deciders,” in this case the Product Manager and VP of User Experience & Design.
The winning sketches were two competing ideas. We ambitiously decided to “rumble” and create two prototypes for our first sprint. Tying back to our long-term goal, we built our prototypes to help users understand how Tophatter works, be excited to try it out, and trust us.
We titled our two prototypes Snag and Spree. Snag had a subtle design and tried to ease anxiety by only showing upcoming auctions. The goal was to give new users time to browse and teach them to set reminders on products that they’re interested in bidding on in auction. Spree took more elements from the gaming industry and had a personable interactive tutorial that explained how to navigate Tophatter while building trust and credibility. The highlights for each are that we delayed registration until after checkout, asked for push notifications in context, and tried to instill trust by educating how Tophatter works.
Day 4 — Prototype
Divide and conquer
In crunch time, we only had time to develop the two sketches that were chosen without any changes. We unfortunately did not have time to discuss the key takeaways from our other solutions and iterate and improve on the winning solutions.
In the future, we will try to carve out more time for a group discussion during the storyboard stage.
We decided to build our prototypes on Sketch and use Keynote for animation. While there are many robust prototyping tools, we chose Keynote since it had the ease and functionality we needed to be able to quickly build a realistic facade.
When building the two prototypes, we did a good job dividing roles and responsibilities. We had Designers/Engineers working on the mock-ups and animation, Product Manager writing the content, User Researcher creating the interview script, and Engineers setting up the video for user testing.
The main challenge was balancing the sequentialism and staggering timeline of different roles. For example, people dedicated to animation could not start working until the design mock-ups were ready. Bottlenecks like these in the flow could be remedied in future sprints.
However, we got it all done and built not one, but two realistic prototypes in the span of 6 hours.
Day 5 — User Testing
Challenge assumptions — don’t design within a vacuum
We had six people from our target demographic come to our office on Thursday for user testing, whom we recruited on Craigslist and qualified through a survey.
There is no replacement for direct observation and interaction with people using our product. Unfortunately our nine-person team can’t crowd in a room and see what’s happening without making the interviewee uncomfortable. So as Sprint recommends, we set up a fake research lab. We had two cameras in the interview room that displayed in our sprint room: one that showed the interviewee’s hand interacting with the phone and one that showed their face and the interviewer.
We sat as a team and watched every interview. We took notes on post-its on how people interacted with our prototype. We aggregated our mass of sticky notes at the end and found key themes and patterns. We realized we made a lot of assumptions about what is useful and delightful that were challenged when we saw actual interactions.
It was the first time many of our engineers had seen live user testing. Observing in real-time gave us encouragement and made what we’re building seem much more tangible and valuable.
What did we learn?
Knapp describes two types of outcomes from Design Sprints: efficient failures and flawed successes. An efficient failure is when the solution does not work but because the solution failed fast, the team is able to save months of engineering work and extraordinary cost. Flawed success is when the prototype is heading in the right direction but needs to be further refined.
Like most things in life, our outcomes were not black and white. Both of our prototypes were a combination of flawed successes and efficient failures: the beginning and the end of each solution was strong, but the middle (i.e. the clarity around bidding mechanics) still needed to be improved.
I initially went into the sprint thinking that we would come out with a winning solution wrapped up neatly in a big bow. But we had an ambitious question to solve for and ended up coming out with more questions.
From the sprint, the most value we derived was figuring out what problems to focus on. We were able to look at the problem holistically and get insights that we would not have otherwise gotten. Even insights that did not make their way into the solutions were valuable and changed some of the fundamental ways we approach product development and interactions with other teams.
While we did not walk away with a winning solution, we were able to create a clear roadmap and prioritization of the key problems with onboarding and ultimately, position the Onboarding team for a successful quarter.
What happens next?
In Sprint, the process ends abruptly after user testing, but we found we needed a partial sixth day to regroup and absorb the takeaways and feature ideas for our roadmap. The Friday after our sprint, we had a post-mortem pizza party. It was valuable to have a retrospective and recap what worked and what we could do differently for next time. The celebration aspect was also important — we just spearheaded a new initiative and a different approach for Tophatter!
Using these new insights, we’ve customized our sprint process to fit Tophatter’s needs and are launching our next Design Sprint for the Buyer Experience team on Halloween (in full costume, of course). We’re going to come into the sprint with an answerable question and hope to leave the sprint with targeted solutions and a victorious “Yes!” feeling. We are paving the path forward to be more design and customer-centered and provide a great product that people understand, trust, and enjoy using.
👋🏾 Get to know the people and ideas shaping the products we use every day. Subscribe to Noteworthy — the product & design newsletter written by the Journal team.
