Remote design sprinting at Stack Overflow

My hobby is coaching. I spend my evenings coaching youth and high school athletes in Track & Field & Cross Country.

I approach much of my design thinking in a similar manner as my coaching; prescribe appropriate frameworks to elicit a response, always look for opportunities of improvement, and be ready to adapt the toolkit to novel problems.

At Stack Overflow, I was tapped to lead our product team using design sprints to explore broad new concepts. We are specifically interested in helping developers grow by providing new ways for them to participate and connect on our platform.

In this lengthy article, I will talk about how we used design techniques to explore problems and test solutions quickly, and how I adapted a framework to creatively answer critical business questions facing our team.

The Challenge

Our team is remote. We have several different product segments and a very passionate and opinionated meta community. We also have many different stakeholders with strong opinions of how we can expand our services to developers and help them find better careers. Our product is tested, proven, and well known. So why upset that balance?

Our product team wanted to swing for the fences. We had tried several different design techniques in the past to move projects along ~ from whole product games to vision quests. While these proved useful to generate a lot of ideas rapidly, they did not meet our needs when it came to validating key ideas or aligning the team on solutions.

Using Google’s Design Sprint Kit as a backbone, we adapted their workflows to our unique remote circumstances.

Meetings that are supposed to be in person, now had to be remote. This required using tools in different ways.

Tools we used: Trello, Figma, Google Documents, Hangouts, Respondent Screenflow, Github Pages, Stacks

We are also spread across the U.S. and all have different working hours.

Locations & Time Zones: Des (California), Max (New York City), Courtny (Indiana), Piper (Ohio), Kirti (New York City), Elaine (New York City)

The Sprint Plan

We had five days to work on our design sprint. This was an intentional and necessary time box. We wanted our team’s full attention and efforts focused to achieve the best outcome, while delivering maximum value.

Time is precious, so we were stingy during the sprint. We were even more aggressive and squished down the traditional 6 hour sessions into a 4 hour time block each day. We found that being remote and async allowed individuals to work individually within the rest of their workday as they deemed fit.

The initial plan consisted of 5 days. Each day was broken down into a “theme”.

Our schedule roughly looked like this:

• Day 1: Create a shared understanding
• Day 2: Brainstorm & sketch all the thingz
• Day 3: Decide what to build
• Day 4: Build the damn thing
• Day 5: Test it and see what people think

There are also different roles we filled during the sprint. It’s key to have a Sprint Lead, Decider, and a Technical expert. It was a bonus that we were able to have Marketing, Design, and User Research also involved.

• Sprint Lead: The sprint lead facilitates and runs all of the sessions. The lead prepares all content, guides the team through exercises, and keeps everything on track and organized. In our situation, I served as the sprint lead.
• Decider: The decider role is very important. This person makes the final decisions on product direction and ideas. They need to listen to their team and be able to build consensus among team members. Aligning folks on shared vision is their goal during the sprint. We had one of our Talent Product Manager, Desiree Darilek serve as the decider during the sprint.
• Tech Expert(s): These people advise on technical matters, but also assist with building the final prototype. They are key members of all design activities. Max Horstmann & Kirti Thorat, developers on the Talent product served in this role.
• Marketing Expert(s): Provided guidance on product positioning. Elaine Wang our Product Marketing Manager filled this spot.
• Design Expert(s): Gave guidance on user experience considerations, as well as helped with final designs for the prototype. Piper Lawson and I provided support here.
• User Research: Our resident research specialist, Kristina Lustig assisted us with all the planning for our user research sessions and facilitated the discussions with users on the final testing day.

The ideal team here is no more than 6–8 people. Too many people can stagnate decisions. The sprint’s goal is to reach decisions quickly, prototype those decisions, and test them.

Day 1: Shared Understanding

Before we started our sprint, we needed to lay down ground rules for all involved and what the expectations were. We also needed to pick the brains of the subject matter experts we have in our company, and reflect on past research and activities.

Set a long-term goal

“Why are we doing this project? Where do we want to be in 1 month, 6 months, or a year from now?”

We spent time as a team discussing and understanding each other’s perspectives. It’s very important for each person to actively listen to one another. We designated a note taker so we could reflect on the discussion.

Here’s an example:

Our Ideal outcome — a strong direction for our product. How do we “10X” our offering to developers?

Problem — Can we provide more value back to our developers when it comes to engaging with companies, jobs, each other? Can developers enrich their careers and opportunities by using Stack Overflow? Will Stack Overflow be able to deliver on that vision?

Sprint questions

Once our team had a shared understanding of each individuals perspectives, and we aligned on ideals and problems, we moved into identifying what questions we wanted to answer during the sprint.

Sprint questions are also a time to be honest as a team and identify what hurdles will be in your path as you try to innovate.

In our case, internal pressures and the desire to be conservative with our core product offering was identified as the biggest challenge to any new ideas we presented.

The core questions your team should be asking:

• What questions do we need to answer in this sprint?

• To meet our long-term goal, what has to be true?

• Imagine we travel into the future and our project failed. What could have caused that?

Idea review

We had a lot of previous research, archived design sessions, and insights to reflect on. We also have had failed projects from the past, and we want to learn from those mistakes.

Leaning on this previous knowledge gave us a head start on our solution, as well as helped us make more educated assumptions.

We were able to identify common trends and themes reappearing across all of these different pieces we had. Retroactively reviewing these findings, design documents, and conversations sped up our process and allowed old ideas to be brought back into the spotlight.

Map your solutions

Once the team identified the goals and failures, it was time to start thinking about possible solutions.

For most people I find that they already have a solution pre-baked into their head. It’s usually attached to a specific feature, or core feature but it is rarely completely detailed. Much less an end-to-end story.

By providing your team with a structure you can begin to scaffold more detail.

Our team chose to take a “story based” approach. Its a simple concept that most anyone can begin to break their solution down with.

What’s the beginning, middle, and end of your story that solves our long term goal?

Each individual took some time to prepare a short story. Then we presented to one another. The design sprint facilitator pulled together common themes and highlighted particularly strong ideas. The team worked together to build the narrative.

Stakeholder Interviews

Speaking with stakeholders about this shared story was the next step.

This was less of a presentation of the team’s work, and more about our team asking questions that filled in the gaps of the story. These people helped our team identify pitfalls, reflect on old initiatives, and uncover additional opportunities.

When doing this session, here are a few tips.

Take the time to be opinionated about the product direction. Assume brand new features exist that help tell your story. As an example, you might ask an ads stakeholder: “In what ways do you think companies would want to promote their content [we do not have this today]? Do we feel like this would be intrusive to the user’s experience [meta may dislike it]? Why and why not?[help the Stackholder consider both sides]”

Asking hypothetical questions allows you to temper new ideas rapidly with these key people, and also gets their minds working to address new (and sometimes potentially disruptive) ideas.

For our company, these are the team members and product areas we had represented:

• Puneet Mulchandani & Sean Bave— Talent strategy

• Noah Meiri & Yi Shan — Ads

• Joe Friend — DAG context, what’s the mindset

• JNat — Dev community, Cogro/Meta mindset

• Donna Choi— Dev community, user motivations (Public Q&A, Teams, Jobs)

Homework: Product Teardowns

At the conclusion of the day, we broke down as individuals to work quietly. We asked each individual to find products and services that could serve as an inspiration for this design sprint. Each person was to prepare to share their homework on Day 2.

Day 2: Brainstorming & Sketching

On day two the team members were focused on creating a design solution sketch by the end of the day. This is basically a visual representation of the core features that will bring our story outlined in Day 1 to life. Day 2 was composed of activities that helped focus the team on rapidly generating a lot of ideas and exploring them.

Review & Refine Story

We started off the day reviewing the long term goal, sprint questions, solutions map, and all the expert interviews. Was there anything that we should change in our story based on those sessions?

Lightning Demos

For the lightning demos, we compiled a list of all the products and services people brought to show and tell. We had each individual give a 3-minute tour of the product or service, and highlight exactly what features are standout to them.

For these lightning demos, each team member presented on Google Hangouts, while a note taker compiled all of the interesting features and concepts each person covered.

Sketch

The sketching activities took up the majority of the day. Here we chose to focus on a couple of different sketch activities, although there are several methods for your team to try and use. We chose to use “How Might We’s”, “Crazy 8's” and then move onto the Design Solution Sketches.

Beginning with How Might We’s allowed us to take the problems we would face and pose them as opportunities for design. This is a common design activity that guides people to believe the answer is out there, may or may not work, and reminds people that the sprint is about teamwork and riffing on other’s ideas. The team is working to create opportunities versus narrowing down solutions too early.

Our team came up with a couple of broad “How Might We” statements, and then began providing their individual solutions for each.

How Might We examples:

• How might we curate content?
• How might we introduce new content types?
• How might we guide developers careers?
• How might we connect the developers and employers?

How Might We’s organized on a Trello board

This activity spawned many ideas, but even more rich conversations and exploration about how we may execute. This was precisely what we were going for! Seeing everyone’s HMW’s opened the field for discussion. We then started organizing each of the HMW’s into broader category buckets so we could see some of the common ideas our team shared.

In fact, this exercise took so much time we did not get the chance to dive into Crazy 8’s sketching. This wasn’t necessarily a bad thing. The Crazy 8’s exercise is in place to challenge team members to rapidly sketch 8 distinct ideas on paper. This wasn’t mission critical since our team was aligning behind some of the shared ideas being presented during the HMW exercise.

One comment here as the facilitator is: don’t shut down good, productive discussion. Our team was really on a roll creating and detailing some of the HMW themes. So I called an audible and let us go an hour longer on that segment to make sure we captured everyone’s thoughts and insights.

We ended the How Might We session by allowing each team member to vote on their favorite HMW cards on the Trello board. Each individual only got three votes, and with nearly 60+ different ideas this was difficult. However, it forced each individual to focus on the core story they wanted to tell and that they felt emphasized our goals as a team.

We began to see alignment.

We then closed off the day creating our Solution Sketches. This activity focused on defining the key visuals of product ideas in action. The team members would create a sketch focused on the major keyframes that would need to exist for their solution to be brought to life. Once their sketches were completed, they emailed them to me where they were compiled into a single Figma document.

Some things to keep in mind while doing a solution sketch:

• Make it self-explanatory. It needs to explain itself without you presenting it. If people can’t understand it on it’s own, then it probably won’t make sense when it’s polished either.
• Keep it anonymous. Don’t put your name on the sketch. We want to evaluate ideas — not people.
• Ugly is fine. The sketch doesn’t have to be fancy and polished. It just needs to have enough detail, be thoughtful, and complete.
• Words matter. Don’t use “text will go here” headlines. Think about the copy you would use. Strong writing is an important product element.
• Give it a catchy title. The title helps us keep track of the different solutions. It’s also a way of drawing attention to your big idea.

Bonus: Lightning Research Interviews

We had the luxury of testing these solution sketches, and receive some early user reactions to them. This was a luxury because it provided some great high level feedback and insights moving into Day 3 where we decide on a final design to prototype. This was done completely remote and reminded me a lot of testing paper prototypes with users in person.

Day 3: Deciding

Decision day is the most critical to the sprint. This is the day your team will decide on what to ultimately build and test in order to answer your sprint questions.

Before our team dug in, we spent time at the beginning of the day reviewing work to date. This was a good time to collect comments and feedback on the brainstorm and sketching activities you did previously so we could improve that process in the future.

Heat Mapping the Design Solutions

Our first activity of the day was reviewing the design solutions each team member created. This review session was coupled with a heat mapping exercise and speed critique session.

All of the sketches were placed into one Figma document that was shared with all team members. Team members design solutions were kept anonymous from one another. Each team member was asked to do the following heat mapping exercise:

• Don’t talk.
• Look at a solution sketch.
• Put dots besides parts that you like, commenting with a simple “+1”
• If you really like an idea, put 2 or 3 “+1”.
• Comment on parts you really like.
• If you have a concern or question, comment on that part.
• Move on to the next sketch, and repeat.

Design solutions placed into a Figma document allowed for multiplayer polling, commenting, and discussion.

Speed Critiquing the Design Solutions

Once the heat map was completed, we moved onto the speed critique.

The overall goal of this session was to create a record of promising ideas for the prototype. We didn’t need to debate if they should be included, we just presented each idea. This was a purposefully focused time period (30 minutes) and it helped to have a dedicated note taker, allowing the sprint leader to narrate the solutions to the team.

For each design solution we did the following:

• Set a timer for three minutes.
• The sprint lead narrated the sketch.
• The sprint lead called out standout ideas that have +1’s around a theme.
• The team noted any ideas the sprint lead missed.
• The note taker wrote down the standout ideas and gave each idea a simple name, such as “Animated Video” or “One-step Signup”.
• Review concerns and suggestions that were noted.
• The solution creator remained silent until the end. (“Creator, reveal your identity and tell us what we missed!”)
• The solution creator revealed any missed ideas that the team failed to spot, and answered any questions.
• The user researcher shared the feedback provided by users during the lightning interviews
• Move on to the next design solution, repeat.

Straw Poll

Once we finished critique, we moved into aligning concepts. This activity focuses on getting folks on the same page.

We reviewed our long term goal as well as sprint questions. We were looking to identify the big, risky ideas that would have potential.

Each team member was asked to write down their favorite ideas shown during the critique. This could be an entire solution, or pieces from different solutions stitched together.

Once everyone was done, we went around to each individual and revealed their choices and they explained their votes.

Ultimately, it is the Decider who determines what ideas the team will explore in the prototype. This is all based on input from the team, but the decider is there to solve any disputes and get the team aligned on the solutions to be explored.

Des (our Decider) helped us identify what we wanted to focus on as a team:

• Introduce new content types for exploration

• Figure out how we can curate content for users.

• Enhancements to company pages

Prototype Storyboard

Our team now had solid direction for the prototype. The task now became breaking these high level concepts down into executable steps. This is the time we began to figure out exactly what the prototype would look like.

Here’s what we did:

1. Decide who the “storyboard artist” will be. This is the person(s) responsible for writing, note taking, and sketching ideas the team comes up with. We used a combination of Google Document + Figma to assemble the storyboard and any accompanying sketches.
2. Create up to 10 storyboard frames. You can always use less, but max it out at ten. This will help to figure out how a user will experience the prototype, while also keeping the scope tight.

Imagine an opening scene for the prototype. What would be the natural starting point for users? An example opening scene: “User clicks through a new post on Hacker News titled Stack Overflow introduces new community features”.

It was important to keep the following in mind:

• Work with what we have. Resist inventing new ideas. Work with the ideas we have already come up with.
• Don’t write copy together. Either use headlines from the sketches or defer copywriting until tomorrow.
• Include just enough detail. The storyboard should include just enough detail that nobody asks “what happens next?” or “what goes here?”, but don’t get too specific. It doesn’t need to be perfect today. We can decide this tomorrow.
• The Decider decides. Ultimately the Decider makes all final decisions. If we reach an impasse, we defer to the Decider and move on.
• When in doubt, take risks. The sprint is for testing risky ideas with huge payoffs. Skip the easy wins for bold bets. This isn’t the time to be conservative.
• Keep it to 15 minutes or less. This will ensure we don’t overbuild and it will help us make sure we leave enough time for customers to think aloud and answer questions.

Storyboarding involved hand sketches, Google Documents, and Invision boards for collecting inspiration.

Day 4: Build the Prototype

Now the team moves onto the build phase. It is here that the designers and developers involved in the sprint work together to breathe life into the sketches and documents created by the team. Their efforts provide the tangible vision to test with users.

We started off the morning with a quick check-in; assigning roles and breaking down tasks.

Our team decided to build the prototype using our user interface library, Stacks and depended on Github Pages to serve the content. Using Github allowed the designers and developers to collaborate on the project as it was being built.

Day 5: User Testing

Day 5 is when our team placed our vision in front of users.

Our prototype was ready and publicly accessible.

We were very interested in seeing their reactions to the project we had completed. The insights we gained from these sessions would inform the net big steps for our product.

Given the developer focus of Stack Overflow, we recruited seven developers of varying experience levels and familiarity with Stack Overflow to test with. Kristina Lustig ran our research sessions and provided a script to walk each of our users through.

Sprint team members were encouraged to sit in on the sessions, and were allowed to hear the user’s feedback and see the reactions to the prototype.

Wrapping it all up: Findings

Following the research sessions, our team walked away with several findings that we could use. While we weren’t provided a definitive strategy, we did spot major recurring themes that will inform our product trajectory.

As the sprint lead, it was my responsibility to wrap up insights from the research into a nice package.

Closing

Leading a design sprint is intense. It requires a lot of preparation and active participation by everyone in the sprint.

This can lead to exhaustion. Providing adequate breaks between sessions was important. Each day we got better at monitoring when to take a break, let the mind rest, and come back fueled up and ready to tackle the next batch of work.

Don’t be afraid to take the traditional design sprint model and tweak it to your teams needs. Adapting the processes with some handy collaboration tools helped us bridge the physical-digital divide.

I am optimistic about Stack Overflow using this model more in our product development process.

It has certainly been a beneficial experience for our product team, and it was fun to go outside the box and think big!