Being a Product Design Scientist

A workflow for rapid, hypothesis-driven design cycles


Its no secret I advocate an approach to product design that blends design thinking with a scientific method, as a way to solve specific problems. The idea of using “science” to improve products is not a new one, but its not always well-understood how to best apply these ideas to the various disciplines involved in creating a great product. This can be especially true for designers, where we are used to dealing with concepts that are perhaps more open-ended (most people don’t naturally associate “concrete evidence” with a creative design process).

On a recent project, I worked with a small team to explore various ways we might bring some of these ideals to our own product design process here at Salesforce. The team was quite happy with what we learned, so I thought it’d be useful to share some of those lessons here.

Getting ready

This workflow assumes you already have a strong understanding of your target customer, and the specific problem(s) you’re solving for. This means you actually have them written down somewhere, and you’ve collected sufficient evidence to prove both your persona and problem statements to be true. If you don’t have this, take a few steps back and go do it. In addition, there are a few simple questions to answer in advance that will help you be prepared for the project:

  • What’s the overall goal for the project? What does success look like? How will you know when you’re done?
  • Who’s the team? Do you have all the necessary roles covered? Are those people able to dedicate the necessary time to the project?
  • What are your project constraints? Time? Technical capabilities?
  • What’s the schedule? Mapping out the project across a calendar can help insure you’re tracking towards the deadlines at an appropriate pace, and also watch for parallel tasks that might conflict.
@jina got crafty and put together this awesome calendar for a recent project.
  • What prior work/research has already gone into trying to solve this problem (either within your company, or elsewhere in the industry)?
  • Do you have a project space? We’ve found war rooms to be incredibly effective at facilitating collaboration within a team, but we’re experimenting with other arrangements as well.

Kick it off

Now that you’ve got all your ducks in a row, let’s get this baby moving! But how? I’ve found a Google Ventures-style design sprint to be one of the best ways to kick off a new project. A well-executed design sprint brings some amazing benefits that should carry forward throughout the project:

  • Everyone works together. The particular activities prescribed for a design sprint level the playing field for designers and “non-designers” alike. You should be using this to your advantage to bring in engineers, product managers, researchers, prototypers, and other stakeholders into the product design process. Getting them heavily engaged in the very first week dramatically increases the likelihood they’ll be both willing and able to contribute in a meaningful way going forward.
  • You move quickly. Our goal is to get into a consistent, weekly cadence. However, for a team that might not be used to iterating at this pace, it can be a tall order. Since almost every minute of a design sprint is accounted for and laid out in front of you, it can help build the team’s confidence that such a pace is possible and get you through that first cycle.
  • You hone your understanding of “good enough.” You have a single day to prototype, which never feels like enough, especially for us perfectionist types. However, after seeing just how much you can learn with that embarrassingly rough prototype, you start to understand why spending any more time on it would have been wasteful.
  • You set a precedence for testing with users. As designers, we should never go more than a couple weeks without putting our ideas in contact with real users. The design sprint process says loud and clear, “we test our ideas with users at every step.” You’re really forced to hold that feedback as the most important input into further iteration because time doesn’t allow for anything else.
We put all our work out in the hallway so people walking by could check out our progress.
  • You force a reaction. Design sprints are such a visible, high-energy activity, that they will always attract the attention of those around you and almost force a reaction out of them. Sometimes you have to really put yourself out there to get important people to pay attention and react, and those reactions aren’t always positive. When they are — great! But quite frankly the negative reactions are more valuable. If you’re headed down the wrong path, you’re better off knowing now (we call it “inducing vomiting early”).
  • You have fun. Design sprints are a lot of hard work, but they’re also a lot of fun. Breaking up the intense work periods with delivered lunches, periodic breaks, and a couple open houses (days 2 and 5 are really good for this) sets a great tone for the project and the team.
Getting [other people] heavily engaged in the very first week dramatically increases the likelihood they’ll be both willing and able to contribute in a meaningful way going forward.

Get into a rhythm

Design sprints are great and all, but doing design sprints back-to-back over the duration of a project just isn’t a sustainable pace. You need to get into a rhythm that the team can maintain week after week. A good weekly cadence might look something like this:

  • The prior Friday — Plan for the week.
  • Monday through Wednesday— Build the prototype.
  • Thursday — Test your prototype your target audience.
  • Friday — Synthesize your learnings and do a retrospective on the week.

This still carries forward many of the important principles from the first week, but relaxes things a bit so as not to burn everyone out.

Planning

Proper planning is an important step that will determine whether you end up making the best use of the upcoming week. The most important question you need to answer will be, “What do we want to learn this week?”

Start out by stating your solution hypothesis (you should have some idea of what your prevailing hypothesis is from the design sprint). Ex. By giving sales people the ability to magically transfer their thoughts into their phones after a meeting, they will need to spend less time inputing data into our system after work. What assumptions are you making that must be true for this hypothesis to hold true? Ex. We can build the technology to read people’s minds, and users wouldn’t be freaked out by magical mind-reading capabilities. Which of those assumptions would you consider the riskiest? Which of your assumptions have the best chances of blowing your whole idea up if proven wrong? Your entire week should be focused on proving (or disproving) that assumption.

The most important question you need to answer will be, “What do we want to learn this week?”

Prototyping

I use the term “prototype” here loosely. A prototype can be almost anything. It can be code, but it can also be a series of screenshots, some sketches, or maybe even a survey. It doesn’t necessarily need to look like your solution, or even a piece of it. A good prototype is anything that helps you determine whether your assumption is correct with the least amount of effort.

Designing a good experiment is an art in itself. Focus on testing the assumption, not the product (the difference is slight, but important). Take a stab at whatever you think is best, and carefully evaluate the effectiveness of your prototype choice at the end of the week. As time goes on, you’ll get better at this.

A good prototype is anything that helps you determine whether your assumption is correct with the least amount of effort.

Testing

Like with the design sprint in the first week, its a good idea to pre-book your testing sessions. Even before you have something to test. This serves as an excellent forcing function for the team, and ensures your efforts are focused on the deliverables necessary to support your tests.

Most of our tests were simply 30-60 minute sessions with users, conducted remotely over GoToMeeting. Everyone should attend these. If necessary, setup an observation room separate from the facilitator and invite anyone who has anything to do with the project you’re working on (not just designers). Someone should be taking detailed notes, and everyone else should be writing key insights, questions, and quotes on sticky notes.

Lajja is a research intern that’s been helping us with our user sessions.

Tip: GoToMeeting makes it easy to record your sessions, but its really difficult to go back and dig through hours and hours of video. Write a timestamp in the corner of each sticky note so you can easily go back to that point in the video if you need to get additional context or an exact quote later.

Synthesis

The point of this step is to be able to clearly articulate (for the team, as well as others you might share with) what you learned during the week. This part can be time consuming, but incredibly rewarding when you’re able to walk away with concrete, useful insights.

If everyone did their part and created good sticky notes during the testing, you can start by clustering these notes and calling out themes around the largest clusters. Have a discussion about each of these themes, and add any additional thoughts to each one as they come up. This is the most efficient way we’ve found to distill insights out of a full day of testing.

Pulling out key themes after a day of research. Our research participants were color-coded, making it easy to remember who said what.

Repeat

Go through the whole process again, starting with planning. Figure out what needs to change about your solution hypothesis, given what you now know. Sometimes a new assumption will have become apparent throughout the week, that needs to be tested immediately. If not, go back to your original list and move down to the next riskiest assumption.


This workflow should hopefully give you a foundation for driving your product design process around a series of hypotheses. With each iteration you’ll build more evidence and increase your confidence in your proposed solution. As this happens, the fidelity of your ideas will increase and what you test will become more detailed. Like using a microscope (because after all, we’re scientists), you’ll move from making coarse adjustments on big knobs to finer adjustments on smaller knobs.

We’d love to hear about your experiences implementing this type of workflow within your teams. This entire process is in itself, an experiment. Your own learnings serve as valuable evidence that help us iterate and improve.

Cover photo by Jose Betancur. Creative Commons License.


Follow us at @SalesforceUX.
Want to work with us? Contact
uxcareers@salesforce.com
Check out the
Salesforce Lightning Design System

One clap, two clap, three clap, forty?

By clapping more or less, you can signal to us which stories really stand out.