How To Create A Kick-Ass Product: User Studies Are Cheap And Easy

This is part 3 in a series about how to create kick-ass software products. You should probably read part 1: Use an Inter-disciplinary process.

User Research has this official-sounding tone to it. It probably requires someone with a lot of training in psychology, someone who understands the legalities of using human participants, and… what types of companies help you recruit participants? Actually, user research is not hard and doesn’t have to cost money.

Ok, high quality, professional user research requires years of training, experience, and an unbiased selection of participants. But simple user studies are easy. You don’t need a lot of people. You don’t even need a representative sample of users.

User studies with a few participants and conducted by a novice is still better than having no user feedback.

I mentioned the most common mistake entrepreneurs make in part 2 of this series. Not getting user feedback early is the second most common mistake entrepreneurs make.

Startups often think that it’s risky if they conduct user studies too early and receive negative feedback, since the feedback could be biased due to the incomplete prototype. On the contrary, a startup cannot afford to conduct user studies too late.

One of the startups I consulted with conducted user studies several months into product development, and uncovered some fundamental problems that would require a pivot to fix. Because they’ve already worked on this product for so long, their choices were to scrap months of work and lose investor confidence, or to release a product the know will fail. If they tested their ideas earlier, they could have pivoted before sinking in months of time and money.

So how does one conduct inexpensive user research as a novice? Here are a few pointers to get you started:

Anyone can participate in a user study, and you only need a few

Most consumer products can be tested on friends and family. Even B2B products can be tested if you provide the participant a little background so they become the “actor” and pretend to be your target audience. I’ll explain that more in the next section.

If your product requires domain expertise to use, or it’s confidential, you can always call on your coworkers. At Apple, I was responsibe for proposing new feature concepts for Safari. To conduct user studies on my designs and prototypes, I gathered feedback from coworkers who had clearance but were not working in the same area as me.

Through trial-and-error, I found that a study size of 3–5 is optimal for rapid iterations. When I gathered 19 of my Apple coworkers in a room, the list of feedback was overwhelming and the loudest opinions dominated. When I tried to test my designs 1-on-1, it was time consuming and hard to distinguish between personal taste and usability issues. A group of 3–5 allowed me to calibrate each person’s feedback with the group, and was cozy enough to allow all personalities to be heard. This streamlined my process so that I could create a design, get feedback from 3–5 people, iterated on the design, then get feedback from a different set of 3–5 people.

If you’ve exhausted your immediate network for study participants, there are other ways to meet strangers for user studies without breaking the bank. When I was designing Watcha Doin’ and pushed for external feedback, the PM attended a $15 event that allowed him to interview 3 participants. Even with just 3 people, several patterns of usability issues immediately became clear due to their feedback.

Watcha Doin’? Available on the App Store

You can test things you haven’t built yet

Any product can be tested before it is complete, and a lot of products can be tested before they even exist. The key is to create specific hypotheses and use still images and some good old slight-of-hand.

A targeted question allows you to test a feature without designing the rest of the product. When I was working with Second Life in 2008, my team was proposing a recommendation system that had recommendations by category, user ratings, a map view, and a similar categories graph. We wanted to test “can users can find an event fitting their interest?”, but without building any of it.

We asked study participants to roleplay a persona. “You are ___, a new user in Second Life. You love karaoke, so you’re going to find a karaoke location.” We drew each screen in the activity-finding workflow on paper. And when the participant “clicked” on a part of the paper, we would manually switch it out with a different piece of paper. When we didn’t have screens for some actions the participant took, we simply asked the participant to try something else (and took notes on that usability issue).

We drew many “screens” on paper and used them to conduct user tests.

This scrappy approach allowed us to conduct 13 user studies early in the design process. We quickly realized that the feature was bloated, and adopted a minimalistic HUD UI without needing to change a line of code (because we haven’t started development yet!).

Second Life recommendation HUD at its default, minimized state

One last note: user research data informs your decisions, but the data does not decide for you. Your participants are giving you first-impression feedback, and you may decide that some feedback is irrelevant because you are sacrificing first impressions for long term retention. How do you know when to listen to the user feedback and when to set it aside? It’s part art, part science, but if you use the inter-disciplinary approach I described in part 1, you maximize the information you have in choosing the right approach.

This was part 3 in a series about how to create kick-ass products. Here are the other parts:

Part 4: Great UX balances emotion, efficiency, power, and simplicity

Part 5: Tips for shipping extremely cheap & fast


Part 1: Use an Inter-disciplinary process

Part 2: Cull your MVP ruthlessly