User Research in Startups

How designers can make research powerful!

User research? Who has time for that when we gotta go Lean, pivot, and be Agile with a unicorn coder/designer stylizing disruptive game-changery? Well, despite such (slightly overblown) constraints and pressures, it’s still quite valuable to have user research somehow folded into the startup business. A recent online discussion with members of the KP Dim Sum crowd began with the following prompt:

Given the limited resources in a startup context, how do you conduct user research activities and foster a culture of supporting research, as a part of a “design process” overall? What are some methods and techniques?

Being a recently hired design leader myself, I initially requested user research as one of my immediate hires, much to the puzzlement of my superiors…and purse-string holders. That request was greeted with surprise. “What? Not a visual artist or prototyper?” Nope, I needed to deeply understand the wild, alien context of virtual datacenter analytics before I can determine a good design direction.

My own philosophy is “I can’t design without research.” We ended up contracting out the research aspect, which has worked very well, defining high level personas & workflows of our existing product. I mandate that everyone on the product team be present at the final research presentation, and ensure that everyone has unfettered access to the reports, videos, participant profiles, etc. I also encourage everyone to attend at least one user interview session via some remote communication tool, to foster understanding and learning.

But how about others in the startup world? What are other design leaders’ approaches to ensuring user research is not neglected and continually supported? Indeed, there has been a noticeable increase in the number of job postings around user research, as this seems to be gaining traction, in addition to hiring digital designers focused on visuals and interactions. Perhaps this is a good thing, this growing recognition of the value of user research.

Thankfully, Erika Hall has recently published a wonderful short book entitled Just Enough Research which offers startups directly applicable guidance on how to get rolling with research as part of the startup’s efforts in building the right product or service. As Erika says:

Getting more entrepreneurs and developers (and even designers) to understand and value research is my personal mission. I’ve been doing a workshop on collaborative research specifically this year, because I think it’s easy to lose a lot of value when the people who have the best understanding of the research aren’t also the ones doing the design and development.

The other great thing about research becoming a team-wide activity, is that it actually fosters collaboration and group participation, a true win-win for everyone!

Regarding the finer points of conducting user research, there are some factors to consider as cautionary notes:

1) Self-reflection bias. There’s some risk of the designer him/herself testing their own concepts, so you should be beware of potential issues with how concepts are being presented, the manner and rhetoric of the questioning, and so forth. Being as neutral and objective as possible is vital. Of course, nothing is perfect but self-awareness is the first step!

2) Testing across geographies. News flash folks! The world is NOT comprised of “Silicon Valley” techies sipping Pumpkin Spice Lattes in their Teslas while tapping on their iWatches. Gasp. Shock. I know!

As Dallas-based entrepreneur Caroline Jennings helpfully suggests:

We tend to think that at least urban US is very much the same from one place to another — well it’s just not so. The needs, expectations, and navigation preferences are totally different.
One of my favorite techniques is to pick a friend in every city to be a spearhead of a focus group. You would be amazed at the completely different viewpoints generated by different geographies.

3) Paper prototyping. There’s some controversy around the validity and utility of this approach for gathering user inputs. Sparked by this article published by Google Ventures, a good discussion ensued on the merits and pitfalls. Erika Hall supports this method, stating that:

I have used paper prototypes successfully to test information architecture, interaction design, and interface language — I have never used them to ask for feedback or gotten anything amounting to encouragement.

David Sherwin, formerly of frog and now head of UX at, argues:

With paper prototypes, you can only test for comprehension. If that’s part of the feedback you need to commit or tighten an experiment you’re going to run, then it’s a valid method to use, but I only use it in research in tandem with other stimuli or methods to answer important questions.

Continuing further, David says:

The philosophy we use(d) at frog and that I am using now at is, anyone on our team can participate in and run research, as long as they’re being coached by someone with experience in the process, and has an eye towards rigorous analysis and output.

In terms of approaches to user research, there’s a variety available per situation and data gathered. Anand Sharma, who made the creative personal analytics website April Zero, offers this account on his two primary models of conducting user research:

One is looking through analytics — things like bounce rates, funnels, signup and payment percentages, etc. Especially at scale, understanding and tuning these numbers is really important. Things that are higher or lower than you would’ve expected are a good opportunity to dig deeper.
And then it was very helpful to have people try things out every few iterations. You can get helpful data from almost anyone, it’s just a matter of asking the right questions (or none and just watching). I would just be at a coffee shop working and invent something new and turn to the random person sitting next to me and have them play with it for a minute and see what happens. It would quickly be obvious whether something needed more work or was going smoothly.

4) Recruiting and research goals. This is a highly critical component, ensuring you’re seeking the properly matched folks for your study, to reveal the insights sought, with clearly defined goals guiding the study progress.

Erika agrees this is the hardest part:

Yes, recruiting is absolutely the hardest part, after formulating clear research questions. What are your goals? What are the questions you need to answer to better meet those goals? What resources do you have in terms of time, team, budget, and expertise? That will tell you what activities to undertake.

And, just to relay one final story of user research in a startup context, Peter Cho of Inkling explained an evolution of how research has helped shape product development process:

Our experience at Inkling the past 5 years has had ups and downs. Early on it was difficult for research to influence products because for better or worse we believed in ‘genius’ design — designing the product we liked best, intuitively. Now we make a big effort to test concepts in discovery and validate them before they go into delivery. We also do recurring usability testing to inform our backlog.

Getting user research done in a startup context is a veritable challenge, with many pressures and demands competing at the forefront. However, research is not only essential, but it is quite doable, to ensure the right product or service is being developed for the right audience. Hopefully this summary of our recent online discussion is the start of your own journey into making research a keystone element of a customer-centric startup strategy.

Special thanks to Deena Rosen, currently VP of UX at Opower, in helping to co-facilitate the original online #seminar on “user research in startups”.