Secrets to Fast and Affordable User Research

Hopefully anyone reading this article knows what user research is, and how important it is in building a truly successful product. But for those that need a refresher or some reasons to convince other people of the importance of research, here they are:

• Research de-risks your product. Effort here will exponentially reduce effort in the future, either in the form of preventing effort on features users don’t need or by optimizing new designs up front.
• Knowing more about your users is never detrimental. At Crafted, we’ve done hundreds of hours of user research across dozens of clients, and there has never been an instance where we haven’t learned something valuable.
• Research doesn’t have to be slow OR expensive! There are methods to get insights quickly and affordably.

In this blog post, we’re going to go over the best strategies for lean user research as well as tools and techniques for getting feedback on a budget.

What is User Research?

For those that are new to user research, it can be surprising how expansive the research space is. A common misconception is that user research is just talking to users; doing interviews, focus groups, etc. While talking to users is extremely valuable, it is not the only way to learn more about your product and how people are interacting with it.

There are two broad categories of user research: qualitative and quantitative. Ideally, you should be doing both to create a comprehensive understanding of user behavior. User interviews fall under qualitative research; that is, anything that provides feedback that cannot be measured. Quantitative research is the opposite. It captures any research that returns measurable data.

Many research techniques fall under both categories, where you might get both measurable responses (e.g, “50% of users said they preferred option A to option B”) and unmeasurable responses (e.g., user responses on why they prefer option A to option B).

Ultimately, research is anything you are doing to collect insights into user behavior both within your product as well as across the market space. This spans everything from validation (testing new product features/looks), to evaluative (feedback on existing product), to generative (asking users about their habits within the market space, independent of your product).

How to Collect Feedback on a Budget

1: Understand Your Needs

The best way to save money is to not spend any in the first place. Don’t go into user research blind and end up wasting your resources for answers to questions you didn’t have or that you already knew.

The first step to take when embarking on user research is knowing what questions you need answers to! (Hint: these are often different from the questions you will actually ask users). Defining your top unknowns will let you prioritize your resources across the appropriate research efforts. There are several ways to approach finding these questions:

• Business: Oftentimes a good place to start is looking at your top business goals and figuring out the biggest unknowns around how to meet them. Trying to sell more product? Your top questions might be how many users are purchasing, where the biggest dropoff is in the sales funnel, or why a user is choosing your product over another.
• User: One of my favorite alternative ways to go about this is through the lens of the user; identifying the biggest user goals, the top opportunities for improvement around them, and what the biggest assumptions are around those improvement opportunities. This method also helps tie directly to development efforts for more measurable ROI.

Knowing your question (or questions) will help you narrow in on what sort of testing and tools are needed. The way to do research cheaply and effectively is to take your question, and think of the most basic knowledge you would need to answer it. Need to know why users are skipping out on a step? A simple follow up survey with an open text box could get the answers you need. Want to know how users are moving through your funnel? A pricey analytics platform is likely not needed if you can find a cheap session replay tool.

2: Identify Your User Base

You likely already know who your target user is, as well as how specialized they are. But, depending on the questions you are trying to answer, you might need to narrow — or expand — your focus. Knowing how flexible your recruitment efforts can be will greatly impact how much spending is needed.

The biggest differentiator is new vs. existing users. Do you need to know how people are already interacting, or do you need to know how to get new users to buy your product or interact with your platform? Existing users are relatively easy to source from, provided you collect user data (which if you are not, think of ways you can integrate this into your product without being overly disruptive). New users will involve some level of recruitment, which we break down tools and methods for below.

If you do need to recruit, think of how specialized your recruitment needs to be. Often, answers to questions do not necessarily need an expert’s context and this will greatly expand your potential user pool. Think about a website selling surgical equipment; you will need someone with surgical knowledge to tell you if you have the correct information presented, but you can ask anyone to interact with the site and see if they can find how to filter and proceed to a checkout (even if they don’t know what they’re buying).

Data is only as valuable as the people it is coming from; make sure you are researching with the correct personas and can trust the information you get back.

3: Make it Happen (Tools and Strategies)

Check out our below guide on qualitative research strategies, ranging from free/affordable tools to more expensive, robust options.

Check out our below guide on quantitative research strategies, ranging from free/affordable tools to more expensive, robust options.

While staying lean, always show appreciation for your testers’ time. Additionally, compensation can help you by reducing the risk of a no-show. One of the advantages of a tool like UserBob is that users are paid through them.

Here are some ways to compensate testers on a budget:

• Secure a discount or free use of your product for a period of time (ask your sales team!)
• Small gift cards, such as a $5 gift to Starbucks or Amazon, work well.
• In one case, we worked with a non-profit that truly had zero budget. We sent “thank you” cards. Something is better than nothing.

Case Studies: User Research on a Budget

Crafted is a big advocate for user research, so it is something we bring to all of our clients, big or small. Having resources is great, but often our clients have to be scrappy and get creative with how we do user testing. Here are some examples of how we got great user feedback on tight budgets:

Case Study #1

Goal: When working on a mobile app with one client, we had a clear goal from our stakeholders: the sales team wanted customers to think the product was “modern”. This is a question that we could answer quantitatively. The “why” in this case didn’t matter. We just wanted to test how a user assessed the appearance of the product within the first few seconds.

• Question: How do users interpret the appearance of this product?
• Method: Impression test (or 5 second test) with adjective word bank

Users were shown a screen for five seconds, then asked to select which 3–5 words from a list of 20 they thought best described the screen.

• Tool: Lyssna
• Recruitment: Through Lyssna
• Cost: $20 for twenty users (repeated)

Learnings: The goal was directional. With each iteration we weren’t striving for the level of certainty you would want in a medical trial. Instead we looked to see, “were more people describing” it as modern in one test vs. another. Over the course of several iterations, we went from a design that was scoring at less than 1/20 users, to designs that were consistently scoring at 10/20. Further, the other words users were describing were trending towards positive adjectives, rather than the negative ones.

Case Study #2

Goal: On the same project mentioned in Case Study #1, for the same screens, we wanted to understand if users understood the difference between two types of account balances. This was a question that required a “why.” Simply knowing users preferred one design over the other was not helpful (we did know that the preferential testing showed that users preferred lower information density). We needed to understand if there was a reason behind the preference.

• Question: Why did users express a certain preference across these screens
• Method: Talk aloud, unmoderated usability test (3 minutes)

Users were shown lightweight prototypes of the app’s home screen and account screen. They could click into it. In the prompts for the test, after collecting their initial impressions, we asked them specifically to define those two pieces of data.

• Tool: User Bob
• Recruitment: Through User Bob
• Cost: 10 users @ 3 minutes per test = $30

Learnings: Our unmoderated tests yielded the same preferences observed in other tests. However, we did validate that our hypothesis was correct: there was a knowledge gap partially driving the preference. However, some users did understand the nuance between the two numbers, but still preferred not seeing both on the summary view. They simply didn’t find it useful to them, especially when the additional details were only a short press further into the app.

Case Study #3

Goal: Our final example comes from a different client. While it was a large client with a sizable budget, we were trying to introduce some evaluative research cycles and needed to operate as cheaply as possible while proving out the value of research initiatives. The primary goal we were trying to meet was improving the conversion rates of their ecommerce platform. While we had great analytics, there was very little qualitative data to go off of. We knew what the problem areas were, but what we needed to know was why they caused dropoff.

• Question: What is causing user dropoff at various points in the funnel?
• Method: Talk aloud, unmoderated usability test (3 minutes)

Users were given a link to the live website and asked to go through it as if they were a real purchaser, primed with the goals we knew our users typically had when purchasing. On each page, we asked questions about visibility of features and asked them to perform certain common actions to see if any struggle occurred.

• Tool: User Bob
• Recruitment: Through User Bob, targeted at users above 55 to accurately represent the client’s primary user base
• Cost: 16 users @ 8 minutes per test, split across desktop and mobile = $140

Learnings: We found that some of our primary product offerings were going completely unnoticed, especially on mobile devices. We also learned that while users engaged with filtering functionality easily, we were not offering the correct filters that users wanted. Users wound up having to search manually and became frustrated by the process. Both of these learnings informed targeted design improvements for those areas that would have otherwise been overlooked.

Research Going Forward: Repeat, Repeat, Repeat

One-off user research efforts can be informative, but will quickly become outdated. Treat your first user research cycle as the beginning of many! This is not meant to discourage those attempting user research by saying you should immediately develop an intense and rigorous research methodology. Any user research you can do is better than none; start where you can, and move up in incremental steps from there. If that’s one-off research campaigns focused on validating new features, try expanding into some generative research during those campaigns. If you have monthly user touchpoints, try expanding to bi-weekly. As your product grows, your research efforts should as well.

Going back to your questions, take a look at which questions might be one-off events vs. what will be continual and repetitive research needs. Got KPIs? Those are the perfect way to kick off regular data collection for measuring product success. Understand what your key product questions are, and make sure you have some way of constantly receiving feedback on them.

Finally, don’t forget to involve the whole team in any research efforts. This includes the rest of the Balanced Team (product management and engineering), too. Even if not everyone is actively analyzing data or performing interviews, everyone should have access to the same data and insights (we like using Dovetail for this) This will create consistency across your product in terms of customer experience and team priorities. Getting buy-in from the entire team when deciding on a research strategy will also help in managing resources and finding internal support for any future efforts. For more information on how to socialize research so everyone can access data and insights, check out our user-centered design white paper.

Conclusion

User research is incredibly important in building a truly successful product. And the good news is getting started on collecting feedback from your users doesn’t have to break the bank. If you need more help in implementing the strategies and tools discussed in this article, reach out to Crafted’s team of design experts and we’d be happy to connect!

Toolbox & Resources

Tools mentioned:

• Amplitude
• Attention Insight
• Calendly
• Clarity
• Fullstory
• GCal
• Google analytics
• Google Forms
• Google Meet
• kardSort
• Hotjar
• Hubspot
• Lyssna
• Maze
• Neurons’ Visualeyes
• Optimal Workshop
• Optimizely
• Pendo
• PostHog
• Qualtrics
• Smartlook
• Survey Monkey
• Twilio Segment
• Typeform
• Userbob (comes with recruitment)
• Userbrain (comes with recruitment)
• User Interviews (comes with recruitment)
• Usertesting (comes with recruitment)
• Zoom

Additional Resources

• Neilsen Norman on Research Methods
• User Interviews Research Tools Map
• Think Like a UX Researcher, David Travis and Phillip Hodgson