User Testing Like It’s your Middle Name

Why user testing and gathering end-user input matters for startups and designers creating products.

We all know how important getting customer feedback from the end user is, right? (insert silent pause here) Over the course of my career in user experience design I’ve come across this preconception that user testing is too expensive or requires a lot of effort. I disagree. Involving the end user during early concepting and studies helps product managers and product designers to build more engaging products and learn to speak their customer’s language instead of creating things on gut feelings and intuition. Let me explain why…

"Everyone’s opinion is a hypothesis for research”

Gathering User Research & Validating Concepts

User testing goes way beyond usability flaws and validating the visual design output of an app, website or service. Sometimes it’s not so much on which patterns are being used, however more so if you’re solving the right problem for a customer. Often times we’re quick to assume that we know our personas and users really well, however do we? When was the last time you conducted an actual user interview and validated some of the hypotheses formed by internal project stakeholders?

Solving User & Business Needs vs “Creating Features”

Let’s face it, we’re all in the experience economy. Consumers now have higher expectations from product companies and expect the best for their time and money. Therefore the following needs to happen to solve any hiccups in your product:

  1. Define the problem you’re solving for and how it contribute to the business and provides value for customers. Getting intimately familiar with this helps you prepare the questions you need to ask your customers or curious about why things are the way they are.
  2. Gather any existing data from CRM services or analytics to observe trends on user behavior to have something to compare against the findings you gather from the user test.
  3. Form hypotheses on what drives their behavior. This would give you an opportunity to figure out whether or not you understood the problem correctly and if theres more to the matter.

As designers we’re accustomed to do as much problem definition in an effort to uncover pain points and challenges people have today while using our products and be the ones to have empathy for them. Building something in hopes it will solve a problem for your customer is a risk. Next up, I’ll take you through how you can run a test in-house.


Create a Guide or Agenda Prep

Some light-weight housekeeping will assist you in the process whether you’re conducting the study alone or with a colleague in-house. Given the complexity of the amount of questions to be asked and information to be gathered you’ll benefit from some sort of a guide to stay on track.

The guide or agenda you prepare will show signs of a confirmed list of the people who fit the persona(s) you’re creating a product for, jobs to be done or user stories and goals for the user to complete with the designs you’re planning to put in front of the user. Given the duration of the test and the amount of participants I highly recommending prioritizing and strategically mapping these out accordingly.

Crafting your Prototype(s)

There are variety of ways you can create a prototype. The cost of a prototype can range on what you’re looking to learn and how much time you have for going from prototype creation to testing. You’ll need to create one of these whether or not you plan to run the study in-house or off-site with a third party.

Design Firm IDEO has a saying that resonated really well with me in the past and always comes to mind…

If a picture is worth 1000 words, a prototype is worth 1000 meetings.

Often times if you’re running low on time you might be able to get away with some rapid prototyping techniques (i.e. paper prototypes). In reality you might need to a higher fidelity prototype for bigger questions to be asked and require transitions designs to communicate them quicker. An important thing to note is that often times UX designers and prototypers will use a combination of a browser based HTML/ JS prototype or a simple invision protoype.

Or… you might be able to get away with testing all of your participants with one of these options. Often times you’re also able to add on analytics tracking on a prototype and are able to quantify the interactions happening. This would make for a strong prototype to get the most learning you get from it. With your prototype being done you’ll be able to approach user testing from different angles.

Running the user test in-house

So, you’ve recruited participants that fit your customers profile, prototype(s) done and you have you’re guide and decided to test in-house. One of the most overlooked things moderators do not do is level set with their participant and mention why were testing them.

Do stress that its not an evaluation on them personally, but on being able to make sure the right problems and pain points are being addressed. One of the things that could drive happiness or motivate them on participating is mentioning the reward at the end. Folks have used things such as a meal, gift cards or company swag or all three of these. At this point you want to preface the surface with your participant about what you’re looking to learn, why it’s important and that this test is not an assessment on them, but identifying on how close we are on solving a problem.

After getting through one participant leverage the already created prototype by reusing it with each participant and have someone from your team present to take notes while you moderate.

Wrapping up test and what to do next

Teams often document key measurements they were looking for (both quantitive & qualitative) and highlight the key summaries in the style of a matrix chart or a powerpoint for project stakeholders and decision makers. Additional artifacts to include are trends in behavior, notes around why things are important to the user, or assumptions that were right or wrong should go into your document of choice.

Here’s a quick tip: In addition of documenting the findings and insights of what you’ve gained from the test — take it a step further by noting the very turn out of your user test sessions and what can lead to better sessions in the future.

It’s a wrap!

Thanks for reading. Do you have any best practices or methods not outlined here for demonstrating the value / conducting a user testing session and gaining valuable insights? Let me know your thoughts. Tweet me at @designedbyjay

Show your support

Clapping shows how much you appreciated Jay Ramirez’s story.