Findings from the first user testing of Q
You only need to test your product with 5 people, we got 32 and here’s what we learned.
I won’t exactly tell you what Q is yet or how it works. I am sure you can guess but you shouldn’t have to, that’s precisely why we are testing Q with people at every step of the design process. What we discover along the way as we build Q will be shared here.
“the best results come from testing no more than 5 users and running as many small tests as you can afford.” — Neilson
Jakob Neilson points out that you only need 5 people to test your product. His theory is that as soon as you collect data from a single test user, you gain insights. The more you test the less you learn because the first user will most often have already identified the usability problems with the design.
Testing the landing page
The purpose was to test with users the clarity of the message on the landing page and discover what could be improved. The test was a link that led users to a landing page (photo above) with a button that went to a survey. The 2 key questions we wanted to answer were, does the user understand what the offer is and what this landing page is about? We decided to leave 2 other key questions for the next round of testing, does the user know what they can do and why they should care?
As you can see from the above photo there is little description about the product. That was deliberate for two reasons 1) we are starting small and building the design, branding and messaging of Q in steps 2) this was an opportunity to ask users some market research questions about current applications used for dating and meet ups.
User testing results
We decided to shop the test around to a few friends considering the odds of people having time or energy these days. To my surprise, 32 people completed the testing. The results of the test validated both Jakob’s thesis that over time the same issues will continue to reappear, and provided insights for the next round of design changes.
As you can see, it’s pretty unclear. Here are some of the guesses (which are pretty good and comedic): fancy Linkedin, a quiz site, a platonic dating site. Most importantly, there are many responses that say no idea. This means that there is not enough description to answer the question for a user as to what the product is about and why they should the care.
Another objective was to get insights into what problems people currently face when using existing dating or meet up apps.
As you can see, many of the common themes emerge: swipe fatigue, lack of integrity, ghosting, disingenuous intentions. Some of these things may not be able to be addressed with design and technology because it is not a problem to be solved solely by design and technology. These are deeper societal problems in the way people behave and interact with each-other alongside existing personal experiences. However, design and technology can help change the user experience. The most important insight that we gained from market research about people’s current problems with existing dating and meet up apps is simply, more testing needs to be done and further questions need to be asked to them.
Take the example of swiping, even from speaking to a handful of people it’s pretty clear there is a general feeling of ‘swipe fatigue’ (which is precisely why it may be time to change things up!) Partly this feeling is due to a consequence of the users experience of interactions both with the application feature that provides the ability to swipe left or right and from the experiences with the people he/she interacts with. User testing can be done over time to see how by removing a swiping feature, it changes the way people interact and ultimately, how they feel about their experience meeting people and using the app. This would be one step towards validating if swipe fatigue is a problem and if an alternative design solution could work.
We don’t have a minimum viable product yet but we’ve identified and prioritized initial features and by initial, I mean 1–2. We will test assumptions about these features with a focus on addressing the questions of what a user can do and why they should care. No matter where you are in the software development lifecycle it’s key put your idea out there right away and invite people into the design process. Conduct many small tests over time and build out your product.
Until next time.