A few bucks for your thoughts? My experience as a user tester for apps and websites.
As a UX designer looking to strengthen my skills in user research, I signed up for UserCrowd and User Testing in April 2019. Becoming a tester seemed like a good way to understand how researchers use these platforms to get feedback for their digital products. Both of these websites pay people for giving their feedback on different websites and apps.
As a tester, I have deepened my understanding of the types of questions user researchers deem important to ask their users. There are a few ways I believe participating on these platforms as a tester has made me a better researcher.
1. I recognize the type of language researchers are using for their questions. Most researchers use very neutral language when asking a question. They are thoughtful about the language they are using in order to generate responses that aren’t just what the researcher may want to hear. Voice and tone are crafted to create questions that avoid leading the user in a direction that may compromise survey results.
2. Questions are short, easy, and answerable. I have never felt overwhelmed by what a question is asking of me. Questions steer away from the rhetorical in favor of the specific.
3. Question order is considered. Particular attention is paid to the order of questions in each survey I have taken. Researchers avoid ordering questions in ways that may prime the user to answer each subsequent question a certain way. Opinion based questions are usually asked earlier on in surveys, while factual information is given later. This keeps me as the tester from allowing these facts given by the researcher to influence my opinions.
UserTesting is the stricter of the two websites. The approval process to become a tester involves a short video training and a trial run of your testing skills. UserTesting has been frustrating because even though some of the surveys can pay $10 for about 30 minutes of your time, most surveys have a barrier to entry — and this is after you have already passed their trial run. Before I can begin each test, I am usually asked 2–5 questions in order to qualify for each test. These qualifying questions often ask about my occupation, my income, or other personal life questions. According to UserTesting, this allows companies to get feedback from testers that are most similar to their current user base. 90% of the time, I am not qualified to take the survey after answering the qualifying questions.
UserTesting is also more difficult to complete because of the length and audio components. A typical test involves a series of questions and short instructions to perform. Usually I will be asked to record my voice as I navigate through a website or app, thinking out loud about my experiences. At the end of the voice recording portion, I am usually asked to write down some answers to some follow-up questions.
UserTesting requires that I am in a private, quiet place in order to record my voice, which makes it more difficult to complete these tests regularly, but I also see the immense value in the platform from the researcher’s perspective. The audio files received by the researcher give some insight into what a user is thinking as they are navigating a website or app, while the written questions at the end allow the tester to give their feedback on their overall experience with the digital product.
UserCrowd, on the other hand, does not ask qualifying questions. I am brought straight into the survey, and most surveys take 2 minutes to complete. Many UserCrowd surveys have asked simply about my preferences for one visual design over the other. For example, the last test I participated in gave me three different website landing pages, and asked which design I preferred. The main differences were aesthetic, with different stock photo backgrounds or illustrations. The follow-up question asked for my reasoning behind my choice.
Other questions I often receive on UserCrowd surveys ask about my understanding of what the purpose or intent of something is. It is common to get questions like, “What do you think clicking this button will do?” or “Based on this landing page for this website, what is the main product this company sells?” or “Would you personally sign up for this service?” Because of these types of questions, many of which I see repeated on different surveys for different digital products, I get the impression that the researchers on UserCrowd are interested in the opinions and feelings of their users, and want to improve their designs in order to align with what a user expects from their digital products.
I wouldn’t consider either of these websites to exactly allow testers to “strike it rich.” Getting selected for these surveys is its own skill and requires time and dedication — I’ve noticed on both platforms that over time, I have gotten chosen more often as I have learned how to be a better tester. Both websites give you a feedback rating which shows you how well you are doing on your surveys according to the researchers. Both platforms are transparent about the earning potential of their testers, with UserTesting stating: “While UserTesting is a great way to earn a few extra dollars on the side, it won’t make you rich. The number of opportunities you receive will depend on a number of factors, such as your demographics and your quality rating.”
From my own experience, the following makes a tester more valuable and more likely to score high-paying surveys:
1. Read through questions and tasks in their entirety. In cases where a question is ambiguous, I mention so in my response and why, and then answer to the best of my ability.
2. Don’t accept every test. In cases where I feel I am not able to give valuable feedback on a digital product, I decline to take a test. This usually happens when the website or app is about something I am completely unfamiliar with. It’s not worth it to accept tests you are unsure about completing well, because your review written by the researcher may suffer as a result.
3. Share honest, real opinions. Researchers aren’t looking for advice on what to change or for their testers to act like experts. Instead, the most useful feedback to give comes from taking on the perspective of an average customer, and sharing your experience. Sharing emotions (“This webpage is overwhelming and I’m not sure what to click on”) is better than sharing advice (“You should change this font and make all the buttons bigger so that I can find them more easily.”)
4. Do your best to complete tasks. If a test asks you to find a certain button, but you don’t find it right away, it’s much more valuable to the researcher if you spend time searching for the button, explaining why a task took you a longer time than expected to complete, rather than just skipping over a task.
Overall, I do recommend becoming a tester to those looking to strengthen their user research skills. The most valuable part of becoming a tester has been the development of my understanding of what user testing looks like, and what makes a test valuable and worthwhile to the researcher in order to improve the design of digital products.