User Research and Fast and Rigorous User Personas— CXL Review

This is the part 7/12 in my series reviewing the Conversion Optimization Minidegree, provided by CXL Institute.

Over the past week, I had the opportunity to delve into three other courses that make up the Conversion Optimization Minidegree: “User Research”, taught by Megan Kierstead, and “Fast and Rigorous User Personas”, taught by Stefania Mereu.

Here is a brief compilation of my main lessons!

User Research

Initially, Megan Kierstead explains that User Research, also called design research or UX research, has as its main objective to understand who the user is and what their needs are to create better and better products and experiences.

Unlike what happens with market research, which focuses on knowing the buyer and what he or she wants, the objective of user research is to understand the real needs of the user.

When done well, this type of research reduces risk, saves money, identifies opportunities for delight customers, and increases user retention and satisfaction.

Note: It is noteworthy that a user survey is not the same thing as usability testing. The scope of user search is much more robust and can provide even more insights for the organization.

How to define useful questions

1. Define your organization’s goal:
   Megan recommends that you start the search for your organization’s goal. If you’re clear about what you want to achieve, it’s much easier to measure whether the insights you gained from the user survey really were meaningful.
2. Use “who, what, when, where, why, and how” to help you develop questions:
   If your company wants to reduce the number of support requests by 50%, for example, ask yourself:
   > Who: who are opening tickets? Are the same users contacting you many times or different users?
   > What: what is the content of tickets? Are there any patterns?
   > When: How many calls are received per week? How often?
   > Where: Are tickets related to a specific device, location, or browser?
   > Why: Why are users having difficulty? Are the instructions and documentation clear?
   > How: How are tickets being resolved?
3. Define whether you will use open-ended or closed questions:
   In general, exploratory research with many uncertainties uses open-ended questions. Closed questions, in turn, should be used when you already have enough data to hypothesize. In these cases, the research must confirm or deny these hypotheses.
4. Understand the project stage:
   It is recommended that research begin in the early stages of the project when changes can be made relatively easily. As code begins to develop, changes become more complex and more expensive, as they take time and many refactorings.
   Understanding the stage of the project also impacts which research method will be used. A project that is at a very early stage cannot use data analysis and heat mapping, for example.

Recommended reading: When to Use Which User-Experience Research Methods

Attitudinal methods: surveys and interviews

Megan continues the course by going deeper into two attitudinal methods: surveys and interviews.

It is noteworthy that these are not the only methods available. The diagram below, authored by the Nielsen Group, provides an overview of the available user search methods.

Interviews are excellent exploratory generate research. It helps you discover things you didn’t even know even when you didn’t even know how to ask the best questions. Surveys, on the other hand, are excellent for taking insights that already exist and trying to generalize them.

Interviews pro-tips:

• Think of the interview as a conversation. Your goal is to make the users trust you and speak their minds;
• Set your interview goals;
• Create a template for data collection based on the goals you set;

Template recommended by Megan

• Conduct the interview and ask “why”;
• Bring a partner if possible: while one person focuses on conducting the conversation in a natural way, the other focuses on collecting the data.

Pro-tips surveys:

• Don’t see surveys as mere forms. They are great for gathering information and validating existing insights.
• Be careful with coverage errors, sampling errors, measurement errors, and non-response errors.

Behavioral methods: tests

Usability tests fall under behavioral methods since the person conducting the test must observe what the user is doing.

For Megan, this type of testing works much better when you already have tasks defined that really need to be tested.

Pro-tips tests:

• Build out your list of tasks;
• Create a test plan and write your tasks;

• Choose your approach and an appropriate tool;
• Decide what is important to measure: an alternative is to use post-test forms to understand the level of difficulty users felt in completing a given task.

Different ways to do a test:

• Remote x In-person
• Qualitative x quantitative
• Scripted x open-ended
• Moderated x unmoderated

How to synthesize data:

Now that you’ve gathered a lot of data, it’s time to understand how to synthesize it and turn it into action.

1. Organize: Create a replicable, flexible organization system that can be reused later.
2. Categorize and encode your data: you can use software, but often post it do the job.
3. Combine different types of synthesis: complex or risky decision processes require combining more than one data source to mitigate risks.

Creating useful research deliverables:

After synthesizing all the data into insights, you must create deliverables that can be used across your organization. This is the case with personas, empathy maps, customer journey, and task analysis.

What makes a good persona:

• Communicates the most relevant information about your main users;
• Helps build empathy;
• It includes direct quotes and is based on the challenges your users face.

To go beyond: UX Mapping Methods Compared: A Cheat Sheet

Fast and Rigorous User Personas

Continuing studies in the field of user research, the “Fast and Rigorous User Personas” course, taught by Stefania Mereu and Eric Taylor, challenges the idea that creating user personas takes a lot of time or reinforces stereotypes.

Through a data-based framework, the course reveals how to write actionable and unbiased surveys, recruit respondents, validate and organize the collected data, and analyze them to build archetypes.

According to Alan Cooper, a user persona is “a precise descriptive model of the user, what he wishes to accomplish and why”. This means that it is not an exact approximation of the user, but a model that reveals their main motivations.

For Stefania and Eric, a good persona provides relevant and actionable information that can really be used towards organizational goals and that facilitates communication between different areas.

The framework suggested by the instructors for building personas has three steps:

1. Collect data;
2. Identify groups;
3. Build archetypes.

In practice:

1. Collect quantitative data:
The first step is to collect quantitative data. This is the way suggested by Stefania and Eric because it allows the researcher to have control over the type of information that will be collected through the surveys.

Your persona is only as good as your data. Therefore, the questions on your survey must be relevant, actionable, and unbiased.

Consider building a persona for a car sales app.

• Irrelevant: “How has your car driving experience been so far?”
• Relevant: “How has your car shopping experience been so far?”
• Unactionable: “What is the MPG on your current car?”
• Actionable: “What is your desired MPG on your future car?”
• Biased: “Would you rather buy a car from a dealer or from a stranger?”
• Unbiased: “Would you rather buy a car from a salesman or from a private seller?”

2. Build your survey:
There are many tools for building surveys, including free options like Lime Survey and Google Forms.

Stefania recommends that you use tools that allow open-ended questions because that will give you more flexibility.

3. Recruit Respondents:
There are a few possibilities when it comes to recruiting respondents.

You can apply the search on your own website using tools like Hotjar or hire tools like Mechanical Turk.

In general, you need to have between 300 and 1000 participants to gain more confidence in your results.

If this number is not possible, it is worth combining different data sources to increase reliability.

Pro-tip: As there is a monetary incentive for respondents, some users may be interested in taking the survey just to earn the reward. To avoid polluting your data with a person who is answering randomly, Stefania suggests inserting questions as a specific answer to filter out only those who answered correctly.

4. Start the survey after validating the data with a small sample:
Before starting the survey, conduct a pilot test survey to identify mistakes (10–20 people).

Another good tip is to share preliminary data with a co-worker to ensure a second assessment of what has been done so far.

5. Organize your data and identify factors:
Start by organizing your data. A good practice is to use the so-called “Tidy” data:

• Each row is a response (one unique participant);
• Each column is a variable (the answer to a question).

Then simplify your data using Exploratory Factor Analysis (EFA), where the answers to the questions are reflected in factors. For example 16 questions from a survey can be summarized in 3 factors.

Your aim is to identify the influence each factor has on each of the questions.

By the end of the analysis, you will have simplified the questions and will have a sense of how much each respondent cares about each of the factors.

6. Identify clusters:
After factor analysis, it is much easier to understand how to identify clusters.

When you have few answers you can do this process manually. To cluster many responses, however, you will need a computer.

This clustering helps you understand the main differences between the groups.

Recommended Analytics Platforms:

• IBM SPSS
• R: if you’re interested in learning R, learn interactively with the swirl.
• wessa.net
• GNU PSPP

7. Build your archetypes:
Now that you’ve surveyed the users, identified the most important factors, and created the clusters, it’s time to start building your archetypes.

Stefania recommends that you start building archetypes from the users’ needs.

To do this, start the analysis from the factors you defined and only then increase the profile with the other information available in the survey.

In the above example of the car sales app, personas can be concerned about three different aspects. These three profiles should be the starting point for further information.

It’s also nice to include real quotes to show how these users think about a particular topic. Another cool feature is creating a word cloud.

Extra tip: use different data visualization types
Viewing quantitative data in different ways (word cloud, user journey, empowers the document that represents your personas. Some cool features are: Shiny, Wizard, D3.js, Tableau, and Google Data Studio

8. Next steps:
In general, you can continue using the same persona for several months. Despite this, it is necessary to be aware of factors that can change them.

This article was originally posted on fefy.com. It is part of a series of 12 posts sharing my main learnings during the Conversion Optimization Minidegree, provided by CXL Institute.

You can read the previous articles on the links below.

• Week 01: 5 conversion rate optimization best practices to increase your results
• Week 02: The (almost) mathematical formula behind millionaire salespages
• Week 03: Neuromarketing, emotional content strategy and influence design
• Week 04: Google Analytics for Beginners: an almost illustrated guide
• Week 05: Landing Page Optimization and Conversion Research
• Week 06: Google Analytics 4 and Google Tag Manager — CXL Review
• Week 07: User Research and Fast and Rigorous User Personas
• Week 08: Heuristic Analysis, Google Analytics Audit and Voice of Customer Data
• Week 09: A/B Testing Foundations, How to Run Tests, and Testing Strategies