Here’s why user research does not happen often enough

Over the past month we’ve been talking to many product and UX leaders about the state of user research in their organizations. Apart from anecdotal conversations and interviews, we collected over 100 responses to a short survey which results may surprise you.

We presented some of the findings at the Design and Research Ops Summit in New York last week, but wanted to share an in-depth look at the results and hopefully shed some light onto why most organizations struggle to do effective customer research often.

In the survey we asked four questions and here’s what we learned:

Question 1: Do you think your team does enough user research?

A common sentiment shared amongst 80% of the UX professionals we surveyed is that their team doesn’t do enough user research. This is not surprising at all, we have seen similar results in research done by Patrick at ProfitWell with a larger pool of respondents.

Most product teams would agree that understanding their customers is the most important activity they should be focusing on however, when it comes to prioritising learning over delivery it seems that very few organizations are truly intentional about it.

We wanted to understand the reasons behind this apparent contradiction so we followed up by asking:

Question 2: What is stopping your team from doing more research?

We received over 80 responses and the vast majority pointed to 3 reasons: Budget, time and a general lack of support from the leadership. Here’s a small sample of what we heard:

“There are multiple reasons for not being able to do more research. Lack of resources is definitely a contributing factor but tight timelines from business units along with not involving UX earlier in the process would be the biggest factor”

“Colleagues who are convinced they already know what our customers need”

“Time pressure (although we are wasting a lot of time developing features to test them live, only to watch them fail)”

“What’s stopping us is mostly low budget and daily firefighting to keep the product running. Prioritisation happens at the business leadership level. So it is, in the end, all about the vision of the leadership and not the user.”

“Some PMs would rather build and hear about problems later.”

Working in an environment where research is said to be valued but it is not given enough time and resources to flourish can feel very dysfunctional at times and ultimately can become a source of stress and anxiety for those who are hired to do the research itself.

We also wanted to understand the levels of satisfaction around the research that gets actually done so we asked:

Question 3: Are you satisfied with the way user research is used in the development process?

Responses were closer to 50/50, but we still see a majority of respondents saying they are NOT satisfied with the way user research is being used in the development process. So far we have a significant portion of people who are not happy with the amount of research that is done and not happy with the way it is used.

Regardless of the size of your business or your role, somebody is doing some kind of customer research in your organization, and that person is most likely to feel the same way. What is your business doing about it?

We also wanted to understand what, according to the respondents, could be done in order to remedy the situation. Here’s what they said:

Question 4: What would need to change for you to feel satisfied with the way user research is used?

“Including the product and engineering folks in the research phase”

“Team of developers is old school in my company, I used to work with more open minded (and very young). They took user research seriously”

“Need to sync development team with design team to understand metrics and research results”

“Decisions are made with a revenue first approach with user research second. Typically a revenue number is tied to a business request without having adequate data in place to determine how the user’s experience would be impacted. While we are all employed to make the company money, I feel there needs to be better analytics on the user impact in order to make sure their experience isn’t negatively impacted at the same time”

“Sometimes, developers ignore changes suggested as a result of user research just to gain a couple of hours of dev”

“Building user centric products would need to be prioritized over meeting deadlines for product releases”

“If UX research was anchored at the top and everyone agreed on its usefulness. That happens in some projects, but we are far from the level of UX maturity that would ensure it is done and sought after systematically.”

“User research is too often done AFTER development has begun, thus it’s used to confirm uninformed biases formed before any meaningful research is done.”

Most responses pointed to a lack of team alignment, lack of awareness about the importance of research and ultimately to a development process that seems to be more focused on deadlines and deliveries than on the quality of outcomes.

Question 5: How much do you think other team members (outside of the product team) are aware of the insights generated in the user research process?

56% of respondents believe that people outside of the product team know very little or nothing about insights generated from user research. This seems to indicate that not only product teams are missing out on important insights but also the rest of the organization is not learning from the scarce user research that takes place.

So far it is not looking very positive so we decided to ask about what is actually working.

Question 6: Have you implemented any tactic that has helped you engage your team with user research insights?

“Yes. Assumptive meetings. Having people come together to say what they know (assume) about a product, and then compare to real user research results. It can be humbling and eye opening”

“Lunch and learn sessions and trying to get stakeholders from other teams to participate in research and ideation.”

“We’ve taken to making posters that highlight the key points from research and the implications for development. They are quick to produce and putting them up is making research front of mind.”

“Including devs upfront design sprints”

“We organised a ‘Ground zero Day’ for the entire company, where 80% of the employees went to the field and met customers and our partners to gather first hand feedback and insights”

“Doing weekly learnings sessions from User Research interviews”

“Anytime I hear a team member say “I think this would work best” or “I want the user to do this,” I re-affirm that personal opinions or personal wants/needs have no validity in our decision making. Backing up decision making with real data and user feedback is how we make choices.”

“User testing of competitor products”

“I’ve made sure to include engineering more in research reviews and created a research database to allow anyone in the company to access research results at any time”

The responses to this question were interesting. We got a lot of people flat out saying “no”, they hadn’t implemented any kind of tactic to increase engagement. The vast majority of the responses were centered around meeting with others face-to-face early and often throughout the entire development process.

Why is it so hard to embed customer research in the development process?

We think the challenge starts at the very early stage of the business. When you’re small, research is typically left up to the founder(s) because they wear multiple hats and typically haven’t hired any UX practitioners. At this stage, user/customer research typically consists of customer interviews, feedback emails, and basic surveys — relatively simple methodologies that have decent potential for surfacing insights. The biggest challenge for founding teams of small companies is transferring knowledge to the rest of the team; it’s one thing to mention insights in a monthly meeting, but it’s difficult to build that knowledge into the fabric of your company.

Medium-sized companies usually have more specialised roles that do user research, whether they call it that way or not. For example, you would typically have marketing, customer support, design and product folks executing on a variety of methods that are usually more sophisticated. At this stage, the challenges become more about transferring knowledge between different teams and ensuring research insights are incorporated into the product’s decision-making process.

Having more people doing research in silos does not equal more insights. The politics around what type of research is useful and who “owns” customer understanding start here.

Large enterprises specialise even further, hiring specifically for user researchers and building teams dedicated to user/customer experience. Methodologies used in earlier stages continue to be used, and in-house or original methods of research may be developed to increase the team’s capabilities. At this stage, you’ll encounter the same challenges as before, in addition to challenges brought about by internal politics, legacy processes and the inevitable complexity of large scale-teams.

Evangelising research is a long-term game

At each stage, you will find that most people try to solve the problem by evangelising research. The focus tends to be around communicating the importance of user research, however, most product and research teams grow frustrated as their evangelisation process seem to build awareness but not tangible change in processes and resources.

It takes many years to move an organization through the different stages of UX maturity.

Although evangelising user research it is indeed a strategy for aligning the organization around the customers, it also largely depends on the ability of the evangelist to tell stories and use data to prove the value.

If the evangelist is not able to execute on research successfully, the chances of succeeding at convincing others about the importance of research are also reduced.

Flipping the script — prioritising Research Operations over evangelising research.

It is natural to think that the business needs to reach a certain level of UX maturity before they begin to improve the processes behind the research. And on first consideration, this makes sense — why focus on optimizing research when you’re still figuring out how to incorporate research into the development process?

Here’s why — focusing on improving your current research process and eliminating inefficiencies in the process might actually help you reach UX maturity faster. By investing time, effort, and money into research operations, user research can be done more often, more efficiently, and more effectively. This increase in the research quantity and quality may expose more of the organization to the value of user research, resulting in increased investment into the research team (whatever that means in your company) as a whole.

As most organization changes, you can start small by identifying what processes can help you and your team do better research faster. Here’s a couple of areas you can explore:

• Customer segmentation — can you find the right customers to talk to?
• Protocols for engaging with customers — does your company have rules and processes for reaching out to customers? Can you make it easier?
• Recruitment — do you have a consistent and ideally low cost channel to recruit participants for testing?
• Properties and events — Is your team tracking the right product metrics? Do you know which actions your users take?
• Data access — does your team have easy access to the data they need? Do you have a place to share all user research insights easily?
• Protocols and support material to educate non-researchers — do non-researchers receive any training about utilizing research insights?
• Communication channels to share research and gather inputs — are internal feedback loops a part of your organization’s communication strategy? In order words, can different people in the business share inputs and ideas with the product team without overwhelming them?
• Tooling — do researchers have the tools and technology they need to succeed?

Interested in finding more about what Research Ops is and how to get started at your organization? Here’s a couple of links that can get you started:

👉Coming next: How do you know Research Ops is working? Some of the metrics you can start measuring — Subscribe to this blog to receive updates.

Originally published at blog.nomnominsights.com on November 19, 2018.

Here’s why user research does not happen often enough

Over the past month we’ve been talking to many product and UX leaders about the state of user research in their organizations. Apart from anecdotal conversations and interviews, we collected over 100 responses to a short survey which results may surprise you.

We presented some of the findings at the Design and Research Ops Summit in New York last week, but wanted to share an in-depth look at the results and hopefully shed some light onto why most organizations struggle to do effective customer research often.

In the survey we asked four questions and here’s what we learned:

Question 1: Do you think your team does enough user research?

A common sentiment shared amongst 80% of the UX professionals we surveyed is that their team doesn’t do enough user research. This is not surprising at all, we have seen similar results in research done by Patrick at ProfitWell with a larger pool of respondents.

Most product teams would agree that understanding their customers is the most important activity they should be focusing on however, when it comes to prioritising learning over delivery it seems that very few organizations are truly intentional about it.

We wanted to understand the reasons behind this apparent contradiction so we followed up by asking:

Question 2: What is stopping your team from doing more research?

We received over 80 responses and the vast majority pointed to 3 reasons: Budget, time and a general lack of support from the leadership. Here’s a small sample of what we heard:

“There are multiple reasons for not being able to do more research. Lack of resources is definitely a contributing factor but tight timelines from business units along with not involving UX earlier in the process would be the biggest factor”

“Colleagues who are convinced they already know what our customers need”

“Time pressure (although we are wasting a lot of time developing features to test them live, only to watch them fail)”

“What’s stopping us is mostly low budget and daily firefighting to keep the product running. Prioritisation happens at the business leadership level. So it is, in the end, all about the vision of the leadership and not the user.”

“Some PMs would rather build and hear about problems later.”

Working in an environment where research is said to be valued but it is not given enough time and resources to flourish can feel very dysfunctional at times and ultimately can become a source of stress and anxiety for those who are hired to do the research itself.

We also wanted to understand the levels of satisfaction around the research that gets actually done so we asked:

Question 3: Are you satisfied with the way user research is used in the development process?

Responses were closer to 50/50, but we still see a majority of respondents saying they are NOT satisfied with the way user research is being used in the development process. So far we have a significant portion of people who are not happy with the amount of research that is done and not happy with the way it is used.

Regardless of the size of your business or your role, somebody is doing some kind of customer research in your organization, and that person is most likely to feel the same way. What is your business doing about it?

We also wanted to understand what, according to the respondents, could be done in order to remedy the situation. Here’s what they said:

Question 4: What would need to change for you to feel satisfied with the way user research is used?

“Including the product and engineering folks in the research phase”

“Team of developers is old school in my company, I used to work with more open minded (and very young). They took user research seriously”

“Need to sync development team with design team to understand metrics and research results”

“Decisions are made with a revenue first approach with user research second. Typically a revenue number is tied to a business request without having adequate data in place to determine how the user’s experience would be impacted. While we are all employed to make the company money, I feel there needs to be better analytics on the user impact in order to make sure their experience isn’t negatively impacted at the same time”

“Sometimes, developers ignore changes suggested as a result of user research just to gain a couple of hours of dev”

“Building user centric products would need to be prioritized over meeting deadlines for product releases”

“If UX research was anchored at the top and everyone agreed on its usefulness. That happens in some projects, but we are far from the level of UX maturity that would ensure it is done and sought after systematically.”

“User research is too often done AFTER development has begun, thus it’s used to confirm uninformed biases formed before any meaningful research is done.”

Most responses pointed to a lack of team alignment, lack of awareness about the importance of research and ultimately to a development process that seems to be more focused on deadlines and deliveries than on the quality of outcomes.

Question 5: How much do you think other team members (outside of the product team) are aware of the insights generated in the user research process?

56% of respondents believe that people outside of the product team know very little or nothing about insights generated from user research. This seems to indicate that not only product teams are missing out on important insights but also the rest of the organization is not learning from the scarce user research that takes place.

So far it is not looking very positive so we decided to ask about what is actually working.

Question 6: Have you implemented any tactic that has helped you engage your team with user research insights?

“Yes. Assumptive meetings. Having people come together to say what they know (assume) about a product, and then compare to real user research results. It can be humbling and eye opening”

“Lunch and learn sessions and trying to get stakeholders from other teams to participate in research and ideation.”

“We’ve taken to making posters that highlight the key points from research and the implications for development. They are quick to produce and putting them up is making research front of mind.”

“Including devs upfront design sprints”

“We organised a ‘Ground zero Day’ for the entire company, where 80% of the employees went to the field and met customers and our partners to gather first hand feedback and insights”

“Doing weekly learnings sessions from User Research interviews”

“Anytime I hear a team member say “I think this would work best” or “I want the user to do this,” I re-affirm that personal opinions or personal wants/needs have no validity in our decision making. Backing up decision making with real data and user feedback is how we make choices.”

“User testing of competitor products”

“I’ve made sure to include engineering more in research reviews and created a research database to allow anyone in the company to access research results at any time”

The responses to this question were interesting. We got a lot of people flat out saying “no”, they hadn’t implemented any kind of tactic to increase engagement. The vast majority of the responses were centered around meeting with others face-to-face early and often throughout the entire development process.

Why is it so hard to embed customer research in the development process?

We think the challenge starts at the very early stage of the business. When you’re small, research is typically left up to the founder(s) because they wear multiple hats and typically haven’t hired any UX practitioners. At this stage, user/customer research typically consists of customer interviews, feedback emails, and basic surveys — relatively simple methodologies that have decent potential for surfacing insights. The biggest challenge for founding teams of small companies is transferring knowledge to the rest of the team; it’s one thing to mention insights in a monthly meeting, but it’s difficult to build that knowledge into the fabric of your company.

Medium-sized companies usually have more specialised roles that do user research, whether they call it that way or not. For example, you would typically have marketing, customer support, design and product folks executing on a variety of methods that are usually more sophisticated. At this stage, the challenges become more about transferring knowledge between different teams and ensuring research insights are incorporated into the product’s decision-making process.

Having more people doing research in silos does not equal more insights. The politics around what type of research is useful and who “owns” customer understanding start here.

Large enterprises specialise even further, hiring specifically for user researchers and building teams dedicated to user/customer experience. Methodologies used in earlier stages continue to be used, and in-house or original methods of research may be developed to increase the team’s capabilities. At this stage, you’ll encounter the same challenges as before, in addition to challenges brought about by internal politics, legacy processes and the inevitable complexity of large scale-teams.

Evangelising research is a long-term game

At each stage, you will find that most people try to solve the problem by evangelising research. The focus tends to be around communicating the importance of user research, however, most product and research teams grow frustrated as their evangelisation process seem to build awareness but not tangible change in processes and resources.

It takes many years to move an organization through the different stages of UX maturity.

Although evangelising user research it is indeed a strategy for aligning the organization around the customers, it also largely depends on the ability of the evangelist to tell stories and use data to prove the value.

If the evangelist is not able to execute on research successfully, the chances of succeeding at convincing others about the importance of research are also reduced.

Flipping the script — prioritising Research Operations over evangelising research.

It is natural to think that the business needs to reach a certain level of UX maturity before they begin to improve the processes behind the research. And on first consideration, this makes sense — why focus on optimizing research when you’re still figuring out how to incorporate research into the development process?

Here’s why — focusing on improving your current research process and eliminating inefficiencies in the process might actually help you reach UX maturity faster. By investing time, effort, and money into research operations, user research can be done more often, more efficiently, and more effectively. This increase in the research quantity and quality may expose more of the organization to the value of user research, resulting in increased investment into the research team (whatever that means in your company) as a whole.

As most organization changes, you can start small by identifying what processes can help you and your team do better research faster. Here’s a couple of areas you can explore:

• Customer segmentation — can you find the right customers to talk to?
• Protocols for engaging with customers — does your company have rules and processes for reaching out to customers? Can you make it easier?
• Recruitment — do you have a consistent and ideally low cost channel to recruit participants for testing?
• Properties and events — Is your team tracking the right product metrics? Do you know which actions your users take?
• Data access — does your team have easy access to the data they need? Do you have a place to share all user research insights easily?
• Protocols and support material to educate non-researchers — do non-researchers receive any training about utilizing research insights?
• Communication channels to share research and gather inputs — are internal feedback loops a part of your organization’s communication strategy? In order words, can different people in the business share inputs and ideas with the product team without overwhelming them?
• Tooling — do researchers have the tools and technology they need to succeed?

Interested in finding more about what Research Ops is and how to get started at your organization? Here’s a couple of links that can get you started:

👉Coming next: How do you know Research Ops is working? Some of the metrics you can start measuring — Subscribe to this blog to receive updates.

Originally published at blog.nomnominsights.com on November 19, 2018.