Generative AI: The Perfect Colleague for UX Researchers

Over the past year or so I’ve used ChatGPT almost every day to support my work, gradually increasing from a simple question here and there to multi-hour conversations with thousand-word prompts. While I don’t think AI could replace UX Researchers any time soon, I do think that those who are under-utilising it are at a severe disadvantage to those that are actively working to master it. In this article I break down some of my favourite methods, prompts and overall learnings from my experience using generative AI to support my work.

Everyday Conversations:

I see ChatGPT like a colleague with near-infinite wisdom and bandwidth. In my world of business applications, domain experts can be hard to come by and are typically short on time. But ChatGPT is an expert in almost every domain, it always has time for me and is always in a good mood, no matter how basic my questions might be. In that way it can often act like a PM, providing quick insights into our product space (finance, supply chain, commerce, etc.), our users and our competitors. It can also act a little bit like an engineer, explaining complex technologies in simple ways, and helping me to understand what is and is not technically feasible for our products. Finally, it can also act as a research colleague, off whom I can bounce ideas around research methods and plans, or even more thought-provoking discussions around the implications of AI.

Of course, these AI colleagues are only as good as the prompts you give them, so I want to discuss how this research colleague, in particular, helps with (i) accelerating existing research methods and (ii) unlocking new methods, before (iii) touching on some of my concerns and cautions around its usage.

Accelerating Existing Methods:

Planning:

Whether you think you know the best method for a study or you have no idea where to start, it’s always good to ask another researcher for their opinion. Here, I like to give ChatGPT as much context as possible, without giving away any sensitive information.

“I’m a UX Researcher on a team that develops [explain product]. Right now, we’re working on [explain feature or problem].”

Explain what you’re trying to do:

My team wants to know:

- How do people… (e.g., complete a certain task, feel about AI, etc.)

- Which features we should prioritise…

Tell it what you want:

- What method do you think I should use?

- What do you think of my current plan?

I recently carried out a study that required somewhat of a new method, as we wanted to see how users might interact with two different AI voice assistants, without really having two different versions built to get feedback on. I explained my ideal method and the limitations restricting it, and ChatGPT helped me to come up with an alternative plan.

Execution:

Once you’ve settled on a method and overall plan, use generative AI to streamline the execution.

My team wants to understand the jobs-to-be-done for a Manufacturing Manager in a medium-large company. Can you provide some assumptions on this persona’s JTBD?

I want to validate and refine these JTBD with a number of Manufacturing Managers. Can you help me to build a screener and questionnaire?

This works even better if you have examples from previous studies (personas, screeners, etc.) that you can attach as a reference, to help keep things consistent.

Analysis:

Once you’ve run the study and collected your data, use generative AI to help you analyse it! Of course, here you need to be particularly careful about information that is sensitive to your company and your participants. Make sure to remove any personally-identifiable information (PII) and organise the data in a way that makes it easy to copy/paste or upload. I recommend numbering the participants (P1, P2, etc.) and providing any available demographic information (e.g., their industry, job title or years of experience). Then, if you number their responses the same way (Q1: P1, P2… Q2: P1, P2…) you can ask for correlations between the responses and demographic information.

I carried out a survey/series of interviews where I asked people to…Can you help me to analyse the responses? I’m looking for key themes, supported by quotes directly from the data.

Are there any correlation between the responses to Q3 and the backgrounds of the various participants?

How do you think I should present these insights to the team? Do you think there are any other insights that might be interesting to them?

Of course, you need to be careful of inaccuracies or exaggerations here. In my experience, the process of cleaning and organising the data beforehand should help to give you a sense of trends and supporting quotes to look out for in the generative AI’s responses.

That number seems a bit off, I counted more/less.

I don’t agree with that theme being very prevalent in the data. How did you arrive at that?

Are there any quotes that did not fit into a theme but that you think would be important for the team to know about?

The content is good, but I want you to change the format so that the supporting quotes are placed after each theme.

Unlocking New Methods:

I recently completed a project where I mapped hundreds of product features to individual user tasks and the jobs-to-be-done (JTBD) that these tasks fall under, along with the D365 apps our personas use to complete them, the hats they wear while doing so and the types of AI that would benefit them. This enabled me to create an Excel spreadsheet that my team can filter to see which types of AI would have the biggest impact on our users, and which personas and JTBD would be most affected.

Due to the sheer number of apps and personas involved, this type of project would have traditionally required hours and hours of workshopping and async communications with several domain-experts (typically PMs), and likely would have failed due to the overwhelming amount of information and opinions on it. However, I was able to feed the information into an internal generative AI tool (similar to ChatGPT), ask it to map it all together and then get domain-experts to review this instead. Of course it took time to figure out the right prompts and to get things working consistently, but by documenting my learning and prompts, other teams could likely achieve this in a fraction of the time.

Simplified versions provided here:

[insert hundreds of ERP features/tasks defined by the customer success team].

Can you create a number of ‘hats’ (Analyst, Data Orchestrator, etc.) that users might wear as they complete these tasks?

Which types of AI (automation, content generation, etc.) would support each of these hats?

Here are the personas and JTBD we’ve already researched. Can you map each task to the relevant persona, JTBD and hats they might wear as they complete them, as well as the types of AI that would support them?

I mentioned that I turned this into an Excel file that my team could easily manipulate to make data-driven decisions. In reality, the file ended up being a little too big and technical for people to use, so I typically manipulated it for them and provided the insights. However, generative AI has helped to save the day in this regard too, as I recently built a custom GPT that allows the team to ask questions about the data in natural language. Now that the GPT understands what each of the Excel columns represent and how it’s mapped together, team members can simply ask questions like “Which AI feature should we focus on next?” and it will tell them, along with guidance on the personas and JTBD to consider when doing so.

Concerns

From what I’ve seen, concerns about AI in UXR typically centre around two topics; concern about our own job safety, and concerns about the accuracy and trustworthiness of generative AI. I believe it will be a long time before AI can replace UX Researchers, if ever. The methods above are examples of how we can accelerate existing methods and potentially even unlock new (or unfeasible) ones, but the core decision-making, data collection and interpretation remains very human. As for research quality and accuracy, I understand the concerns. However, even if AI-enabled UXR is only 60–70% accurate, it’s better than nothing in my opinion, especially given the pace at which we need to operate today. Particularly for qualitative research, the inherent subjectivity makes it impossible to get a study 100% accurate, so I’d rather do four or five studies at 60–70% accuracy than spend that time aiming for perfection in one or two.