A Practical Guide to Using AI for UX Research Analysis

UX / Design Research as a field is grappling with Artificial Intelligence (AI) and its uses at the moment. As a community, UX researchers need to be keenly aware about weighing up the ethics and pros/cons of using any research approach or tool. This article explores a number of key factors UX Researchers would be sensible to consider when using AI for analysis in their work. We’ll also step through some ideal use cases for an AI based UX research assistant.

While the introduction of AI into your work may seem like a big change, just remember, UX researchers are experienced at managing new technology; it’s our job.

We are also always juggling conflicting obligations about how we do our research. Whether it is the push of the limited time we have available to us in agile environments to design, run, analyse and report on our research; or the pull of our ethical obligations to our research participants — we often find ourselves as the meat in the sandwich, conflicted by our competing obligations. Using AI is a tempting tool to ‘speed things up’, we just have a few things to consider beforehand.

Considerations, considerations

So have you had a discussion about using AI with your Product Owner (PO) lately? Or maybe you just feel the pressure to increase the ‘productivity’ or ‘efficiency’ of your research approach by using AI. There are quite a number of considerations you’ll need to take into account before relenting to the push and pull to take the plunge.

First though, as with everything we research, the context of use is key to understanding what considerations need to be assessed. Some professional networking groups for UX researchers on places like LinkedIn and Facebook have been talking about using AI to do all sorts of things: from producing personas, writing reports, transcription of research sessions plus lots more. In this article, I focus on using Generative AI (like ChatGPT) to analyse primary qualitative research data from your research sessions in order to uncover themes and insights.

When you need to analyse a huge amount of qualitative data, it is tempting to simply grab and drop these data into ChatGPT to spit out some insights. Generative AI can do this, but what things do we need to consider before doing so?

Here are just a few:

Organisational risk appetite: Is your organisation a bank, insurer, or government?

Some organisations will be more open than others to using Generative AI tools for research analysis (and certain AI vendors than others) due to issues these tools have with handling data. Get your legal, privacy, cyber etc. experts involved before giving it a go with the organisation’s, customer’s or research participant’s data.

Informed consent: Do your consent forms inform research participants how the particular AI vendor will be recording, storing and using their data?

Some vendors will use this data to train their AI models, some may have humans reviewing the data and others may use it for other purposes. Each AI vendor, as well as their different products, can vary quite substantially. You need to let potential participants know in the consent form how their data will be handled and used.

Deidentification: Even if participants are ok with how the AI vendor will use their data now and in the future, you need to make sure that the information entered into the Generative AI tool is free from personally identifiable information (name, date of birth or even a combination of personal information that might identify participants). Deidentification should be part of your standard practice as a researcher regardless.

Ability to input — exclusion of ‘unsafe’ topics: Once you enter data into Generative AI tools some may block you from being able to analyse it due to their built in moderation system. Recently, we were analysing data about the experiences of veterans (the good and bad) and ChatGPT wouldn’t allow us to analyse the data or send some of the analysis to colleagues due to the nature of some vivid descriptions of war. Some tools allow this to be by-passed but using Generative AI analysis might be harder if you are researching tricky, sensitive but legitimate topic areas.

Ability to input — number of tokens: Some Generative AI tools are limited in the number of tokens (kind of like parts of words, numbers and punctuation — the AI version of character limits) they can process at once. If you have long transcripts then you may need to break this up over several entries. While this has improved recently with ChatGPT, it does have a limited memory of these tokens so the AI tool may forget what you entered earlier, unknowingly biasing your results.

Ownership of input and output: The ownership of both the input (what you enter into a Generative AI tool) and the output (what the Generative AI tool responds to your input with) is not always clear. In many cases, it rests with the Generative AI tool. So if you build your product with analysis provided by an AI, does that company have a stake in your product?

Overlay this with the unclear status of how Generative AI tools could be trained on copyrighted material and this could be a problematic aspect to consider.

Accuracy of output: AI can be super confident about things it makes up. I once asked Google’s Bard for the latest news about a big pending financial merger. Bard said the merger was approved by the regulator on X date, but it had in fact been blocked on that date by the regulator. Not only that, Bard outlined the next few dates of news stories about the merger even though these events hadn’t occurred yet. The moral of the story, we can’t necessarily trust the output of these tools as yet.

Bias and misunderstanding: Bias is built in via the data selected to be used to train AI. For instance, the Generative AI tool may not pick up on culturally different uses of terms in its analysis of your research data. For instance in Australia terms like ‘sick’, ‘deadly’ or even something commonly misused like ‘mickey mouse’ will be interpreted using a US data framework. At best it is possible you may miss some insights as a result and at worst it misunderstands this cultural use of the terms and goes off on a well meaning tangent, skewing your analysis. Eg. The content of your research was not ‘sick’ unwell people, but a ‘sick’ amazing new skate park facility.

Environmental: It can use a huge amount of energy to run an AI program and its analysis. It’s been reported training GPT-3 with 175 billion parameters, consumed 1287 MWh of electricity and resulted in carbon emissions of 502 metric tons of carbon dioxide equivalent. Equivalent to driving 112 internal combustion engine powered cars for a year. A single request in ChatGPT is said to consume 100 times more energy than one Google search.

Generative AI tools should be powered by certified green energy (wind, solar, hydro, and not biogas or cooled by the oceans). Bard says “Google is committed to achieving 24/7 carbon-free energy for its data centres by 2030” but is this accurate, telling me what I want to hear, and will this commitment eventuate?

Fingers crossed it will.

As you can see, it gets a little messy and these tools have plastered some of these shortcomings on their tools to warn users.

Wild west? Well, yes.

However, with every day comes new advances and refinements of different AI products’ capabilities and their accompanying business models. It seems, from my perspective, to have been the wild west of late and AI companies appear to be using somewhat of a move fast and break things approach due to each firm wanting to establish a first mover advantage over others. (sidebar: companies like Apple have shown the first mover doesn’t always create the best and most sustainable business model. They have shown that waiting, learning and refining before betting big can be very successful so I challenge the gold rush mentality we currently seem to have in the AI space ¯\_(ツ)_/¯ ).

However, in a gold rush situation, things like safety, ethics and effectiveness can be less of a focus. So as a UX researcher with your mind often required to address these aspects, what are you to do?

Some considerations have been met by some vendors

Image: AI can be seen as a bit of a mysterious black box, and it is to be honest

At the moment there seem to be limited offerings covering all the considerations I’ve mentioned and if they do then it can be at varying degrees. However, I do recognise that organisations used to working in the B2B and B2G environment, such as Microsoft, seem to have covered a number of key areas with their Azure OpenAI product*. Azure OpenAI is essentially a ‘ring-fenced’ version of ChatGPT where it seems the lawyers have gotten involved and said ‘we need to get a bit more serious about this’.

Where the free version of ChatGPT retains the inputs and outputs of its interactions with customers, and could use this for training their model, Azure OpenAI clearly states

“Your prompts (inputs) and completions (outputs), your embeddings, and your training data:

• are NOT available to other customers.
• are NOT available to OpenAI.
• are NOT used to improve OpenAI models.
• are NOT used to improve any Microsoft or 3rd party products or services.
• are NOT used for automatically improving Azure OpenAI models for your use in your resource (The models are stateless, unless you explicitly fine-tune models with your training data).
• Your fine-tuned Azure OpenAI models are available exclusively for your use.”

There’s even a way to exempt human moderation of the content that is entered into its system. Now, I am not endorsing the use of Azure OpenAI, and you need to do your own homework, but their product and business model is an evolution of what has come before it.

The environmental aspects don’t seem to have been covered by Microsoft yet (between 2025 and 2030 apparently according to their statements) so, as mentioned earlier and just like Machine Learning, Generative AI will still use much more energy than your everyday Google search. So I challenge Generative AI operators to use accredited green power from sustainable sources (i.e. more solar, hydro and wind and not biogas nor ocean cooled servers)

So if we were to use Generative AI for analysis, what’s the use case?

Some people have advocated using Generative AI to produce themes and insights from collated data. But ask yourself, how familiar would you be with the data if you left it to the machines?

Data analysis is a great way to really familiarise yourself with the data plus how do you know if the insights are actually accurate (remember even Generative AI providers warn users not to rely on the data as it may be inaccurate).

One use case I see for Generative AI and UX researchers to work together on analysis seems to be where researchers analyse the data, produce themes and insights then check in with their Generative AI research assistant. This way you can see if Generative AI has any different ways to look at the data or additional themes. It can also be a great way to find different vocabulary or phrasing for your insights. When someone is in a UX team of one especially I can see the benefits. However, again, checking the data afterwards to see if these additional insights hold up to scrutiny within the data is still an important step.

Overall, there are some important considerations when looking to use Generative AI tools to analyse your research data and with some of the established players using their experience to evolve the product and business model AI could be a realistic research assistant but user beware, do your homework to make sure the tool matches the need (like any good UX researcher would 🙂)

*I am not endorsing the use of Azure OpenAI, and you need to do your own homework etc etc