UXRConf 2023 — Part 2

UXRConf 2023, one of the most important UX research conferences, was held last week from June 7–9. The sessions from the final day covered a variety of topics, from the impact of AI on the field of UX research to improving data collection on research projects.

I’ve recapped some of the sessions I attended on the final day below. If you would like to read through some of the amazing insights from the first day of the conference, feel free to visit part 1 here: https://medium.com/vaz-titude-of-experiences/uxrconf-2023-part-1-2264ba9cbe8a

How AI models will change UX Research

Savina Hawkins, a senior UX researcher specializing in artificial intelligence, kicked off the first session of the final day of the conference with a talk that outlined how new developments in AI will impact the field of UX research. Savina has over 10 years of experience in both applied and academic research and 7 years of experience specializing in artificial intelligence.

Here are some of the key highlights from her talk:

AI will soon become a skilled colleague you can work with at the workplace (Image Credit: Screenshot from Savina Hawkin’s presentation)

• 🧠AI will become our smartest colleague.
  Given the current capabilities in the latest advancements we’ve seen with AI, it’s likely that these AI models will increase in linguistic intelligence in the near future and permeate into workspaces.
• ⚠️However, these models can’t do everything.
  We have already heard of instances where large language models (LLMs) have produced factually incorrect and biased data. This happens because these models produce statistically likely data based on prior input. And at their current levels they aren’t exactly reliable sources of information.
• ⚡They will impact tasks centered around language input and output.
  This would impact our workflows as UX researchers, but not our jobs, as these models become mainstream in our workplaces. What are some ways this could manifest in? For one, it will become easier to create deliverables as the task of wordsmithing can be passed on to such tools, allowing us to focus more on collaborative decision making.
• 🔮Constitutional AI might be the promise of a harm-free, pro-social future.
  Constitutional AI is a set of ethical principles used to train chatbots so they are more socially and emotionally aware. Once trained, this could result in AI agents becoming mediators, thus enabling human dialog.
• 💡LLMs will begin to write API queries on their own.
  APIs are a standardized way different pieces of software use to communicate with each other. The results of this capability could mean that all tools will soon be accessible through AI agents, and hence the agent would be a single resource to rely on than using multiple tools to explore data.
  What this would also imply is that these agents could run data queries for us researchers, automating the manual effort required in analysis. They might also be able to take in more inputs into analysis than what we are currently constrained to.

My takeaway from this talk was that the use of AI in the workplace is inevitable. Rather than seeing it as a threat to our job functions, it is more important to see it as a tool that can be leveraged to improve the work we do.

If history has taught us anything, it’s that resisting technological advancement is ultimately pointless. And if we don’t take a seat at the table and start weighing in on the design of these tools, we could lose the ability to advocate for the designs that actually (em)power us to do our best work.

Garbage In, Garbage Out? Getting Good At Data Collection

Rachel Ceasar talked to us about reflecting on our data collection processes and made a case for using a scientific method approach to how we go about conducting UX research. Rachel is a UX researcher at the Culture of Health + Tech Consulting and is an Assistant Professor at the University of Southern California’s Keck School of Medicine.

She walked us through some of the elements we should try baking into our process and things to be mindful about as we go about planning and carrying out research. She contrasted the typical UX research process against the scientific method and pointed out how the scientific method allows room for error. She further drove the point home by telling us about how using the scientific method in one of her projects helped her team with finding the appropriate sources of information during data collection.

Key Highlights:

Check for errors at every stage of the process (Image Credit: Screenshot from Rachel Ceasar’s presentation)

• 🚨Errors are possible at every stage of the research process.
  Breaking down the research process into multiple steps and reflecting on each step can help you mitigate possible errors.
  Once you have come up with your research questions, reflect on whether you have the right ones? When creating your protocol, ask yourself if your team has the life experiences of the target population or if it would be better to bring someone in to consult on your work? Are you asking the right questions, in the right way? Are you excluding anyone based on the methods you plan to use to collect data? Are you protecting your participants?

The steps in the scientific method (Image Credit: ThoughtCo.; https://www.thoughtco.com/steps-of-the-scientific-method-p2-606045)

• 🪴The scientific method allows room for error.
  Another way to obtain error-free results, is to give yourself room to make errors. Are such opportunities present in your process? Rachel pointed out that the typical UX research process assumes biases are not present and doesn’t allow room for error. However, the scientific method provides us with the opportunity to make errors and to improve on it.
• 🎯Aim to minimize biases throughout the process.
  Does your process include steps to reflect or refine your biases? Rachel mentioned that she makes use of existing frameworks, such as a social identity map, on her team to identify the biases they bring to their work. Leveraging existing tools can help improve the quality of data you work with.

This talk resonated with things I heard throughout my HCI degree — test and iterate! It made me realize that these are principles you learn about in the design thinking process, and if you come from a background like mine, you are always encouraged to create that low-fidelity prototype and test it before you create something more solid. However, when it comes to the research process, there isn’t a well-defined counterpart to this, and in the industry setting, especially with tight timelines, it’s likely that we don’t plan for opportunities to make errors. In this situation, the suggestion for adopting the scientific method in UX research does feel like a valuable lesson and one that I will try incorporating in my future research projects.

It’s ok to get it wrong and to create opportunities for that.

Researching Personality in Automation

Lauren Stern, Director of Global Insights at iRobot, talked to us about researching personality in automation. Lauren has held various research positions throughout her career with iRobot. Through this talk, Lauren spoke to us on the importance of measuring social perception in automation and if we’re listening to the correct emotional experiences based on the context users are in.

She demonstrated that designing for the personality people create for their robots has business impact. However, most products that leverage autonomy are not designed this way, even though these products are fostering more social thoughts and experiences for us. Below are some of the key highlights from her talk.

Key highlights:

The various stages in researching social perception in robotics (Image Credit: Screenshot of Lauren Stern’s presentation)

• 📏Measuring social perception in robotics is studying a robot’s social role in their interactions with humans, and the reaction of users to it based on the attributes of the robot. Incorporating this into UX research can have business impact as it leverages the fact that people tend to give their robots a personality (like a name/nickname), making for a more intuitive experience.
• 🔍Discovery: In this stage, you are thinking about experiences that don’t exist yet and can thus heavily rely on your imagination. Conducting activities that are not based in technology, such as puppetry (where the robot is a puppet), can help elicit more feedback from end users interacting with it at this stage.
• 📃Definition: At this point, you are outlining requirements that define a product’s direction. Ask yourself, what should users feel when they interact with your product? This can help come up with personality characteristics of the product that function as requirements for it.
• 🧠Model Development & Training: This stage involves model refinement through the ML training process. Incorporating user feedback at this stage can improve confidence in a model, and one way to do so is by collaborating with your data team to get this feedback. A potential activity to use here is device perspective taking, where a participant behaves as a device, letting you know about the ways in which they expect it to react.
• ✏️Design Iteration: As you iterate on your designs at this stage, incorporate social perception feedback into the usability testing you perform. You can do this by asking participants to compare between design ideas using criteria like ‘helpfulness’.
• 📢Ongoing feedback: As you continue to refine your product, test it periodically against the personality characteristics you came up with earlier, and ensure you see improvements against those metrics.

I really liked learning about this framework as it focused on building intuitive human-robot interactions by keeping the human at the center of how the product is to be designed and packing it into the way research is conducted for such products.

It was wonderful to be a part of this conference and to listen to people’s thoughts on such a wide variety of topics, see where our industry is headed and meet more people from the community. It’s definitely served as a lot of food for thought for me!