Lean Research — Part 1 of 2 (or “How emotional data can reduce your exposure to PDF research reports”)

(Part 2 of this series, on database design, can be found here)

Hello fellow designer, I hear you’re burnt out on research. You’re right, no one reads anymore. So your manager scoffed when you wanted to show a highlight reel from a recent series of interviews? And yeah, that one quote isn’t actually all that representative of the voice of the user. Accurate user research is really hard. Synthesizing the totality of a user experience is a tall order.

Remember when we all thought data visualizations would rescue us?

Unfortunately, as an industry we’ve also turned our noses up at data. Mostly for good reason. Quantitive data simply cannot currently capture the broad record of our experience as users, as humans. But it still remains, in a lean design environment no one is reading those PDF research reports, creating an awkward tension of somehow over-producing ineffective research artifacts. So how do we solve the problem of efficiently conveying qualitative information in a lean research setting?

I’d like to introduce you to emotional data.

Emotional data is exactly what you’re thinking. It’s data about emotions. It can include things like whether a user feels confident navigating the menu of an app for the first time, or whether a research participant is feeling calm while walking around a public space. Emotional data can be generated by a person, in the form of biological data, literally a physical reaction, or it can be observed by a researcher, by taking note of the way a user is speaking, or by looking for specific behaviors. Emotional data can help clarify and quantify ‘why’ a user acts by providing another channel of information that is easy to analyze.

Emotional data can be a powerful tool for design research, especially as you start to dig into lean or agile UX research.

Emotional data can also be captured through text or visuals, like this emoji-based survey from the fitness app, Runkeeper.

Pretty sure it was 34 degrees out

This screen is from a very cold jog I took in Helsinki. After you notice how slow of a jogger I am, you’ll notice that with one question, RunKeeper is able to learn an important quality about me, the user, without making assumptions or performing extensive correlations.

And that’s the crux. Emotional data is quantitative, but capable of capturing qualitative elements, like language or a state of being.

It sits in this interesting middle space between the density of qualitative, ethnographic-style research and the strict logic of databases, where context can easily be lost.

Instead, emotional data can provide both summary and meaning. For example, using analytics on your website may provide drop-off rates that you can shape into a story and create a user journey from, but those numbers are not necessarily capturing intention, and they’re certainly not telling you ‘why.’

But if we were to add facial recognition to a usability test, we might see that the user is simply bored or instead we might see that she is frustrated, and those two emotions result in two very different user journeys.

Emotional data provides quantitative translations of qualitative research. The advantage in using emotional data in your design research is that it allows you to cut through noise and systematically streamline the thoughts and feelings of many, many users so that you, as a designer can easily integrate that feedback into your creative process, just as you might with usability data. Emotional data threads the needle between anecdotal bias and a lack of context.

Capturing Emotional Data (cheaply)

While one can capture emotional data using biometric devices, or facial recognition, we’re going to focus on a low-cost method: the synthesis of expressive emotions, through categorizing, or coding, of text and behaviors. Let’s take a look at some sample survey responses from participants.

Phrases participants have used

Coding is the act of applying a descriptive label, or a code, to synthesize similar types of text or behaviors. When you’re dealing with language, this method assigns a meaningful category to all of the language you’re encountering, either in interview transcripts or surveys.

There’s a bit of bias to this method, which can be mitigated by having more than one researcher code the list

The words and phrases with an emotional or affective quality to them can be further pulled out of their context, and condensed down to one word, without losing much of their individual meaning. Remember how Silicon Valley was reportedly on the lookout for poets? This is another example of how former English majors can improve your product design cycle.

These key phrases are all condensed down to the word “familiar”

Once you have a list of synthesized terms, count the coded terms to find ratios or volume.

Coding essentially boils down to categorize and count, and is not all that different from synthesis techniques that many design researchers already use. What is different is the focus or lens of emotion, which will affect your initial coding schema and the efficiency of your analysis.

Services like Crimson Hexagon or Radian6 are already doing this at scale using sentiment analysis and natural language processing. But you can borrow their big data methods and apply it to your own collection of qualitative data.

This method applies to behavior as well. For example, last summer I conducted a think-aloud usability test for some new UI elements. I used lookback.io to record video, and when I reviewed the video I looked for participant’s responses, marking them against a rubric for confidence and sentiment. Were they smiling? Did they furrow their brow? Did they ask for help?

Usability test using Lookback.io

That sounds like a metric ton of work, right? Not at all Lean. But I’d like to argue that with a strategic framework for how you’re collecting data, you can significantly speed up the process, while still gathering the important pieces of information. That framework will consist of a clearly articulated hypothesis, and then units of measurement for the hypothesis written out as a rubric.

This rubric captures ability, sentiment, and the emotional quality of confidence

So in this example I coded the user testing videos based on whether the user was able to complete a task, but then I also looked for behaviors that expressed confidence as well as positive or negative responses to the UI elements I was testing. I knew what I was looking for, based on a hypothesis, a product goal, and previous research.

This is really important. Emotional metrics shouldn’t be chosen arbitrarily. Our emotional range is vast. And so these methods only really become useful if you’re employing an informed focus.

A Final Word

Emotional data is an emerging field, and I believe it’s coming up in the next cohort of tools that we’ll all be using. In these early days though, we have the space, as creatives and as designers, to play with this information and shape how it will be used. We can build tools to harness emotional data within design, or even take a moral stance on how we use such deeply personal information.

In the next part of this series I’ll talk about using emotional data in the context of a design research system, and that’s when things get really fun. Stay tuned.

One clap, two clap, three clap, forty?

By clapping more or less, you can signal to us which stories really stand out.