Stove, Oven, or Washer?

Can different researchers find the same patterns?

QRCA is the Qualitative Research Consultants Association. Its mission is to help qualitative researchers share stories and ideas with each other. Their monthly Qcast covers a lot of ground with a variety of speakers. I had the honor of speaking 13-Dec-2024, titled Patterns in the Noise. (Registration is required, but you can access the recording free.) For more in-depth answers, there is a two-part course on emergent data analysis/synthesis.

Here are four questions from the session chat.

1. How does bias enter into the work of emergent analysis/synthesis?

Background: Emergent synthesis is different from top-down synthesis. In top-down synthesis the groups that form represent concepts already known to the researchers or within the org or the industry/field. Emergent synthesis seeks to get away from the solution, org, industry, and researcher point of view, and instead represent the unfiltered cognition of participants, specifically in relation to the goal, purpose, intention they are addressing. I usually talk about emergent synthesis as a tool to help us resist bias in the knowledge we generate for our orgs. Working from the bottom-up forces us to focus on what a concept means to the participants, not what it means to us. It is for strategic research, not usually tactical research, like evaluative studies.

Yes, bias can still show up with emergent data synthesis. It often enters the work because a team member misunderstands or misinterprets a cultural reference. You can reduce bias by making sure a team member who is steeped in the same/similar culture as the participant does the listening session and especially the data analysis/synthesis.

Because of the nature of our brains, you will never be completely free of bias. This is why our mentors keep saying “keep and open and curious mind.” It gives us a better chance of accepting variety in other’s approaches, and it helps us resist defensive reactions when someone points out our bias.

2. My experience is that even with emergent analysis/synthesis, with different researchers, different patterns emerge. Is that your experience?

In my experience, different researchers come up with basically the same patterns. It’s because I hold certain things constant in my emergent approach, so we can concentrate on fewer variables. I restrict synthesis to only thee types of concepts that are recognizable to everyone on the team:

• 💭Inner thinking
• ❤️Emotional reactions
• 🧭Personal rules

We avoid trying to analyze opinions, preferences, or explanations how something works, or of processes. In the listening session, we get to the root of these, going back in memory to where the inner thinking, emotional reactions, and personal rules are.

The analogy I use is that I can ask you to go into someone’s apartment and list all the furniture, the appliances, and baskets/bins, but not the plants, not the blankets, nor pet toys and bowls, nor anything else. I can go into that same apartment and list all the furniture, the appliances, and baskets/bins, and my list will be pretty much the same as yours, except for what either of us missed seeing clearly.

1. The first step of the process is about listing the furniture, the appliances, and the baskets/bins. In research, I mean inner thinking, emotional reactions, and personal rules. This means grabbing some text from the transcript that represents one particular inner thinking, then grabbing another bit of text the represents another inner thought, and so on. You put them in a list.
2. The second step is untangling things. Like, is the sofabed a sofa or a bed? Or, did the same sofabed appear to you three times, because you kept going past it in your walk through the apartment? You only need it on the list once, so you consolidate the repeats. In research, the participant might come back to a particular bit of thinking or a particular emotional reaction several times, to further explain the nuances of it, or to emphasize it.
3. The third step includes writing a summary of the item, so we don’t have to re-understand or re-untangle it again later. Also, this summary forces us to telegraph the meaning that the person meant to communicate. I use a rigid formula for writing the summary so that it’s really easy to work with all the summaries in the emergent synthesis phase, where we find patterns by focus of mental attention.

When we are both finished making our lists from the same apartment (transcript), we will have the same things listed, except where we misinterpreted what an item was, or didn’t realize it needed untangling.

This sets us up with lists that are fairly reliable.

What about the emergent synthesis phase? The emergent synthesis uses the affinity technique of looking at people’s focus of mental attention. That sounds like it would be open to interpretation?

Again, we can use the analogy of the apartment. The location of the object can imply what it is used for. Likewise, in a listening session, following the whole story of what the participant will show you all the focuses of mental attention that they pass through.

In the apartment, furniture with drawers in a bedroom might be used for storing clothing versus furniture with drawers in the eating area might be used for storing tableware. (You would ask the apartment occupant. Maybe it’s art supplies. 😁)

In a transcript, inner thinking about deciding whether to confront someone who nearly caused an accident has the focus of mental attention for that participant of teaching them about the danger, and an even higher focus of mental attention of making sure they will never repeat their dangerous behavior near anyone else. You know these focuses of mental attention from what the participant is telling you. You are asking.

This means each researcher going through the same recording/transcript has a strong chance of coming to the same conclusions about a participant’s focuses of mental attention.

I do make sure my teams review each others’ work and discuss interpretations. This introduces the next question:

3. Do multiple people work on the affinity mappings at once, or is it best for one person to work on it?

I think the stronger case is when multiple people work on the affinity mappings. If you are a team-of-one, it is good practice to sit down with anyone willing to go over your understanding of participants’ focuses of mental attention. Or hire a contractor to go through all the recordings/transcripts and then review your focuses of mental attention and discuss. You need someone.

Generally I have a team of 2–4 people doing the emergent synthesis. Usually we are in different time zones, so our work takes place serially, with reviews together. There is a variety of ways to collaborate on the synthesis work, so other logistics might work better for your team.

Side note: An important point to remember here is that, with emergent synthesis, each summary that is added can change the meaning of how summaries in an existing group relate to each other. In other words, if I add a summary about deciding whether to report someone to an authority in hopes the authority can cause them to change their dangerous behavior to the group about decide whether to teach a person about the danger of their behavior, and we already have 10 summaries in that group, it might pull some summaries out of the group. Potentially it could split into three groups of summaries:

• Decide whether to confront someone (or if it might result in an altercation)
• Confront someone about their dangerous behavior
• Ask an authority to confront someone about their dangerous behavior

With emergent behavior, the groups will keep shifting as you add summaries, until some of the groups reach “data saturation” and tend not to shift frequently.

4. Is this different from digging for “underlying needs and motivations” from individuals that drive their behavior & attitudes? If so, how?

It differs in the framing. Often qualitative research follows more well-known market research guidelines. Hence much qualitative research is framed by a solution or an idea. If you frame a study by what a person is addressing, their intention/purpose/goal (not “using a solution” or “reacting to an idea for a solution”), then they might be the same.

For example, studies about buying decisions are often framed by what the solutions offer, the features, costs, availability, etc. This limits your understanding of the person. Instead, a study about what a person is addressing, say taking care of their mom while she recovers from surgery, allows for participants to reference much more than a digital solution for monitoring which medications have been taken. It can include mechanical solutions (e.g. pill boxes), social solutions (e.g. stories from siblings about a childhood drug reaction), memory solutions, manual solutions, and also thinking that ranges farther from just the monitoring of medications, such as keeping Mom from getting too depressed, helping Mom do her physical therapy to recover, keeping Mom nourished, mentally active, comfortable, respected, etc. You will collect a bigger range of inner thinking, emotional reactions, and personal rules. You will be able to find focuses of mental attention across participants that will be intriguing to your org. (I might have lost you there, so skip ahead. 😉)

Here’s another way needs & motivations differ. Needs & motivations are are not as clearly recognizable as inner thinking, emotional reactions, and persona rules. In trying to learn more about this throughout my career, I see a lot of variation between teams (and team members) as to what a “need” is and what a “motivation” is. There are circular definitions, but nothing clear and widely referenced. Needs tend to get expressed as explanation and scene setting, and motivations are a mix of preferences, opinions, and generalized inner thinking. Not always, but often enough. It wasn’t helpful to teams trying to truly understand people. They could understand process, but not people.

I came at strategic research with the language of cognitive empathy, so I used the components that form cognitive empathy: inner thinking, emotional reactions, and personal rules.

Do “needs and motivations” drive “attitudes?” What are “attitudes?” What are “attitudes” used for?

I think perhaps “attitudes” has a history in market research. Orgs want clear pointers from market researchers for engaging ways to speak to people about their solutions. They want to be sure of “product market fit.” Ultimately, orgs want to de-risk their strategic and their tactical decisions about the solutions they are making.

User experience pokes it’s head in the door here. User experience is about how a solution helps to support someone who is addressing their purpose/intention/goal. There are lots of solutions a person employs toward this purpose/intention/goal. An org needs to be aware of these and offer something the other solutions don’t — not in terms of features, but in terms of supporting people’s inner thinking, emotional reactions, and personal rules in their efforts to address their purpose/intention/goal. Supporting a broader variety of thinking styles opens innovation and growth, especially if other solutions harm it.

User experience is about how a solution helps to support people’s cognition as they address a particular purpose/intention/goal.

I honestly don’t know that attitudes play in the user experience arena. This is the ground where inner thinking, emotional reaction, and personal rules bring clearer understanding so that orgs can offer differentiated support for what people are experiencing as they address their purpose/intention/goal.

I’m willing to believe that “needs and motivations” drive “attitudes” usefully for market research. 😊 I need to hear from the market researchers. (I have already heard from many ex-market-researchers, grin! 😆🫣)

Hope this helps!