How to Stop Mismeasuring the People

A guide to research-driven storytelling for progressive change

Katniss, rebel hero of The Hunger Games, is the deep story of many Millennial voters. 
Find out how to find that out, and why that’s important for progressive campaigns and causes.

Here’s a distressing fact: Audience research methods used for progressive causes and candidates are lagging behind corporate America’s advanced marketing research practices. It’s distressing (to me, at least) because it indicates progressives aren’t pursuing every fair advantage we can get. And that’s shameful, given that we have to overcome the right’s undemocratic advantages — gerrymandering, voter suppression, constitutional provisions that favor rural votes over city people’s more numerous votes, not to mention the occasional help of KGB hackers.

For the past 20 years, sophisticated corporate marketers have been gaining market advantage by improving their ability to find the often-unconscious, emotional stories that actually determine their audiences’ views of reality. This is only possible by integrating traditional research with newer forms of psychology-based and cultural research, as well as so-called big data analysis aimed at understanding and quantifying what the audience is saying. But political audience research and opinion polling have made little similar progress.

The gap is now so big that the traditional approach to research sometimes seems stuck in an antique century of hunches, with consultants brainstorming messages based, at best, on incomplete data — or, at worst, no data at all — about their audiences’ deep mindsets. At the same time, sophisticated consumer businesses have moved into the near-future by integrating newer technologies and disciplines — neuroscience, pop-culture analysis, story collection, online content analysis and so on — that actually make it possible for strategists to understand the personal worldviews and unconscious emotional associations that are the driving forces of human behavior.

The last few years have seen a rapidly growing understanding that traditional ads, dry policy summaries, and talking points simply do not engage audiences — especially online — the way storytelling does. Yet, virtually no candidate or cause uses (or knows how to use) the modern research tools that actually help creative people frame and structure narratives. This means our best political communications strategy — telling effective stories — is left unsupported by the modern research tools and protocols that help determine which narrative frames and stories are going to be most effective in activating an audience.

Of course, lightning strikes occasionally. Naturally gifted storytellers emerge with authentic personal narratives that audiences embrace instantly. The waitress-to-congresswoman tale of Alexandria Ocasio-Cortez is the best recent example. And, of course, there was Barack Obama. But, as we learned from Obama, a movement can’t rely on candidates and causes having true stories that closely resemble rags-to-riches Hollywood fantasies.

Right now, Trump’s irrational careening from one racist, sexist idea to the next, and the Republicans’ complicity, is helping rearrange the American political landscape in favor of progressives. In the 2018 midterms, backed by big majorities of women and sizeable majorities in cities and suburbs — and powered by the highest turnout rate since 1914 — Democrats won the popular vote for the House by 53.1% to 45.2%. This 8.8-million-vote edge out of 111 million total ballots cast was the biggest midterm margin in history.

Finding the stories that unite us

The way to capitalize on this moment is to assemble the very large coalition needed to make fairness and equity permanent features of our democracy. This is a feat that ultimately will require the broad consensus needed to end partisan gerrymandering, stop voter suppression, overturn Citizens United, make Roe v. Wade unassailable and get around the Electoral College so our presidents are elected by a majority of voters — just to cite a few examples. But the Democratic Party and progressives remain ill-equipped to erect such a big tent because the research tools being used are totally inadequate. The questions all tend to be binary, partisan policy queries — as if there were only two ways to think about any given aspect of life. Worse yet, demographics continue to be viewed as important in politics, when psychographics have been proven far more useful.

Most of our pollsters and researchers work hardest at finding differences; segmenting audiences into smaller and smaller slices based on income, age, race, geography, education. The underlying assumption of the methodology is that people are rational actors, easily split into groups based on measurable differences. Another assumption is that all people think their way to decisions in similar and similarly rational ways. Yet, we know that most decisions are based on emotional reactions and those emotions, generally, cross demographic lines to be found in people from very different backgrounds. Despite this core truth about human beings, there is little or no focus on finding the unconscious, emotional forces that frame our values, define our views of reality and drive virtually all our political decisions.

Research aimed at finding what we agree about, as opposed to what divides us, is all too rare. Put simply, our research approach gives us no insight into the big, unifying narratives that could bring together 60% or more of the country.

So what are we missing? What do we need to not just win, but win big? To not just flip the House, but flip the culture, upend the system, and create an entirely new normal?

No One Can Tell You Their Unconscious Motivations

Current political research is almost totally dependent on 20th century quantitative and qualitative research methods: national telephone polls and in-person focus groups where, for the most part, rational questions are posed in a way that demand binary answers (No, I oppose that policy.), or ratings on some artificial sliding scale (My attitude toward Donald Trump is “very unfavorable.”). There’ve been some steps forward with observational research — like The Peoria Project, which segmented people based on their values. But too few campaigns are applying these or any other advances.

Traditional research methodology is extremely useful. But everybody in the research business is painfully aware of the limitations of these methods. Polls, for instance, are severely limited by the need for questions that make the answers quantifiable. No such question can be sufficiently open-ended or complex to include an answer that the pollster never anticipated. So the information collected may be valuable in the way it quantifies the answers, but the answers will always be limited by the imaginations of those posing the questions.

Focus groups, of course, have different problems. First, we all know the answers we hear may or may not be representative of widely held public opinion. Yet, even the most sophisticated of us tend to act as if they were. Otherwise, why would we spend hours in dark rooms behind one-way glass, eating bad food in far-flung places? Another well-known focus group problem is the way in which group dynamics function to give ownership of the group to the loudest and most aggressive person, not the one who most closely represents the most widely held opinions and beliefs.

Both these methods are indispensable. But no business with any resources would dare formulate a core communication or bring any product or idea to market today based solely on focus groups and polls. Neither should any progressive cause or candidate. Especially not if the goal is to tell the emotional story that will be most effective at influencing the behavior and beliefs of the audience, both in the short term and the long haul.

The real problem with using research that completely depends on asking people questions is this: People can only tell you what they are conscious of. And any good psychologist can tell you that people’s beliefs, reasoning, and behaviors are most powerfully influenced — and, in the end, explained — by associations and emotions that are partly or wholly unconscious, which is why the vast majority of people cannot explain their motivations to anyone, least of all to pollsters.

Informing campaign storytelling with the audience’s own stories

We find the most motivating emotional narratives in the stories that people tell to themselves about how the world works and why it works that way. Sometimes people are aware of their internal stories. Most often, not. One informed view of such interior narratives can be found in sociologist Arlie Russell Hochschild’s book “Strangers in their Own Land,” a voyage into what she calls the “deep stories” of impoverished Tea Party supporters in western Louisiana. This excellent book became a momentary fixation of many progressives when it appeared in 2016, just in time to help explain why people might vote for Donald Trump.

Hochschild’s approach has two problems for candidates and causes. First, it took her five years of extensive interviewing and in-person observation to construct her conclusions. Second, while she created an important, authentic and singular explanation for her subjects’ seemingly inexplicable and counterproductive behavior — behavior clearly not in their own best interests — she offered no suggestions about how to influence their beliefs for their own good. (This is not a criticism of her, merely an observation. If you haven’t read the book, you really should.)

Fortunately, there are quicker approaches available that are crafted to help campaigns and causes tell their stories in ways that will help create the most powerful connections with audiences. There are various applications of implicit association testing (IAT). And there are differing ways to locate and measure implicit associations, including what’s called “indirect elicitation” work that seeks initial responses from people in non-verbal forms such as pictures, for example. There are cultural audits that probe for the common threads of the pop-culture narratives being consumed by mass audiences. There are a wide range of “story collection” approaches involving in-depth, one-on-one interviews. And more.

The important thing is not so much which of these methods are chosen to measure implicit associations. Rather, as my friend and colleague Frederika “Riki” Conrey, the social data scientist, says, it is that we “agree that artifacts of the implicit system — affect, emotion, and unconscious association — are the important things to measure.”

Exploring the Interior and Exterior of Audience and Culture

IAT is one approach to exposing our implicit associations. Another is a methodology developed at Harvard in the 1990s by marketing professor Gerald Zaltman — ZMET or Zaltman Metaphor Elicitation Technique, which uses imagery and lengthy psychoanalytic interviews. (Here’s a link to overviews of IAT and ZMET,) Both approaches look deep inside people’s minds to find the unspoken feelings and unconscious associations that most profoundly affect them.

A cultural audit looks in the opposite direction — outside the individual to the most-consumed stories of pop culture. Those external stories — films, music lyrics, books, magazines, paintings, poems, newspapers and so on — permeate and define the shifting fabric of our culture, inevitably forming the context for and reflecting, in a variety of ways, the interior stories through which we each make sense of the world.

Taken together and carefully integrated with polling and focus groups, these methodologies form a new, holistic approach to audience research that can significantly increase the reach and impact of any progressive campaign with storytelling that intersects the audience’s own stories.

One excellent example of applying the ZMET to progressive politics is some recent work on voting done by Olson Zaltman (OZ), a marketing research firm founded 21 years ago by Professor Zaltman at Harvard and Professor Jerry Olson at Penn State. Since then, OZ has conducted studies for many of the biggest corporations in the world. Their proprietary methodology focuses on using images collected by each research participant as the starting point of lengthy, complex conversations based in the techniques of psychoanalysis, that locate the deep emotional metaphors each participant uses to understand the topic being studied.

OZ put Millennial voters and non-voters through its proprietary process. While they did not do enough interviews to arrive at statistically reliable answers, their findings illustrate how valuable the approach can be. OZ found, for example, that young non-voters seem united in feeling that voting distances them from their own communities. Non-voters expressed “anxiety over getting muddied and socially disconnected by voting,” OZ reports. Voters, to the contrary, saw casting a ballot as a heroic act that connects them more deeply to their communities. Voters, OZ tells us, “consistently expressed a rebellious hope in the face of overwhelming odds” that voting, while held in wide disregard by Millennials, might still produce positive results.

Slide from Olson Zaltman’s demonstration study of 2018 midterm Millennial voters.

Interestingly, underscoring the connection between the interior stories in their minds and the external stories in pop culture, voters consistently described themselves as identifying with the epic heroes in hits like Stars Wars or The Hunger Games. One typical young voter told OZ:

“I identify with Katniss [the young hero of The Hunger Games] trying to give the people in my community a little more dignity and love and trying to make the world a better place. I feel like voting is an expression of this, and I feel connected to my community and making things better, however difficult it may be, when I vote.”

Clearly, these kinds of insights, metaphors, and storylines are directly helpful to campaign creatives who must craft narrative frameworks with the highest chance of impacting the audience’s behavior — in this case, voting. The language and images that emerge from these non-traditional research methods can be tested, ranked and honed using traditional polls and focus groups. At this point, this kind of iterative, multi-disciplinary process offers the only proven way to locate authentic stories and guide the way they are structured, designed and written to deliver the greatest potential impact on behavior.

This kind of iterative, integrated research approach has been in use for more than a decade for commercial marketing. It is just now beginning to be tested in progressive communicating.

So far, the most extensive use of these approaches has come from philanthropist and progressive activist Tom Steyer’s organization, which was successfully engaged throughout the 2018 midterms on numerous fronts. Steyer’s teams pressured the Democratic Party establishment to impeach Trump, worked to increase Democratic turnout in key districts among youth, ran a parallel get-out-the-vote campaign with older voters supporting impeachment, and looked beyond to find the most powerful and unifying progressive narrative for the 2020 elections. While much of what Steyer’s group learned can’t be released yet, the approach outlined in this essay located the strongest narratives — stories that consistently moved quantitative results in the right direction — toward more favorable audience rankings, more persuasive impact, more changed or changeable minds, more volunteers and more votes — by double digits. Steyer is the Democratic Party’s biggest publicly identified donor, putting at least $120 million to work in the 2018 midterm cycle.

The key is for researchers and political strategists to pay attention to the ideas and storylines that light up all the relevant research approaches. What’s needed are concepts that are deeply embedded in the audience’s version of reality, spark emotional identification with a campaign’s substance and goals, climb the charts of pop culture, draw people together in focus groups and, ultimately, move the numbers in polls. These are the winning ideas.

Almost eerily, this multidisciplinary approach unfailingly seems to find the most effective, relevant way to connect with an audience.