We Shouldn’t Blame Data for Bad Campaign Messaging
Data can’t salvage bad campaigns, but it can help good ones. And for us to recover after 2016, we’re going to need all the help we can get.
In early 2014, the Democratic National Committee began building a new data science team at party headquarters, and I had the privilege of leading that team. Its creation was part of Project Ivy, the DNC’s plan to leverage data and infrastructure built by the Obama campaign to support downballot candidates and prepare for the 2016 presidential race. Our team had many accomplishments during the years that followed, but we were never given the chance to fulfill our most ambitious goal: making data a key part of the Democratic Party’s strategic planning.
I say this not to air old grievances or poke at wounds that have barely begun to heal. My team and I kept a low public profile and labored behind the scenes, and these days I prefer to look forward rather than back. But it is hard to stay quiet when a string of news articles and opinion pieces in recent months have blamed the use of data for our losses in 2016. So for the sake of preserving the very real progress we have achieved, I want to offer a response to these accusations.
The common premise of these arguments is that Democrats lost because we chose to rely on data instead of developing a messaging strategy that would appeal to voters. But this claim fundamentally misinterprets the role of data: data is a tool for campaigns, not a strategy. Absolutely nothing about a data-driven campaign precludes the development of a winning message, and it would be absurd to try to run a campaign without one!
(Not to say that this doesn’t happen — in politics, the absurd is a pretty regular occurrence, obviously.)
In fact, the people who work with campaign data love good messaging as much as anyone, and can even help in that effort. We can run surveys to understand how the public sees the world, test messages to see which resonate most with voters, then help campaigns tailor those messages for particular audiences. This is not an alternative to the way messaging was done before the Obama campaigns, it’s just an evolution. The field of modeling and experiments grew directly out of more traditional survey research, and many of the people who have led campaign analytics programs (myself included) came out of the polling industry and see our work as a natural extension of that methodology.
To show what this looks like in practice, I’ll share an example from my own work. In 2012, I was part of the team that developed the strategy behind the campaign for against Proposition 32 in California. You probably never heard about Prop 32, but it was actually the second largest campaign in the country that year: at $135 million in total spending by both sides, it dwarfed every Senate and House race and was only eclipsed by the presidential campaign. Working as part of a team coordinating the efforts of dozens of labor unions, we built a data-driven program of targeted messaging that directed a budget of more than $60 million and produced more than 20 million individual voter contacts.
During that campaign, our challenge was to fight a deceptively-worded measure backed by the Koch brothers and other wealthy conservative donors. The proposition was presented as an ideologically-neutral, good-government policy, banning both corporations and unions from spending payroll-deducted funds on political purposes. Of course, only unions actually use payroll deductions (in the form of membership dues) for advocacy — corporations simply never put the money in their employees’ paychecks in the first place — so the real impact would have been to cripple unions politically and take away a major resource for Democrats.
In early summer, we were down by over 25 points in some polls, but in November we won by more than 13. How did we do it? Well, we can’t take credit for all of that shift — opinions on ballot propositions are almost always more fluid than candidate preferences, since the stabilizing cue of partisanship is not immediately obvious. But we built a campaign based on messaging that was proven to be effective, customized the targeting of these messages to fit different audiences, and collaborated with partners who built field and advertising programs to deliver these messages to the most receptive voters through the right channels for each individual.
Our campaign’s message development started with traditional polling, which helped us to gauge the opinions and attitudes of the electorate. Working with voter file-matched samples, we were able to supplement those responses with other information about voters, and use those combined resources to develop a set of draft messages about Prop 32. We then ran a large-scale modeling survey which included experimental tests of the effectiveness of each of the proposed messages, both independently and as a counterargument to messages our opponents were likely to use. That helped us to narrow down those messages to the most effective set, and with additional data from another survey we were able to dig down into those results to identify the best messages to use with particular voters.
We found, for example, that messages about the partisan impacts of the proposition were the most effective of any we tested with registered democrats, but had no effect on unaffiliated voters. Messages informing respondents about the measure’s backers, however, did not have the same partisan split, but did have a substantially larger impact among the most politically-engaged voters. We then built persuasion targeting models to identify voters most likely to be moved by the specific messages we were using, and were even able to use our models to select the message most likely to work with each voter. These modeled predictions and overall patterns were used to develop an optimized outreach strategy that combined highly-effective messages with fine-grained targeting. In post-election analyses, we found that the direct impacts of the campaign helped to change the minds of hundreds of thousands of voters in that race.
The Prop 32 campaign was a great example of how data and messaging can work in harmony with one another, and serves as a good contrast to what some are taking away from 2016’s results. The idea that data and messaging are somehow mutually exclusive is not only incorrect, it’s entirely backwards: campaigns are most effective when they combine strong messages with data-driven targeting. That was a winning recipe in 2008 and 2012, and it’s what our team hoped to repeat last year, not just in the presidential race but for Democratic campaigns up and down the ballot.
But that relationship only works when we’re asked for our help, and in 2016, those responsible for the party’s messaging were never all that interested in what we had to offer. In more than two years at the DNC, I had exactly one face-to-face meeting with any of the party leaders whose offices were thirty feet above ours. That meeting was in the wake of the 2014 debacle, when our team was asked to produce an internal report on what went wrong that November.
Our analysis was done in advance of the official postmortem that was shared with the press, but as far as I can tell, none of what we came up with ever made it into that report. In our recommendations from that report and subsequent proposals, we pushed for the DNC to launch an ongoing, nationwide survey research program. That program would not only have provided the party with updates on where specific races stood, but also would have also helped to develop a stronger party-wide messaging strategy at the national, state, and even local levels.
From our defeats in 2010 and 2014, we knew it was a challenge for us to win races without Barack Obama’s name on the ballot, and so we strongly recommended that the party do more to understand voters and find better ways to connect. We also envisioned that a DNC-led program would help individual candidates and state parties run their own surveys and learn more about their states and districts, so they could better speak to the things their electorates cared about most. This was something that we had failed to do well in recent years, too often focusing our messages on our own priorities rather than those of our constituents, and ultimately leaving even our most partisan supporters lukewarm about our candidates.
Surveys alone would not have been enough to reverse this trend and build a winning message, but they would have given us somewhere to start, and that would certainly have been better than the alternative. In the absence of data on public opinion, we are forced to rely on instinct and anecdote to develop our messages, with no way to tell whether they are connecting with voters until it is too late to change course.
(And yes, in a perfect world, each of our candidates would come with a voice and message that wins over voters without ever having to field a poll or hire a consultant. But a Barack Obama or a Bill Clinton — or a Ronald Reagan or even, dare I say, a Donald Trump — only comes around once every decade or so. The rest have to learn to speak to voters somehow, and good data can help them do so in a way that reflects their constituents’ views and priorities.)
Our team’s proposals from 2015 were certainly not the only option for the party to build a data-informed messaging strategy, and we saw them as just the start of a much longer conversation. We wanted to help, but unfortunately, that conversation never even got off the ground. Not only were our proposals tossed aside, but we never had any substantial conversations after that. The distance between our programs was so great that when the Wikileaks scandal broke last summer, we might as well have worked for a totally different organization — there was not a single reference to our team or any of its members in even one of the stolen emails.
I do not want to overstate the importance of this rift. DNC Comms was just one of many messaging operations across the party, just as our team was only one of many data and analytics shops in Democratic politics. But we did have as good an opportunity as anyone to try to do something about the party’s messaging problem, and we let it slip away. Would it have made any difference? I honestly do not know. But it sure would have been nice to try.
In any case, all of the DNC’s operations were taken over by the presidential campaign in July, so they were in charge of the party’s messaging from that point on. (The wisdom of that arrangement is a topic that merits its own post, which I’ll save for another time.) Having not been in the room, I have no special insight on how data may or may not have been used by the campaign’s messaging strategists. But I do know what the DNC’s more limited data told us prior to the takeover, and I also know what their messaging strategy looked like in practice, and I can say that the former did not obviously recommend the latter.
The campaign’s own data might have looked different from ours, or they may have reached a different interpretation of similar data, or they may not have even looked at data at all. I don’t know enough to say which. My point is not to second-guess the campaign’s strategy in hindsight — that horse has long since turned to glue — but to say that the use of data should not be held responsible for ineffective messaging. Data can be used to produce good messaging, it can be used for bad messaging, and it can be used for no messaging at all. The data itself is agnostic on that decision.
In 2016, the messaging behind one big data-driven campaign (and many smaller ones) was not enough to win. And that sucks. It hurts to lose, and it is scary to think of what comes next, and we are angry because it feels like we should have won but someone somewhere messed up. But while we struggle to process what happened, we cannot misplace blame on the tools that were used in the course of that failure.
When a plane crashes, we do not abandon air travel for good and start building more ocean liners. (We would not even have those if we took the same lesson from the Titanic.) Instead, we investigate what went wrong and work to fix it. And the same goes for campaigns: losing a campaign is a miserable thing, but it is also an opportunity to learn from our mistakes and get better on all fronts.
In 2016, the Democratic Party’s messaging strategy did not work out like we hoped, and the use of data did not fix that. That is all we know for sure at this point. The relationship between those two aspects of the campaign is still a mystery for all but a handful of people, but it is clear that there is room for improvement. Should we start by collecting data that helps us to better understand how voters think and what they are most likely to respond to? Or should we flip a coin to determine which data-free messaging guru to invest the party’s future in? Is that really even a question? I certainly hope not.
So now, as we look ahead and start to plan, we have to recognize that the “debate” between data and messaging on a campaign is a nonsensical one. The reality is that we are not going to win in 2018 and 2020 with either data or messaging. If we hope to turn things around and start winning again, we are going to need to do both.
