Your Data Will Win the White House
Predictive analysis expert Ken Strasma on how data crunching can mean the difference between the Oval Office and political oblivion
Ken Strasma is a pioneer in the field of predictive analytics in high-stakes Presidential campaigns. He served as the national targeting director for John Kerry’s presidential campaign 2004, and again for President Obama’s 2008 campaign. Strasma’s firm, Strategic Telemetry developed the predictive models that propelled then-Senator Barack Obama to a dramatic victory over Hillary Clinton in the Democratic primaries, and then a historic winning campaign for the White House.
In 2012, Strasma launched HaystaqDNA, a firm that has been on the forefront of bringing political-style microtargeting to commercial clients. HaystaqDNA has worked for multiple Fortune 500 companies, and boasts a commercial client list with a combined market capitalization of over $600 billion.
With the race for the White House reaching boiling point, Strasma came to Second Home to discuss how the use of data can mean the difference between the Oval Office and political oblivion.
It is kind of amazing even to me, when I look at scores predicting people’s issue attitudes, that things that you would not expect to be predictable, are.
I come from a campaign management background, and when I was working in political campaigns I found there was a lack of targeting data — the campaigns were just speaking to everyone. So that’s what prompted me to, first, work for an organisation in Washington DC that provided geographic targeting to campaigns. In 2003 I left to form my own company, and was fortunate enough to sign up with John Kerry’s presidential campaign in the summer of 2003, which was a great case study for political micro-targeting, individual-level targeting. At the time, everyone expected Howard Dean to run away with the democratic nomination — he was way ahead in terms of fundraising and in the polling.
The first contest in the US nominating system for president is the Iowa Caucuses. It is an odd beast because it’s not something where the winner is the person who gets the most votes, the winner is the winner of 1,800 separate contests that happen that night.
So in each precinct in the state of Iowa, which is roughly 1,800, an election is held, and they get to allocate a certain number of delegates to their stay convention, which then elect the 40 delegates that go onto the national convention. So there really is a premium on having really precise targeting at a very small geographic level.
As you can imagine, when you’ve got more than 1,000 precincts, there’s no way you could actually conduct a poll in each one of those areas. But what we could do was conduct a large sample poll state-wide and use those results to build predictive analytics models that predicted how everyone else on the voter file — the list of all the registered voters in the state of Iowa — would’ve answered those questions if we’d been able to survey them.
So we use a number of statistical machine-learning algorithms from basic regression and segmentation through nearest neighbour and neural networks and some of the other machine-learning algorithms, combine those together — because we’ve always found that we get more robust models when we use multiple different algorithms, any predictive model is going to have some level of signal and some noise, error and correct prediction — if you’re using significantly different modelling algorithms, the error will cancel out and the correct prediction will begin to reinforce itself — it’s why we combine these multiple methodologies. At any rate, we were able to correctly predict how people were going to vote in the Iowa caucuses and move resources around where we needed to. When you’ve got a system like that where there’s a limit to the number of delegates you can elect, it’s important to not waste your votes. If you’ve got a precinct that elects three delegates, it doesn’t matter if you have 90 out of 100 that vote in that precinct or 900 out of 1,000, you’re still going to get three delegates — the number is fixed regardless of turnout.
“In 2004 we were able to correctly predict how people were going to vote in the Iowa caucuses and move resources around where we needed to.”
Howard Dean had large support mostly among young people. So he was polling as if he was a head in the state of Iowa because of big support on college campuses, largely. But his vote was concentrated and he had a lot of wasted votes. We were able to figure out where Kerry should go in order to win the next available delegate in various ones of these precincts and were able to surprise in that election.
[That] really is the key in the Iowa caucuses, it’s not about the 40 delegates, which is about 1% of the total to the national convention that are elected, but it’s about getting momentum in the media story. [That’s] why we had to keep secret the fact that we knew we were going to win from about November on. We knew we were on track to win but we wanted to keep that a secret so that it would be a surprise in the press, and then based on that, John Kerry went on to win in November.
We did micro-targeting for his campaign in the general election, but it was used not across the board and with mixed success. But based on that, we did a lot of other campaigns including the successful campaign for the democrats to take back congress in 2006 — did a lot of micro-targeting for that — and then signed up with Barack Obama in 2007, where he had the same dynamic going on that existed for Howard Dean, with a lot of anti-war young voters and the press was writing pretty much the same stories. They had learned the lesson in Howard Dean’s campaign, and they said, ‘Okay, Barack Obama’s going to lose because his support is all concentrated in a few specific areas’.
But because we knew that and were able to game out what the caucus math would work, we were able to use our resources to boost our vote total where we needed to in order to get those votes out, and also do things like encourage young people to vote at their home address, their parents’ address instead of their college address, things like that that would help distribute our vote more evenly.
We went on to win the Iowa caucuses for Obama in ’08 and had a very welcome change in the political attitude for the general election where the micro-targeting was a key part of all voter contact for the campaign. So used to great effect in the general election, really caught the press attention in the US after that, which led in part to my firm spinning off to do a fair amount of commercial predictive analytics and big data because people wanted the ‘Obama secret sauce’ as many of them call it. So we’ve done a number of other projects, though we still keep our hand in politics.
No serious high-level campaign in the US would be without micro-targeting at this point.
In terms of campaign technology, I like to think of it as there’s two things that you can do with campaign technology: you can talk to voters in a different way, and you can decide who you’re talking to. Those two sort of leapfrog each other as new techniques are developed for talking to voters, deciding who you’re going to target with those — the question of who you talk to sometimes tends to lag behind.
In the start of campaigns, audiences were often self-selected. If people were giving stump speeches, it’s the crowd that shows up to hear the candidate so you don’t really worry about who you’re targeting because it was people who turned up to hear the candidate talk. But that doesn’t really happen that much anymore, although some candidates like Obama in ’08 and Bernie Sanders this time, still can attract crowds of tens of thousands, but by and large it’s direct voter contact — reaching people where they live and work.
“Candidates like Obama in ’08 and Bernie Sanders this time, still can attract crowds of tens of thousands, but by and large it’s direct voter contact — reaching people where they live and work.”
There have been only a few major sea changes in that end of the technology equation — how you talk to people. Television, starting with Eisenhower’s presidential campaign, was a very untargeted media. For a long time television was considered one of the least targeted, people would buy national television.
Over the last few presidential cycles people have been able to figure out how to target television on a state-by-state level because of the electoral college system in the US where — as we discovered to our sorrow in the Al Gore campaign, where we lost by 538 votes, and I was on that campaign — it’s not the winner of the national popular vote who’s elected, it’s the winner of the electoral votes from each individual state who are elected.
So buying a national television advert in a presidential campaign doesn’t make any sense because you’re paying to reach voters in New York, California, Texas — huge states whose outcome is not in question in any seriously competitive election.
So in the last few cycles campaigns have been buying local media markets in order to focus on particular states. More recently, campaigns have been getting smarter about what shows and what networks to buy on — they’re able to say, ‘I don’t just want a certain number of gross ratings points, but I want specific kinds of people. I want to be able to look at my voter file and predict if I buy an add on programme A vs programme B, am I going to reach more undecided voters?’. That’s something where micro-targeting technology combined with television ratings data can be very powerful — it’s helped us target broadcast television. Further innovation is addressable television — more and more people having set-top cable boxes, the cable companies are able to serve ads to actual individual voters, and so that’s a perfect marriage with micro-targeting technology. We can provide a list of however many voters we want to reach through television ads, and if the cable company has the ability, they can then serve an ad directly to those voters. Right now, only about 30% of televisions in the US are reachable via direct addressable ads, but that amount is growing rapidly — it was barely any just four years ago.
“We can provide a list of however many voters we want to reach through television ads, and if the cable company has the ability, they can then serve an ad directly to those voters. Right now, only about 30% of televisions in the US are reachable via direct addressable ads, but that amount is growing rapidly — it was barely any just four years ago.”
There are other technologies that are coming along, for example, in ’08 for the Obama campaign, I believe we were the first campaign ever to advertise in a video game. That was fun because someone, I can’t remember who, came up with that idea, and it was something that we wanted to keep secret. The press didn’t stumble upon it for several weeks — we had the ads running in various video games before anyone figured out we were doing it. Now it’s more power for the chorus that people are doing that.
In-app ads in mobile phones are also a new, emerging technology, as are digital ads in general. In the ’08 Obama campaign it was considered reasonably innovative to be targeting ads by particular websites. Campaigns would do things like target news sites where people who were interested in politics were going for their information.
One of the problems with that is that’s a self-select audience that’s fairly high information, so someone who reads the New York Times goes to the New York Times website probably isn’t that undecided voter that you really want to reach, much more likely to be someone who follows issues and makes up their own mind. So finding those voters on some other site that isn’t serving up political news is much more valuable.
Over the last couple of years it’s been possible to do cookie matches of lists of individual voters, so we can take a micro-targeting model and come up with a list of, say, 50,000 target voters in a particular state, run them through a service that will see, ‘Have these individuals registered at any websites? If so, what are the available cookies that will allow us to serve ads to them, not just on the site where they registered but elsewhere as long as that cookie exists on their computer’. That process was fairly expensive and fairly slow — it would take a few weeks and cost thousands of dollars at a minimum to do.
Recently we’ve been able to upload an entire national voter file ahead of time with a number of issue models. So we have models predicting attitudes on things like gay marriage, marijuana reform, the Tea Party in the US, voter likelihood, partisanship, ideology, and those are already pre-cookie matched, so it’s possible for a campaign — including a very small campaign — to go and buy a certain number of targets and serve digital ads directly to them.
“We have models predicting attitudes on things like gay marriage, marijuana reform, the Tea Party in the US, voter likelihood, partisanship, ideology, and those are already pre-cookie matched, so it’s possible for a campaign — including a very small campaign — to go and buy a certain number of targets and serve digital ads directly to them.”
Just like advertising in video games was considered innovation in 2008, I’m sure there’s something that someone is going to do to advertise in the 2016 election that none of us have thought of yet. Then people like me will scramble to figure out how to target those people. It tends to be that whenever there’s a new technology, it’s pretty much a shotgun approach and then people have to figure out how to target it better.
There are things like promoting tweets on Twitter that people are just now figuring out how to target successfully. Whenever a new technology like that emerges, it takes a while to figure out how exactly to use it to the best effect.
People don’t want to feel like they’re being blatantly targeted, and also I’m sure all of us have had [the] experience of being incorrectly targeted, especially online. When the targeting is [really] obvious it kind of loses its effect. Practitioners are always walking that line between cool and creepy, and not wanting to do things that violate someone’s privacy or even feel like that even if they’re not. That’s an important line we have to walk.
“Practitioners are always walking that line between cool and creepy, and not wanting to do things that violate someone’s privacy or even feel like that even if they’re not. That’s an important line we have to walk.”
A couple of years ago I began wondering, ‘Is there a way to take that prediction and move it forward in time a little bit?’.
Being able to predict ahead of time that that lifestyle change is likely to happen, allows us to get the message to the appropriate people at a time when it’s going to resonate with them. There’s a direct parallel in politics with having children, because people’s attitudes change. Before you have children you’re not thinking about school districts as much, there are a lot of other issues — you might be taking a longer-term view on issues like the environment once you have children, the way you can talk to someone changes a lot once they have children. That’s the information that on a voter file is generally going to lag a year or two — consumer data files will have presence of children flags, but it takes a year or two for that data to filter through.
It is kind of amazing, even to me, when I look at scores predicting people’s issues attitudes — things that you would not expect to be predictable, are. They aren’t with 100% accuracy, it never would be, humans are interesting, quirky beasts and we all have our own attitudes, but generally speaking you can have a prediction that someone is more or less likely to hold a particular attitude and those models work surprisingly well.
One of the ways that works is this notion that you are like your neighbours, people move to particular neighbourhoods where there’s likeminded individuals. That holds, though, in more than just geographic space.
One of the machine-learning algorithms that I mentioned that we use is called ‘nearest neighbour’, where you look at the people nearest to someone, most similar, but that’s not meant necessarily geographically — it can be similarity on any data point that you have about the people. Are they similar in terms of income, in terms of education, in terms of race, gender etc? And you find the people who are most similar to a particular individual.
Then there are various different variations on that technique, where you might take a certain number of the closest people and have them, in effect, voting and say the most common response from them, or you could do a weighted average based on the level of similarity.
There are other things like classic logistic regression where it’s just a simple formula where you’re saying, ‘As someone’s income gets higher, they get more conservative’. There’s segmentation where you split a list into different groups and you say, ‘The people above $50,000 annual income vs the people below’. Those sorts of techniques fall down on things that don’t have a direct linear relationship to something.
In the US at least, it’s accepted as a truism that the richer you get, the more conservative you are, and it’s true up to a point. But there is a point at which, as some people put it, ‘You’re rich enough to be liberal’ — you’re no longer all that concerned about taxes, it’s not like you got a pay raise and suddenly you’re paying more in taxes. You’ve got enough to live on and you can make decisions based on other things.
We’ve also found that comparing income and education is a key predictor. A rich person with an advanced degree is very different from a rich person without an advanced degree. Similarly, income and age have a big difference. So someone in the US who is making $50,000 is very wealthy if they are 21 years old, and they’re not very wealthy if they’re 60 years old. So we like to look at age by income.
Another thing that relates to income that we look at is housing patterns. In the US there is more racial diversity here. We found in president Obama’s primary history that there were very different racial attitudes in different states. The people in very white states tended to be more open to voting for then senator Obama than more integrated states, which kind of surprised us.
“We found in president Obama’s primary history that there were very different racial attitudes in different states. The people in very white states tended to be more open to voting for then senator Obama than more integrated states, which kind of surprised us.”
The nominating process started in Iowa and New Hampshire with very small minority populations, and we didn’t know, right up until the day of the Iowa caucuses, if people were really willing to vote for the African-American candidate. The campaign manager asked me the morning of the Iowa caucuses, ‘Are we going to win?’, and I said, ‘Yes we are, unless everyone’s lying to us’. Because if you call someone up and ask who they’re going to vote for, the socially desirable response is to say, ‘Yes, I’m going to vote for senator Obama’.
If [people] are harbouring racist notions and didn’t want to go for the African-American candidate, they might not say that. It’s known in US politics as ‘the Bradley effect’ after Los Angeles mayor Tom Bradley who was ahead in the polls when running for governor, and then on election night lost by a significant margin. He was the first serious African-American candidate for state-wide office in California.
Ever since then, people were very concerned about the Bradley effect, and it’s played out in a number of elections. People would tell pollsters they were more likely to vote for an African-American candidate than they actually were.
So that was a big concern for us, the morning of the Iowa caucuses. When the results came back in they were exactly as predicted based on our survey data. No matter how careful we were about building the models and testing the models — and we always have a random hold-out sample of survey responses that aren’t used in building the model so we can see, ‘Did the statistical models accurately predict those surveys?’ — we do surveys after we’ve built the initial models to make sure the models are predicting those correctly. But all that verifies is we’re able to predict what someone will say on a survey question, not what they’ll do. Very often, as evidenced by the Bradley effect, what someone says and what they do is different.
So it was a big relief the night of the Iowa caucuses when we found that people actually were voting pretty much how they said they would.
In 2012, alot of Republican pollsters got the election very wrong. There were lots of pollsters who were saying, ‘Mitt Romney is going to win the election’. They weren’t fudging the numbers or lying about what they had, it was just a question of what their sample was. You can only poll whatever sample you decide, and not everyone votes, so you have to have some assumption about who is going to be in the electorate.
“In 2012, alot of Republican pollsters got the election very wrong. There were lots of pollsters who were saying, ‘Mitt Romney is going to win the election’. They weren’t fudging the numbers or lying about what they had, it was just a question of what their sample was.”
The Republican pollsters were saying, ‘The electorate is going to be more white and older than it was in 2008’. And it was, to a certain degree, but Obama was able to turn out a lot of his initial winning coalition and had a lot of young people, a lot of African-Americans turning out — more so than the Republican pollsters expected in their sample. So if the electorate had been what they thought it was going to be, their predictions would’ve been correct, which is one of the challenging things in polling.
A lot of pollsters will have a likely voters screen asking, ‘How likely are you to vote?’, so they don’t have to go in with a pre-conceived notion. The problem with that is that people lie about that. It’s like asking, ‘Are you going to vote, or are you a bad citizen?’ — everyone knows that you’re supposed to say you’re going to vote. So people grossly exaggerate their likelihood of voting.
We’ve seen that comparing our polls to actual election results, we’ve even gone back and surveyed people and asked, ‘Did you vote in the last election?’, when we actually know that from a voter file, and a large percentage will consistently say that they voted in the last election. They’ll also say they voted for the winning candidate. Whoever won, more people say they voted for the winning candidate. I don’t know if they’re lying or they’re just misremembering it — people want to think that they were with the winning candidate. On the UK example — and as just an observer, having not been directly involved — it sounds like the polls were measuring attitudes towards the major parties in general rather than the decision or issues on which people made their voting decision. So leadership and the economy, when people were asked about which party they supported, people thought the Tories were better on those two issues. When they asked, ‘Which party do you support in general?’, more people said Labour.
So people’s hearts might have been with Labour, but when they went to vote, the issues that they were making up their minds on were the ones that pushed them in the other direction.
That’s another tricky thing to poll on because if it asks someone, ‘What issues is going to drive your voting decision?’, people will stop and think and give you the idealised version of their decision-making process.
I have great sympathy for the pollsters that got it wrong because in many ways they were right — the polling in the UK was probably right in terms of people’s attitudes towards the parties, just not picking the correct question for picking people’s votes. Just as the Romney pollsters were correct with their version of what the electorate would’ve been.
“I have great sympathy for the pollsters that got it wrong in the UK because in many ways they were right — the polling in the UK was probably right in terms of people’s attitudes towards the parties, just not picking the correct question for picking people’s votes.”
One of the things that’s nice about micro-targeting is we’re often in the field at a much larger scale and more often than a traditional poll. A traditional poll might have a long script — maybe 20 minutes of keeping someone on the phone and asking a lot of detailed issue questions and a sample of 400/500 people.
In our micro-targeting surveys we try to keep it under a minute and we might be calling 10,000 people. In the case of the Obama election, in every one of about 20 battleground states We called 10,000 people a week, every week from June through November. So we were able to keep a constant track of the polls for the electorate, which helped us to be able to react to external changes in the dynamic of the race, and also to not overreact.
Two of the biggest things that drive public opinion in the US presidential election are the party conventions and the debates between the presidential candidates. Absent some big surprise, black swan event, those are the things you can count on to move polls. If you look at a graph of polling over time, you can spot the conventions and the debates — that’s when there’s a spike one way or another.
So we had these daily calls, we were calling a mix of new voters and calling back people who we’d spoken to before, every week, and the biggest shift we saw was when John McCain picked Sarah Palin for VP. At the start of the week there was a huge spike among Republican women and Independent women that were suddenly much more likely to support the McCain ticket.
We were able to launch communications to that group because Sarah Palin’s record was not what that group wanted in a presidential candidate. Her issues were the direct opposite of the Independent women and, to a large degree, even the Republican women. So we had phone calls going out and mail ready to go, although the mail didn’t even have time to hit before public opinion shifted. By the end of that week, after Sarah Palin opened her mouth and people heard what she believed in, there was a slingshot effect and independent women were actually less likely to support John McCain than they had been at the beginning of the week.
“By the end of that week, after Sarah Palin opened her mouth and people heard what she believed in, there was a slingshot effect and independent women were actually less likely to support John McCain than they had been at the beginning of the week.”
So that was a case where the micro-targeting and the ongoing surveys allowed us not only to identify a group where there’s a problem and the message that would be effective at moving them, but also not to overreact. If we had believed they actually had this long-term sustainable bump based on the Palin pick, the campaign might have made decisions about pulling out of particular states or other things that would’ve been an overreaction and a strategic mistake.
In terms of the number of variables, that’s one of the things that trips up practitioners when they sometimes start to do this for the first time — especially as software has gotten cheaper and there’s open source software available where people can build models for themselves that used to take software costing tens of thousands of dollars.
There’s a tendency to throw as much data as possible at a problem, and if you use a fairly basic algorithm, say you’re using logistic regression — where you’d say income x 0.5 + age x -0.9, whatever — one variable makes you more likely to be democratic, one makes you less likely to be democratic, add them all together. There’s a temptation to throw more and more data at that. I totally understand that, I always think more data’s better, but if you have enough variables compared to the number of surveys you have, it’s possible for the model to just memorise the data that it already has.
There are techniques that can automate the process of deciding which variables to use. So penalised logistic regression is one where the algorithm tries to find the model that not only makes the most accurate prediction, but makes the most accurate prediction with the smallest sum of the coefficients for the indicators. So it’s automatically going through the process of trying to make a good prediction without using too many different predictors. There’s a number of techniques like that that can guard against the curve-fitting. It’s something that oftentimes trips people up when they’re trying to do it for the first time because you just assume [that if you] throw more variables in it will do well, and if you don’t have a hold-out sample and you’re just looking at the graph of your predictions, it looks beautiful.
A lot of users of models don’t know what to expect from a model, so they might see a model [that] secures the graph of the model prediction and here’s the actual results — that’s pretty good as long as the actual results go up along with the model score, that’s good. And you don’t expect them to be exactly alike. In fact, if I see something where the test results exactly match the model, then I know there is a problem — somehow the test was polluted with data that was used in building the model. So if a model looks too good to be true, it probably is.
I think there’s potential for backlash if people realise how they’re being targeted, and there’s potential for candidates to misuse it — although much less than there would have been if we didn’t have fortuitous timing about the internet arising at the same time as this technology was arising. So there are stories about candidates that would, in a state-wide race, might say one thing in one media market and another thing in another, safely knowing that the newspaper that covers the speech over here is not going to be read by the people over here. So candidates in southern states who would have two different opinions on segregation depending on if they were on the northern half or the southern half.
Even recently in the state of Missouri — it’s pronounced ‘Missoura’ in the southern part of the state and ‘Missouree’ in the northern part of the state. So candidates would have TV ads saying it differently — there’s nothing wrong with that, it’s not like you’re pretending something different, you’re just speaking the language of the local people.
“There’ve been concerns about, ‘If you can predict what someone thinks about an issue, isn’t that going to allow a candidate to just lie about their positions and just pretend that they agree with you about everything?’
There’ve been concerns about, ‘If you can predict what someone thinks about an issue, isn’t that going to allow a candidate to just lie about their positions and just pretend that they agree with you about everything?’. While I would like to, naïvely, believe that all candidates are good and no one would ever do that — that’s the faith part of it — but my evidence part of it is more that they’d get busted.
Now there’s going to be someone with a cell phone taping a speech, and the minute a candidate takes a position, it’s out there in the public record. Candidates can’t speak out of both sides of their mouths anymore, so there’s much less danger of a candidate misusing micro-targeting predictions in order to tailor their message to a particular individual. I would like to think you can hold an elective official’s feet to the fire by having videotape of them saying something, is going to be more powerful than a press interview where they said it. Although still, people do change their positions.
What is the future for predictive data analysis?
The two big advances I expect are speed and data sources. So on speed, when I first started doing this for John Kerry it would take about two weeks to build a model. Now, it can generally be turned around in a day — an hour or so for crunching the numbers and then a few hours for quality control and checking it and shipping it off.
We’ve developed and tested automated technology where instead of having IDs shipped to us from a call centre — transfer a file one evening and we load it up and crunch the numbers, it’s actually delivered in real time as the call is completed via an API. We can update a model — and we’ve tested this in California, the highest population state in the US — and are able to have a model that updates in less than a second when we have one new ID. So a new ID comes in and in less than a second it’s reflected in scores.
The implications for that can be very big when you’re starting to do a phone bank with lots of callers, either paid or volunteers, or door-knocking where it used to be that door-knockers would have a clipboard with a printed sheet and they knock on the doors, they go back to headquarters, if you’re lucky it’s data entered that night or the next weekend, if you’re unlucky it’s thrown away and found in a box after the election.
A few years ago there would be a few campaigns that would consider themselves very cutting edge, would invest in palm pilots, if anyone remembers when that was a big thing — ‘It’s a computer that I can hold in my hand!’ — but it would cost hundreds of dollars and people worried volunteers were going to run off with the palm pilot. They still had to bring it back, put it in a cradle and sync it back to a computer.
But now, any volunteer you’re going to have is going to show up with a smartphone. Everyone, virtually, has access to a computer they can hold in their hand that can be connected to the web at all times. So instead of marking, ‘Voter Joe Smith cares about the environment’, on a clipboard, they talk to Joe Smith and say, ‘Hi, I hear you’re concerned about education’, and Joe Smith says, ‘Yeah but I really think global warming is the biggest thing’, and so they mark ‘environment’.
The technology exists, and I think will be in widespread use fairly soon, that by the time that volunteer walks to the next door, the score for Sarah Jones — where they might have been predicting that she was an education voter — will have been updated based on the input from the previous election and not just for that particular walker who’s going from these doors, but all these hundreds of walkers who are out knocking on doors everywhere. Real-time micro-targeting, I think, is one of the big innovations that’s made possible by much faster computers and better, omnipresent internet connections.
“I don’t think there’s any one method that will replace phones as the gold standard — people are going to have to use a combination of online, telephone, text message etc in order to get a representative sample.”
I don’t think there’s any one method that will replace phones as the gold standard — people are going to have to use a combination of online, telephone, text message etc in order to get a representative sample.
One of the technologies with big potential, in my mind, is social media data mining. So we spend all this time and effort begging people to tell us their opinions. In effect we’re calling someone up and asking, ‘What do you think about this issue?’, and they’ll say, ‘It’s none of your business, that’s way intrusive, why should I answer?’, and then they go and tweet a picture of what they had for breakfast.
So people are volunteering this tsunami of information on social media that’s out there freely available, there’s just two big challenges with this data. It’s unstructured, it’s not like someone is filling out a form where the question is what you had for breakfast, it’s they may or may not have shared a picture of what they had for breakfast.
So you have to have computers that are smart enough to parse out that kind of information — called natural language processing — where they’re able to look at a Facebook post or a tweet and come up with what that means.
We’ve done a lot of research at my firm about sentiment analysis on social media, which is tricky because a lot of social media is sarcastic, and people can very easily understand what that means, computers do an awful job of it. One of the examples I’ve been using for a couple of years — picking on Sarah Palin again — is, ‘I love Sarah Palin’, by that phrase someone loves Sarah Palin. ‘I love Sarah Palin like I love getting root canal’ — to a computer it mentions a politician, it says love, okay, fine, to a human you know that’s not the case.
It changes so fast — language, especially language on social media — [it] changes fast enough that you couldn’t set up your natural language processing now and have it still working a month from now. The process that we use is having humans classifying things to build the algorithms that have a computer do it, and then the humans have to check it and you flag the ones that you’re getting wrong and go back and rebuild rules for the ones you’ve gotten wrong.
The other half of the equation in order to use this data is matching it to an individual identity. Because it’s fine to know that kitkat187 has this opinion, but matching that to whoever that is in order to be able to get them a piece of direct mail or knock on their door and encourage them to vote, is another matter.
“There are lots of ways of figuring out someone’s identity from social media that a lot of people probably wouldn’t think were that good. One of them is that most people forget to turn off the location-aware features on their phones.”
There are some ways you can go about it, and here, again, it’s the balance between cool and creepy. There are lots of ways of figuring out someone’s identity from social media that a lot of people probably wouldn’t think were that good. One of them is that most people forget to turn off the location-aware features on their phones. So if they’re tweeting from their phone, the chances are the latitude and longitude is available in the metadata for that tweet — so you can tell where someone tweeted from. I know and sort of disapprove of the fact that my phone knows where I am all the time, but I don’t go in and turn it off. There are so many apps that it’s useful for it to be tracking me that I don’t really bother.
So a trained focus group moderator will listen to the people and get a sense of, ‘This is what concerns people about a particular candidate and this is what motivates them’. Being able to discern that sort of thing in an automated process is something that I think artificial intelligence is going to get better at. Part of that will be social media listening and data mining, where it’s more like a conversation around a table in a focus group, that the computers are listening to what people are saying and discerning what their concerns are.
Another technology that we refer to as active collection passive response, is where you’re using different technologies from people not in response to specific questions. An in-use example now is eye-tracking software where you can have someone watch an ad or see a picture of a piece of direct mail and see where their eyes go. So instead of asking someone, ‘What really caught your eye in this piece of direct mail?’, you show them an image and see where their eye went and in what order. That sort of information is useful.
Once again, getting into the creepy, there are biometric responses that go with emotional responses. So if someone flashes a picture of a candidate on a TV screen in front of a room, you can see things about pupil dilation, body heat etc, that give you a sense of their response to that person.
There will be a lot of potential for abuse of that sort of thing, you could have hidden cameras in a movie theatre watching people while an ad before the movie comes out, and the facial recognition matches it to those people and says, ‘These people had a negative response and those had a positive response’.
I think there would be, rightly, a backlash against that technology if people knew that was being use. But as opt-in technology I think people might be open to that. If they said, ‘We’re doing this focus group and we’re going to show you ads. With your consent we want to be able to let this camera see where your eye is going and how you’re responding to these different candidates and collect data that way.
What do I hope will not happen in the next 10 years? I fear, listening to myself, that it would be my list of what I think is the coolest stuff that technology should do, is all the stuff that I hope doesn’t happen.
by Ken Strasma, CEO of HaystaqDNA
This talk took place st Second Home, a creative workspace and cultural venue, bringing together diverse industries, disciplines and social businesses. Click here to find out who’s speaking here next.
