The CNN/ORC poll is garbage
To estimate the rate at which something occurs in a population, statisticians attempt to take a representative sample from the population. The sampling methodology is one of the most important steps in the process. Representative is the operative word. If a sample is not representative of the population you are trying to describe, the statistics produced will not provide reliable information about the true population.
In the United States presidential race, the population is the people who will vote in the presidential election. Let’s first look at the United States eligible voting population by age range:
This shows 45% of the voting age population is under 45. However, it’s well known that younger voters do not turn out at the same rate as older voters. If we look at who reported that they voted in the 2012 election, we see a slightly different story:
Instead of 45%, we see that only 38.6% of actual voters in 2012 were under 45.
Now, let’s look at CNN’s sample by age group:
I did not find the raw numbers for each age group; but, we can estimate these from the reported sampling error.
CNN sampled too few voters under 35 to admit a sampling error for this group, despite the fact that this group represents about 30% of the voting age population. We can estimate that the sampling error for this group would be about +/-10% from the surrounding context. What did CNN’s sample look like then?
We can see that, though voters under 35 represent about 30% of the voting age population and around 23% of likely voters, CNN only used around 14.5% in their “representative sample”. Likewise, those 65 and older only represent 20% of the voting age population and about 22% of likely voters; but, in the CNN sample, they are a whopping 34.2%. Note that more voters under 35 voted in 2012 than voters over 65. The CNN sample included around 2.35 times more voters over 65 than under 35.
Due to imprecise reporting by CNN of their sampling error, their numbers don’t quite add up. So, we can only infer roughly what their sample size for each age group was. If anyone knows how to access the raw data, we could be more precise. But, let’s do the same exercise with only two groups, under 45 and over 45. CNN/ORC reports a +/- 7.0% sampling error for voters under 45 and a +/- 4.0% for voters over 45. This would imply 196 voters under 45 were sampled and 598 voters over 45 were sampled. But, 196 + 598 = 794. The total sample size is reported by CNN as 779. So, CNN is rounding down their reported sampling error. But, let’s assume they sampled 196 voters under 45 and 598 voters over 45. What does this look like?
What would a proper sample look more like?
And if CNN wanted to promote the voices of voters under 45 equally to those of voters over 45? They could take a representative sample of the eligible voting population. This split looks like this:
Now, let’s compare CNN/ORC’s overall 3% for Johnson with his 8.4% showing in the IBD/TIPP poll conducted at the same time.
Notice Johnson polling at 16% among voters under 45. Assuming voters under 45 represent 38.6% of likely voters, like they did in 2012, this block alone, without any votes from those over 45, would translate to a 6.2% share of the vote.
CNN has a disclaimer that they do some weighting to try to correct for their atrocious sampling methodology. But, these methods are not public, to my knowledge, and the disclaimer contains a bold lie about voters under 35:
Some subgroups represent too small a share of the national population to produce crosstabs with an acceptable sampling error.
There are about 1.5 times more potential voters under 35 than there are voters over 65. But, CNN was able to produce an “acceptable sampling error” for voters over 65 who they egregiously over-sampled.
Conclusion
Politics is a game of perception and enthusiasm. The Democratic establishment would like to push the narrative that a vote for Jill Stein or Gary Johnson is a waste and that support for 3rd party candidates is fading or collapsing. CNN’s use of this poll with its dubious methodology (and disclaimer which includes a blatant lie) raises serious ethical questions about the pollsters who produced it and the organization that promotes it as reliable information about the likely voting population.
One may reasonably conclude that the purpose of this CNN/ORC poll is to steer perception rather than to measure it. This poll is garbage, especially if young voters turn out in spite of it.
