First, let me correct something I said above when I implied you were arguing the Harvard-Harris poll is meaningless. That’s not what you were ever doing so that was inaccurate on my part. Other people are doing it, but you seem to be skeptical of the results, which is something different and entirely reasonable.
Second:
“Secondly, the methodologies you describe that somewhat control for self selection were not the ones necessarily used by many of the polls,”
Again, fair point, but that’s where I go back to the fact that I have yet to see any experts or anyone in the polling industry attack the HH methodology. That doesn’t necessarily mean it’s controlling for bias, but it at least raises a rebuttable presumption it’s sufficiently sophisticated.
“…it would be interesting to do a meta analysis of those using in person random selection for accuracy.”
Over at 538, they did an analysis along those lines. I don’t consider Silver and friends to be the end-all-be-all of statistical analysis given some pretty heavy biases they’ve been showing since going “big time,” but I do think they’re at the top end of the spectrum. You can see they don’t analyze Harvard-Harris, but give individual (and middling) grades to Harris and Harvard (based on only one poll so, meh). Additionally, predicting political races isn’t the same thing as gauging popularity so I’m not sure how useful the analysis is for this discussion, but at least it’s something.
“I bet the major problem with those polls was untruthful answers.”
That’s my guess, as well. At least as far as the presidential election, it would help explain why the polls narrowed whenever Donald Trump wasn’t getting blasted by controversies of his own making i.e. people were embarrassed to publicly declare for him when he was making an ass of himself, but were less bashful when he kept his mouth shut and especially when Hillary Clinton was the target of criticism.
