Addendum, Uber Poll. For Stat Geeks

The Ferenstein Wire likes to be wonky about statistical methods, so here’s some notes for folks who wanted to know more about the survey.

Google Surveys is an established method for conducting public opinion polls. Since the 2012 election, it’s been used by the New York Times, Nate Silver, and a number of established media brands to do opinion research. Google displays survey questions to a representative sample of Internet users, and then re-weights the results to reflect the broader US population. I’ve used to replicate a number of Pew studies on a range of political issues.

Surveying citizens about proposed laws can be tricky. Respondents, even if they know absolutely nothing about the law, will answer based on familiar names or question ordering. The best way to overcome these issues is to run the survey with different wording and response options (what methodologists call the “robustness” of a survey”).

In each of 4 surveys( here, here, here, and here), about 2/3rds of respondents supported Uber and opposed regulation, if they have an opinion.

When we had a response option that allowed users to say they either didn’t know or didn’t care, most respondents (60%) had no opinion. This is prety typical of online surveys, which have much higher ‘no opinion’ response rates than face-to-face or phone surveys. This is why it’s standard practice to give respondents the option to fill out an alternative view point in a short-form text box.

Under this condition, 13% of respondents supported an Uber regulation and 27% opposed (or about 67% with an opinion opposed the regulation). Given that less than 50% of citizens vote in an election, the “no preference” option is also a nice indicator of how many citizens actually think about an issue. In the case of Uber, it’s roughly the number that voted for Mayor in the last election.

Secondly, we also tested different wording options. In one survey, “Mayor de Blasio” was mentioned, along with his reasoning (to relieve traffic congestion). In a second survey, there was no mention of de Blasio or traffic congestion, in case the poll could have been biased.

Personally, I thought adding in de Blasio’s reasoning about traffic congestion would have biased the results in favor or regulation, but the results were robust to this wording, indicating that New Yorkers with an opinion on Uber have a stable and well-founded belief.

The Ferenstein Wire endeavors to be as transparent and methodologically sound as possible. Feel free to comment on this post in case you think we went wrong somewhere.