A critical evaluation of public opinion polling and majority decision making.
Public opinion polling is all around us, we see polls in the climate debate, we see polls about war, we see polls about holidays and food, we see polls about love, and we see polls about hate. These polls are designed to provide information about what a majority of people believe, prefer, or understand.
Using public opinion polls, we can track the changes in American majority mindsets over time, we can see the public believing more in anthropocentric climate change and growing more concerned with its negative effects.
There is value and validity in these surveys being produced and distributed, but there are also structural limitations that inhibit the use of surveys. Moreover, there are real world outcomes of surveys that deserve discussion.
I argue that the tools of public opinion collection fail to capture real representations of public opinion, and these tools constrict the diversity of individuals into the diversity of groups, which in turn limits the amount of potential solutions we allow ourselves to consider.
It is people who determine what survey questions will be valuable to ask, and people decide in what form the questions will be asked. People are inherently biased by their own reality. You cannot know about, or ask questions about something that is not within your own cognitive reality.
For example, imagine a survey designed by political activists that aims to understand how the elderly feel about healthcare. If applying good practices, the surveyors will interact with and talk to the elderly population in question to better understand their world view in order to ask more relevant questions. However, the questions asked by the surveyor will always come through the ideological lens of the surveyor. The surveyor, myself included, will never be able to truly understand what it is like to have lived to the age of 80 through conversation alone.
For example, let us look at the following survey question:
“What is your favorite Thanksgiving dish?”
- Green Beans
- Mashed Potatoes
- Candied Yams
- Dinner Rolls
My answer would of course be stuffing. However that is not the point, the point is that my ability to ask this question was immediately limited by what Thanksgiving dishes are normal to me, or prevalent in my culture. Your ability to answer this question in a meaningful way is limited by your sharing of an association for these foods. Imagine if your family eats duck or pork for Thanksgiving, and it’s your favorite part of the meal. How does this individual respond? Do they go to their second favorite? Choose arbitrarily? Who is to say.
Another example, please state your level of agreement or disagreement with the following statement.
“Reforming the immigration system and securing America’s borders will be a primary factor in who I choose to vote for in 2020.”
4- Neither Agree nor Disagree
When different populations or individuals are asked this survey question, they must first interpret what the question is asking. The word “immigration” will pull dramatically different concepts into different people’s minds. Consider different media environments, immigration as a concept is shaped by some people to be a caravan of dangerous criminals. Other media environments contribute to an image of immigration as refugees fleeing worse circumstances. Therefrom, depending on the media environment of an individual, the meaning of the word immigration, and therefore the survey question, will be different.
If this is true, and the meaning of a survey question changes based on the ideology of the surveyed, then we cannot assume the data collected demonstrates an equivalent response across a population. However, because of survey questions such as this, I can tell you that Republicans care about immigration 6% more than Democrats (84% vs 78%). In these numerical figures, there is again a conveyed assumption of equivalency. Democrats care about the same immigration as Republicans, just 6% less.
This is the misrepresentation. There are not only 2 sides or 2 options, there are many different points of view. There are different levels of importance, there are different reasons for importance, there are different aspects of immigration that are more or less important, there are different interpretations of immigration that lead to different perspectives about its importance. This data does not tell us this complexity, that is not it’s goal, but therefore this data does not represent public opinion, because public opinion is not a majority agreement with a single sentence statement.
A lack in equivalency does not invalidate the use of surveys as a valuable tool to better understand our world, but it is important that we understand how surveys should be used in order to protect ourselves from the survey’s misrepresentation of reality.
Academics and social scientists use surveys with the intention of drawing information from the real world in order to test hypotheses and glean new understandings. If we know that Republicans care 6% more about immigration than Democrats, then we can use that data as a jumping off point to help understand where this difference comes from. Or scientists could observe how this difference in importance actually manifests itself in the behaviors of each respective group.
Results in quality academic surveys exist within a logical framework for what the results demonstrate and mean. For example, Dunaway, Branton & Abrajano (2010), use data tracking the importance of immigration by state. In using the survey, the scientists look for differences in the data between border states and non-border states. Tangentially, the scientists evaluate the media environment of those states as well as the demographic makeup of each state.
Through this process, this paper is able to demonstrate the influence of the media on the public’s perception of immigration. If you live in a border state, the news media available to you will spend more time discussing immigration, and therefore, via the agenda setting theory, more people will perceive immigration as being the most important issue facing them politically. Interestingly, for those living in a non-border state, an increase in immigration focused media will have a greater effect than a media increase in a border state.
Notice how the article is not using the data to further an agenda or support a policy position. The importance of immigration does not dictate what should be done, it demonstrates what happens when the media behaves in a certain way. In essence, the scientists are able to circumvent the diversity in question response by not placing value on the question itself, and instead focus on contextualizing the (change in) data into a functioning theory for human behavior.
Other organizations who focus on public polling aim to present the data in a usable format for individuals and organizations to use at will. Again, these polls are valuable, they help people understand their communities and make informed voting decisions, but they exemplify the constricting nature of the pre-created answers. Additionally, these organizations often neglect to contextualize the data, and enable representations of reality based on imperfectly perceived survey questions.
Let’s use another example, this time “Views about abortion by state (2014)”, notice how there is no context for this data. These results are not grounded in a logical framework that outlines a purpose or meaning of the data. This poll is specifically designed to take a snapshot of public opinion and provide people with real usable data, but how real is it? The lack of complexity in answer choices (legal in all/most cases, illegal in all/most cases, and don’t know) forces a respondent to self-analyze and decide which side of a constructed line they fall on.
Returning to a previous line of discussion, abortion does not mean the same thing to all this survey’s respondents. Someone encased in a media environment filled with fake statistics on the prevalence of third trimester abortions does not have a naturalized image of abortion that is at all similar to a women’s rights activist. How could they. These people are not answering the same survey question, and yet we use this collected data to try and understand how an entire population thinks about abortion.
To add to this, because we lack guidance from the author, our interpretation of the survey results are largely subjective. Consider Georgia who, in this survey, demonstrates a near perfect split with 48% believing abortion should be legal and 49% believe abortion should be illegal. Without further investigation, research, and contextualization, this survey does not tell us what people would like to see happen, nor does it encapsulate their perspectives about abortion. While it does demonstrate a general polarization about the issue, the responses and beliefs between the dichotomy are not equivalent. What the 49% believe should be illegal is not the same thing that 48% believe should be legal.
The presentation of this data solidifies an image of a false duality that leaves no room for nuance. You either support abortion, or you oppose abortion. It is easy to consume this data, it is easy to chalk up the residents of Georgia into one of these categories, but that is no way to make policy decisions. If you use this data to try and identify what a state would like their abortion laws to look like, then you are pushed towards total legality or total illegality.
Consider the potential drawbacks of following survey results as a guide for where people want the country to go politically. Climate change is bearing down on us, and in our incessant polling of Americans’ views on climate change, we have constructed arbitrary limitations for our solutions. Consider nuclear energy, and as an example: “40 Years After Three Mile Island, Americans Split on Nuclear Power (2019)”. In this instance, the surveyors are allowing for two possible decisions to be made when deciding how to combat climate change. There should either be nuclear power, or there should not be nuclear power. Yet if we are to combat climate change around the world, then we must enable ourselves to make contextualized decisions based on community by community cases.
To run with this example, consider a heavily forested area with a population in the tens of thousands. In providing this population with clean electricity we might not have the capacity to construct a wind or solar farm without first removing hundreds of acres of habitat from the local wildlife. We cannot continue to rely on fossil fuels if we are to attain the necessary close-to-zero emissions needed to lessen the incoming negative effects of climate change. What then, is this community to do?
Perhaps the most sustainable option for this community is a small nuclear power plant, but if we allow public polling to dictate the nuclear rhetoric in the coming years, then we may very well be inhibiting this community from achieving carbon neutrality. Conversely, nuclear energy will not be viable in all situations, and the risks associated are ever present and terrifying. The point is, the path towards clean energy should not be controlled by falsely constructed dichotomies in public opinion. Everything is contextual, and the “public opinion” about anything will shift dramatically when the question is changed from a hypothetical agreement to a specific plan of action.
“Overall, do you strongly favor, somewhat favor, somewhat oppose or strongly oppose the use of nuclear energy as one of the ways to provide electricity for the U.S.?”
“Government engineers, nuclear physicists, and local city planners have, over the past 2 years, devised a plan to provide clean energy to this community through the construction of a small nuclear power plant located 100 miles outside of the city limits. This distance, paired with the thick bedrock of the local geologic landscape, will minimize any potential risks associated with the power plant. All nuclear energy will arrive via armed escort and all nuclear waste will be shipped to a national government recycling facility within 24 hours of its depletion. If this city is going to achieve carbon neutral energy production by 2025 then this plant is the best option. Would you say that you are strongly supportive of this plan, somewhat supportive of this plan, somewhat opposed, or very opposed.”
Would you answer differently? Would your friends and family? This is not to say everyone is going to be more supportive, the 100 miles statement alone will probably increase opposition in some way, but the context changes the calculations. The top survey question is not equipped to dictate the decision making process for a real world, contextualized nuclear power plant.
You don’t need a majority position to enact a policy solution or change a culture. A solution does not “win” when it achieves or maintains the majority public opinion. Policies happen because groups of people work together to define their future and the path to it, then they put in the leg work of executing on their plans. We should not measure the success of civil rights activists based on the changes in public opinion around hot topic issues, we should measure their success based on the positive change they create in people’s lives.
Pay attention to the polls you see around you, and look past the presentation of hard statistics. See the survey design and ask yourself, is this a fair question to ask, are they extrapolating past what the data allows, is the data honest, and what are the intentions of the survey? There will not be a magical tipping point where suddenly enough people believe in climate change and then change happens. Just because you see a “majority” of people support one option does not mean that option is the best, or that the options on the survey are the only options in reality. We live in a complex world, we need not allow others to trivialize this complexity on our behalf.