On Economics
Published in

On Economics

What do polls actually say?

How to interpret a poll like a statistician

1/ Suppose you ran a @Twitter poll asking

“Should @RW_UNP be PM?”

Suppose, If 151 people answered, and 85 (56.3%) said “Yes”.

How should you interpret the results?

Could we conclude that a *majority* of #SriLanka think “Yes”?

2/ Not everyone in #SriLanka is on the internet.

Of that not everyone is on @twitter.

Of that not everyone will see and answer your poll.

So, No.

Instead can we say,

“A majority of my @twitter audience think ‘Yes’” ?

Possibly.

3/ Some people in your audience think “Yes”, others “No”.

The 151 who answered are a subset of this audience.

(For now, let’s assume this subset is a random sample.)

What we need to find out is, what proportion of the audience think “Yes”, and if that is >50%.

4/ Intuitively we know that the “Yes” proportion is around 56.3%.

But it could also be 56% (i.e. 56.0%). Or even 49%.

Theoretically, even 10% or 90% might be possible.

Intuitively we also know that 56% is more likely than 49%. And 49% more so than 10% or 90%.

5/ Suppose the actual proportion was 56%, and suppose we picked 151 random responses from my audience.

The responses follow a Binomial distribution.

We can use Microsoft Excel’s BINOM operator, to make various Binomial distribution calculations.

6/ If we picked 151 reponses, any number between 0 and 151 could be “Yes”. 0 and 151 are possible, but unlikely. What values are more likely?

95% of the time we would get between 73 and 96 “Yes” responses.

7/ Similarly, if the actual proportion was 49%, 95% of the time we would get between 62 and 86 “Yes” responses.

Note, the observed 85 responses is within the range. But only just.

8/ Now, if the actual proportion was 10%, then 95% of the time we would get between 8 and 23 “Yes” responses.

Hence, while 85 is theoretically possible, it is highly unlikely.

9/ Now, suppose we ask “What is the range of actual proportions for which 95% of the time, our observation (i.e. 85) can be observed?”

We know that 56% and 49% fall in this range. And 10% does not. But what is the actual range?

We can use Excel’s “Goal Seek” to find this.

10/ The Upper Bound of our range of actual proportions will have 85 at is lower end. And the Lower Bound, vice versa.

We get 48.5% to 64.3% as are range of possible “actual proportions”.

11/ In statistics speak [48.5% to 64.3%] is the 95% confidence internaval of our 56.3% estimate.

Alternatively, we could write, 56.3±7.8%

7.8% is known as the margin of error.

12/ So, can we say “A majority of my @twitter audience think ‘Yes’” ?

No. Because there is a small chance that the actual proportion is 49%.

Let’s discuss some FAQs.

13/ FAQ: Why 95%?

95% implies that there is a 95% chance that the actual proportion is within the [48.5% to 64.3%]. There is a 5% (known as “Significance”) chance that it is outside.

Statisticians think 95% Confidence is “sufficiently” large.

14/ FAQ: What happens if we have more samples?

If we had 10x as many or 1,510 responses, of which 850 said yes, then our 95% confidence interval would be [53.7% to 58.8%] or 56.3±2.6%.

In this case, we could say “A majority of my @twitter audience think ‘Yes’”.

15/ FAQ: What about “let’s assume this subset is a random sample”?

This assumption could indeed be false. For example, if only RW supporters answer the poll.

However, in many cases this is not an unreasonable assumption.

16/ FAQ: Could we conclude that a *majority* of #SriLanka think “Yes”?

No. Like I said, Not everyone in #SriLanka is on the internet. Of that not everyone is on @twitter. Of that not everyone will see and answer your poll.

So, No.

--

--

--

Articles on Economics by Nuwan I. Senaratna

Recommended from Medium

What Makes Great Wine… Great?

Fewer Dashboards, More Analysis

Visualizing Beethoven’s Oeuvre, Part I: Scraping and cleaning data from IMSLP

Guide to Data Chats: Convening Community Conversations about Data

Mapping Hard-to-Count Communities for a Fair and Accurate Census

Lessons learned from a data science project on meme popularity

Types of data visualization

New open data: Fusion Lifestyle publishes data to ‘remove the barriers to participation’ among…

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Nuwan I. Senaratna

Nuwan I. Senaratna

I am a Computer Scientist and Musician by training. A writer with interests in Philosophy, Economics, Technology, Politics, Business, the Arts and Fiction.

More from Medium

My favorite interview question

The Secrets of Creating a Vibrant Culture For Your Teams.

Don’t Make This Mistake If You’re a Tech Coach