Reporting statistics (health, science, surveys, opinion polls)
Even brilliant reporters get numbers wrong once in a while. This workshop session is about avoiding pitfalls when interpreting surveys, health research, science, and social science studies.
- When reporting on changes (increase/decrease), put the numbers into context and decide whether the change should be described in percentage or percentage points.
- When the change is more than 100 percent, don’t use percentage. Say “double,” “2.5 times,” and so forth so that the audience understands it better.
- When the size is small, a change in percentage is deceptive. Report the actual figures as well.
- Report the sampling error as well as the sample size.
- Be mindful of “the range” when the results are generalized to the entire population.
- When the results are broken down to a specific demographic (subgroups), be very careful (in general, the smaller the sample size, the bigger the sampling error).
- Sampling methods matter. A study of 2,000 high school students randomly chosen across the country is fundamentally different from 2,000 students from two schools in a city.
Things reporters should ask
- Are you presenting preliminary findings or something more conclusive?
- What’s the sample size and what was the sampling method?
- Was there a control group?
- What is the limitation of your findings?
- If other researchers try to replicate the study, do you think they will see the same results? (Has this study been replicated by other independent researchers?)
- [If appropriate]
Who funded the study? Where did the money come from?
Recap lessons from Making Sense of the News
Understanding scientific research (and the news)
Recommended resources for journalists
- CNN’s transparency questionnaire for polling standards (CNN)
- Regression analysis: A quick primer for media on a fundamental form of data crunching (Journalist’s Resource)
- Statistical terms used in research studies: A primer for media (Journalist’s Resource)
- Polling fundamentals and concepts: An overview for journalists (Journalist’s Resource)
- Scientific method: Statistical errors (Nature)
- Science isn’t broken (FiveThirtyEight)
- Understanding Hypothesis Tests: Significance Levels (Alpha) and P values in Statistics (Minitab Blog)
- Spurious correlations (tylervigen.com)
- An unhealthy obsession with p-values is ruining science (Vox)
Recommended free online courses
- Math for Journalists: Help with Numbers (Poynter Institute)
- Understanding and Interpreting Polls (Poynter Institute)
- Probability and Statistics (Stanford University)