How to lie with data, Part 3, or Fully vaccinated account for 91% of deaths in the country
The story
In July 2022, the far-right website wnd.com published an article titled “Fully vaccinated account for 91% of COVID deaths in one country”:
Originally, the article was published by expose-news.com, and then reposted in multiple other sources. “One country” in this case was the UK. A scary chart was included:
Tweets, facebook posts, and all the usual media spread followed.
The data behind the story
The original data comes from the dataset provided by the Office for National Statistics, specifically the edition for the 1st quarter of year 2022: “Deaths occurring between 1 January 2021 and 31 May 2022 edition of this dataset”.
The sheet of interest is called “Table 8”, and it contains monthly counts of registered deaths for unvaccinated and vaccinated people by age group. The deaths involving Covid-19 are separated into their own category.
After loading the data, filtered to the same date range as the article, and calculating the same ratios, we get the following table:
And we can build the same chart:
So why are so many vaccinated people dying? The answer is, unsurprisingly, because there are just so many vaccinated people. I can’t put this any better than this fact check from Reuters:
A spokesperson for the Office for National Statistics told Reuters via email: “Looking at the proportion of deaths among the fully vaccinated without accounting for the vaccination rate is highly misleading. For instance, if everyone were vaccinated, 100% of people would die vaccinated.”
Let’s look at the ratio of vaccinated to unvaccinated people in in the UK. I used the dataset from Our World In Data, filtered to only retrieve the UK data, and only the metrics I’m interested in — the number of vaccinated people, and the total population. I calculated percentages for vaccinated and unvaccinated people.
Plotting this data:
Hold on, you may say. So, close to 80% of the UK population were in some degree vaccinated by May 2022. Then why does the vaccinated group still contribute around 90% of Covid-19 deaths?
Because things are not quite so simple. The groups are very different not only in size, but in other important characteristics, such as age, presence of other medical conditions, risk factors, etc. Older and more medically vulnerable people were vaccinated first, but even so, they remain more at risk of getting severely ill and dying than young unvaccinated people. The vaccinated group has a lot more old and vulnerable people than the unvaccinated one. We can’t directly compare numbers of deaths from two populations that are so dissimilar.
The cited article is a very blatant example of comparing the incomparable. The produced metric looks credible, because it’s so easy to calculate and check for yourself, but it doesn’t make any sense.
Metrics that make sense
What makes sense, however, are the mortality rates within our groups of vaccinated and unvaccinated people. The same dataset from the Office for National Statistics shows those rates in Table 1. The metric is called “Age-standardized mortality rate / 100,000 person-years”. After filtering the data to only Covid deaths, and only two vaccination categories — “Ever vaccinated” and “Unvaccinated” (because we plotted the same two categories above), we have this table:
Mortality rates (age-standartized) are consistently lower in the vaccinated group. Let’s plot this metric:
This metric shows a very different picture. However, age-standartized mortality rates (ASMRs) are not as easy to calculate or understand, as simply taking a ratio of two numbers.
Sometimes, absolute numbers do make sense, and that’s when you’re looking at the total numbers for the period. The dataset from the Office for National Statistics lists total numbers of covid deaths by vaccination status in Table 6:
Let’s plot this data as well:
In every age group, the unvaccinated population contributed the largest number of deaths attributed to Covid-19. The correct numbers do not lie.
Summary
In this example, absolute counts of deaths from two populations of incomparable size and demographics were directly compared against one another. So, today’s lesson on how to lie with data is:
- boldly compare the incomparable.
- maximize the impact by representing the numbers visually.
- as always, omit the metrics that are actually relevant.
The code (Python + Pandas) I used to load, transform and plot this data can be seen here:
https://github.com/j-bennet/talks/blob/main/2022/uk-covid-deaths/uk_deaths_by_vacc.ipynb
If case you missed my previous articles:
