How to mislead with graphs
In an earlier post, I wrote about publication bias, which can adversely affect synthesis in knowledge translation (KT) — and by extension, our perception of truth on a given subject.
There are several other forms of bias that can arise throughout the research process, each harmful in its own way. This week, we’ll explore a form of bias that can occur during dissemination. Dissemination, commonly referred to as “end-of-grant KT”, includes the distribution and sharing of information and research results. Often, dissemination takes the form of presentations and publications, many of which feature graphs and figures.
While graphs can be a valuable tool to help summarize data into a compelling story, they can also inadvertently be used incorrectly or worse, to mislead. This can make results appear different than they actually are and/or lead others to draw incorrect conclusions. So here are a few tips to look out for in your own work, and when interpreting the results in the work of others.
Caution: the following graphs are of good data, used incorrectly by me.
1. Axis and scaling
A common misleading feature in graphs is a skewed scale of the y-axis.
Say, for example, we were looking at the annual population growth of Winnipeg. The two graphs below show the same data for the population of Winnipeg from 2013 to 2015, and they both tell the story of a population that is increasing.
At first glance, the top graph tells the story of a population that appears to have doubled in size from 2013 to 2015. Just look at the difference in size of the bars! But a closer inspection reveals its y-axis scale starts at 685,000 rather than zero, which biases the graph to show huge increases relative to the population, when that’s simply not the case. The bottom graph is a more accurate depiction of the information because it shows the population has increased, but represents the growth proportionally.
Data source: http://www.winnipeg.ca/cao/pdfs/population.pdf
2. Missing data points
Something similar can be said about problems with the x-axis. If some data are missing, as is the case in the following example, they can tell a different story than when all of the data are presented. The two graphs below use the same data source, looking at the number of individuals assisted by food banks in Manitoba. The top graph only shows data for the years 2008, 2012 and 2015, while the bottom graph shows data for all years between 2008 and 2016. Because of some missing information, the two graphs tell different stories. The first appears to show the number of individuals assisted steadily increasing year over year between 2008 and 2012. Meanwhile, the bottom graph is a more accurate reflection of reality, showing more volatility from one year to the next.
Sometimes data just doesn’t exist for certain years. When that’s the case it should be highlighted. Similarly, if you ever notice data is missing, you should be critical, questioning whether it’s just not available or if it’s purposely being left out.
3. Misused pie charts
What’s wrong with the chart below? Pie charts are for slices which, together, make up 100% — i.e. the whole pie. If the numbers don’t add up to 100%, the pie chart has been misused. When categories are not mutually exclusive like below, they would typically be better represented as a bar graph.
Data source: http://www.statcan.gc.ca/pub/11-631-x/11-631-x2016001-eng.htm Proportions of individuals 65 years of age and over with selected chronic conditions, Canada 2008, 2009.
4. 3D graphs
In general, 3D graphs are misleading. They throw off proportions and make things look big or small depending on the angle. Here is the same pie chart, now in 3D. We already know I am using this pie chart completely incorrectly, but when it is presented in 3D, the data are even more skewed. The “arthritis” piece in the foreground looks bigger than “high blood pressure” in the back just because of the placement.
In this day and age, with abundant false information [and alternative facts] readily available online, it is important to be critical of information being presented to you. Question how the evidence is being displayed and where the data came from. Similarly, when presenting and sharing your own results make sure you are not misleading others unintentionally.
For fun examples of misused graphs in Fox News check out: http://simplystatistics.org/2012/11/26/the-statisticians-at-fox-news-use-classic-and-novel-graphical-techniques-to-lead-with-data/
About the author
Gwenyth Brockman is a Research Assistant with the George and Fay Yee Centre for Healthcare Innovation in the Knowledge Translation Platform.