What’s behind the sausage wars? Three questions to ask of any contested claim
Only 6 months ago, UK tabloid newspaper The Sun was proclaiming that a ‘Rasher of bacon a day is deadly’. This week its front page declared that ‘Bacon’s safe — no ifs or butties’. We’re all used to this constant swinging opinion on foods from ‘safe’ to ‘dangerous’ and back in the media — but why does it happen, and how can we get at the truth behind the claims?
While both headlines are somewhat exaggerated, the change did reflect the conflict between this week’s recommendation by a group of scientists that people could “continue their current consumption of both unprocessed red meat and processed meat”, and standard health guidelines such as eating less than 70g a day. Harsh criticism by other ‘boffins’ (to use Sun-speak), such as “These papers and conclusions represent the most egregious abuse of evidence that I have seen” and “These authors I think are wrong about absolutely everything”, led to the argument becoming the story.
I pity the poor public trying to make sense of all these arguing scientists, an unedifying spectacle that does little to improve trust in health advice. So who should we believe? When confronted with some contested scientific claim, I try to find the answers to three fundamental questions -
* Why am I hearing this?
* What does the evidence show?
* Are the claims trustworthy?
Why am I hearing this?
The authors of the controversial paper behind the headlines were a group of fourteen researchers, headed from Canada but from seven countries, who do not represent an official body. They had already published their general ‘NutriRECS’ approach to developing nutritional guidelines, in which they were extremely critical of current efforts, saying “Guidelines issued by several organizations, including authoritative governing bodies, suffer from major limitations in their trustworthiness, relevance and usefulness for practice.”
They also claimed that “guideline development requires consideration of patient and community values and preferences,” including “individuals’ predisposition to either favour (like) or not favour (dislike) something such as red meat.”. Their approach, using a formal, almost algorithmic, data-led methodology for both collating evidence and reaching a recommendation, was clearly intended to be disruptive.
They produced a huge study, comprising hundreds of pages in total, and the Annals of Internal Medicine, a reputable US medical journal, decided not only to publish it but promote it with a bold press release.
Publication was greeted with strong criticism from people and organisations which have made their reputations in this area, such as the Harvard School of Public Health, and there was even an (unsuccessful) attempt to halt publication: David Katz, derisively referring to the authors as ‘statisticians’, formed a group that ‘ took the unprecedented step of asking the Editor in Chief and editorial office at the Annals to withhold publication of these papers pending a full review of related concerns’.
My conclusion: I am hearing this because a group seem to have set out to disrupt the standard approach to setting nutritional guidelines, the medical media promoted the story, the recommendations contrasted with standard guidelines, and criticism from others encouraged media coverage.
What does the evidence show?
Nobody has claimed that the evidence presented was either novel or surprising. They estimated, for example, a 7% reduction in lifetime risk of bowel cancer associated with 3 fewer servings (of 50g each) per week, which is corresponding to a 16% reduction for a daily reduction of 50g. This is very similar to the 18% previously estimated by the World Cancer Research Fund.
However, they took the vital additonal step of converting these relative risks into absolute risks. For example, Cancer Research UK estimate a lifetime risk of bowel cancer of around 6%. Therefore we can estimate that the lifetime risk of daily bacon-eaters is around 7% (since 6% is a 16% relative reduction from 7%). So out of 100 daily bacon eaters, we would expect 1 extra case of bowel cancer in their lifetimes. That’s from around 1,500,000 bacon sandwiches.
We’ve used this way of communicating the risk for many years, including explaining this to school children in 2011, and it was gratifying to see BBC TV adopting this established technique.
[Note: the Annals papers say there is a 2% lifetime risk of developing colorectal cancer compared with the 6% figure from CRUK, producing far smaller estimated absolute risk reductions. The 2% figure is based on a standard Globacan source, which uses a different methodology to that of CRUK and only covers life below 75. To be honest, this seems odd, given that cancer is a disease of old age.]
As discussed in detail in a fine article by Julia Belluz in Vox, the novelty in the new study is their use of the ‘GRADE’ scale for assessing ‘certainty of evidence’, which was designed to assess medical interventions, but is now being more widely applied: more than 110 organizations endorse it as a means of judging the quality of the underlying research. The GRADE scale strongly downplays the kind of observational studies that are the main basis for nutritional guidelines, and the researchers also ignored animal and laboratory studies — this appears reasonable, given they were not seeking to show if meat is carcinogenic but were solely interested in magnitudes of risk.
They concluded that all the evidence provided either ‘low’ or ‘very low’ certainty. The BBC reported me as saying “This rigorous, even ruthless, review does not find good evidence of important health benefits from reducing meat consumption. In fact it does not find any good evidence at all.” They could not conclude confidently that meat is either safe or dangerous: “We’re not saying there is no risk, we’re saying there is only low-certainty evidence of a very small reduction of cancer and other adverse health consequences of reducing red meat consumption.”
Crucially, they also reviewed the evidence on why people ate meat. They found “Low-certainty evidence suggests that omnivores are attached to meat and are unwilling to change this behavior when faced with potentially undesirable health effects”.
My conclusion: weak evidence suggested small magnitudes of harm associated with eating red and processed meat. And people quite like eating meat. We also happily live our lives exposed to things that we could not classify as either safe or dangerous: or example driving a car, or just getting out of bed in the morning.
Are the claims trustworthy?
The final recommendations of the NutriRECS team were:
Recommendation for Unprocessed Red Meat
For adults 18 years of age or older, we suggest continuing current unprocessed red meat consumption (weak recommendation, low-certainty evidence). Eleven of 14 panelists voted for continuation of current unprocessed red meat consumption, whereas 3 voted for a weak recommendation to reduce red meat consumption.
Recommendation for Processed Meat
For adults 18 years of age or older, we suggest continuing current processed meat consumption (weak recommendation, low-certainty evidence). Again, 11 of 14 panel members voted for a continuation of current processed meat consumption, and 3 voted for a weak recommendation to reduce processed meat consumption.
These conclusions are reasonable, given the approach taken by the authors, but unfortunately they did not make clear that a ‘weak recommendation’ means: Recognize that different choices will be appropriate for different patients, and that you must help each patient arrive at a management decision consistent with her or his values and preferences.
The coverage therefore suggested far more certainty than seems to have been intended by the authors, who were remarkably ambivalent, both in terms of the minority voting in favour of recommending reduction, and the extent to which individual preferences should come into any recommendation.
So how could two groups of ‘experts’ come up with such different conclusions, given they broadly agree of the evidence? David Aaronovitch in the Times identified the critical underlying issue behind the ensuing conflict: whether we take an individual- or a population-based approach. Essentially, the authors point out that any absolute risks are small from an individual perspective, and may generally be cancelled out by the enjoyment of eating, and the bother of changing habits. But these small benefits can be important from a public-health, population-wide perspective, since a lot of people making a small change, that only reduces their risk by a personally-negligible amount, can add up to thousands fewer cases of disease.
That’s what has generated the disagreement. It can be perfectly reasonable for guidance to be given by authorities, and it can also be perfectly reasonable for individuals to ignore it. Both can be ‘right’.
My conclusion: the claims were trustworthy, given the approach that the NutriRECS team had publicly announced. But the recommendations were very poorly communicated: they did not make clear that a weak recommendation meant that the decision should be individualised to each person.
Overall, personal, conclusions
While we are in a disruptive mood, it may seem reasonable to ask the seemingly-naive question: why do we need these paternalistic guidelines in the first place? The approach of our Winton Centre is that people should be provided with the evidence in a trustworthy, balanced, and comprehensible way, and allowed to make up their own minds. Which would require taking into account how much we each relish a good sausage.
There are other issues. If we really want to change behaviour, then exhortation is not the most efficient way — a recent Cambridge study showed that simply providing more vegetarian options in cafeterias substantially decreased meat consumption. And the NutriRECS team have been criticised for ignoring the environmental impact of meat: of course that’s not what what the study was about, but for me it’s the biggest reason for cutting back on meat.
So what would I recommend, were anyone trusting, or foolhardy, enough to actually ask me? With such low-quality evidence, that nevertheless points to potential for harm, a precautionary approach might be reasonable. So if I were talking to someone who was a moderate meat consumer, I might say something like:
“There is some evidence of small increases in risk, but there is no good evidence that the meat will cause serious harm to you. If you want to be on the safe side, then cut down. And that will also help the planet”
************************************************************
[I am grateful to colleagues in the Winton Centre for Risk and Evidence Communication for their insightful comments on an earlier draft]
