Hi Seth
Aayush
31

Aayush,

Great questions: On the 97% number, what should be made of those that do not express an opinion?

Their methodology was to use crowdsourcing to rate the abstracts only — it was over 20,000 papers, and reading the entire papers was perhaps too much to ask. However, more than half of the papers rated as “no opinion” from an analysis of the abstract did endorse the concept of anthropogenic global warming in the full paper (as self-reported by the paper authors in follow-up email interviews). There was also a decrease in the percent that declared a position in the abstract over time — the presumption is that, as the consensus became established, fewer papers felt compelled to take a position in the abstract itself.

The 97% number has been replicated by other means — a different set of researchers did a survey looking at the public positions of climate scientists, and again found 97% support. You can see more information about the projects here: https://skepticalscience.com/tcp.php?t=faq

You are right to ask this question — it was just too much detail for me to go into in the blog post.

As for the drawbacks to Hill, like anything, it can be applied poorly, and because it doesn’t spit out a number at the end we need to be on guard to make sure we are not being lazy in our application. (Of course, the trouble that science has been having with p values shows that quantification alone is not enough protection from self delusion.) So, yes, I have seen at least one attempted application where a blogger was using it to prove that GMOs are “bad”, but never went into a scientific definition of what “bad” is. It was smoke and mirrors, and it was pretty obvious. But it’s an example of mis-use.

A specific detail of the criteria that is often discussed is the dose-response relationship — for some systems, there may not be a linear relationship (for example, a threshold change may exist). But if you declare and evaluate that weakness going in, I don’t see it as any worse of a problem than the difficulty of doing experiments on the planet.

Thanks for reading!

Like what you read? Give Seth Miller a round of applause.

From a quick cheer to a standing ovation, clap to show how much you enjoyed this story.