The other side of Bayesian statistics
Our final class reading is an opinion piece by Kruschke (2013) on the failings of traditional null-hypothesis significance testing (NHST). And it sure is opinionated. According to the author, you might think that NHST is the statistical antichrist. While this may seem extreme, there is almost certainly an underlying truth to it: conventional NHST has been elevated to an uncomfortable level of precedence and something needs to be done about it. The author’s answer to this is a concept not usually covered in traditional statistical classes: Bayesian statistics.
The reading begins with Kruschke providing a general overview of the failings of modern research and researchers. He notes that researchers consistently cite Bayesian methods but then rarely use them and apply them. Further, research always, always seems to come back to the p-value. I covered this topic in an earlier post and it has recently become a major point of contention in the research community. Two major problems with the p-value, Kruschke (2013) notes, is that it does not allow for trade-offs among the parameters and causes inaccurate confidence intervals. Further, traditional research seems to overly pigeonhole the data with assumptions and constraints. Additionally, it is so fragile that there are a world of other corrections that are needed, such as Bonferroni and Tukey, in order to account for false alarms created by variation with the p-value.
Perhaps Kruschke’s biggest problem with NHST is that it punishes people for exploring the data and thus is a “feeble foundation for empirical science” (Kruschke, 2013). If it is as bad as Kruschke claims it is, then yes, this is certainly an issue, especially framed within our running theme of intimately and broadly understanding your data. Kruschke’s answer to this is Bayesian statistics, which is (apparently) an analysis well worthy of the complexity and context of data.
In a very general sense, Bayesian statistics, according to Kruschke, overcomes these problems. It is an analysis that is easy to customize to specific situations. It is flexible and useful for scaling and categorization. It handles its power analysis and replication probability significantly better than NHST; it handles the uncertainty that comes with the p-value. It uses prior distributions along with the data to create a solid framework and makes inferences easy (Kruschke, 2013).
All of this considered, and even among the author’s fervor, I couldn’t help but wonder that it felt just a little too easy, just a little too perfect of an answer. I am by no means in a rush to defend NHST, but I feel its fundamental of “I’m not going to say there is a definite cause, just that there is probably something going on here” is still worthwhile. There is still room to be highly aware of context and confounds within NHST. It’s not perfect, but it’s not worthless. In light of this, I felt that the best way to be fair (or maybe petty) was to examine the completely unmentioned downsides of Bayesian statistics. And as luck would have it, there is an equally-passionate objection article by Gelman (2008).
Funny enough, Gelman mentions that his anti-Bayesian objections stem from an idea he had for an April Fool’s blog, which is probably indicative of his feelings towards the concept. His first major objection is that Bayesian methods are considered “an automatic inference engine” that can handle any set of parameters(Gelman, 2013), which should be a red flag to anyone with experience who understands that different methods are useful in different situations. This is also the concern I had; it does seem like Bayesian statistics are purported as a sort of God-tier fix to all sets of data, at least by Kruschke (2013). But to be fair, it is completely reasonable to be passionate about a fix to something you view as a major problem.
His second major objection is that Bayesian statistics use prior distributions as subjective states of knowledge. His concern here is that research should be concerned with objective knowledge and it’s unclear how to analyze subjective knowledge. Bayesian methods are quick to make the computations and analysis elaborate instead of focusing on information available in the data (Gelman, 2013).
Gelman covers many other objections, but he makes it clear that he has no problem with the mathematics of Bayesian methods, just the idea that the goal of statistics is to make optimal decisions. Either way, he is not happy with the prevalence of Bayesian methods: “I can’t keep track of what all those Bayesians are doing nowadays…but I wish they would all just stop already and get back to doing statistics the way it should be done, back in the old days when a p-value stood for something, when a confidence interval meant what it said, and statistical bias was something to eliminate, not something to embrace” (Gelman, 2013).
Overall, it’s certainly an interesting discussion, and even though at times it may seem better suited for a contentious subreddit, there is no doubt that modern research needs challenged.
References
Gelman, A. (2008). Objections to Bayesian statistics. Bayesian Analysis, 3(3), 445–449.
Kruschke, J. K. (2010). What to believe: Bayesian methods for data analysis. Trends in cognitive sciences, 14(7), 293–300.
