Supplementary reading for Scenes from a Replication Crisis.

Guinness, Agriculture, and Inferential Statistics

• The origins of hypothesis testing: Student (1908); Fisher (1922); Fisher (1925); Neyman & Pearson (1933).
• Historical context on hypothesis testing by Box et al. (1987); Halpin & Stam (2006); Lenhard (2006).

File Drawers, P-hacking, and Statistical Death Rays

• Publication decisions and statistics: Chandler (1957); Sterling (1959); Sterling et al. (1995); Fanelli (2009).
• The “methodological paradox” of theory testing in psych: Rozeboom (1960); Meehl, P. (1967); Tversky, A., & Kahneman, D. (1971).
• A lack of power: Cohen (1962); Cohen (1990); Cohen (1992); Cohen (1995); Ioaniddes (2008)
• The “file drawer” problem: Rosenthal, R (1979).
• Campbell’s law: Campbell, D. T. (1979).
• It’s called p-hacking: Simmons et al. 2011; Head et al. (2015).
• Most published research is false: Ioannidis, 2005; McCarthy et al. (2008); Begley & Ellis (2012); Button et al. (2013); Szucs & Ioannidis (2016).

Replicating Psychology

• ESP exists (p<0.05): Bem (2011); Commentary by Wagenmakers et al. (2011); Failed replication by Galek et al. (2012); Supposed replication by Bem et al. (2015); Why anyone listened: Bem (1967).
• Evidence from Doyen et al. (2012) that unconscious priming [Bargh et al. (1996); Aarts et al. (2002); Cesario et al. (2006)] does not replicate.
• Daniel Kahneman’s open letter.
• Evidence from Hagger et al (2016) that ego depletion [Baumeister et al. (1996); Sripada et al. (2014)] doesn’t replicate.
• Evidence from Wagenmakers et al. (2016) that facial feedback [Laird (1974); Strack et al. (1988)] does not replicate. Context: Schacter & Singer (1962); Duclos & Laird (2001); Maslach (1979); Marshall & Zimbardo, 1979
• Fraudulent activity: Diederik Stapel; Jens Förster; Marc Hauser; Michael LeCour
• It’s a replication crisis!?: Klein et al (2015); Open Science Collaboration (2012); Open Science Collaboration. (2015); Nature survey.
• No, it’s not a replication crisis?!: Stroebe & Strack (2014); Gilbert et al (2016).

Circularity, Salmon, and the Human Brain

• The possibly seductive allure of brain stuff: Weisberg (2008); McCabe & Castel (2008); Farah & Hook (2013); Weisberg et al. (2015).
• Reverse inference: Aguirre et al. (2003); Poldrack (2006); Poldrack (2011). Examples: This is your brain on politics; You love your iPhone. Literally.
• Non-independence: Cureton (1959); Bennet et al. (2009); Vul et al. (2009); Vul & Kanwisher (2010); Commentary by Lieberman et al. (2009); Poldrack et al. (2009); Fiedler (2011).
• Data sharing and the fMRIDC: Fox & Woldorrff (1994); Van Horn et al. (2002); Van Horn & Gazzaniga (2013).
• Software problems: Eklund et al. (2016); Eickhoff et al. (2016).
• Best Practices: Poldrack (2008); Poldrack et al. (2016).

Confounding Variables

• The job crisis in STEM is either a myth [Charette (2013)], terrifyingly real [NAS (2005a); NSF 2015], or fraught with bad definitions [Casselman (2014); BLS (2015)]
• Is there a shortage of STEM expertise in the US ? Yes [NAS (2005b)]. No [Teitelbaum (2014)].
• Adjunct-ivitis; Government Funding Trends