Small sample sizes in neuroscience
Making the rounds on Twitter is a brand new article, published today, on why it’s a bad thing that neuroscience studies have low statistical power. (The article in question is Button et al., 2013, with full text available here.)
Statistical power refers to the likelihood that an experiment will be able to detect an effect statistically, should it be present. Number of observations (in this case, number of participants) plays a role. For instance, if I tell you that I’m a very lucky coin flipper and most often flip heads, it’s likely that if I flip five coins and get four heads you won’t be very impressed.
But if I then proceed to flip 500 coins and get 400 heads, you’ll probably bankroll my trip to the next casino. Even though my head-flipping ratio is the same in both instances (80%), the number of observations in the second example provides a much clearer picture of my coin-flipping ability. The second experiment is much more powerful.
When most researchers consider power, it’s in the sense of “if I don’t have the power to detect if my experiment has an effect, I ought not do it.” However, Button and colleagues (2013) make another interesting point: If a large number of low-power studies are run and the successful ones are published, an inaccurate picture of the actual size of the experimental effect is painted.
Unfortunately for neuroscience, many studies carried out in this domain feature very few participants. Being friends with a few neuroscientists, I can tell you partly why this is: This kind of research is both expensive and time-consuming to conduct. And although many of the people I know work very diligently to avoid power issues in their research, it seems like the rest of the field may still suffer because of these challenges.
You’ll have to read the article for the whole story. Check it out!
Button, K. S., Ioannidis, J. P. A., Mokrysz, C., Nosek, B. A., Flint, J., Robinson, E. S. J., & Munafò, M. R. (2013). Power failure: Why small sample size undermines the reliability of neuroscience. Nature Reviews Neuroscience.
