How can you tell if a study is good?
In the years that I wrote Evidence-Based Fitness, one of the most common questions I got from readers was, “How can I tell if a study is true/good/worth it?”
When I look back on this experience, it’s a question that I never really had a good answer for, but it’s a question that bounced around in my head thanks to you, the readers, for asking it. How can someone who has little to no training tell if a study is good? Do they need to get a Master’s? Or a PhD?
I think if you want to get really efficient at something, you have to train at it. If you wanted to get really FAST at reading research and knowing whether it was good or not, you would need to train for it; and I’d be happy to be the coach that gets you there.
I’ve seen the free online guides on how to read research. I’ve read most of the books on how to read research. And even with a PhD, I hated every one of them. If I put myself into the shoes of someone with not a lot of experience, those guides and books made me not want to do it.
The reason why I hated them was because they made it look like the skill was in the tiny details: Details like, “Was this research sponsored?” or, “How many people were in the study?” or even, “Were the results statistically significant?”
In my near two decades of peer review, I’ve rejected a lot of manuscripts. Rejection rarely happens because of one single flaw in the tiny details. Rejection happens because of major flaws; because something BIG is wrong or inconsistent.
That BIG thing, usually has to do with the original research question — the entire reason for why the study was done in the first place.
So 10 years after starting Evidence-Based Fitness, I finally have an answer that I’m happy with to the question, “How do I know if a study is good or not?” and that answer is, “Learn how to recognize and analyze the question.”
The best news about this answer? You need literally zero statistical knowledge to do it. Literally zero. And yet, it’s what will get you from struggle-bus, to streamlined the fastest; because when you get good at it, it will eliminate your need to spend any time on the vast majority of irrelevant research out there.
I can’t thank you enough for putting this question into my head. I’m sorry it took so long for me to come up with a good answer.
If you’d like to learn more, I wrote a short book with a long title to get you on your way. It’s available on Amazon and it’s called, “Question the Start and End with the Question.”
As always, you can follow what I’m doing at criticalmass.ninja and on Medium
