There are two kinds of stories that we can’t resist. The first is a story about sex. Anything even vaguely related to boning down fascinates us, which makes sense because sex is great. The second are stories about common foods that scientists have suddenly discovered are not just boring produce, but might actually cure us of our ills as well.
From the New York Post to the BBC, media everywhere is agog at the idea that tomatoes might be able to save your sperm. The problem is, that’s not nearly as certain as the news articles suggest.
Tomatoes for infertility sounds like a dream come true, but in reality the evidence doesn’t really stack up.
Tomatoes probably aren’t going to supercharge your sperm.
The study that has everyone drinking tomato paste for their little swimmers is what’s known as a randomized controlled trial of lycopene supplements for sperm quality. Essentially, the scientists randomly split up 56 healthy men into two groups to receive either lycopene or a placebo control. After 12 weeks of, uh, semen collection, they found that the men who’d received lycopene had improved a bit on two measures of sperm quality while the men who had taken the placebo did not, and concluded that lycopene appeared to be beneficial.
What does this have to do with tomatoes? Well, tomatoes contain quite a bit of lycopene — the amount used in the study, 14mg per day, is similar to eating 1–2 large tomatoes a day. So the idea is that eating a few tomatoes might be enough to get the benefits of this trial, without the expensive supplements.
The problem is, the trial had quite a few issues that never made it to the media. It’s quite likely that the results are not very meaningful after all.
The first thing that the news stories missed is that the study actually failed to find an effect for their main outcome of interest. The pre-specified primary outcome for the study — the most important thing that the scientists were interested in investigating — was motile sperm concentration, which lycopene didn’t improve. In fact, out of twelve aspects of sperm quality that the scientists tested — from semen volume to DNA damage — only two had any statistically significant differences at all.
In fact, one thing you could say from this study is that lycopene doesn’t help with the most important aspect of sperm quality, and has no effect on most other measures as well*.
But it gets worse from here. If you look at the table above, you’ll see that the statistical comparisons are comparing these groups against each other. Basically, they measured each group before, and after, and tested whether there had been any change within that group’s score.
This sounds like a minor point, but it’s actually really important. When you run a trial, you often see that people are improving even if they’re not being treated — it’s commonly called the placebo effect. There are a variety of causes of this, but a simple solution: use a control group. By having a group of people who aren’t being tested with the supplement, you’re eliminating the possibility that your results are entirely down to luck.
And this study did have a placebo control. Except, they didn’t use it!
By only comparing the differences within groups (i.e. comparing the lycopene group only against themselves) the study made the results almost useless from the get-go. It’s impossible to know if the improvements seen in the lycopene group are due to the supplementation itself, or if random chance was behind the benefits.
On top of this, the study didn’t correct for multiple comparisons. I’ve written about this before — when you run lots of statistical tests, it’s a bit like flipping a coin: sooner or later you’ll see heads. What this means is that running statistical tests is about probability. When a paper reports that a test is significant, what they mean is that there is a low chance that this finding could’ve happened through pure happenstance. But the trouble is, if you run enough tests, you’ll eventually see a positive result, even if the truth is that there is no effect (what’s known as a false positive result). There are statistical methods to correct for this, but the lycopene paper didn’t use any of them.
So not only are we not sure if the improvement in the lycopene group was down to the supplements or luck, we also don’t know if the improvements were real at all! It might just be that the researchers flipped enough coins to get a result, even though there was no actual impact of the lycopene at all.
Not quite the “tomatoes boost sperm quality” narrative you heard in the media, is it?
It is, of course, possible that tomatoes are the next big thing for sperm, but it’s worth being cautious. This was a small study that definitely didn’t prove anything either way, especially when you consider the issues with statistical analysis.
That being said, the current evidence for lycopene as a sperm energizer is not great. This study is the biggest to date, and as we now know it didn’t find many benefits at all. At this point it seems quite likely that lycopene is a total waste of time as a supplement for infertility, although we’d need a bigger trial to be totally sure.
Why was this study written and reported so positively when it didn’t really find much? Well, a clue might be in the funding body: Cambridge Nutraceuticals, the company that makes the patented LactoLycopene supplement used in this trial. The authors are very clear that the funder played no role in their study design or implementation, but it’s interesting that seemingly fairly boring results have been hyped up so much across the globe.
Maybe I’m just being cynical.
If you’re worried about your fertility, eschew the tomatoes and go see a doctor. Chances are they’ll help you a lot more than some lycopene.
You can now listen to Gid on the Sensationalist Science podcast for your weekly dose of scientific shenanigans and media muddling:
*Note: It’s also worth noting that even the authors acknowledged that the improvements seen in those two variables might not be clinically significant — they found a numerical difference using statistical tests, but there might be no improvement in terms of actual outcomes like fertility.