“Caffeine can be both a treatment and a trigger for migraine headaches, which makes it difficult for sufferers to know how much to sip. Experts believe caffeine helps block adenosine, … it’s also an ingredient in many over-the-counter migraine drugs. But, counter-intuitively, some migraine sufferers also say consuming caffeine can bring on their debilitating headaches.”
TIME reports about a new scientific study on coffee and headache in people who suffer from migraines. The story opens with some common sense that coffee might trigger headaches or alleviate them, suggesting that the study unraveled what coffee does for headaches. But no.
The study only investigated the triggering effect of caffeine. The researchers had asked 98 migraineurs to record health behaviors and headaches for six weeks. Caffeine intake was obtained from coffee, soft drink, and tea consumption, and counted as servings per day.
The main findings of the study are shown in this figure taken from the scientific article (some details masked for focus).
The odds ratio expresses the odds of having migraine headaches for each number of servings compared to the odds when caffeine intake is zero. The dots are the odds ratio estimates and the lines the statistical uncertainty around them. As all the lines cross the dotted line, it cannot be concluded that the odds ratios were statistically different from 1. Hence, it cannot be concluded that higher intake of caffeine increased the odds of having headaches.
But that was not where the study ended.
The graph does show that the odds ratios seem to get higher with more servings per day. There might be a trend. Researchers can test a trend by finding a line that is closest to all dots. Evidence for a linear trend suggests that drinking more coffee increases the risk of headaches.
But the article does not report about a linear trend. It reports about a non-linear trend. The authors fitted a curved line and found that the test for this quadratic trend was statistically significant. It was the only statistically significant finding in the study. The quadratic trend was not only found for having more headaches the same day, but also the next day. Drinking more coffee increases the risk of headaches. Finally.
But that should not have been the end of the study.
The observation that drinking more coffee also increased the risk of headaches the next day was interpreted as confirming their finding. I think it should have raised doubts.
Below the x-axis in the last graph, we see the number of headaches that were found for each number of servings. These numbers are much higher than 98, the number of people that participated in the study.
The graph shows the caffeine intake for each day of the 6 weeks the 98 participants kept their diaries. Together, they documented 4,467 days during which they had 825 headaches. The authors did use a statistical method that deals with multiple observations from the same persons, but that may not have been enough.
How much does your coffee intake vary from day to day? Do you drink no coffee on some days and four cups on others?
The participants didn’t. The authors write that “at baseline, 20% of the participants reported that they typically do not consume caffeinated beverages, 66% reported consuming 1–2 servings per day, and 12% reported intake of 3–4 servings per day.” They also observed that “all participants had caffeinated beverages on at least one day during the study, resulting in some within-person variation in exposure”
Some within-person variation? I read this as maybe not enough variation. I read this as that the people who drank zero or one serving per day are different people than those who consume more than three servings a day. These people differ in their caffeine intake and likely in many other aspects that were not taken into account. The people on either end of the scale may have totally different lifestyles, be incomparable, a problem that statistics cannot overcome.
It might be that coffee triggers headaches only when you drink too much of it, but this study doesn’t convince.