Great claims require great evidence.
The claim that social media and screen time cause depression and suicide therefore needs evidence of the highest calibre. Yet this kind of evidence does not exist.
Jean Twenge’s newest paper, “Increases in Depressive Symptoms, Suicide-Related Outcomes, and Suicide Rates Among U.S. Adolescents After 2010 and Links to Increased New Media Screen Time”, published in Clinical Psychological Science, sets out to provide such evidence - but falls short of this discernible goal. While the paper links social media and technology use to depressive symptoms and suicide rates, the links are so weak and inconsistent they could be artefacts of statistical error. Having re-run parts of Twenge’s analyses on a more recent dataset, I also found no compelling evidence for her claims. Researchers and non-researchers alike need to therefore treat the article’s claims with caution.
Weak Links
While Twenge finds a link between social media use and depressive symptoms, this link is very small. Social media use only explains 0.36% of the covariance for girl’s depressive symptoms; less than half a percent of the depressive symptoms a female student reports can be predicted by knowing how much social media use she reports. 99.64% of her depressive symptoms have nothing to do with social media use. The link is so small, it could well be due to statistical noise. Common method variance, for example, is statistical noise in a dataset because certain people react to measurement tools, for example personal questions in a questionnaire, in a specific way.
Inconsistent Links
The paper only finds a link between depressive symptoms and social media use for girls, not for boys. For boys, social media use only explains 0.01% of the covariance for depressive symptoms. It does not account for 99.99% of depressive symptoms. This is non-significant and extremely small. It is unclear why the effect is only present for girls; we need to be cautious of saying that social media use causes depression, if the effect does not hold for half of the analysed population.
Questionable Links
After reading the paper, I wanted to examine the data myself. I analysed the newest addition to the Monitoring the Future dataset, re-examining some of the paper’s basic claims. Monitoring the Future is an annual questionnaire given to US high school students. While Twenge analyses the data collected between 2010 and 2015 in her paper, I analysed the data collected in 2016. While this is a smaller dataset, it still includes 21,694 students. As I am working on a different dataset, I am not refuting what Twenge found, I am solely pointing out whether the effects that she finds exist in are more recent edition of the data. Furthermore, due to time limtations, I am only analysing the Monitoring the Future dataset and not the YRBSS dataset Twenge also analyses in her paper; the YRBSS contains an hourly measure of screen time, and shows larger effects on suicide related outcomes (accounting for 1.7% of its variance).
I encountered my first problem very early on in my analysis, the question Twenge chose to measure social media use is highly problematic. Students could answer that they use social media ‘never’, ‘a few times a year’, ‘1–2 times a month’, ‘once a week’ or ‘almost daily’. The overwhelming majority chose ‘almost daily’, so there is very little variance that the further analyses can account for. I do not know how students responded to the question prior to 2016, but I dare to guess that most students also used social media ‘almost daily’ in 2014 or 2015.
I, however, continued the analyses and examined the link between social media use and depressive symptoms, loneliness and low self-esteem. I found correlations of 0.013, 0.011 and 0.012 respectively. Social media use therefore only explained 0.01% of the depressive symptoms, loneliness or self-esteem of students in 2016, again 99.99% of depressive symptoms are not explained by social media use. This is a tiny effect and does not merit extensive discussion.
Problems with Using Weak Links to Draw Conclusions
It is problematic to draw grand conclusions with widespread implications using such weak and inconsistent links. As humans we have a bias to confirm our previous hypotheses. Currently public debate is focused on social media’s harmful effects, it is therefore easy to use Twenge’s evidence as confirmation that social media is overwhelmingly harmful. We are, however, still missing concrete scientific evidence that this is the case. If you want to delve deeper into this topic, you can read my previous blog post.
Twenge’s paper fails to account for many other factors that could have lead to an increase in adolescents’ depressive symptoms in the past decade. Teens can suffer because of their parents economic situation, bereavement or stress about examinations or the future. Furthermore, the public discourse around mental health has changed, leading to differences in diagnosis and increasing levels of confidence about sharing mental health issues.
We need more good quality, open and replicable science before we can start making grand claims about social media’s effects. Great claims require great evidence - and great evidence has not yet been found.
Thank you to my colleagues Professors Andy Przybylski and Patrick Markey for their helpful and insightful comments. Andy has recently published a paper investigating the link between social media use and well-being, read more about it here.
