Imagine that you are flipping through channels on a flight from Boston to Chicago. What is the probability that you will learn something?
I inflicted this (unplanned) “experiment” on myself earlier this week so that — you know — you don’t have to.
What I saw was not that encouraging:
- On MSNBC, the “Tiger Mom” was talking about “why some cultural groups are more successful that others.”
- On Fox, a doctor was selling “A Short Guide to a Long Life.”
- On CNN, twin brothers talked about their dieting experience: one brother stopped eating sugar, the other brother gave up fat.
- On CNBC, an analyst was asked which stocks were worth buying. He named several firms, and offered a deep justification: you should invest in these firms “because they are well-managed.”
- A journalist on some other channel asserted that the Tiger Mother “has the answer” to success (I wish I was kidding).
This is not a rant about standards for evidence — though they ought to be much higher.
If your study has a tiny sample size, if your data is so noisy that it can only serve as a parable, if your motivation for “research” was to become famous… Then I am not saying you necessarily must not be on television. You might still inspire or entertain people.
Besides, even you wrote a well-executed study, there would still be potential statistical pitfalls, unanswered questions, and issues with applicability outside of the original context. Imperfect studies can, and will be, covered by the media. Fine.
What is disturbing, though, is that many journalists apparently considered the above to be “newsworthy.”
When two twin brothers go on a diet (did I say they did that for one month?) then it is just not news. Sorry.
To talk about the experiences of two people, as if a no-sugar or a no-fat diet might be worth emulating, is irresponsible.
When Amy Chua and her husband were interviewed about their book, most of the conversation was about… You guessed it: “Are you sure you aren’t racist?” (I’m paraphrasing here.) You might worry, as sensible journalists do, that
Many parents bestow or withdraw affection depending on how well their children are achieving, producing millions of young people without secure emotional foundations, who pine for any kind of approval.
You would not learn much about these drawbacks, though, if you listened to the authors describing their ‘noble mission’ to elevate those who never understood why it is important to work hard.
I just do not think that what we have today is a good model:
- A journalist identifies a catchy or sexy story or personality.
- She lays some “information” in front of us (like: ‘these two brothers went on competing diets’).
- She lets the participants interpret / sell their story.
- She does not encourage viewers to be skeptical, does not warn the audience that the conclusions may not follow from the premises, does not check “data quality,” and does not question whether the guests are even trying to answer the original question in an honest fashion.
I am not arguing that the media has the responsibility to turn viewers into forensic statisticians.
But I believe that things can be done differently.
I am hopeful that these guys will do a great job writing actual news stories and putting them in context.
Still, I wonder what would happen if the following chart appeared on television, without any sarcastic comment attached to it:
I’d hope nobody would conclude that the blue line and the green bars are related. But how much money would you bet on it?
So, let me conclude this post by not answering the question in the title. Apparently, it is now fashionable to just ask a question, make a few related comments, and then ignore it… I obviously do not know whether fast media or fast food is more unhealthy, but it is probably not a bad idea to watch the intake of both.