Making Sense
On Sam Harris, free speech, and the scientific method
Sometimes, Sam Harris’s Making Sense podcast really bugs me. Truly, I think Harris is a smart guy — I appreciate his podcast and in particular I applaud his focus on important but underrepresented content such as effective altruism, existential risk, privacy, child sexual abuse, and more. And because of those great things, it’s probably all the more disappointing when he flubs.
In no example is this more obvious than in Harris’s podcast with Charles Murray, who is known fairly widely as a racist, and also as a pseudoscientist. Murray is known for this so much so, that the top hit for a search on him leads you to a page not from Wikipedia, but from the Southern Poverty Law Center denouncing him. Harris, in hosting a conversation with Murray, was less interested in the latter’s actual views on race and science, but rather wanted to engage more on free speech, cancel culture, and political correctness.
Sam Harris on free speech and the far-left
To Sam Harris, these latter phenomena are anathema: Harris sees himself as a champion of free speech, and sees any and all attacks on free speech as deeply illiberal and as attacks on our ideals as Western society. If I were to try to synthesize Harris’s entire anti-PC thesis as succinctly as possible, it would look something like this:
The far-left believes in ‘moral truths’, where if a fact seems morally reprehensible, it must be untrue, and PC culture attempts to inject moral arguments above rational arguments.
In reality, it might play out something like this, as it did during his interview with Yasmine Mohammed: To the far-left, any criticisms of Islam are racist and Islamophobic, and therefore wrong, and that supersedes ‘good-faith’ criticisms against Islam such as asserting that hijabs are oppressive to women, and that female genital mutilation should be denounced.
To many — myself included — these are reasonable points, or at the very least reasonable matters to discuss. Along these lines, Harris goes out of his way to provide a platform for those who he believes have been unjustly ‘cancelled’ by the far-left. His guests have included the likes of Yasmine Mohammed and Bari Weiss, who have both experienced the sharp ends of attacks from the left — their would-be (and should-be) allies.
But as they say: freedom of speech is not freedom of reach. And Harris, by hosting these guests, is implicitly giving them and their views a platform — and quite a big platform at that, Making Sense has been as high as #11 on the iTunes podcast charts, and his other podcast Waking Up has been as high as #4. And it was on Waking Up that Harris hosted Charles Murray back in 2017. Harris’s naming of that episode — Forbidden Knowledge — hearkens directly back to his worldview that no topic that is off limits for discussion and exploration and that moral arguments shouldn’t suppress ‘real’ facts.
On its face, I understand and am sympathetic to this position. However, where I believe Harris errs is that he takes Murray’s proposed ‘facts’ at face value, and doesn’t really seem to dig into them enough.
Barry on breaking science
There are many breakdowns of how Murray’s work features faulty logic and veers into pseudoscience, the most accessible of which is this piece from Vox which came directly on the heels of the airing of Sam Harris’s podcast episode. This led to quite a squabble between Harris and Vox, ultimately culminating in an intense but rather uninspired sparring match/joint-podcast episode between Harris and Ezra Klein — then Vox’s editor-at-large. In the original article, most of Vox’s piece was aimed downstream of Murray’s work: in short Murray is overextending (by far) what the evidence shows. However, I feel inclined to aim upstream: why abide by the assumption that Murray’s use of the scientific method is sound?
Much of science today sits upon the foundation of ‘statistical significance’, which can be thought of as the likelihood that a certain observation occurred just by chance. So when a scientific publication comes out with an amazing new finding and it’s buttressed by the line “results were statistically significant with p<0.05”, what they are saying is that the likelihood that there finding occurred due to chance is less than a 5%.
Below is my all-time favorite and by-far most-cited xkcd cartoon; It illustrates how our concept of statistical significance can bump up against the sheer number of hypotheses we can test.
Tying this back to the foundation of statistical significance, we see that if you arbitrarily test twenty hypotheses, you can expect that one of the twenty (i.e. 5%) would pass a significance test with a 5% threshold! As xkcd humorously points out, news would certainly pick up only on the anomalous (but expected) result. Even more confusingly, you could arrive at a similar conclusion using twenty different colors of t-shirt, instead of jelly beans.
As you can see, if we wanted to, just by testing enough hypotheses, we can make anything seem ‘true’. This also works further downstream — if you test the same hypothesis on an endless number of data sets, eventually one will come up with the answer you’re looking for. This is called data dredging or p-hacking, and it doesn’t necessarily have to be malicious. An experimenter’s hypothesis may have been proven wrong, but they run just one more test, and then just one more, instead of throwing out the data and starting anew (as they should be).
But if someone were malicious, it’s easy to imagine the tests someone can run: differences amongst races in English grades, math grades, SAT scores, mandated state testing scores, reading levels, IQ, college GPA, high school graduation rates, etc. etc. etc. And eventually one of these will ‘stick’ and overcome the threshold of statistical significance. P-hacking can eventually get you almost any answer that you’re looking for. That’s why you end up with books such as How To Lie With Statistics. To be clear, I have no evidence that Murray did any p-hacking or other chicanery to get the results that he did, but it is also not hard for me to imagine that he would.
People like Charles Murray and by proxy Sam Harris like to hold up the scientific method as infallible — but that isn’t quite true! In fact, the scientific method is extremely fallible, and while we may have strong principals and rules around how to perform experiments, our rigor in considering what is a good hypothesis has become extremely lackadaisical. Murray is almost certainly on the wrong side of science, and Harris — enamored with the idea of platforming someone cancelled by PC culture, ignores that fact (to give it a very generous read).
In the case of Murray, the question boils down to this: Does it make a person racist to publish an inconvenient truth about differences in IQ tests amongst races as Harris purports, or can a person who is racist find enough data and enough tests to p-hack their way to justifying their racism as Vox does? I for one, land on the side of Vox. How about you?
For more on p-hacking, bad science, and why good science is hard, check out this piece from FiveThirtyEight:
And for an example of a valuable podcast from Sam Harris that explains why it is hard for me to drop him, check out this episode:
