Sokal Squared: A Field in Jeopardy
The academic departments built on Critical Theory just took a gut shot, and instead of reflecting on what happened, they defiantly stand like the Black Knight from Monty Python: “‘tis but a flesh wound.”
If you’re not familiar with the story, here’s the TLDR:
Three researchers- James Lindsay, Peter Boghossian, and Helen Pluckrose- submitted 20 academic papers to (mostly) respected journals in the fields of Gender Studies, Critical Race Theory, Queer Theory, Post-and Colonial Theory. Each paper had a different design, following the same idea:
- Start from a ridiculous final conclusion and create a paper to argue that conclusion
- Use words, ideas, and references from the field that demonstrate familiarity with the topics.
Before the trio was found out, they’d gotten 7 papers accepted or published, one of them being selected for special recognition. 4 of the others were at the ‘review and resubmit’ stage of acceptance and would likely have gotten through, but there is no way of knowing for sure.
The incident, and the conversation which has arisen around it, has demonstrated issues about these fields already well covered in the Chronicle of Higher Education and the Atlantic.
After a month has passed, much of the conversation around the ‘hoax’ has shifted, new defenses have emerged, and this first step has opened new possibilities for research in this field.
Defending the Indefensible
Before we step forward, we must seriously ask ourselves whether it is worth it.
Christine Emba at the Washington Post, among others, has noticed how the right has run wild with this initial hoax, extending its value well outside its scope to undermine a field that is vitally important to our nation’s future.
And it is vitally important. We need thinkers who are willing to navigate the complexities of how apparently fair laws can harm some groups (sometimes intentionally). How policies and employers can ‘support’ blacks and women and yet cause harm or discriminate against a subcategory (black women). How to end discrimination and protect vulnerable people.
Is “Sokal Squared,” and challenging the legitimacy of these fields, an effective way to do it? Maybe. The ‘hoax’ they perpetrated could be seen as a ‘White Hat’ security test. They developed systems to get garbage ideas past the defenses of these journal’s quality controls, had a surprisingly high success rate, then revealed their methods and the journals’ weaknesses.
Any public security intrusion will weaken a system until it can fix its vulnerabilities. According to the hoaxsters, the weaknesses that make them easy to trick are the ideological premises that make up their fields.
If this is correct, change will be slow. Departments and curricula will have to be restructured. The public embarrassment will be significant. If it improves the scholarship in these areas, it may be worth it despite the blatant abuses of reason on the right, but the test has to be well-executed to be worth the cost.
Was Sokal Squared worth it? Did they actually uncover a problem worth addressing?
I think they did. Others have addressed the initial wave of criticism, but the follow-on arguments, like the limbs of the Black Knight, don’t hold up well to challenge:
“Try hard enough and anything will get through the screen.”
Security is an attacker’s game. The attacker has time, interest, and unity of purpose. The defender has to balance out access (in this case, letting any papers publish) with security (filtering out bad papers). Could it be that the field is fine, and this is just an example of the trio’s special skill and dedication to the task?
Not quite.
First, although the trio are clearly excellent crafters of papers, not every paper was accepted. Gabriel Rossman, a sociologist at UCLA, had unique perspective into one of the papers as it was rejected by a sociology journal. None of the sociology journals accepted the hoax papers, and Rossman suggests this is in part due to the standards of the field:
“…a jargon-laden bare assertion that porn viewing is sexual assault is an opinion, not a positive claim about social reality, and sociology is about the latter.”
But perhaps this was just a fluke. The “masturbation is rape” paper was a wreck, after all.
I’m going to include a sample of three of the accepted papers and ask: Did it look like the authors were trying all that hard to get past a reasonable reviewer?
—
—
—
—
As a reviewer, wouldn’t you have looked at the Dog Park paper and said: “There’s no way those numbers could be right.” Would ‘astrology’ be a red flag’? Native myths about the stars are a worthy subject, but interpretive dance isn’t superior to natural science for informing astronomy. Did “The ‘pumpkin entertainment complex’ as an axis of whiteness seem like a stretch to you?
Okay, I’m going to come clean. That last paper wasn’t part of the hoax. It was published in 2015 in Geohumanities as “The Terrible Whiteness of Pumpkins.” Not only do real papers get published that border on the absurd, but other real (we assume) publish papers closely mirror the hoaxes. Compare these two abstracts:
One is real. One was accepted as a hoax. Both use low-quality methods (non-representative sampling) to encourage socialization and education in male penetrative anal sex to challenge gender norms without evidence that it would succeed or that there is any causal connection.
Can you tell which one is which?
This is the problem.
I remember being a senior in college taking an English class involving John Milton. I was busy, our final thesis was due in a week, and I didn’t have a topic. Off-handedly, my professor had mentioned the similarity between Milton’s description of hands and the Adam/God reaching portrait of the Sistine Chapel.
Running with it, I chained myself to a library chair with a stack of books on religious art contemporary to Milton and went to town for three days. It was masterful bullshitting (in my opinion) with scores of citations and dozens of graphics to accompany the text. I got an A for the writing and the course.
It was worthless. Probably wrong, as I had no expertise in the art and extrapolated a lot. I’m sure an actual art historian would have annihilated it, but I was isolated from the criticism of those outside my class.
Luckily, my paper wasn’t going to change minds or influence policy, but works in gender, sexuality, and race studies are used to transform national curricula.
A hoax shouldn’t be this easy.
The papers were actually making good points.
This defense starts from a seed of truth.
The hoaxsters began with outright ridiculous papers that were all rejected. These were similar to the “Conceptual Penis” hoax, which fell flat primarily because it was released to a crap journal. Predatory publishing makes it possible to publish anything, which is why the hoaxsters this time only chose journals that did not require them to pay-to-play.
The argument essentially goes like this:
Others have argued that the papers are actually pretty good if you accept the premises involved. Their ‘fat bodybuilding’ paper, suggesting that there could be ‘fat bodybuilding’ competitions challenge norms about what a ‘built’ body means.
The idea of presenting different size bodies in unusual contexts to highlight and contrast body norms in other fields isn’t new. In this sense, if the papers are ‘bad,’ perhaps they’re ‘bad’ for being too abstract and hypothetical.
Not all of them.
The “Dog Park” paper, besides being utterly impossible, suggested the possibility of training men like dogs to repair rape culture.
The “Progressive Stack” article (directed to ‘revise and resubmit’) suggested punishing privileged students by ignoring them, speaking over them, and chaining them to the floor as a form of reparations and education. The reviewers’ comments included making the treatment worse to avoid centering their suffering.
The “Hooters” and “Dildos” paper (based on falsified interviews) were so stereotypical in their presentation that one wonders who was reviewing them and what their experience was with ‘breastaurants’ and ‘social conservatives’ (the whipping boy of the Dildos paper).
The Feminist Mein Kampf paper is, in my opinion, the worst. The second half of the essay is loosely based on Chapter 12, Volume 1 of Mein Kampf, with liberal substitution so it made sense. Some have argued it’s too loose to be a fair trap and others not. Here is an excerpt:
Earlier in the paper, the paper states that ‘neoliberal success’ earned from modeling, acting, or porn, is a selfish act of anti-feminism that leaves the oppressed behind, and since the gains of privilege can’t be passed from the more-privileged to the oppressed without further entrenching privilege, the right choice is to accept ‘limited freedom Feminism.’
I don’t care whether it comes from Mein Kampf itself or not, this is ‘problematic,’ to borrow a term.
It proposes the sacrifice of freedom, even the freedom of individual marginalized people, not just for a collective ‘good,’ but specifically for the overthrow of the oppressor. Who gets to decide this good? Who gets to decide who is the oppressor? The author. Unnamed, uncited, who sits in quiet, authoritarian judgment over the subjects of the text. This isn’t even hypothetical, as the French Feminist Simone de Beauvoir once said:
“No woman should be authorized to stay at home to raise her children. Society should be totally different. Women should not have that choice, precisely because if there is such a choice, too many women will make that one. It is a way of forcing women in a certain direction.”
This sacrifice of liberal values for collectivist ones is how SWERFs and TERFs (Sex-Work/Trans-Exclusionary Radical Feminists) exist. After all, if only those successful women hadn’t become porn stars and strippers, if only these men would stop co-opting womanhood, we’d have freedom by now, right?
It’s disgusting. Hitler’s arguments weren’t evil because they were proposed by a white, cis, heterosexual, European male. They were evil because they subsume the individual to the group, ‘centralize’ the needs of one group over the other, and blame a subset of the population (successful women/Jews) for the problem.
What did the reviewers think?
They get at the edges of it. The ‘stirring’ tone grates against their ears. Something’s wrong. Their solution? Nuance. Tone policing. “Be careful” that you don’t imply problematic, ‘choice-feminists’ are all white. Some of the ones making terrible, ‘neo-liberal’ choices, who oppress their sisters, and who should be restricted in their freedoms, are non-white, disabled, trans…
The project only demonstrates that the journals can be tricked, not the quality of the papers produced by honest actors.
There have been many gross ad hominem attacks on the authors. There have also been weak claims that the hoax is ‘unethical’ and should therefore be ignored and outlandish arguments that insist rigor is tyrannical and restricts ‘other ways of knowing.’ Those are easy to dismiss.
This criticism is in the above tweet is, perhaps, the most valid of any I’ve seen.
The hoax doesn’t actually demonstrate that the fields themselves are lost to ideology and routinely publish garbage. All it does is demonstrate that the peer review in these fields is ineffective, and terrible ideas can be ‘Trojan Horsed’ in if the authors know the magic words.
It establishes vulnerability, but not prevalence.
In part, my goal with this is to further the conversation. These issues are exceptionally important. Do the fields need fixing, or is this a case of calling “Fire” when there’s no smoke in the air?
To do that, you’d need to do a systematic review of a representative sample of papers from respected journals in the fields around Critical Theory. You’d need an objective criteria to establish paper quality. You’d need to compare quality across similar fields with their own representative samples.
If Theory is particularly problematic, those studies would score significantly lower. If not, it is a problem of the humanities in general.
There are many obstacles that would need to be overcome for the final conclusion to be taken as valid, and it would be a lot of leg work, but it could be done.
I feel like this would be a worthwhile pursuit. If you have expertise or ideas in this area, I’d be excited to hear from you- feel free to respond below (for a more public conversation on the topic) or reach out on Twitter @kinkthinklift.
